AI systems present a governance challenge that is unprecedented in scale and complexity. Central to this challenge are the emergent properties of advanced AI systems: state-of-the-art, large-scale AI systems are beginning to demonstrate behaviors that even their developers did not design or predict. As systems become increasingly capable, it will become difficult to forecast how they could be misused, or the manners in which they will interact with society.

Severe and systemic power imbalances magnify the challenge of governance. State-of-the-art AI systems carry substantial commercial advantages, but their development requires enormous sums of capital—attracting primarily private rather than public sector or academic investment. As a result, leading AI and technology companies are racing ahead in research, using their resources to influence policymaking, and hastily deploying untested technology in the interest of securing (or maintaining) market dominance. Unless stronger oversight and more robust laws are implemented, we face the possibility that unfettered development of AI capabilities could lead to incidents of large-scale discrimination or harm.