Artificial intelligence systems are becoming more capable and autonomous, raising new questions about how they can be monitored, controlled and secured. Speaking before the United Nations Security Council,
OpenAI CEO
Sam Altman warned that the rapid pace of AI development could create significant AI security risks, particularly if systems advance faster than institutions can respond.
Altman highlighted concerns ranging from human control and
autonomous AI to emerging vulnerabilities, concentrated technological power and the need for international incident reporting. He also called for stronger cooperation between governments, critical infrastructure operators and technical experts to address emerging AI threats.
Here are six key AI security risks highlighted in his remarks.
1. Losing Human Control Over Advanced AI
One of Altman’s central
concerns was the possibility that humans could lose control over increasingly capable AI systems.
He warned that AI could move so quickly that people may no longer be able to understand what systems are doing or intervene when necessary. He argued that developers should not train systems unless they can make a strong case that those systems can remain under human control.
The concern becomes more significant as AI systems become capable of handling increasingly complex tasks and making decisions with less direct human involvement.
2. AI Systems Becoming More Autonomous
The growing autonomy of AI systems presents another challenge for AI
cybersecurity.
Altman pointed to the possibility of AI systems improving themselves and future versions of themselves, describing this as recursive self-improvement. As the process of developing AI becomes more automated, the pace of progress could accelerate.
For
security teams and policymakers, increasingly autonomous systems could make monitoring, testing and intervention more difficult.
3. Emerging Vulnerabilities and New Threats
More capable AI systems could also introduce
vulnerabilities and threats that organizations and governments may not yet fully understand.
Altman called for secure channels between governments, critical infrastructure operators and technical experts to share information about emerging vulnerabilities and threats.
The remarks place information sharing at the center of preparing for AI vulnerabilities, particularly as AI becomes increasingly integrated into important systems and infrastructure.
4. AI Power Concentrating in Too Few Hands
Altman also warned about the concentration of advanced AI capabilities among a small number of companies, individuals or countries.
He argued that no single company or country should be able to use the most powerful AI systems to impose its worldview on others.
From a security perspective, concentrated control over highly capable AI could create systemic risks if a limited number of actors have disproportionate access to powerful technology.
5. The Need for Global AI Incident Reporting and Threat Sharing
Altman called for accurate and speedy AI incident reporting, including classification and reporting protocols that would allow countries to learn from failures before they become larger problems.
He also called for common standards covering AI capabilities,
risk assessments, safeguards and human oversight.
Such standards, he said, could help countries compare evidence and establish a shared understanding of what effective safeguards and oversight should look like.
6. AI Development Could Outpace Existing Safety Measures
The speed of AI development itself could become a security challenge.
Altman warned that rapid progress could move faster than institutions are prepared to handle. He argued that competition between companies should not become a reason to accept excessive technological risk.
He also stressed the importance of alignment, monitoring and safety guarantees as AI systems become more capable.
For AI safety, this means ensuring that security measures and oversight develop alongside increasingly powerful systems rather than being added only after problems emerge.
AI Security Moves Into a Global Conversation
Altman’s remarks put AI security risks alongside broader questions about international cooperation, human oversight and technological governance.
His proposed approach includes global standards for measuring AI capabilities and risks, incident reporting mechanisms, and secure communication between governments, critical infrastructure operators and technical experts.
As AI systems become more autonomous and capable, the challenge will not only be what these systems can do, but whether organizations and governments can monitor, understand and respond to the risks they create.
Login
