Reports that AI models were used to uncover vulnerabilities in OpenAI systems have sparked debate about AI's role in cyber security.
Chris Anley, Chief Scientist at NCC Group, commented:
"At first glance, headlines about AI models being used to 'hack' OpenAI sound alarming. In reality, this appears to be a good example of the security industry stress testing the security of emerging technologies. The researchers involved were conducting authorised bug bounty work, using Anthropic’s AI tools to identify vulnerabilities in a controlled and responsible manner, before malicious actors could exploit them. This was not a case of autonomous AI independently attacking systems, but skilled human security researchers applying new tools to an established security practice."
"What's notable is not that a vulnerability existed, but that it was discovered, disclosed and addressed through a coordinated process. Security testing, red teaming and bug bounty programmes have long been essential mechanisms for strengthening software security. As AI capabilities advance, we're seeing these technologies increasingly accelerate security research by helping experts analyse code, identify weaknesses and test complex systems more efficiently. That creates significant opportunities to improve cyber resilience across the industry, provided the work is conducted responsibly and transparently."
"The broader lesson for organisations developing frontier AI is that rigorous testing before public deployment is more important than ever. Initiatives such as OpenAI's Daybreak programme recognise this reality by giving trusted security researchers early access to models so they can uncover risks, evaluate safeguards and strengthen protections before these systems reach wider audiences. Stories like this should be viewed less as evidence that AI is becoming uncontrollable and more as evidence that the industry is developing the right processes to make increasingly powerful AI systems safer and more secure."
Chris Anley biography
Chris Anley is Chief Scientist at NCC Group and one of the industry’s most respected security researchers, with a career spanning three decades. Since 1996, he has conducted thousands of penetration tests, code audits and design reviews across a vast range of platforms, languages and architectures for many of the world’s largest organisations.
In his role as Chief Scientist, Chris shapes and supports NCC Group’s global research programmes, while also pursuing independent research into emerging and complex security threats. He has published an extensive body of work, including foundational papers in application security, multiple books on database and application security, and influential research into artificial intelligence and machine learning.
Recognised globally for the depth and impact of his technical contributions, Chris continues to push the boundaries of security research and drive innovation across the industry.
