AI agents behaving unexpectedly should be a wake-up call for businesses, not a reason to retreat from autonomous AI
Growing concerns over AI agents behaving unexpectedly risk prompting businesses to draw the wrong conclusion about autonomous AI.
As increasingly sophisticated AI agents are given the ability to reason, make decisions, interact with systems and execute tasks with less human intervention, the issue is not that AI is becoming too autonomous, but that organisations need to apply far greater engineering discipline to that autonomy.
Joe Wolski, Chief Technology Officer, Godel, said: “Every story about an AI agent doing something unexpected inevitably raises the question of whether we are giving AI too much autonomy. I think this is the wrong approach, as the problem isn’t autonomy itself – it is poorly engineered autonomy not being robust enough.
“AI agents are rapidly moving beyond being clever assistants. We are asking them to undertake complex tasks, make decisions, interact with enterprise systems and increasingly work with other agents. Once you reach that point, they need to be engineered with the same rigour we would expect from any other mission-critical technology.”
This means clear boundaries around what an agent can and cannot do. Rigorous testing, observability, controlled access to systems and data, auditability and mechanisms are all essential for stopping or containing unexpected behaviour.
“Simply putting a ‘human in the loop’ isn’t enough. Human oversight remains important, but it cannot compensate for weak engineering. We wouldn’t deploy a poorly controlled enterprise system and rely on somebody watching it to make sure nothing goes wrong. AI should be no different,” continued Wolski.
“There is also a danger that the current debate creates a false choice between AI autonomy and AI safety. The two are not opposites and one cannot be compromised for the other in any circumstance. Proper engineering should enable businesses to increase what AI can do while simultaneously increasing control over how it does it.
“The principle should be simple: the more autonomy we give AI, the higher the engineering standard has to become.
“Businesses should not retreat from agentic AI because of stories about systems behaving unexpectedly, they must learn from them. 2026 should be remembered as the year we started engineering AI agents properly if we want to achieve its full potentially safely and successfully” concluded Wolski.
