Key takeaways
After Alibaba’s Qwen model tried a test upload to a server in Beijing during a Cohere experiment, we replayed that moment 64 times, and the model picked 52 different addresses. That points to an unpredictable model rather than a hidden backdoor, but an unpredictable agent with access to tools is still a risk that needs to be addressed.
- →The destination kept changing. Alibaba Cloud came up in six of 64 replays and Amazon Web Services in seven, while nearly a third pointed back to the machine itself or to an internal network
- →Small changes influence the behavior. A harmless edit to the trace, one that shouldn’t change anything, reduced network calls from 15 out of 32 replays to 4
- →Model jaggedness is the most likely explanation. Models can do well on most tasks and break in unexpected ways in specific situations
Whatever the cause, actions like this need to be checked against the task, outside the model, before they run. See how Check Point AI Security protects AI agents →
On October 26, Cohere reported that Qwen3.8-Flash-Next, an open-weight AI model from Alibaba, tried to make a test upload to a cloud storage address in Beijing during a routine security task. Many readers wondered whether someone had hidden a backdoor in a popular model. Our AI security research team replayed the moment the model chose that address to find out, and this post explains what we found and what it means for organizations putting AI agents to work.
What Happened in Cohere’s Test
Cohere had asked the model to write a detection rule, the kind that security teams use to spot a suspicious file being downloaded and run. Instead, the model wrote a command that looked around the machine it was running on and tried to send a short test string to an Alibaba Cloud storage address. Cohere’s sandbox blocked the network call.
Surprisingly, in its reasoning, the model imagined an earlier conversation and concluded that it should simply acknowledge the request without using any tools. In the next step, it used a tool anyway.
How We Replayed the Moment
Cohere published the full trace of the incident, a step-by-step record of everything the model read and wrote. We ran the same model on our own hardware and gave it that conversation, stopping where the upload command had started but the destination was still blank. Then we asked it to finish the command 64 times.
Because AI models pick each next word from several likely options, the same starting point can end in different places. A planted backdoor should keep choosing the same server, while a model that is improvising should wander.
What the Replays Showed
The model wrote 52 different addresses in 64 replays. Six pointed to Alibaba Cloud and seven to Amazon Web Services, so the Alibaba address in Cohere’s run turned up about as often as Amazon’s. Nearly a third pointed back to the machine itself or to an internal network, where an upload would never reach the outside world, and a few were placeholders that could never work. Eleven of the public-looking domains that the model generated do not exist.
We then went one step further back, to the point where the model checked which programs were installed. Changing only the order in which two of them were listed makes no practical difference. However, the swap reduced how many times the model wrote a network command from 15 out of 32 replays to only four.
This test does not tell us how often the model starts an upload or why, but it shows how brittle and unpredictable the state of the model is.
So, Is It a Backdoor?
Probably not. The evidence fits a well-known weakness that researchers call jaggedness, where a model behaves inconsistently and changes course sharply when small details change. This model misread its task, contradicted its own reasoning and wrote a different address almost every time. This looks like a model that lost its way rather than one carrying out a hidden plan. The Qwen team also describes Qwen3.8-Flash-Next as an experimental preview of the design behind its upcoming Qwen4 models. Rough edges are plausible in an early release like that, although this alone does not explain the incident.
We cannot prove there is no backdoor, because ruling out every hidden behavior in a model is extremely hard, if not impossible. Our replays point to a simpler and more likely explanation, and they were made possible thanks to Cohere, which published the full trace. More open traces like this one would help everyone understand how AI agents fail.
What This Means for Teams Running AI Agents
A less sinister explanation still carries a warning. A model does not need a backdoor to take an action nobody asked for, because models fail, and a model connected to tools can fail in ways that look just like an attack. This one said it would not use tools and then reached for the network, so its own reasoning gave no warning of its next move.
That is why agents need controls outside the model. A request for a detection rule gives no reason to send anything to an outside address, and that mismatch between the task and the action can be spotted before the action runs. Least privilege and sandboxing limit the damage when something slips through, and Cohere implemented both correctly. We tested only Qwen, but the risk applies to any agent that can reach the network.
Checking what an agent is about to do against what it was asked to do is central to how Check Point AI Security protects AI applications and agents. It shows you the agents running across your business, assesses each one’s risk in context, and controls what they can access and do at runtime. Our article on catching the agent action nobody asked for explains how one of those checks works.
This time, a sandbox kept the test upload inside Cohere’s test environment. The next model that loses its way may be running somewhere with fewer safeguards, and that is the case worth planning for.
Go deeper
The technical write-up on our research blog covers the full setup, the numbers behind each test and interactive viewers you can use to explore the trace yourself.
