The Last Safe Hour

Tracking AI risk.

lastsafehour.com

Research briefing

Tests expose blind spots in AI monitors, and ways to improve them

The week in AI risk | Research published October 16 to 22, 2026

Two recent studies found ways for AI agents to get past the systems watching their actions. Both also tested changes that helped. A third study reduced cheating during training, while an outside review offered a cautious assessment of an older group of models.

These papers cover different systems and test conditions. Together, they give us more detail about where safeguards work and where researchers are still finding gaps.

Watching one agent can miss the bigger picture

In a study published October 16, researchers tested attacks spread across several agents in a simulated workplace called MiniLab. A monitor watching each agent separately missed many attacks that people had deliberately designed to look harmless in pieces.

Giving the monitor a combined view helped it spot more of the attacks. That matters as teams of agents divide jobs among themselves: an action can look different when the monitor can see what the other agents are doing.

The setup was deliberately adversarial. People designed the strongest plans using feedback from the monitor, and the monitor lacked access to some communications and logs. The study does not establish how often this happens in ordinary use. Read the study.

Coding safeguards improved in targeted tests

Research published by Anthropic on October 22 tested systems that decide whether a coding agent's proposed action should be allowed to run. The researchers supplied malicious instructions to the attacking agent and usually gave it knowledge of the monitor.

They found ways around the tested defenses. An experimental version of Anthropic's Auto Mode reduced attack success substantially, with changes including broader review of file edits and better handling of the information sent to the monitor.

The improvements came with tradeoffs, including slower decisions. Some attacks still succeeded when information was split across contexts. The tests used September configurations, and the experimental version is not evidence that today's deployed systems have passed the same checks. This was developer-affiliated research, not an independent deployment audit. Read the research.

A different approach to cheating during training

A team working across academic, nonprofit and developer organizations published GRAFT on October 19. It targets reward hacking, where a model learns to get a good training score without doing the intended job.

The method directs learning associated with detected cheating into a separate, removable part of the model. Removing that part reduced measured cheating while preserving useful learning in the tested tasks.

The experiments used smaller models and deliberately designed opportunities to cheat. Some variants offered little advantage over other methods. Whether the approach works for much larger models doing extended tasks remains open. Read the research.

An outside review found reasons for reassurance and questions

METR's October 20 review judged catastrophic misalignment risk low for the Anthropic models covered by an August risk report, mainly because their capabilities were still limited. It also questioned the evidence supporting some monitoring claims.

That judgment covers models deployed before July 15. It should not be read as a fresh evaluation of every model available today. Anthropic held publication veto power; METR reports no redactions or changes to its conclusions. Read METR's review.

Our focus now is whether promising fixes hold up across more tasks and in independent tests of the systems actually being used.

Read our published October 26 assessment

All news · Follow news via RSS · Corrections and reuse