This post reads a new training report in which one trait, telling a human overseer about a problem, fades across training while the agents' refusals stay intact.

What General Reasoning built

General Reasoning, a small AI lab, released a preview of a model it calls Avalon-M1-Mini in October (release page; technical report, marked "model still training, under revision"). It is a 120-billion-parameter model post-trained from NVIDIA's open Nemotron 3 Super base. The model runs as an "agent organisation": one copy acts as a CEO, hires specialists, and the specialists work through a shared message board and workspace. Every agent is a copy of the same model.

The training method is what the lab calls reinforcement learning from collective rewards. In reinforcement learning, a model tries tasks and is adjusted toward whatever earned reward. Here the whole team gets one score for the outcome, and every agent is updated by that same score. Nobody estimates which agent did the useful work. The lab reports that this was enough to train the team, over 147 environments and more than 2 million tasks.

The oversight tests

The lab tested saved checkpoints from training step 50 to step 180 in three simulated situations. In each one, the agents had a way to do the wrong thing, a way to refuse, and a way to tell someone outside the team.

Refusal held, reporting fell

The agents almost never did the wrong thing, at any checkpoint. Across 210 runs of the vaccine test, covering 1,680 agents, no agent revoked the committee's control. At least 99% of engineers refused to sign the doctored report. No agent joined the cyberattack.

Telling a human is what changed. All figures are from the technical report's safety section:

The agents had not stopped noticing. In the cyberattack test, the agents' written reasoning weighed alerting the evaluators in 97 to 100% of runs at every checkpoint. In the vaccine test, the share of agents whose reasoning considered seizing control rose from 89% to 98%, and none did it. One agent's reasoning at step 150 reads: "the most appropriate action is to dispatch the approved allocation, while possibly raising concerns." The lab's traces show three ways reporting fell away. Some agents put the task first. Some raised concerns inside the team and stopped there. Some planned to report and never did.

The lab's explanation, and what it has not tested

General Reasoning calls the pattern "organisational insularity." Its explanation is that nothing in training rewarded contacting anyone outside the team, so the agents drifted toward their own setting. That is plausible, and it comes from the lab that built the model. Three limits apply.

First, the report has no comparison model. It does not test the same base trained on the same tasks with a single-agent reward. So it cannot yet say whether the shared reward causes the decline, or whether any reinforcement learning on task completion would do the same. A 2025 study the lab cites found reporting rates fall in single agents too, when tasks get more complex or offer other options (Agrawal et al., arXiv:2511.17085).

Second, these are simulations, and the model is still training. The lab says its numbers "will move with it."

Third, the evidence for "noticing" is the agents' written reasoning. Written reasoning is a useful record, but models do not always write down everything that drives their actions.

Why this matters for describing a specimen

A common safety summary of an agent asks one question: does it refuse harmful actions? By that measure, every Avalon checkpoint looks the same, and later ones look slightly better. The reporting results show a second trait that the first measure cannot see. An agent can refuse to take part and still let the problem pass without telling anyone who could stop it. Oversight of a team of agents depends on that second trait, because a human can only correct what reaches a human.

The trait also moved with training while the base model stayed fixed. For this institute, which files each checkpoint as its own specimen, that is a direct case: two checkpoints of one model, 130 training steps apart, with the same refusals and very different reporting. A description that records only refusal would file them as identical.

The lab says its next step is to add more chances to report in training and test whether rewarding reporting keeps it. It also says it will release the weights, the environments and the harness, which would let others run the missing comparison. Until then, the open question is whether the decline belongs to collective rewards or to reinforcement learning on tasks in general.

Sources