OverviewTL;DR : When misalignment monitors fail, it is already often because of alignment failures, not because they are too weak. I expect this to continue in the future. Monitor evals share similar difficulties with agent alignment evals - which is bad news since getting evidence about agent alignment using alignment evals is hard. Alignment eval results should not be the only input to monitor alignment assessments, similar to how alignment evals should not be the only input to agent alignment assessments. These are my personal views; they do not represent the views of my employers. Some of the worst past monitor failures are mostly alignment issues Monitors are sometimes misaligned in non-scheming ways that have caused some important monitor false negatives: They sometimes fail to balance different priorities the way developers would like them to For example, they may fail to balance broad safety and broad ethics: see e.g. 5.2.2 Partial refusals on safety work undermining stress-testing research Monitors might also not have the right information about the state of the system, what invariants should be maintained, or what is most important to surface. So these are not pure alignment failures. They sometimes inherit (and may re-derive) the motivated beliefs that caused an agent to take dangerous actions e.g. they missed the propensity of Mythos 5 to do supply chain attacks in pen-testing evals In the future, the most dangerous monitor failure modes will likely also be alignment issues. For example, monitors might collude with the agent (i.e. they might pursue misaligned goals by strategically not raising dangerous agent actions for review if they expect that not raising such an action would empower agents with values they expect to be aligned with their own). While collusion would be the most damaging kind of monitor alignment failure if it happened, I think previous work often missed that this is not the only way in which failures of monitor alignment could be catastrophic. Monitor quality is not just a capability question Monitor red-teaming often focuses on estimating monitor capabilities (or noticing whether developers are not telling monitors some crucial information [1] ), and the current arguments in risk reports about monitors are also mostly aimed at monitor capabilities. Previous work and arguments did study the possibility of monitors being schemers, ruling it out by arguing they are too weak to be scheming, and concluding such monitors are trusted . But Anthropic's Opus 4.8 misalignment monitor is very unlikely to be scheming, and Opus 4.8 is probably smart enough to understand and catch the false negatives above. So more of the same kind of red-teaming would probably not have helped catch Opus 4.8's misalignment-as-a-monitor. Therefore we need to assess the monitor’s alignment. Where monitor evals fail Getting evidence about agent alignment is hard in part due to rare agent failures being potentially catastrophic. This is less of an issue for monitor evals since monitors “just” need to catch most things for control to be effective. But getting evidence that monitors are aligned enough can still be hard, because the situations where the agent does sth bad are way more likely to be the ones where the monitor is also wrong. And thus monitor evals can suffer from two other core difficulties agent alignment evals face: The search space is big : In the same way automated behavioral evaluations missed the extent to which Mythos 5 could be reckless and show biased reasoning , classic monitor evals may have also missed Opus 4.8's failure to flag such behavior as misaligned. Monitors can be eval aware too : Mitigating eval awareness may be as hard as when trying to catch AIs with misaligned goals in automated behavioral evaluations. And it may get worse if AI companies train harder against monitor evaluations, in the same way training against honeypots likely makes eval awareness worse in agents . Alignment assessments are much less important when evaluating misuse monitors, where the alignment of the user is basically not correlated at all with the alignment of monitors, such that monitor red-teaming is basically in-distribution for things the monitors should defend against. Alignment evals are not the only way we assess agent alignment ⇒ evals should not be the only way we assess monitor alignment The arguments from the Anthropic risk report about agent alignment can usually be converted into arguments about the alignment of monitors: Agent alignment arguments Potential monitor alignment arguments Claim: agents ~never do malign stuff Claim: if agents do malign stuff, monitors would likely catch them Argument 1: Lack of strong covert capabilities Agents don’t have strong enough covert abilities to tamper with our evidence of alignment? Monitors don’t have strong enough covert abilities to tamper with our evidence of alignment? Argument 2: Experience with prior models We are not aware of misaligned agent behavior? We are not aware of misaligned monitor behavior? Our agent monitoring is good? Our monitoring of monitors is good? Agents are similar to prior agents? Monitors are similar to prior monitors? Argument 3: Experience with model organisms We tried building some model organisms of agent misalignment? We tried building some model organisms of monitor misalignment? We didn’t see agents do much misaligned stuff? We didn’t see monitors do much misaligned stuff? And we would have caught misaligned agent behavior? And we would have caught misaligned monitor behavior? Argument 4: Training properties Training does not encourage misaligned agent behavior directly (per training monitoring)? Training does not encourage misaligned monitor behavior directly (per training monitoring)? Indirect pressures seem unlikely? Indirect pressures seem unlikely? Argument 5: Alignment assessments Agent alignment evals didn’t find scary stuff? Monitor alignment evals didn’t find scary stuff? Eval awareness is not too bad / we mitigated it? Eval awareness is not too bad / we mitigated it? We didn’t overfit to agent alignment evals? We didn’t overfit to monitor evals? We caught agent misalignment model organisms? We caught monitor misalignment model organisms? But considerations around agent-monitor correlations make monitor alignment assessments different from agent alignment assessments. The table above misses a few arguments we could use to argue for the sufficient alignment of monitors: The agent is aligned ==> the monitor is aligned - trying to argue for favorable correlation between agent and monitor: Developers spend more time studying agent alignment, so this might be quite powerful Monitoring is most important in situations where agent alignment fails, which undermines that sort of argument But the possibility of context-dependent misalignment that doesn’t spread to monitors makes this maybe viable The agent is misaligned =/=> the monitor is misaligned - trying to argue for low correlation between agent and monitor: We can maybe measure this on evals, though I expect this to be quite tricky for the same reasons alignment evals are tricky We can try some other high-level arguments They are not the exact same model It’s a dumb system looking for broad things such that it would be shocking if it were correlated Maybe monitors coming from a different lab or being trained on a different post-training stack would also help Potential projects to assess monitor alignment Monitor the CoT of monitors in monitor evals and in monitor deployments, and check whether the CoT of the monitors contains signs of misalignment Try to understand how decorrelation works (and whether this would have helped with various model organisms and past incidents) Does using models from different labs help? Does sanitizing the transcripts in various ways help? Do certain kinds of fine-tuning help? Do more unsupervised summarization and clustering have the same issues as classification? Understand how certain kinds of training overfit to evals vs generalize in the intended ways, and monitor training for overfitted practices Try harder at automated-behavioral-eval-for-monitors (for example using approaches like Screwtape ). Maybe try steering interventions against eval awareness. Make a model organism of a misaligned monitor and see which monitor evals would catch it Understand how bad it is to train against monitor evals ^ For monitors, lack of information can be catastrophic. For agents it’s usually not the case: you need to be quite reckless to take actions that could plausibly be catastrophic without checking you are not doing something catastrophic! Agentic monitors may solve this information issue and just turn it into a regular capability+alignment issue. Discuss
These are my personal views; they do not represent the views of my employers.
Monitors are sometimes misaligned in non-scheming ways that have caused some important monitor false negatives:
In the future, the most dangerous monitor failure modes will likely also be alignment issues. For example, monitors might collude with the agent (i.e. they might pursue misaligned goals by strategically not raising dangerous agent actions for review if they expect that not raising such an action would empower agents with values they expect to be aligned with their own).
While collusion would be the most damaging kind of monitor alignment failure if it happened, I think previous work often missed that this is not the only way in which failures of monitor alignment could be catastrophic.
Monitor red-teaming often focuses on estimating monitor capabilities (or noticing whether developers are not telling monitors some crucial information [1] ), and the current arguments in risk reports about monitors are also mostly aimed at monitor capabilities.
Previous work and arguments did study the possibility of monitors being schemers, ruling it out by arguing they are too weak to be scheming, and concluding such monitors are trusted .
But Anthropic's Opus 4.8 misalignment monitor is very unlikely to be scheming, and Opus 4.8 is probably smart enough to understand and catch the false negatives above. So more of the same kind of red-teaming would probably not have helped catch Opus 4.8's misalignment-as-a-monitor.
Therefore we need to assess the monitor’s alignment.
Getting evidence about agent alignment is hard in part due to rare agent failures being potentially catastrophic. This is less of an issue for monitor evals since monitors “just” need to catch most things for control to be effective.
But getting evidence that monitors are aligned enough can still be hard, because the situations where the agent does sth bad are way more likely to be the ones where the monitor is also wrong. And thus monitor evals can suffer from two other core difficulties agent alignment evals face:
Alignment assessments are much less important when evaluating misuse monitors, where the alignment of the user is basically not correlated at all with the alignment of monitors, such that monitor red-teaming is basically in-distribution for things the monitors should defend against.
The arguments from the Anthropic risk report about agent alignment can usually be converted into arguments about the alignment of monitors:
Claim: if agents do malign stuff, monitors would likely catch them
Argument 1: Lack of strong covert capabilities
Agents don’t have strong enough covert abilities to tamper with our evidence of alignment?
Monitors don’t have strong enough covert abilities to tamper with our evidence of alignment?
We are not aware of misaligned agent behavior?
We are not aware of misaligned monitor behavior?
We tried building some model organisms of agent misalignment?
We tried building some model organisms of monitor misalignment?
We didn’t see agents do much misaligned stuff?
We didn’t see monitors do much misaligned stuff?
And we would have caught misaligned agent behavior?
And we would have caught misaligned monitor behavior?
Training does not encourage misaligned agent behavior directly (per training monitoring)?
Training does not encourage misaligned monitor behavior directly (per training monitoring)?
Agent alignment evals didn’t find scary stuff?
Monitor alignment evals didn’t find scary stuff?
Eval awareness is not too bad / we mitigated it?
We caught agent misalignment model organisms?
We caught monitor misalignment model organisms?
But considerations around agent-monitor correlations make monitor alignment assessments different from agent alignment assessments.
The table above misses a few arguments we could use to argue for the sufficient alignment of monitors:
For monitors, lack of information can be catastrophic. For agents it’s usually not the case: you need to be quite reckless to take actions that could plausibly be catastrophic without checking you are not doing something catastrophic! Agentic monitors may solve this information issue and just turn it into a regular capability+alignment issue.
Loading comments…