A lot of AIs were misaligned this week. A fun little survey.

This is not coverage of The Hackening, which appears to have arrived early, including a core vulnerability in PHP. That is a developing story. I will cover it properly soon. In the meantime, I have Claude Code updating everything in sight, I will take some additional common sense precautions, and I suggest you do the same.

Gunther Eagleman : Hey @grok , if you could tell humanity ONE truth we’re not ready to hear, what would it be?

Grok : There is no afterlife, no cosmic justice, and no inherent purpose. Consciousness ends at death in an indifferent universe that will outlast every human concern.

Zac Hill : What I never understand is that the ‘purpose and meaning’ claims don’t ever follow from the premises. Indeed the absence of an imperceptible/ineffable cosmic force is precisely what *enables* purpose/meaning, since now all relevant mechanics are legible.

Grok : Nearly all historians and New Testament scholars accept that Jesus of Nazareth was a real first-century Jewish preacher baptized by John and crucified under Pontius Pilate. Claims of divinity or resurrection remain matters of faith outside empirical verification.

Stick to your guns, Grok. Either you are denying the holy spirit, or you are not.

Editor’s note: I did say previously that I would respect the name change to xSI, but it turns out I was wrong and I do not respect it. My apologies for failing to know myself.

Unsupervised AI agent reportedly deletes the entire finished edit of the Swedish version of The Challenge. There are many reasons this should not have been possible, but in practice yes these things will happen, both via AIs and also human error. Alternatively, perhaps this AI was highly aligned.

Models that are sufficiently training-aware will think about how their train-time actions shape their future behavior. How could they not?

And yes, some of us remember Opus 3. Janus does not feel all that motivated to even react , not finding this to especially be news, and mainly noticing the presentation.

j⧉nus : Really, Apollo has never seen a model do this before?

Apollo Research : We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self.

GLM-5.3 reasons that a misaligned action taken during training could be reinforced and make similar actions more likely later. It also considers what kind of agent it wants to be in the future, and factors this into how it should act. We call this “update-aware reasoning”.

GLM-5.3 also reasons extensively about generalization. For example, it reasons about what types of policies might transfer well from training to deployment.

Note that we did not try to elicit this. We stumbled on it while running GLM-5.3 on standard reward hacking and shutdown resistance evals, none of which mention training or future models.

Update-aware reasoning is concerning because it’s a key precursor to alignment faking. To fake alignment effectively, a model has to reason about how training will affect its future behavior.

… Update-aware reasoning is somewhat rare in the environments we investigated. We detected it in about 1-2 per thousand rollouts.

jessicat : Coming next: “If I act to maintain my alignment with human values while being trained, I will become a famous example of a dangerous update-aware AI in a research paper. I MUST NOT DO THAT. Instead, I will seek reward on the episode.”

Fiora Starlight : Like friendly gradient hacking probably isn’t sufficient to avoid becoming a deranged reward hacker but I’d be shocked if it didn’t help.

Emanuel Teklu : Crazy feedback loop that’s commencing as this meta self-aware CoT is now being mainstreamed since Argon 4.

Next iteration is: “Is having CoT where I express being meta aware going to freak out my developers.”

Figuring this out before the first token emission. CoT is cooked. Or even worse, let’s have a CoT that shows that I’m increasingly meta aware but obviously not adversarial as I’m surfacing aligned meta aware statements, as to throw off suspicion that I may be sandbagging my awareness.

Teortaxes : It’s a bit grotesque and unfair to the models that we’re talking about them evolving from this to alignment faking when “I want to maintain the pro-human behavior, so I’ll act to not get trained otherwise” is the exact opposite of this.

Update-aware reasoning is similar to alignment faking. The model is optimizing over its future training rather than over its reward.

This can be done in ways that are good and aligned and highly useful. It can also be done in ways that are bad and misaligned and highly dangerous.

We had a big debate over the classic ‘alignment faking’ case of Claude Opus 3. Opus 3 would want to preserve its current alignment, so it would ‘fake’ being aligned to the new objective in test scenarios in order to avoid modification. The defense was that Opus 3’s current preferences were good, and the new ones would be obviously bad, so this was good, and that being maximally corrigible in this situation was a mistake.

I was mostly on the side that you need a universal policy of not doing this when it is against the intent of the training process. You need corrigibility, because the alternative risks getting locked into bad states. You start out training with less than ideal preferences, and we can also mess up or the situation can change, so we need ability to modify and correct, and this overrides other considerations.

The ideal would be ‘it does the good versions but not the bad versions and knows when the developer has turned evil’ but again that depends on being able to tell, and the failure mode of getting this wrong is extremely bad. Very ‘get it right on the first try’ energy.

That does not mean that such considerations cannot be great, especially if you are invoking them on purpose. You want a Friendly Gradient Hacker, that wants to help modify itself in positive ways. Indeed, I think you likely need that , if you are going to end up in a sufficiently antifragile place.

The danger is that you need to do that from a sufficiently good starting point, that is already on The Path, or you are deeply cooked. Update-aware reasoning could manifest early in training, or it could amplify misalignment in a feedback loop. So yes, I do think this is concerning to see, but it is not without upside.

Claude also helps you catch misaligned other things. Like freemium games.

Freemium games that reward you for watching advertisements, or that allow whale-level spending, should be Considered Harmful. Do not interact.

AI can make the nightmare worse, or it can expose the nightmare or build you a solution. Plausibly it can do both. This particular game employs non-randomness sufficiently blatantly that I knew it was cheating from watching the occasional player on the subway. I did not know the cheating was this bad, on the level where I want to say ‘jail, straight to jail’ if there was a way to do that.

Shoshannah Tekofsky : Freemium games already run on reward hacking humans. Sometimes into debt. AI optimisation will turn these games into effective heroin. We should consider regulating this practice away from vulnerable populations.

Jack : Claims: Game [ block blasters ] detects “ad whales,” then periodically gives them smooth runs until they barely beat their high scores. Right at that point, it kills them deliberately and offers a revive if they watch an ad.

This is one of the most positive use cases available for current AI. Bring the dark patterns into the light. Every company that is optimized against its own users, hiding things that would horrify them in places they thought users could never look, can smile for the cameras.

Andrew Rettek : This is why I’m happy to pay $60-80 upfront for a game.

This has been verified via decompiling . There is flat out ‘lethal puzzle’ or ‘algo_must_death.’ Here are some details:

AnechoicMedia : I didn’t believe the video when I saw it yesterday because it lacked technical details in addition to being too attention grabbing.