There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this and that and what Mommy would think and something about bedtime—but I am not so interested in that. What interests me is the punchline "...and therefore you can't have ice cream.". I'm only incidentally interested in the other stuff, insofar as it might determine whether I get ice cream.

In an obvious way, I'm showing rather poor listening skills. I did not understand most of what Daddy said, and arguably I'm even being rude.

However, in another way, this stance has a virtue. The virtue is that it holds fast to what matters, as best I know how, and does not give in to Daddy's framing. It sticks to what is real and concrete, what matters, despite a cloud of justifications. This is certainly not an unalloyed virtue, just as the POSIWID perspective is both useful and fraught . But it matters that although Daddy is a complex system with internal states, his internal complexity can, in one sense, be largely ignored and summarized simply as "an entity that prevents ice cream".

Sure, something something lunch and real food, but those are matters for your concern—what do they mean for me and ice cream? I want ice cream. You've made some sort of background assumptions about what implications "real food" should have, regarding the question of ice cream, and I am not on board with whatever those assumptions may be, even if I cannot immediately give you a detailed exposition of what they are and why I disagree.

In the TV cartoon of Charlie Brown, the adults talk, but only in trombone, like "wah wah wah wah": https://www.youtube.com/watch?v=oHdlS3XZLBo

The kids can understand what the adults are saying, but the two groups mostly don't participate in each other's worlds. The kids interact enough to handle the functional situation with the adult, but they aren't engaging deeply with the adult's considerations except insofar as they bear on the kid's situation.

Your bank tells you: Don't give out your passwords or account information on the phone. We won't ever ask you for that.

That's the advice. They don't say, "Evaluate whether the caller is legitimate, and whether the circumstances they describe are circumstances in which you ought to give them your bank account info.". That's not a good stance. That's hackable. Just don't give out your bank account info on the phone.

In " What just happened? Pragmatism and Pessimization ", Ngo details a history of clever reasons that producing more AI capabilities helps decrease existential risk from AI capabilities. We have had, and continue to have, a variety of such clever reasons, very often coming from people who also admit several percentage points of risk of the technology killing everyone:

To invert BoJack Horseman , when you put your glasses back on, all the differential acceleration just looks like acceleration.

(H/t Kave for the productivization of the idea of skip connections .)

People who are going "wah wah wah wah do more AI R&D" might say things like:

[we can accelerate some specific AI capabilities, and we should do that, so] Which AI capabilities should we accelerate?

[we can accelerate AI capabilities and keep them siloed enough that we get a lead and can invest in controlling those capabilities, and we should do that, so] How do we control the AI capabilities we're making?

And then we get tangled up in considering all that stuff. I conjecture that many employees of Anthropic allowed themselves to be misled in this way. I conjecture that lots of people all throughout the AGI existential derisking space get misled in this way. I conjecture that you've been hacked out of the obvious view—AI capabilities are dangerous, so more is worse—by Dario, by Sam Altman, by money, by your project manager.

My hope is that those people will try on the "wah wah wah" way of listening, sometimes. If you consider yourself in favor of human flourishing, notice that it doesn't make sense to build the thing that would eat everything including humans; and it still doesn't make sense even with a bunch of arguments that may have big holes in them.

To put this another way, I think it's often virtuous to have skip connections in your epistemology. Yes, there's a bunch of details to think about, and the details matter, and they can lead to nonobvious conclusions, and other people may understand things better than you do. So there's a blob of thinking that's good for you to do, a big blob of compute as it were, to think through all those details and considerations.

But it's also good to have some skip connections that mostly go around that blob of thinking, and directly get a say in final considerations. At some point in the past or coming months, you may face some decisions about whether to leave your job, speak out about the existential risk of AI, call for a pause, push back on arguments for advancing capabilities, or advocate internally and pressure leadership to support a pause and shut down capabilities research. You may have a sense like the following:

"AI is very dangerous. Maybe we shouldn't do AI. If someone says we should do AI, maybe they are wrong. (Maybe they are using motivated reasoning that's wrong, but confusing enough to confuse me if I already somewhat buy into their way of seeing the world, and not easy to refute in a short paragraph that can, in one stroke, overcome a network of rationalizations and distortions.)"

That sense is good, whether or not other arguments should change your final decision. In your decision-making, your fundamental values should get a say, and your simpler, more childish understanding of what people are arguing about should also get a say.

You probably agree in theory that naive views of a situation should continue to get some sort of say in final judgements. However, implementing this theory often takes active work and doesn't happen by default . There's a set of forces like social pressure, (reasonable) deference , skillful frame control from leaders, general cognitive miserliness , and Sinclair's Razor , which together make it natural to not track the hypothesis that you should stop increasing AI capabilities. Tracking a hypothesis takes work. In particular, it takes work to do these two things at the same time:

That work is worthwhile. Listening like a child might be a helpful tool in that work.

[The following will contain multiple strawmen of any one person's position. However, all of the following is based on things I have heard from many different people either at Anthropic or defending Anthropic's behavior, in a mix of some first-hand and mainly second-or-more-hand. What these stances have in common is that they all say "wah wah wah wah do more AI R&D". With that in mind, my thread of childish listening hears Anthropic, as a collective, say the following:] We have to make AI because we want to slow down AI. But we can't slow down AI because OpenAI won't slow down. If they say they'll slow down, we don't trust them. Did we talk to them? Nah, we don't trust them, there's no point. And anyway, there's China. And if they say they'll slow down, we don't trust them either. Did we talk to them? Nah. Other people are reckless. We are not reckless. That is why we should advance AI capabilities. At the right time, we will stop. But not before. If we stop before the right time, then we might not stop enough. The politicians, they won't want to stop. Did we talk to them? No, there's no point, they don't understand; we have to make them understand by not stopping, and then they'll listen when we say to stop. Well, we'll talk to them because we need them to enforce export controls because China is bad and reckless, so that we can race to the top. This is different from why other people want a lead in AI. They want a lead because they are greedy and reckless. We want a lead because we are good and responsible. We would do AI safety and we would do nice things with an aligned AI, or maybe our AI should take over the world. The only way to stop is to not stop. That way later you can stop more. Should we say publicly that we want to stop? Eh. Sure, I guess. I mean, it's in some blog post somewhere. What the heck do you want from me? What, am I going to spend 10 hours a day nonstop talking about how I love stopping AI research? I got stuff to do. There's AI capabilities to be developed. Plus, if we say we should stop, Trump might not like us, and then we won't be able to continue our research. Our research is good because it will help get a stop eventually, if it's the case that we get strong evidence that we can't solve alignment. We have to publicly politically support AI capabilities so that later we can support a stop. Dario has a plan. He's good. He'll stop. Our leadership is trustworthy and strategic . Alignment might be pretty doable actually. So more AI capabilities are good actually. Did we argue that? Well, someone here read your blogposts. We believe we have understood your blogposts and taken them into account, thank you for your concern. Argue it? Why bother? We know what is going on, and you do not know what is going on, so how could you know that we won't solve alignment? We have the private information, and you do not. Sure, of course there's a 40% chance we'll kill everyone, but if China does it, then there will be a 70% chance.

That feels like a fully general heuristic for ignoring second order effects? Second order effects matter more than first order effects all the time!

My steelman is that it's a good heuristic when there are intense illegitimate optimization pressures pushing towards making people believe the second order effects matter more than the first. But if that's actually your point then it feels unfair to apply it to AI2040 Plan A and Redwood's work on conceptual reasoning , where I expect these optimization pressures to be weak, and comparable in magnitude to illegitimate optimization pressures in the other direction.

I'm not saying to ignore second-order effects. I'm saying that one's ability to even remember first-order effects is actually at risk in the presence of illegitimate optimization pressures such as human-level persuasion and human-group-level persuasion. And therefore, it's good to learn the skill of maintaining (for yourself, or for your group) one mental mode, inter alia, in which you're able to remember firmly the first-order effects. By "firmly remember" I mean things like:

Also, I don't actually expect these illegitimate optimization pressures to be that weak in those two contexts. That doesn't mean I win the argument without thinking about any details, or that people involved should ignore the more complicated arguments. But I do think it implies that everyone involved should be on their toes in general, and in particular might benefit from sometimes listening to arguments in this childish mode (though also listening to the same arguments in other modes).

I expect it to be applicable to Redwood's work, but not to AI-2040's Plan A.

Maybe I'll write a post on Redwood's conceptual work explaining my position in more detail. As of now, I expect that a more conceptually capable AI would take over OpenAI when the time comes instead of attacking HF.

"AI 2040: Plan A" proposes that we stop all capabilities research, and then we continue almost all of the capabilities research, but also publish all the algorithms we're using.

Let's imagine, for just one minute, that we genuinely believe the human race will be dead in 10 years. What would a sane AI policy look like? Let's imagine:

When non-technical members of the public ask me what we should do about AI, I just look them in the eye and say, "Cruise missiles."

I don't actually think cruise missiles are the best AI policy. I think the military should actually park outside the data centers and start feeding GPUs into wood chippers. Obviously we'd want to get China on board, and have them feed their own GPUs into wood chippers. Can't do this without China's buy in.

(We'd only need actual cruise missiles if some country decides to allow Cyberdyne Systems to continue training a next gen AI. And please, conventional warheads only. It's also critically important that cruise missiles are a government act, because nothing less than an international coalition of superpowers even has a chance .)

Obviously we're going to have to regulate AI research, just like any other WMD.

I think, in the end, some clever person will still figure out how to do it, and get access to enough GPUs. So maybe we're all dead in the end. Maybe our species is just cooked. But I bet we could buy a lot of years if we were willing to have the military start feeding GPUs into wood chippers.

Does this hypothetical plan feel more coherent than "Wah wah wah do more AI research"?

It would be more coherent if policymakers agreed to do it and had leverage against those who didn't. How do we steer American policymakers towards such a plan?

I think there is a key point that is missing here: if someone gets the feeling that you are not taking their arguments seriously because they are misguided in some deep way that can't be addressed at the object level, some bad things could happen:

With the exception of Plan A, which I don't think belongs on your list, I relate to a kind of frustration with highly-complex arguments that end in "we should do capabilities research." Although I think listening like a child is the wrong approach because it can shut down communication: