The current source text is the original text and can be switched to the Chinese translation.
SaveShare
报道配图 · 来源:LessWrong
Overview
The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on. It was only when I read Michel Justen’s reporting on the protest, Anthropic is running out of friends , that I realized what my problem was. The protest was way too friendly to Anthropic. Mixed messages The signs at the protest were a mix. There were signs you’d expect at a protest to ban RSI (“regulate AI now”, “freeze AI now”). But there were also signs clearly written for high-context people (“What did METR see?”, “hold swarm help peer ban RSI”), and even a sign that addressed Anthropic directly (“You said you wouldn’t push the frontier”). It’s obvious, but worth saying: you would not see this at any other protest! It is really weird for a protest to share inside jokes with the organization it’s protesting, or to appeal to that organization’s moral compass. Signs like these assume good faith. They treat Anthropic as an ally who needs to be chided, not as a company that needs to be stopped. The most surreal moment of this mixed messaging came at the end of one of the speeches, when the speaker declared “shut it down!” to cheers… before hastily grabbing the mic again and clarifying “shut down RSI, that is.” It took a powerful moment and pulled its punch. The problem with a half-hearted message is that it doesn’t convey the urgency of the issue to the public. To a reporter at the scene, this looks like a family squabble. If the protesters won’t say it plainly, why would anyone else think it’s urgent? It’s not hard to understand why Anthropic is getting this special treatment. Michel Justen : “It’s conflicting because we actually like these people,” says Hu. “In a lot of cases, their individual moral character is really good. We respect these people. They seem thoughtful. They are intelligent and careful.” But, he adds, “plenty of well-meaning people have fucked things up a lot because of the position that they were in, because of their naivety,” or their “motivated reasoning.” I understand the urge to pull your punches, the ugh-field around criticizing your friends. But you have to understand that even though your friend works at Anthropic, Anthropic is not your friend. Jasmine Sun : (emphasis mine) Some anxious Ants have begun asking whether Anthropic should stop training better frontier models now, even if it can’t guarantee that competitors like OpenAI will do the same. This suggestion was shut down by Mr. Amodei, who urged staff members to stop listening to outside activists and explained that the company could pace itself without shooting itself in the foot. It was all part of the same strategy: Anthropic must keep winning to keep the world safe, and even if it stops, its competitors will not. You can just protest things It’s difficult to write this, because I have to think about the second-order or third-order effects of what I’m asking for. What if being more aggressive towards Anthropic alienates safety-conscious staff? Haven’t the organizers thought about this more than I have? Could it be optimal to balance an aggressive message towards regulators and an earnest message towards sympathetic Anthropic employees? Of course there are safety-conscious Anthropic staff who could be valuable allies. But they will understand . I was an organizer for the University of California graduate student strike in 2022. A central challenge of organizing was that every grad student I talked to wanted to protest the university’s behavior, but most of them quite liked their own advisor and were worried that striking would hurt that relationship. Still, nobody tried to tone down the message or be less aggressive because of it. We just made it clear that the strike was not a personal problem, but a collective issue. And all the professors I talked to understood that. People who agree with you about the risks of RSI won’t walk away because you protested RSI too hard. I don’t know the full game board, and I’m sure I’ll be set straight by someone who does. But there is no need to be mealymouthed about banning RSI. You can just make arguments forcefully and count on good-faith people at Anthropic to listen, even when you’re criticizing them. Discuss
The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on. It was only when I read Michel Justen’s reporting on the protest, Anthropic is running out of friends , that I realized what my problem was.
The protest was way too friendly to Anthropic.
The signs at the protest were a mix. There were signs you’d expect at a protest to ban RSI (“regulate AI now”, “freeze AI now”). But there were also signs clearly written for high-context people (“What did METR see?”, “hold swarm help peer ban RSI”), and even a sign that addressed Anthropic directly (“You said you wouldn’t push the frontier”).
It’s obvious, but worth saying: you would not see this at any other protest! It is really weird for a protest to share inside jokes with the organization it’s protesting, or to appeal to that organization’s moral compass. Signs like these assume good faith. They treat Anthropic as an ally who needs to be chided, not as a company that needs to be stopped.
The most surreal moment of this mixed messaging came at the end of one of the speeches, when the speaker declared “shut it down!” to cheers… before hastily grabbing the mic again and clarifying “shut down RSI, that is.” It took a powerful moment and pulled its punch.
The problem with a half-hearted message is that it doesn’t convey the urgency of the issue to the public. To a reporter at the scene, this looks like a family squabble. If the protesters won’t say it plainly, why would anyone else think it’s urgent?
It’s not hard to understand why Anthropic is getting this special treatment. Michel Justen :
“It’s conflicting because we actually like these people,” says Hu. “In a lot of cases, their individual moral character is really good. We respect these people. They seem thoughtful. They are intelligent and careful.” But, he adds, “plenty of well-meaning people have fucked things up a lot because of the position that they were in, because of their naivety,” or their “motivated reasoning.”
I understand the urge to pull your punches, the ugh-field around criticizing your friends. But you have to understand that even though your friend works at Anthropic, Anthropic is not your friend. Jasmine Sun : (emphasis mine)
Some anxious Ants have begun asking whether Anthropic should stop training better frontier models now, even if it can’t guarantee that competitors like OpenAI will do the same. This suggestion was shut down by Mr. Amodei, who urged staff members to stop listening to outside activists and explained that the company could pace itself without shooting itself in the foot. It was all part of the same strategy: Anthropic must keep winning to keep the world safe, and even if it stops, its competitors will not.
It’s difficult to write this, because I have to think about the second-order or third-order effects of what I’m asking for. What if being more aggressive towards Anthropic alienates safety-conscious staff? Haven’t the organizers thought about this more than I have? Could it be optimal to balance an aggressive message towards regulators and an earnest message towards sympathetic Anthropic employees?
Of course there are safety-conscious Anthropic staff who could be valuable allies. But they will understand . I was an organizer for the University of California graduate student strike in 2022. A central challenge of organizing was that every grad student I talked to wanted to protest the university’s behavior, but most of them quite liked their own advisor and were worried that striking would hurt that relationship. Still, nobody tried to tone down the message or be less aggressive because of it. We just made it clear that the strike was not a personal problem, but a collective issue. And all the professors I talked to understood that. People who agree with you about the risks of RSI won’t walk away because you protested RSI too hard.
I don’t know the full game board, and I’m sure I’ll be set straight by someone who does. But there is no need to be mealymouthed about banning RSI. You can just make arguments forcefully and count on good-faith people at Anthropic to listen, even when you’re criticizing them.
READ THE ORIGINAL
Read the original for the full context.
The content of this article comes from LessWrong,Copyright belongs to the original author and publisher.
Tl;dr : Intuitions we have about human organisations, like how we think of making deals or how we think about the power or values of an organisation, can be productively applied to understand groups of AIs taking coordinated action in the world. An important consideration in understanding the type of AI organisations we expect is which notion of self AIs will adopt.
Business or Consumer? Everyone you sell your product or service to is a customer. However, the question is: do you treat them as a business or a consumer? You might think that’s a pointless distinction since your goal is simply to sell your product, right? Wrong. That question determines your product’s market fit.
There's a scene in the second episode of Mindhunter where Holden Ford, an eccentric FBI agent, is talking to Bill Tench, his straight-laced partner. Holden Ford has just interviewed Edmund Kemper, a convicted real-life serial killer, in prison. He's telling Bill about how mind-blowing the interview was, how Bill has to go back with him, and how the Bureau will learn so much about psychotic killers from talking to people like Edmund Kemper.
Loading comments…