Sometimes the following interaction plays out in public forums:
Lab Employee: I think there is a 20% chance of AI killing us all.
Cope Merchant (Quote Tweet): AI safety people are fearmongering, by pretending they have a hard scientific model, as part of their plan to [unknown]
Rationalist (Quote Tweet): But you see, a probability is just a report of their own epistemic state...
I sympathize with (some of) the words that the cope merchant is saying here. While in our subculture (and possibly Silicon Valley) the report "I think there is a 20% chance that AI kills everyone" means something cousined to "it seems unlikely but far from impossible", that is not all it's conveying to a regular person. To most normies, an ML researcher at a lab sounds like some sort of expert. And an "expert" that states a probability about something within their domain, is not just communicating an opinion on an event's likelihood. There's an implication that the expert arrived at it by some well-understood procedure, because most of the time the probabilities a normie reads are derived in this way (i.e. in rain forecasts, surgery odds, polls.)
This implication affects how most listeners update on the result, and probably does for you in regular life too. For example, if your financial advisor told you that your portfolio had a 0.05% chance of going bust, and then later on claimed that they had just "guessed" and ran no actual calculations, you would probably feel rather angry. Even if they explained that it was truly their subjective probability at the time, the misleading context might have made your subjective probabilities a lot different.
I don't know any great ideas for solving this particular public communications problem in the short term. It's hard to convey the alarming thing that these researchers want to convey, at the precision they want to convey it. Some things you could do if you were inclined:
In the long term we are all dead, so I guess these are better than nothing.
I think giving a probability range is superior to all of the approaches you mentioned. It succinctly intuitively conveys uncertainty in the estimate.
My p(doom) has been 10-90% for some time. It's currently more like 20-90%
This still isn't ideal because it's conflating personal circumstantial uncertainty with irresolvable uncertainty, however you'd try to define that. My numbers could just as easily be saying that barely thought about the issue. They are actually saying I've thought lot about it, in the thousands of hours, and I still think the the knightian uncertainty is that high.
But it's a heck of a lot better than just a single number with both types of uncertainty left unspecified.
"Let me use esoteric language to survive being quoted out of context: I have one omnicidal nightmare for every 4 utopian dreams."
Lab Employee's statement is fine (it's qualified with "I think..."). Cope Merchant erred by assuming Lab Employee implied a hard scientific model.
The financial advisor example is skewed with a very specific point estimate (0.05%) implying hard scientific model precision. Most Lab Employees citing their P(Doom) use round numbers (<1%, 10%, 50%, etc.) implying something more arbitrary. Many explicitly disclaim the impreciseness of their point estimate.
"Quoting survery statistics instead" does the same thing Lab Employee's doing, only diffuses the blame/hubris across multiple people. Cope Merchant is not going to be satisfied by the median expert response when they're certain 'all AI safety people are fearmongering pretending they have a hard scientific model as part of their plan to [unknown]'.
'Weaseling' or 'Refusing to answer' seem worse than saying what you think/mean.

Loading comments…