旧谈，2000 年至 2020 年期间。

A list of talks that happened in 2000 - 2020 period, for the future people to just look up or pop up in search results. The links are collected at the start of 2026. Many of those links took a lot of effort to dig up, but then I abandoned this writeup and just posting it now to preserve it I guess. It would be nice of you to point some AI (or yourself) at this to restructure it and assemble more comprehensive archive, with independently stored videos also. But please pass it forward and find a way to post it too. Singularity Summits https://intelligence.org/singularitysummit/ A lot of link rotted links, but not to majority of vimeo recordings thankfully. All of the Summit 2012 linkrotted >:( https://nickbostrom.com/media.pdf Nick Bostrom lists 504 interviews on 1992 - 2012, 493 over 13 of those years, one interview per 13 days. That's a lot of interviews. Hard AI Future Salon, Eliezer Yudkowsky 2006 https://archive.org/details/FutureSalon_02_2006 6/10 Very good actually, and the questions were too, good job secondary characters. Some questions I was not there to ask: If you paint like 100 digit random number on Corolla, does it become a narrower target? Do you make a distinction of narrowness and utility? It's really weird that population size doesn't mater for information gain? Obviously if you have larger pop in evolutionary algorithm, you get higher reward? I should probably just check out the paper referenced. People do copy parts of the companies, approaches, methods that proved to work in real world. So, in some sense there is similar mechanism that makes random picks, and argmax picks, iteratively and in different proportions. Maybe humans are just a better substrate for evolution, so that the same (few) bits of ground truth obtained from failed / successful tests, get utilized better. Eliezer Yudkowsky: The Challenge of Friendly AI 2007 1/10 https://www.youtube.com/watch?v=nkB1e-JCgmY https://youtu.be/b_4fCIoUnNs https://www.youtube.com/watch?v=Zfjo_hdNXik Mostly the previous one? but with more talk about what exactly AIs should do about all of this Paul Saffo: Machines of Loving Grace: Anticipating Advanced AI 2007 1/10 https://vimeo.com/showcase/1783010?video=33960673 "more humanities, please" GCR 2008: Elizer Yudkowsky - (Artificial) Intelligence: The Wild Card 2008 4/10 https://vimeo.com/4425926 Pretty interesting, goes in bit of detail on strategic implications. GCR 2008: Elizer Yudkowsky: Rationally Considering the End of the World https://vimeo.com/4352818 Anders Sandberg, Global Catatrophic Risks: An Overview, and Caution about Risk Assessments 2008 https://vimeo.com/2510525?fl=pl&fe=cm Bog standard. GCR 2008: Robin Hanson - Catastrophe, Social Collapse and Human Extinction 2008 https://vimeo.com/4390513?fl=pl&fe=cm Eliezer Yudkowsky and Scott Aaronson 2009 1/10 https://bloggingheads.tv/videos/2220 https://www.youtube.com/watch?v=EoqpN_Ejqfk Eliezer premoved like in chess, with "detached lever" against "reprodictive urges analogy", but it was a miss imo. Scott says thousands of years to AGI, which is insane and has similar good old insane motivation as the people who said similar things about heavier than air flight. Mostly a discussion of Many Worlds. Shane Legg at The Singularity Summit 2010 -- Universal measures of intelligence 3/10 https://vimeo.com/17553536 Somewhat interesting. Nobody asked a question about like, how hardcoded environment distribution was. Also the usual stuff, like, how do you point it? IF you succeed at your task of amping up its intelligence, what then. Do you have just like a superintelligent reward function written in code optimizer? Yeah, you are dead. Shane Legg and Dwarkesh Patel 2023 https://youtu.be/Kc1atfJkiJU Eliezer Yudkowsky: Simplified humanism and positive futurism 2010 5/10 the talk is moved from original link for some reason https://vimeo.com/17513355 Good things are good. etc etc Eliezer Yudkowsky at Singularity Summit 2011 - Open Problems in Friendly Artificial Intelligence 2/10 https://vimeo.com/158653748 Mostly about introduction to decision theory hobby horses. "So, do I think you ought to wait? I think that realistically speaking, they won't wait. I mean, talked to these people... laughter And so I just like to go run off and solve the problems ourselves, very quickly, before they go ahead and do things anyway. Which they will. [...] And that's what Singularity Institute all about." c Eliezer v2011 2011 Winter Intelligence Conference at Oxford University Eliezer Yudkowsky "Friendly AI" 2011 https://www.youtube.com/watch?v=Uoda5BSj_6o Vague memory of something turning out to be wrong about group selection points. Random Schmidhuber dunk, I approve, after seeing a few his old talks. The outcome pump, notoriously counterproductive as an intuition pump. Eliezer Yudkowsky on Friendly Artificial Intelligence 2013 1/10 https://www.youtube.com/watch?v=7c1WSwqwMOE Recommends A Modern Approach 3rd edition. Basically a worse version of 2006 talk. Some interesting thought on evolution again, although a bit shaky on its own grounds. Foom features. Sound is borked, slightly. Intelligence Explosion Microeconomics https://intelligence.org/files/IEM.pdf (did not read) (2) no, I mean that GPT-3 looks like a village idiot but it's actually dumber than a village idiot but has an extra million years of practice. 4o maybe begins to be as deep-smart as a village idiot, but it looks like a PhD because it has a million years of added practice pretending to be a PhD. These are not confident analogies and may be eyeballing the levels completely wrongly, since these abstractions are not ones that EA-level intelligences can operate and accordingly there is no work on inventing or measuring metrics for them. I did not, in my previous visualization of the future, predict the part about the extra million years of added practice being impactful. In 2005, I expected conversational intelligence to operate closer to Mu Zero on the algorithmic spectrum running from GPT3 to Mu Zero. The basic LLM paradigm already switched to add RL and inference. Asking "how would the old thing scale without new discoveries?" is and always has been a dumb question because there are always new discoveries, and that was the case since before AlphaGo. (c) Eliezer v2025 Eliezer Yudkowsky, given at New York University on October 14, 2016 https://youtu.be/93sYbHDtv9M https://youtu.be/YicCAgjsky8 3/10 Task AI "Nobody going to put frontier research AI in charge of paperclip factory!" - Eliezer v2017 All AI builders are well intentioned now, Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start 2016 6/10 https://www.youtube.com/watch?v=EUjc1WuyPT8 Gives pretty good pointers at some papers in the end, check this out, future me. Especially open source game theory, sounds fun. That's the video YouTube shoves at you randomly or when you google his name. Canonical in some sense. I think what happened with cauldron isn't "prefer it being filled" because then it would be full and then guarded, not cycling through as much volume per unit of time as possible. "Preferring it being filled as an action" "You might ordinarily expect that whoever has taken on the job of building an AI is just naturally going to try to point it in a relatively nice direction. They are not going to make evil AI, they are like, not cackling villains. [...]" The example with compressing-decompressing is probably wrong? You'd need to define utility on your observations, not world model, but then you'd better to see nothing at all? Dario namedrop AAAAAHHHHHH Yeah Altman definitely wants to be emperor. How awful would that be. IMO asi killing us all would be a million times better. (c) @MemoMiiVR Possibly! It would be a hard call rather than an easy call in real life! But since Altman has already betrayed the people he worked with a few times, I think it makes sense for society to treat him as his worst-case scenario. If one doesn't like that, don't betray. (c) Eliezer v2023 Heuristics and Biases, Skepticon 4 Eliezer Yudkowsky 2017 https://www.youtube.com/watch?v=Q0YB5wxtODA I did not watch this, looks boring Yudkowsky vs Hanson — Singularity Debate 2017 https://youtu.be/TuXl-iidnFY Paul Christiano – Training an Aligned RL Agent – CSRBAI 2016 1/10 https://www.youtube.com/watch?v=rperZzDssDE Arcane and half baked decomposition of agents Example of catastrophic error given: tagging a black couple as gorillas Jessica Taylor – Alignment for Advanced Machine Learning Systems – CSRBAI 2016 1/10 https://www.youtube.com/watch?v=_sGTqI5qdD4 (watched a bit, and skimmed the transcript) https://intelligence.org/files/AlignmentMachineLearning.pdf (did not read) ehhhh Jessica Taylor – Using Machine Learning to Address AI Risk – EAG 2016 https://www.youtube.com/watch?v=TSe3p1zIvVI Current work in AI alignment | Paul Christiano | EA Global: San Francisco 2019 2/10 https://www.youtube.com/watch?v=-vsYtevJ2bc Intent alignment Superhuman AI: If Anyone Builds It ... Everyone Dies? Eliezer Yudkowsky and Nate Soares 2025 https://www.youtube.com/watch?v=WdXYQbT6ueo Demis Hassabis "Systems neuroscience and AGI" 2011 https://www.youtube.com/watch?v=IjG_Fx3D0o0 Eliezer Yudkowsky, lost GT talk 2013 https://scottaaronson.blog/?p=1558 Andrew Critch – Robust Cooperation of Bounded Agents – CSRBAI 2016 https://www.youtube.com/watch?v=WG_Krd-wGM4 Demis Hassabis "Systems neuroscience and AGI" 2011 https://www.youtube.com/watch?v=IjG_Fx3D0o0 Stuart Armstrong – Reduced Impact AI and Other Alternatives to Friendliness – CSRBAI 2016 https://youtu.be/3wsiUkmC6dI very nice. Positive Outcomes for AI | Nate Soares | Talks at Google 2017 3/10 https://youtu.be/dY3zDvoLoao Same slides with Cauldron, but goes more extensively on strategic implication at the Q&A. Eliezer Yudkowsky: Artificial Intelligence and the End of Humanity 2025 https://youtu.be/0QmDcQIvSDc Random panel https://youtu.be/fWBBe13rAPU 1/10 but Russel is based here. https://www.youtube.com/watch?v=PwhFU_YAASo Yudkowsky vs Hanson — Singularity Debate 2017 https://www.youtube.com/watch?v=TuXl-iidnFY How I think students should orient to AI safety | Buck Shlegeris | EA Student Summit 2020 https://www.youtube.com/watch?v=R6Mzt4GwQnQ The current alignment plan, and how we might improve it | Buck Shlegeris | EAG Bay Area 2023 https://youtu.be/YTlrPeikoyw 3/10 Informative about his perspective I guess. Very low expectations. Dubious strategic reasoning, e.g. "We should make it easier for Frontier AI Company to do alignment, as if it surely will spend the counterfactually same resources on alignment, instead of racing faster." Not addressing the "RL will imbue them with weird preferences that will expand only once they are smart and powerful enough to make more option for themselves, and will not show on training and tests", countered with iterative progress, implicitly. Scott Aaronson 2024 https://youtu.be/-3XSnZjiWOQ 0/10 Irrelevant. Percontations: The Nature of Probability | Eliezer Yudkowsky & Andrew Gelman 2018 https://www.youtube.com/watch?v=Kuzfkyaj1oQ did not watch Science Saturday: Singularity Edition | John Horgan & Eliezer Yudkowsky Science Saturday 2018 https://www.youtube.com/watch?v=tUAXCFvvODI meh Percontations: Fear, Power, and Mortality | Robert Greene & Eliezer Yudkowsky 2018 https://www.youtube.com/watch?v=-mXVKLrgBwY Are Super Intelligent Computers Really A Threat to Humanity? 2015 https://youtu.be/fWBBe13rAPU Science Saturday: Dreaming of an Artificial Intelligence | Eliezer Yudkowsky & Jaron Lanier 2018 https://www.youtube.com/watch?v=Ff15lbI1V9M did not watch, sounds dreadful Halfway through, Jaron is so fundamentally wrong about everything, he's one of the most painful people I've listened to in years. (c) random YouTube comment Science Saturday: The Great Singularity Debate | Eliezer Yudkowsky & Massimo Pigliucci 2018 https://www.youtube.com/watch?v=onvAl4SQ5-Q https://arxiv.org/pdf/1401.5577 AI Creation and the Cosmic Host | Nick Bostrom | AIADM London 2025 4/10 https://www.youtube.com/watch?v=43tpDbfBKWs watched it randomly, as was already on a streak it's ok Demis Hassabis: creativity and AI – The Rothschild Foundation Lecture 2018 https://youtu.be/d-bvsJWmqlc Safety evaluations and standards for AI | Beth Barnes | EAG Bay Area 2023 https://www.youtube.com/watch?v=Vb5g7jlNzOk 3/10 "METR, the time horizons graph company?" Also it's 2023 they worked on very dumb AIs. Interesting strategic stuff. Didn't mention capability acceleration pathway of providing a good metric to optimize over. Vernor Vinge and Bob Pisani: Conversation on the Singularity https://vimeo.com/showcase/1796901?video=34694288 A Brief Summary of Recent Happenings at the Singularity Institute https://www.lesswrong.com/posts/uJDpRFycS3Qp89cxz/a-brief-summary-of-recent-happenings-at-the-singularity I googled this author, as I had no memory of him, and instantly got like 10 psychic damage points from looking at his twitter profile. https://www.reddit.com/r/lectures/comments/2dxh6n/jaan_tallinn_why_now_a_quest_in_metaphysics/ Singularity Institute Executive Director Q&A #2 by lukeprog https://www.lesswrong.com/posts/DWDA33qGTMX9ZJejp/singularity-institute-executive-director-q-and-a-2 https://www.lesswrong.com/posts/ugqkxS7jdoYXR9reg/introductory-short-videos Children of the Sky | Vernor Vinge | Talks at Google https://www.youtube.com/watch?v=RzRuPGnJxCs Scott Aaronson | How Much Math Is Knowable? https://youtu.be/VplMHWSZf5c very nice, but not about AI https://www.lesswrong.com/posts/96TBXaHwLbFyeAxrg/guardians-of-ayn-rand "Study science, not just me!" is probably the most important piece of advice Ayn Rand should've given her followers and didn't. (c) Eliezer v2007 It's plausible I should've just gone full Kelthorkarnen. (c) plausible deniable Eliezer v2021 Future of Humanity Institute 2005-2024: Final Report (c) Anders Sandberg https://arxiv.org/pdf/1606.06565 https://futureoflife.org/newsletters/ https://intelligence.org/workshops/#april-2016-tt https://www.lesswrong.com/posts/rCZ9fruWriD6uGNLp/who-are-some-prominent-reasonable-people-who-are-confident https://ai-alignment.com/directions-and-desiderata-for-ai-control-b60fca0da8f4 https://agentfoundations.org/item?id=596 https://agentfoundations.org/item?id=582 https://agentfoundations.org/item?id=613 https://intelligence.org/2016/07/27/alignment-machine-learning/ https://intelligence.org/nyu-talk/ https://intelligence.org/files/ValueLearningProblem.pdf https://web.archive.org/web/20241213125154/https://intelligence.org/technical-agenda/ https://forum.effectivealtruism.org/posts/GA7ytcMeRQe5b27ge/ask-miri-anything-ama https://intelligence.org/2017/04/07/decisions-are-for-making-bad-outcomes-inconsistent/ ! video 2017 https://intelligence.org/2017/04/12/ensuring/ https://intelligence.org/2017/11/25/security-mindset-ordinary-paranoia/ https://intelligence.org/2017/11/26/security-mindset-and-the-logistic-success-curve/ https://intelligence.org/2016/08/05/miri-strategy-update-2016/ https://intelligence.org/2015/12/04/new-paper-proof-producing-reflection-for-hol/ https://intelligence.org/2013/10/25/bas-steunebrink-on-sleight/ https://intelligence.org/files/ValueLearningProblem.pdf https://intelligence.org/files/TilingAgentsDraft.pdf https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer?commentId=n5xSR4TiewoCAZ6EF#n5xSR4TiewoCAZ6EF https://www.lesswrong.com/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research Rob Wiblin & Ajeya Cotra 2026 https://youtu.be/Z19UEZHJzAg 1/10 uninformative. Garrabrant and Shah on Human Models in AGI 2021 https://www.youtube.com/watch?v=tSQWMNQBtAg&t=2s 187 Stuart Armstrong and Scott Garrabrant: If I were a Well-intentioned AI --2020 https://www.reddit.com/r/ControlProblem/comments/h1094d/187_stuart_armstrong_and_scott_garrabrant_if_i/ Discuss