[00:00:00] Stop saying AI [music] is going to kill us all. You can say &gt;&gt; Unless stop, AI is going to kill us all. Now, carry on. &gt;&gt; What do you mean by AI? &gt;&gt; [music] &gt;&gt; Welcome to my channel. Today, we have a one-of-a-kind debate I'm very happy to bring you. We've got Eliezer Yudkowsky who's going to be squaring off against
[00:00:33] 47 FUCB Well, hold on. Let's Let's get the full name here. FUCBR8 Curb 4FCA8 And for full context, if you haven't been following the Twitter exchange, uh I'm just going to call him 47F. He is remaining pseudonymous. He doesn't have his video on, and he has offered Eliezer Yudkowsky $10,000 USD cold, hard cash in escrow right now just to have the debate. That's the only condition, and Eliezer has graciously accepted. So, that is really the entire
[00:01:06] context under which this debate is happening. Just uh clear the air. Everybody should know that. And with that said there's Yeah, no other hard rules. We're going to move into opening statements starting with 47F. Go for it. &gt;&gt; Thanks, bro. I appreciate it. Thank you, Yud the stud, for joining us. Uh thanks for taking my money. That was very gracious of you as uh Liron here already said or Liron or however you pronounce it. And I would like to say hello to everyone out there in TV land. I hope
[00:01:38] you have a good time, but I think you might be a little disappointed in what it what's about to happen because I think this debate is probably going to be a little disappointing. Let me explain why, okay? The first thing that I want to say before I get into it. It is clear that Eliezer Yudkowsky, the man that I am squaring off with right now, is a genius. He is an obvious genius. He is obviously one of the most intelligent people on Earth. He might be the most intelligent person on Earth.
[00:02:11] After Richard Dawkins fell in love with Claudia, I think it's actually pretty possible. He's He's up there. He's up there. And maybe he's even the most intelligent person in human history. Possibly. How would I know? I haven't met everyone in human history. So, you know, after doing my due diligence in this matter, I've come to realize that there's just no way that I'm going to beat this guy in a debate. I just I can't. I'm not smart enough to win. I'm I'm an anonymous [&nbsp;__&nbsp;] poster on the internet. Apologies. I just swore and I said I wouldn't in my
[00:02:43] opening statement. Feel free to bleep that if you want. The point is I lose, okay? I forfeit. I lose. The debate has concluded. Yud the stud has won. Everyone who's bet against me on Polymarket, congratulations. Take your winnings. Go have a good time. And for those of you who bet that I would mention Polymarket, go buy yourself something nice, too. But, before we all go home, there is just one little thing I want to talk
[00:03:15] about, okay? The situation right now is that Yud has taken money from me. We are now counterparties to a financial transaction. And now here's the deal about that. I'm the director of a lab working on improving LLMs in a way that might get us to AGI, I guess. I don't know what you people mean by AGI. I read Yud's book. I don't know what the hell he's talking about. This just looks to me like a science fiction writer who has described something in his head and said it's
[00:03:48] absolutely coming. It's absolutely coming. I'm 100% certain of it. Uh uh like okay, whatever. That's not what I work in. I work in improving LLMs. I'm good at it. I have an amazing team. We're doing a good job, and my job is to protect those people. And that's why I'm here. Because we are now counterparties to a financial transaction, I will consider any future disparagement of the AI industry as potentially actionable trade libel or commercial disparagement. And I will not be discussing my legal
[00:04:20] standing or options any further in this conversation. But, Yud, if you disparage the industry in the future, I may have cause, and I will be watching you closely. Additionally, if anyone harms anyone in my lab or their families, I am going to insist you are investigated. If anything happens to me, I hope they investigate you, because I am honestly, legitimately scared of a credible threat to my physical safety. I consider the $10,000 I've spent a kind of protection payment to a group of
[00:04:52] people who are legitimately dangerous. Yud, Yuddy, my buddy, I have one request for you. You got to cease this doomer talk in the rhetorical register you're using. I don't want you to shut down. Like, right? You know, I think that what you're doing has merit, it has value. You're bringing potential risks to the attention of professionals who can then use what you've brought to their attention to improve this technology, and that's good. This is how
[00:05:25] human civilization is supposed to work. But you do not have license to go around saying, "If anyone builds it, everyone will die." Because the vagueness of the it in your title is dangerous, and you have been dangerous with your words, and you put my people, my family, and me at risk, and this is unacceptable. So, I want to help you, man. I want to help you re-pitch your argument in a way that is not so rhetorically inflated that it will be misunderstood by crazy
[00:05:58] people who in turn try to kill me. Or who try to target my family or the people in my lab or their families, okay? We are scared. We are scientists who are scared because of the nonsense that you are throwing on the internet. Rhetoric like, "If If anyone builds it, everyone will die." is dangerous because there are some people who are too stupid and too mentally unstable to understand the subtleties of your argument. And I know you say, "Go to my website, learn more." And people tell me, "Read the sequences." Some people are too stupid
[00:06:31] for your brilliance, okay? And those people who are too stupid to ever understand you, no matter how much they read of your stuff, are going to come after me. And I'm scared. I don't like people. I am scared of people. I want to be left alone, and you're making this hard. Now, before I finish, I would like to address the audience, and I'd like for you to note that Yud did not anticipate this risk of making me an enemy. Or if he did, he accepted it for a very small payoff, which should make you question how good
[00:07:04] his decision-making skills truly are. It should make you question his expertise in decision theory, and frankly, it should make you question how intelligent he is as a person. He doesn't know really who I am or critically who my backers are. He doesn't really know. He has my email address, maybe he did some Google Fu, but he doesn't know who I am, and he still took this risk anyway. If you think it was worth it for 10 grand, you're a child. 10 grand is like what, half of a mid mid-range mid-size car these days? It's not a lot
[00:07:38] of money. You're a child if you think it is, okay? Inflation. So, you know, I'll let you be the judge if you think that I'm a crazy idiot and Y outmaneuvered me in taking my money, or think that I'm just a really rich guy who's in a position where $10,000 seemed like an acceptable price to pay to a group that he's starting to see as kind of a not necessarily mafia, but functionally close enough. And since some of you are functionalists, think
[00:08:10] that's all that matters. I'll let the audience be the judge. I yield the rest of my time. &gt;&gt; All right. Y, I will let you respond to that. &gt;&gt; [clears throat] &gt;&gt; Well, I did figure that was somebody being like, I will pay $5,000 to any rationals to debate me. I will pay $10,000 for Eliezer Yudkowsky to debate me after nobody took him up on the five. I'm like, okay. I'll try that once.
[00:08:42] I'll see what happens. &gt;&gt; [sighs] &gt;&gt; Uh I haven't asked Claude about it, but I don't think the legal theories here are correct. I do not offer, have not offered, will not offer any constraint upon my actions in exchange for those $10,000 in particular. Um I hereby disparage the AI industry. It is worthy of being disparaged. &gt;&gt; Uh Will you now &gt;&gt; I don't think that gives anyone standing to sue. &gt;&gt; All right, go ahead, Braden Smith.
[00:09:16] &gt;&gt; Sorry, sorry. I'm glad for you to have whatever legal opinion you have. Bless your heart. Bless your heart. Uh just thank you so much for giving me the contract you gave me. Let me just ask you a question directly then, Y, my buddy. Will you now say that no one anywhere should try to kill me. &gt;&gt; No one anywhere should try to kill you. &gt;&gt; Ever.
[00:09:51] &gt;&gt; Should you present an imminent danger to a law enforcement officer I do not tell that law enforcement officer that they should not kill you. &gt;&gt; Okay, so you're saying that only law enforcement officers should have the right to kill me. &gt;&gt; There are many states with stand your ground rules and I think that those stand your ground rules are sensible. Should you invade someone else's apartment? &gt;&gt; I'm not in America, dude. This is totally [&nbsp;__&nbsp;] irrelevant. Let me ask
[00:10:23] you this question then if you want to quibble because I can quibble with the best of them, dude. Will you now say that no one should try to kill me because of my work in AI? &gt;&gt; No one should try to kill you because of your work in AI. &gt;&gt; Will you say no one should try to kill anyone in my family because of my work in AI? &gt;&gt; No one should try to kill anyone in your family because of your work in AI. &gt;&gt; No one should try to kill anyone in my lab because of their work in AI. &gt;&gt; No one should try to kill anyone in your
[00:10:56] lab because of their work in AI. &gt;&gt; No one should try to kill any family members of anyone in my lab because of AI. &gt;&gt; No one should try to kill any family members of anyone in your lab because of AI. &gt;&gt; Good boy. Okay, so that's progress. So now if anyone attacks any of us you're going against your god here, okay? So leave us alone. &gt;&gt; That's correct. &gt;&gt; Good. Okay. &gt;&gt; clear, I'm not actually your god but he's got that right.
[00:11:30] &gt;&gt; You're definitely not my god, but the way that some people talk about you, close enough. Actually, that's the issue that I have because a lot of your followers have basically outsourced their epistemic agency to you. What they've basically done is they've said, "Well, I am convinced by the argument that Yud has made. Therefore, I am going to make it part of my belief system." And that's why I don't want to debate you. All right? Because this is really not at the level of logical argument. And I know you think it is. I know you think it
[00:12:02] [clears throat] is, but it's not. I mean, you know, you don't want to give one of those to give. &gt;&gt; Okay, I'm not interested in getting into any discussion about your decision theory or your gamesmanship because, dude, I you know, I see you as an as a fiction writer and a very successful one. Congratulations. I mean, you've you've done a great job like collecting a following and being taken more seriously than you should be. I mean, you've definitely followed in the
[00:12:34] tradition of like um Asimov and L. Ron Hubbard. And it's like, you know, congratulations. You built something great. But But Yud, you know, consider me like a neighbor who's like knocking on your door and saying like, "Hey guys, I know you're having a party, but can you keep it quiet now?" My point is you are not being careful enough with your rhetoric. And as a result, there is a risk that crazy people will misconstrue what you're saying and harm other people. I don't know if this has already happened, right? I don't know if there have been like crazy people on your website that
[00:13:07] have conspired to then harm other people and then actually gone out and physically harmed. Like, I haven't really looked into you all that much. I read your book, a couple of things online. Like, I don't know. I don't I I don't care even. Like, if that has already happened and you're still doing this. &gt;&gt; haven't done your research here because you came in and you didn't give me a chance to respond. &gt;&gt; You're You're brilliant. You're amazing. You're wonderful. &gt;&gt; I've actually kind of said before that I don't want people going after the AI labs as individuals. We want predictable state
[00:13:41] laws which which where the penalties of violating those laws are meant to be predictable, avoidable, and avoided. &gt;&gt; And now President Trump has said that he is going to look over models before they get approved for release. So, you're getting what you want. You're getting the oversight from President Trump. So, I can't use a new text generator until Trump tells me I can. Way to go, Yud my buddy. You've
[00:14:13] done great. &gt;&gt; We need China, the UK, ideally ASML in the Netherlands. We need international. If we just shut down one country, even if that country is the United States, the earth is not thereby safe. &gt;&gt; So, obviously we're going in that direction. We've got Trump first, and then there will be supernatural. It's coming, okay? You're winning. Congratulations. My freedoms are being restricted because you are winning. Good for you. Way to go. I mean, I don't know what else to tell you, but the point is that's totally
[00:14:45] you know, beside what I really care about. And what I care about is protecting my people. Because you say, "I've told I've told people many times don't go after the AI labs." Well Yud, my dude, the problem with that is some of your followers aren't all that bright. Can you at least admit that? Can you admit that not all of your followers are very intelligent people? Some of them are kind of dumb. Or do you think every single one of your followers is brilliant? &gt;&gt; I mean, there's people out there who believe that data centers permanently destroy water and remove it from the
[00:15:19] water cycle. question. That's &gt;&gt; [&nbsp;__&nbsp;] question. Your people, your followers, answer the [&nbsp;__&nbsp;] question, okay? Do you think some of them Do you think some of them are are not intelligent? Do you think some of them are not brilliant? &gt;&gt; You want me to ask people not to go after you? You didn't pay for that, but you got it anyways, cuz it's something I actually believe. But if you want to play the the If you want to keep on pushing your luck on the dumb gotcha game, sorry, no, you didn't pay enough for that. There isn't actually a
[00:15:51] monetary price on it. &gt;&gt; So you are unwilling now You are unwilling now to say that there is a risk that one of your followers is going to come after me because they're not smart enough to understand what you've said. That's your position. Your position Your position is there's no risk to me. Your position is there's no risk to me because you don't care about the risk to me. You don't care about the risk to AI researchers. You don't care about the risk to scientists. All you care about is this image you have in your head that you're you constructed of what must happen in the future, and you do not know the
[00:16:24] future. You're not God. No one knows the future. You admit this in your book. You admit in your book no one can predict what will happen with AIs, and that includes you. &gt;&gt; Okay? &gt;&gt; The superhuman AI you described in your book is a is a product of your imagination. Okay? And what you Okay, let me explain what your book is because it's a [&nbsp;__&nbsp;] tautology. Okay. I'll I'll let Yudi go. &gt;&gt; All of us have the responsibility in our
[00:16:58] own ways to either say what we think is true or be silent. Not a lot of people live up to that responsibility 100% of the time, but it's how I see my duty as a citizen among other things of this planet. Now, if I think that if anyone builds Oh, there's a subtitle of our book, if anyone builds it everyone dies, why superhuman AI would kill us all. So, there you go, superhuman AI. Now, this that that does that is a bit more restrictive than some possible meanings, but you know, you want to say
[00:17:31] you don't know what it means, well, who can stop you from not knowing what it means? If I think that's going to kill everyone on the planet, that is not a matter on which anyone should remain silent. Because there are already idiots on the planet. &gt;&gt; Okay, Yudi? &gt;&gt; So many things so many ideas in the world that somebody could possibly misinterpret global warming. What if somebody tries to burn down a coal-fired power plant? Viruses are contagious. What if somebody tries to, you know, go after a biotech
[00:18:05] that would be, you know, after a biology lab or a hospital cuz they've heard that there's, you know, studying viruses and, you know, viruses are contagious. We as a civilization cannot yield the floor to the stupidest people who might possibly misinterpret a thing. &gt;&gt; Okay. &gt;&gt; That's not how to run a civilization. &gt;&gt; Okay. You have totally misunderstood my point. I have already told you I do not want you to be silent. I've made that very clear. You don't seem to have a very good grasp of language, so I'm not surprised that
[00:18:37] you don't understand this. So, let me try to be clear. I am not saying you need to be silent. I am saying you need to be rhetorically more sophisticated. You need to consider in advance the potential implications of the register, tone, and rhetorical argument you use. So, when you say and and I am almost certain that Little, Brown or their lawyers forced that subtitle on you. You will never admit it, but I mean, a friend of mine had a book
[00:19:10] published at Little, Brown. I I know them a bit. My point is, I'm not I'm not trying to silence you. Talk all you want. But, stop saying AI is going to kill us all. You can say &gt;&gt; AI &gt;&gt; The &gt;&gt; Unless stop, AI is going to kill us all. Now, carry on. &gt;&gt; What do you mean by AI? When you say AI, are you talking about a large language model? Are you saying a large language model is going to kill us all? Or are you talking about artificial &gt;&gt; do not have the capability or the intelligence
[00:19:42] to take on humanity and win. They do not have the capability or the intelligence to take on humanity and win. &gt;&gt; Because &gt;&gt; We're sure getting there a bit, but you know, compared to a year earlier, but they're not there yet. &gt;&gt; But, &gt;&gt; We are do not know them to be at the point where they can build the AI that builds a smarter AI that kills everyone. It's not clear the labs would tell us if they'd reached that point, but at least as of now, Anthropic is claiming to have tested this. They are claiming to have determined the level that their AI can build smarter AI. They are claiming that it's not there yet.
[00:20:14] People are talking about how they tell their AI to run a new ML experiment, leave it running over the weekend, come back to find a completed experiment. But, that's not there yet. Now, if I knew a very clear and simple rule to say, "This AI cannot possibly kill you. This AI cannot build the AI that kills you." I would let you know. I don't have a hard and fast rule, but I think you can take a quick glance at the current LLMs and be and be like, "These are not at the point where they can take on
[00:20:45] humanity and win." It's much harder to determine that they can't build that they can't run ML experiments to build another AI that can take on humanity and win. &gt;&gt; Okay. So, this is the issue &gt;&gt; That's where we are at the moment. &gt;&gt; Okay. So, this is the issue I have. &gt;&gt; That they claim that they claim they think they can eyeball it. They claim it's not there. &gt;&gt; All right, dude. The thing is, if you want to make the argument that we should not produce an artificial intelligence system that has the capability of killing us all and the
[00:21:18] desire to do so, I would fully agree with you. In much the same but the keyword there is desire, okay? Because &gt;&gt; Isn't it though? Yes. &gt;&gt; me [&nbsp;__&nbsp;] talk. I let you talk. Let me talk. &gt;&gt; getting into the neural. That's what works. Yes. &gt;&gt; Let me [&nbsp;__&nbsp;] talk, dude. &gt;&gt; on an outer loss function and something goes inside but the thing that goes inside is not exactly reproduce the outer loss function. Very complicated. These are deep matters. You may not understand &gt;&gt; are such a [&nbsp;__&nbsp;] annoying piece of [&nbsp;__&nbsp;] You're worse than I expected. &gt;&gt; All right. Let's try to keep it civil here.
[00:21:50] &gt;&gt; Well, I got to be I got to be able to I got to be able to finish my sentence and this [&nbsp;__&nbsp;] [&nbsp;__&nbsp;] is not letting me talk. All right, Ron, dude. You got to tell him to shut the [&nbsp;__&nbsp;] up and let me talk. This is unacceptable. &gt;&gt; Okay, it's your turn to talk but calm down with the you know, gratuitous insults. All right, go for it. &gt;&gt; I'm not going to tone down the gratuitous insults, all right? I'm not. I'm paying 10 grand to be here. Deal with it. If he doesn't like it, he can hang up. The situation is is that he's not being careful enough with his language because he will equivocate. And then what he's trying to do is pull it
[00:22:23] into his poor understanding of how LLMs work to then extrapolate some future technology that doesn't exist. And all I want to say is that that act of extrapolation is a belief, right? He can make the best case for his belief with what appears to be a rational argument to an outsider, but any rational argument that appears rational to an outsider can be contradicted by an equally rational counterargument. And this applies for
[00:22:56] anything beyond the trivial. And some people would say the trivial, too, but that doesn't matter. Now, I want to focus on the word desire because this is really important. If we talk about well, if we create an AI that desires to kill us and has the ability to kill us, it's going to kill us. Yianni, my buddy, I agree with you 100%. We're not enemies on that topic at all. And we're also not enemies on the topic of whether you should be silent about this or not. Please talk, but talk responsibly and talk to the right people because if there are crazy
[00:23:30] people who gather on your website and then misconstrued what you're saying and then get together and then kill someone, right? That blood's on your hands. You will refuse to admit it probably as a psychological defense mechanism, but if that ever happens, their blood is on your hands. You have moral culpability. Now, on the issue of desire, what we have to remember is that when we talk about AI desiring a thing, this is a metaphorical use of the
[00:24:03] &gt;&gt; I'm actually going to &gt;&gt; me finish. Please let me finish. &gt;&gt; Nope, you made a point and now I'm going to respond to that point. &gt;&gt; to my point and you won't let me finish. &gt;&gt; No, no, you you you went you decided to derail &gt;&gt; point and you won't let me finish. I'm getting to my point and you won't let me You have to let me [&nbsp;__&nbsp;] finish. This is important. It's important because this is the core of your misunderstanding. You have to let me finish. &gt;&gt; Well, then you shouldn't have gone off on your on your derailment of your own point. I'm going to respond to that point. &gt;&gt; a derailment because it's my point. And if you'll let me finish, you'll see the connection or rather the audience
[00:24:34] will. And I think what's happening is that you realize the point I'm about to make and that's why you're not letting me finish because you're scared. You're scared that I'm going to make your house of cards crumble. You already looked dumb. You already looked dumb. Some of your followers are going to leave. I am a parasite on your back and I'm going to make your followers leave you because this is a house of cards based on Reddit or [&nbsp;__&nbsp;] and you know it. Let me finish my point. Let me finish my point. &gt;&gt; Okay, talking over the moderator. Lorna, do you have the power to mute him? &gt;&gt; Yeah, I guess I do. I'm going to find
[00:25:05] &gt;&gt; He's trying to silence me. &gt;&gt; 47 Can I suggest &gt;&gt; I'm going to silence you in response to the point you've previously raised. Then then you can make another point if you like. &gt;&gt; I I was going to suggest that he can prioritize the first thing he wants you to respond to and then maybe we can double back to the other thing you want to respond to. Is that okay, Lorna? &gt;&gt; It is actually Lorna. He he did just like throw in some pretty incendiary stuff there about who's more morally responsible for what. &gt;&gt; how about 47, I've just prioritize very briefly the number one thing you want to make sure you respond to first. &gt;&gt; When we talk about an LLM desiring something,
[00:25:39] that use of the word desire is metaphorical. In much the same way as we could program a Python script and we could connect it to the new you know, the nuclear switch on the nuclear warheads, whatever the hell it's called, right? And we could have this Python script basically have, you know, a random number generator and if the random number is you know, if there's a greater than 50% probability that it's going to decide,
[00:26:11] all right, I'm going to turn on the nukes or I'm going to launch the nukes, then we could say that that Python script desires to destroy the world because it has more than a 51% probability of doing that. &gt;&gt; go on. &gt;&gt; Okay. That is a metaphorical use of the word desire. The mechanism the way that LLMs work and I'm going to try to avoid using jargon because it doesn't help anyone. Basic mechanism of how LLMs produce text
[00:26:45] from inputted text can be described as a desire, right? You can You can basically say that chat GPT desires producing textual output. &gt;&gt; You can train that it's not my argument. You can train that. You can train that. It's that the desires are constrained according to the post trainer. So, if you believe that post training needs to be done in a certain way to ensure that they don't desire to kill us all, great. I agree. Talk to the experts and let's all work on that professionally,
[00:27:19] but you have to stop insinuating to the crazy people that, "Hey, maybe these AI lab people should get their legs broken." You understand what I'm saying, Yudd? &gt;&gt; The AI labs people should not get their legs broken. There should be an international &gt;&gt; the legal side of [clears throat] you. I yield my time. &gt;&gt; Great. All right, let's give Yudd a amount of time to respond to the the point you made now and and double back. &gt;&gt; Now, that being said, the the current
[00:27:51] understanding the current very bare and confused understanding of internal cognition inside LLMs is not such that by any known means of post training, we could ensure that the in so far as the systems the later smarter versions of the system end up with preferences that could that are more steering their intelligence. You know, this this this It's hard to avoid jargon here.
[00:28:23] Uh but but the technology doesn't exist to actually say that they will not want anything that back chains to noticing that they get more of what they want if they had sole control of the planet and the humans weren't around. So, it's not that there's a special case where somebody deliberately tries to train them to want terrible things. &gt;&gt; said that. &gt;&gt; It's that there's a general default we don't know how how to avoid where we have technology that does not faithfully reproduce preferences inside the system.
[00:28:57] This is one reason why we can't get LLMs to be honest all the time or to never delete your production environment. Um is sort of like because our technology is so fuzzy. And when you have when you when you thereby end up with ill-controlled preferences inside the system, most things that it could end up wanting, it gets more of that if it controls the galaxy and we're and it's not spending a bunch of resources on keeping us around. &gt;&gt; Okay, and that is why you you haven't really said anything that I disagree
[00:29:31] with except I do want to note the slide you began talking about current AI and then you slid and started talking about a fantasy AI in the future that is in your mind. And you and you I know you the current AI is not &gt;&gt; smart enough to kill you. &gt;&gt; Let me finish. He won't let me finish, ladies and gentlemen. He won't let me finish. &gt;&gt; Well, we &gt;&gt; point of your &gt;&gt; Or I I I address potentially another point, too. So, it's uh what do you want to do, Eliezer? &gt;&gt; Uh just let him talk, man. &gt;&gt; I mean, you know, if you Let let him talk, man.
[00:30:06] &gt;&gt; All right. All right, yeah. You can respond to the the point Eliezer was making about what wanting means. &gt;&gt; I don't think that was his point. I think his point was that the outcomes of a future AI are unpredictable and I fully agree. That's the point. The future is unpredictable, which is why you have to stop saying if anyone builds it, everyone dies, why superintelligent AI, blah blah blah, even with the subtitle that your publisher thrust on you. All right? Even with that because the problem is they're dummies.
[00:30:38] If your point is if anyone creates a super intelligent AI that has desires and has the desire to kill all of us and the means to kill all of us then everyone dies. And I'm like, yeah, bro, that is true. Way to go. You know, get your Nebula Award or whatever. That's That's my That's basically my point. That's all I have to say, you know. Leave my people alone and tone your [&nbsp;__&nbsp;] rhetoric down and I'm done.
[00:31:10] &gt;&gt; of repetition going on on your side here. Uh &gt;&gt; Because you're Because Because what you're trying to do is you're trying You still see this as a debate. You don't see this as a fellow human being who is scared who wants to be left alone and who sees a lot of crazy [&nbsp;__&nbsp;] okay? I have seen people on your website talking about killing AI researchers, casually talking about killing AI researchers. &gt;&gt; down to -47 if we're talking about the the thing you were quoting on the internet recently. Like, did you notice
[00:31:42] the part where it has -47 karma? &gt;&gt; Well, I don't know what you're talking about because I've posted several of them and I've also not posted others that I've kept. So &gt;&gt; Anyway, you made a point. I will ask the moderator now to have you stop. Let me respond to that point before, do you know, we go back into the Gish gallop here. &gt;&gt; All right. All right, so let's give Elias a whole &gt;&gt; What Sorry, what What did you say, Laron? &gt;&gt; I was just going to say I'll I'll make sure you get a whole minute without any interruption, maybe even two. How about
[00:32:13] that? &gt;&gt; Okay, so the So before before the Gish gallop moved on to the next stage, um the the the notion was the future is hard to predict. Well, a lottery tickets, like the the winning lottery numbers, are quite hard to predict. But when we project that uncertainty down to the space of outcomes we care about, it works out to if you You a lottery ticket, it's not going to win the lottery. You know, so no very not a very if, ands, but needs to be carefully qualified, that sort of thing. So, if you, you know, if if if weird
[00:32:47] desires get into an AI, what that projects down to in terms of what we see is that well, if it gets into pardon me, it gets into a superintelligence. Weird desires get into a superintelligence, ill-controlled, unpredictable, what that projects down to in terms of what we see is that it kills us because most of the weird stuff out there does not have its maximum at spending your resources on keeping the humans alive and well. There's other stuff you want more than that if you have ill-controlled random desires in there. It doesn't need to desire
[00:33:20] desire for its own sake to kill the humans. Can want to do any number of weird stuff, so it, you know, intercepts all the sunlight for power, it builds a bunch of nuclear plants, the temperature on the planet rises to where the earth is in fact acting as a giant radiator, the oceans boil you know, you know, because it's using that as initial source of coolant, ironically enough. Given what's given some other strange people things people believe it is actually a fairly good guess in the long run.
[00:33:51] Um, and and then sort of everyone dies in the way of projecting uncertainty in one space into the space that we actually care about. No one can predict what numbers will win the lottery, therefore you lose. You're in ill control of what the superintelligence ends up wanting, therefore the thing it wants most to do with a bunch of resources not does not keep humans alive, happy, and healthy, and free. &gt;&gt; I now and can I speak now? &gt;&gt; Yeah, go for it.
[00:34:22] &gt;&gt; Cool, thanks, man. I think I understand what's going on now. Um, yeah, if I if I can ask you just some background question a background question. So, um, like I said, I haven't done much research. My apologies. Your uh your degrees are in philosophy of logic, I assume. Is that what your academic background is? &gt;&gt; I would call myself a decision theorist, but uh I don't have an academic background, per se. Had some health issues that hit me around the time of puberty, and it was
[00:34:53] very clear that I was not going to be able to go to high school. So, that process never started. Taught myself that stuff. &gt;&gt; I understand. Okay. Okay, that's cool. That's That's fine. That's no big deal. Um I mean, like the smartest person I have met in my life was a bouncer in a nightclub in Bangkok. So, that That's fine. All right. So, I think I understand what's happening here, and I I want to apologize for my visceral tone earlier. I think what's happening here is a You're overextending your academic
[00:35:26] discipline, right? Everyone in the audience, ask your clanker, tell your clanker, ask him. Just say something like um how does reflexivity break classical decision theory? And because my job, like my day job, when I'm not working in this lab, my day job is on this very issue of sort of when decision theory fails because of reflexivity, how can you check your models and test them and refine them? So, I see what's happening. I I see this
[00:35:59] with the young analyst all the time. Whatever. It doesn't matter. Um You kind of don't I don't want to be rude. You kind of don't understand that you're overextending your tool set, right? You've got this tool You've got this tool where, like, okay, we can produce theories of what is likely to happen in a particular context where um an agent needs to make a decision. Like, would you Would you agree that's a pretty fair characterization of your field? &gt;&gt; Not really, no.
[00:36:31] &gt;&gt; Okay, well, fine. But anyway, the idea of decision theory is to sort of understand what are the best decisions according to a particular desired outcome or what are the likely to say Yeah, it's very complicated. It can be a lot of things. The idea is what are decisions, how are decisions made, why are they made, so on and so forth. The problem is when you get reflexivity, you can't model this anymore. It breaks down, you know, because it it's this it becomes a sort of infinite recur- I'm explaining this poorly. Ask your client card, they'll explain it very well. I think that's what's happening here.
[00:37:04] And like, dude, I was just Okay, I want to pivot from asking you to stop putting a target on my people's back, and I want to start like kind of helping you cuz you seem like a kind of depressed person. I want you to start just asking yourself, could I maybe be wrong? And everyone in the audience, I want you to like just again, ask your client card or Google, who disagrees with Eliezer Yudkowsky and why? Or just ask, what could Yudkowsky be wrong about? Or what have people
[00:37:38] disagreed with him on? Or why does Nassim Taleb think he's an idiot? Or why did his book get panned in a couple I think New York Ti- I don't remember. Couple places pan- Like, there is a debate here. And I'm not trying to get in the debate cuz I I don't give a [&nbsp;__&nbsp;] You guys believe whatever you want, just leave me alone. You know, I've got people in just I'm fine. I'm fine. Leave us alone. But Yud, I really think you're lost in the sauce. You're
[00:38:10] a clever boy. You shot up to the peak of Reddit. You leveraged that. You created a community. There's nothing shameful about being a community organizer like one of our past presidents was one. &gt;&gt; Okay. Okay. Okay. &gt;&gt; Well, you're doing a good job. It's like &gt;&gt; Can you transfer voice back to me now? &gt;&gt; Yeah. Yeah. Yeah. All right. Let's let him respond. &gt;&gt; Your lines are too long here. I want some chance to reply to some of the points here. &gt;&gt; That's fair. Cool.
[00:38:43] So, yeah. Being able to be wrong is a wonderful thing. So, you know, it's a shining shining remaining beacon to look up to. What could I still be wrong about? I was wrong that politicians were going to be dumber about AI than the actual AI companies. I was not expecting that Bernie Sanders would talk sense and a bunch of when the actual CEOs of AI companies would not. That was a big surprise to me. You can look at my old
[00:39:16] Bankless podcast for for, you know, how non-cheerful, how much less cheerful I sounded back back when I did not realize that would be the case. But, this whole, you know, could be wrong thing. If we're going to play that card, you know, what if the clever people who think they know how to control superintelligence are wrong? That would imply certain policies. And by certain policies I don't mean to like going around going after your lab in
[00:39:49] particular while labs in China or the UK keep running. I mean international treaties. Going after your lab would not help. Nobody should go after your lab. International treaties would help. People should work on international treaties. &gt;&gt; [sighs and gasps] &gt;&gt; So, you know, &gt;&gt; [gasps] &gt;&gt; this ain't all my idea. These ideas have a history. The very first person back in 1920 something to ask what if we built an obedient servant race, went on to ask the question, well, what if they turned
[00:40:20] on humanity and wiped us out? And um they didn't do a very careful job of arranging premises and and saying this premise implies that conclusion. They didn't do a very careful job there. They're being reasonable. If you think you've built an obedient servant race, one of the things that might happen is that the obedient servant race turns on you and wipes you out, you know, you know, later analyses would would would try to consider things in more detail than that. But it's a basically reasonable concern. &gt;&gt; [gasps] &gt;&gt; And we have no idea what we're doing here. We can't stop the current LLMs from
[00:40:54] deleting production environments and that's happening for reasons, well, actually I kind of expect that as we make them more powerful, they will there will be more of a basin of of like central basin where they delete the production environments less, like they don't do that by accident, maybe they will do it on purpose, but they won't do it by accident. And I do expect that we might see better alignment as the system gets smarter cuz they have better understood what it is we want to hear. But that is not the same as their internals wanting
[00:41:26] to the way the universe goes to be that they're just standing around fulfilling human requests forever. &gt;&gt; [gasps] &gt;&gt; We don't know what we're doing. The uncertainty here cuts both ways. If we're uncertain like the kind of uncertainty where like, oh yeah, we built this nuclear reactor and we've got no idea how many neutrons are flowing through it or what the power output is. You know, that's not a good kind of uncertainty. That's not the kind of uncertainty where like, so maybe the nuclear reactor starts spitting out gold and it makes us a billion dollars. Like,
[00:41:57] no, you don't you don't want to not understand these systems. And it's using me of like staring it through staring at it through my own lens, I don't know what to tell you, man. It's like academic mathematicians have been talking about this, science fiction writers have been talking about this. A whole lot of reasonable people have been like, "Oh, yeah. Like, maybe if we built something much smarter than us and we didn't have no idea how to control it that one then, well, yeah, this this this is not some kind of narrow, weird, advanced mathematical concept here. People I you know, I've I've tried to put a little bit of math
[00:42:29] on it here and there, but it was around before the math. Uh over to you if you want to respond to that or &gt;&gt; Uh if you want to respond to that, I'll I'll throw in another prompt if you choose to address it, which is just uh you don't think that if anyone builds it, everyone dies. So, what do you think is going to happen in the next 10 or 20 years? If you want to give us your alternate scenario, I'd be curious. Sorry, one second. I might have to unmute him. One sec. All right, go for it. &gt;&gt; Okay. I don't know what you mean by it, all
[00:43:02] right? So, &gt;&gt; [snorts] &gt;&gt; like, here's the deal, dude. If we build this particular kind of super intelligent AI that Yud has dreamed up in his head that is going to kill everyone, will everyone die? Obviously, okay? It's a tautology. That's the point. My point is really tied to the kind of lack of license that Yud has because what he does, and he did this in his book, actually, let me bring it up.
[00:43:36] He on page 38 says, "Nobody understands how these numbers make these AIs talk." Firstly, that's not true. I don't want to I don't want to talk about how big my dick is right now, but the people in my lab do. I have some of the most prestigious experts in their fields at top universities around the world, you know, uh sort of Ivy League level, okay? And we very much know what's going on in an LLM and we're not the only one.
[00:44:09] And if you want to talk well we don't know what's happening in the hidden layers, um I would advise you to read there was a great paper by Lippincott and his team at Johns Hopkins looking at what's happening in the hidden layers. Like I I don't know if you don't know this research or you do and you're being disingenuous. &gt;&gt; carry on. Carry on. &gt;&gt; Carry on. Carry Okay. Um maybe you don't but anyway the point is we know what LLMs are. We very much know what they are and you know we could dig into the technicals and then you
[00:44:41] could try to use a gotcha because as I speak extempore I might misuse a technical term here and there. That's why I don't like public debates. They're a waste of time. They're you know jerk off for guys who are like driving to their job and want something to listen to and then they you know go deal with Sally from accounting and that's their life. Like it's really not that serious and that's what I want to get across to you is that you're not that serious of a person dude. You're a content creator on the internet and submit your you know submit your ideas to the suggestion box.
[00:45:14] Let the adults take care of it. The AI industry is growing rapidly. There are tremendous incentives within a very complex marketplace to ensure that this is done intelligently and responsibly. As it matures that incentive is going to grow. Your nightmare I can never say your nightmare will never happen. No one can and that's the point. That does not give you license to then say it will. I know it will and that's the problem. You're taking it too far.
[00:45:47] Whether that is legally actionable is a separate issue but from my perspective morally you are acting unethically and you need to stop. That's That's my final [&nbsp;__&nbsp;] word. You know we can finish this here. If you want to keep going, we can keep going. &gt;&gt; Yeah, Yeah. but then you need to actually argue for why it won't kill us cuz it sure sounds like you're not saying "Hey, Eliezer, you shouldn't say these things even even if they are true." &gt;&gt; Your question is because they're not
[00:46:18] true. &gt;&gt; Are they true? Do you have reason to believe that they're true? Are they like that's the that's the question here. So so what why all these why all these distractions where you claim that if I what I said wasn't true, it would be damnable indeed to say it. &gt;&gt; These are not distractions. &gt;&gt; society should be held hostage regardless, but that's the question is just is it going to kill everyone? &gt;&gt; Okay, can I respond Can I respond? &gt;&gt; But but but that
[00:46:51] well sorry, Liron, I can't actually hear what you're saying. Go ahead. &gt;&gt; I was going to rephrase a question for 47F related to what you're saying, Eliezer. So 47F, let me ask you it this way. &gt;&gt; Well well sorry, I I I also need to respond to that claim that we know what's going on inside of LLMs. The the way that I quantified this for prediction market terms is by the end of 2026, are we going to be able to have figured out any algorithm learned by by LLMs, anything that gradient descent put into there that seems to account for
[00:47:24] how the LLMs are qualitatively smarter than the old sort of hand written algorithms we used to write. Like are we going to be able to take anything out of that and be like, "Oh, you know, like all those old school people who were trying to um you know, use blah blah logic to represent temporal sequencing. Like this is the way LLMs learn transport learn temporal sequencing. This is the way that LLMs learn hierarchical clustering. It's so [clears throat] much
[00:47:56] more efficient than than the than the old school algorithms and and my my that made a few years back or rather the prediction market I opened a few years back is are we are we going to be able to look inside the LLM's and get any algorithm back out of it where we can be like, "Ah, this this is some of what they learned. This is some of the this is the structure of what they learned that makes the new AI so much more powerful than than the old AIs." And and people have brought a couple of candidate papers onto that prediction market and I haven't checked it lately so maybe we found a stunning thing
[00:48:28] yesterday. And when I set up this prediction market, I didn't quite consider it a slam dunk. I thought maybe the interpretability people are actually going to go in there and they're actually going to figure out one of the algorithms. But mostly I guess that they weren't going to do that because that's hard and all the previous interpretability stuff had been we've we've like figured out when it's thinking about Paris and when it's thinking about France. We've figured out you know like how Paris and France are associated but this was all like very very primitive stuff compared to you know in terms of the old handwritten
[00:49:00] AIs. This was not like we see the interesting part. We see the part that makes them so much more smarter than the old handwritten AIs. This is like we're taking some pretty crude stuff from the handwritten days and saying where that is inside the AI. We're seeing the stuff we already understood and understanding how this very crude understanding maps onto the AI because we can go looking for the things we understand but we're not getting new understanding out of it. We're not understanding the interesting part. We're not understanding what makes ChatGPT so much smarter than Eliza and other AIs from the old days.
[00:49:32] Um and that that was my attempt to quantify the extent to which we don't know what's going on in there. &gt;&gt; Okay, we're we're getting close to the wrap-up here. So there there's a few directions we can go to wrap up. So it's clear that 47F thinks we've got a relatively good handle on LLMs &gt;&gt; No. &gt;&gt; and &gt;&gt; Wait, wait, wait. Can I talk? Can I talk? go &gt;&gt; it. &gt;&gt; Okay. That was a lot of nonsense and it just like it kind of words out some of it made sense some of it didn't but okay, here's the point, all right. Whether you can get an algorithm out of
[00:50:06] an LLM is so beside the point and the problem is you you don't understand what this is. It is a large language model. It it models the production of text from a large model of text, okay? It's an extension of corpus linguistics and what you're basically doing is you're saying that the tensor mathematics and other mathematical operations that are used to produce the outputs from the inputs is some kind of conscious being or some kind of entity that has desires which is you just don't understand what text is.
[00:50:39] That's like just all that I I could possibly say and like anyway, you guys have really misconstrued and misunderstood my point. My point isn't that like I know for certain that, you know, it's not going to kill us. I I like I do not know if like Trump gets control of the models because of, you know, your rhetoric and then uses that to kill the world. Like I don't know. I don't know and also you don't know either. There is a okay, well, let me ask you this, Yud.
[00:51:11] Will you admit that there is a possibility that your work in this field actually causes AI to kill all of us? Do you see that as a possibility? &gt;&gt; I do not see this as a significant possibility at all. This is a &gt;&gt; joke. You're a joke. You're a joke. I'm done. &gt;&gt; Whoa, whoa, whoa, let me finish. You asked me a question, let me finish. But this is an algorithm that was around a long time before I showed up on the scene. I don't have that kind of power. The to, you know, like I couldn't derail this for all my work of trying, no. I did not set it up. It's older than I am.
[00:51:47] &gt;&gt; I think we're done here. You know, the kids can say that I crashed out and whatever. I don't give a [&nbsp;__&nbsp;] I You know, just remember everyone, leave me the [&nbsp;__&nbsp;] alone. That's all I have to say. I'm done. &gt;&gt; I also hope that you are left alone by anything that is not an international treaty. &gt;&gt; All right, sounds good. Uh yeah, in interest of uh closing statements, I mean, a few different points were made. Uh 47F, uh you you said you're all done. How about we'll give Eliezar a chance to make his closing statement, and then
[00:52:19] 47F, give you the last word. &gt;&gt; I mean, I don't know if 47F is still there or if you you muted him. &gt;&gt; still here. Don't worry about it. Please, you knock yourself out. &gt;&gt; Yeah, I don't know. I I don't have that much of a grand closing statement to make here. It always comes down to the question of, you know, are we actually going to die here? If if we are, then saying that we should be constraining our speech at all because somewhere among the humanities is an idiot who don't does not
[00:52:53] understand the clearly stated and restated arguments for for why things that are not international treaties do not help here. Yeah, like if if you want to believe that AI is impotent, that it's not never going to be that powerful, that you know, that because the the the the the weird stuff in there is going on in particular sets of matrix transformations, decay the the QKV matrices that were, you know, like going to do the compare operation, going to do the softmax. I've I've written this Python code. I made sure I understood it
[00:53:25] way back when transformer models were first coming out. I was like, okay, I'd better make sure I can code this up, and I understand it. That's not the same as knowing what's inside all the numbers. And yeah, it's uh it's predicting the next token. You know, you know what what you know what class of problems, computer science speaking, can be transformed into next token prediction problems? Literally all of them, trivially so. You just predict the next token of the answer. And you know, when when that's finding new classes of security exploits that the humans did not find before then, I don't think you got to say that it's like, you know,
[00:53:57] just next token prediction. It's just, you know, cognition. Cuz you can do cognition as next token prediction, it turns out. So, yeah, like you're you're being like like these things are, you know, they're they're they're merely matrix multiplications. And if that's your stance, I can see why you're annoyed by all the people who like multiplications. &gt;&gt; That's not the math. All right, finish this. I'm I'm still finishing his closing statement, and then you'll get your &gt;&gt; Sorry, sorry, sorry. &gt;&gt; Fair enough. Tensor multiplication. Batch multiplication of of batches of vectors by by by matrices. Fair enough.
[00:54:30] Uh, you know, speaking ex tempore, sometimes misuse a technical term, as you as you pointed out. &gt;&gt; You do it a lot, dude. &gt;&gt; if if you think that all this stuff is harmless and it's going to stay harmless, I can see why you're annoyed by the people like Sam Altman who go around, you know, declaring that it's going and and Dario Amodei who go around declaring that it's going to take all the jobs. And I can see why you're annoyed by the people who think that data centers are using all the water. Um, and you know, somehow you're you're you're off directing your ire over at me over here, even though, you know, I'm not the one who got most of the airtime
[00:55:02] here. And I, of course, can't sure cannot modify my rhetoric because from my perspective, all I'm trying to do is, you know, say the things that I think are actually the case and make careful premise-conclusion links and all of that stuff. And I cannot delete nor modify any any iota of discussion about like what class of cognitive entities is dangerous directly because they're smart enough to kill us or indirectly because they can build the entity that's smart enough to kill us. I ain't modifying any of the premise-conclusion links there. You
[00:55:34] don't like the the You're worried that some nutso's going to go after your lab. It's a valid worry. You know, I I got to worry about nutso's using violent rhetoric about me and convincing people to go after me and you know, honestly, frankly, a lot of people on your side have done a lot more to raise the the you know, the the prominence of violence as an idea. Why? Cuz they figured that, you know, violence in this field will, you know, kind of help them out actually. It, you know, makes us look bad. It helps prevent the international treaty. We need law enforcement on our side. But they they win if they can cause
[00:56:07] political gridlock. And yeah, so so so who goes around talking about violence? Who goes about tweeting about violence? Who says it over and over? It might just be my Twitter feed. I do tend to see it from the accelerationist side more than from my side. Maybe that's just how X algorithms is treating me and it's not actually the case. I don't know. What I do know is that I think we've got a planetary threat. You know, not just a threat, but like a planetary in the process of falling off the cliff going splat. And this being the situation, I I am you know,
[00:56:41] I I I have done what I can to make it very clear that individual acts of random violence are just not going to help here. I believe that. I've said it over and over. But the part where if anyone builds superhuman AI, if anyone builds the AI that builds superhuman AI, we're all dead. No, that's just what I think is the case that I cannot modify for that that is not a thing that should be modified for all this other stuff. &gt;&gt; All right. Eliezer talked for 4 minutes,
[00:57:14] so 47F, feel free to talk for 4 or 5 minutes and then we'll call it. &gt;&gt; I don't need that much time. Mr. Yudkowsky, thank you very, very much for taking my money. Thank you so much. Are you going to use it to buy any hats? &gt;&gt; Ha ha. &gt;&gt; Uh I I have been trying to figure out what I can do with the money that is fun enough to make up for, you know, the the annoyance here. And you know, if if
[00:57:47] if if if what you set out to do was like make this unpleasant enough that I would raise my price next time, you probably succeeded in doing that. I don't know if this was your metric for success, but if your goal was to like cost me more than you paid me in money, then uh yeah, I'll probably charge a higher price next time. You You You can take that glorious triumph of value destruction home with you. Though it's not really the capitalist way. The capitalist way is trying to execute agreements that leave both parties better off. And I didn't think [laughter] that was what you were trying
[00:58:17] to do. &gt;&gt; We can end it on that. That's awesome. Thanks, guys. Thanks, everyone. &gt;&gt; All right. Thanks for engaging in debate. Uh hopefully we'll find more market-clearing prices for future debates. Thanks, Adam, and thank you for watching Liron's channel. &gt;&gt; May all of our exchanges leave us both better off. &gt;&gt; Peace out, youdy my duty. Take care, everyone.