Nathan Baschez
Why I have a very low p(doom), even though I have a very high p(some bad things happening): - Small bad things will almost certainly happen on the way to big bad things. It will not be a straight shot to extinction. Hugging Face is a perfect example. - Society only reacts to actual harm occurring. For better or worse, it does not react when people (even very smart people) say “hey this bad thing is gonna happen.” - Because of Hugging Face, and to a lesser extent the math discoveries, society is reacting a bit more strongly now. That’s because it has a bit more proof bad things could happen. - Imagine how much more strongly society would have reacted if AI took down a hospital and 20 people died? Or if it took down the power grid for a city block and nobody died but the result was tangible? We would have an instant reaction. - Imo it is a good thing that society reacts slowly, because when it does, it is the most blunt instrument you can imagine. (I use “society” here to refer to governments but also culture / shame / scapegoating generally.) Massive unintended consequences and secondary harm. - AI risk is more like nukes than handguns. Very hard to prevent mass shootings, because guns are cheap and abundant. Nukes, in the other hand, require nation state level resources to create or use. Super powerful futuristic AI is closer to that end of the spectrum, in terms of the level of energy and compute required. It is not a decentralized threat. Yes open models exist, but they need massive data centers to come alive. Especially given the real danger seems to be from unchecked swarms rather than single threads.