AI Safety Is Mostly a Sex Cult

Edit: I.e. are you implying something about the submitted writeup? Or just demeaning her? Or something else?

Neoreaction, “HBD,” and similar movements emerged from some of the same forums. I consider them at least adjacent.

It’s not that I think AI is all upside. We were naive about the net that way and we shouldn’t repeat the mistake. But it’s a technology and all tech is two sided and dual use. If you make a better hammer most people will build with it but some people will bash people’s heads in with it.

I think the biggest single problem I have with the AI safety crowd (other than what I consider dodgy reasoning and magical thinking) is that they put all the emphasis on sci-fi scenarios that are at least very unlikely (if not impossible) while ignoring much more likely and even currently real negative uses of AI.

The AI risks we should be talking about are things like mass manipulation, AI being used to supercharge addiction, a “zero entry level positions” economy that leads to mass unemployment, and monopolization of powerful AI by small numbers of people.

The last scenario is one the safety crowd is working to make more likely. Closing off AI research, outlawing open models, forcing all new innovations and models through gates, those are the things that will lead to the most powerful AI being kept back and only made available to the mega rich and governments.

I’m not sure rule by immortal trillionaire god-kings wielding superintelligence (while outlawing access to it for the rest) is that preferable to the rise of an “un-aligned” independent superintelligence. It might be worse. I’ll take the bear.

most people care more about their social standing than about truth, so they just lie (including to themselves), no biggie.

the "AI alignment" people are for example far more open to granting personhood to sufficiently advanced AI systems than are AI risk denialists. it's not hate of AI that drives the alignment discourse, it's a deeper understanding of our shared reality.

for simpler minds, understanding the risks posed by a superhuman AI is akin to hate, and understanding the degrees of inheritability of various traits is akin to discrimination. simpler minds need the lies in order to not hate, need the lies in order to not discriminate.

The former is mainly about being true to the scientific process of evidence-led discovery, while the latter is... whatever it is (but rational it is not). From a critical thinking perspective, there is no evidence to suggest the current form of AI could achieve superintelligence, and mechanisms that could enable ASI have not been discovered yet. Their perspective, on the other hand, is far more activist.

2. some of that group hangs around HN, there is an amount of (likely uncoordinated) voting/flagging patterns

Happy to fast-forward a bit: I will not take “sat and thought deeply” as a qualification, which would make all other prophets qualified for scientific/engineering judgement.