From Academia to Alignment

Kinda funny a chatbot threatening its users was the breaking point for them to see "AIs would overtake humans as the dominant intelligence". I would've thought it would be benchmarks or AI cleanly solving unsolvable math problems.

I guess that means fear is a better motivator than demos.

Good luck to the ARC. Alignment is not a solvable problem. A paradox cannot be solved.

I've stated like the following:

“Alignment, which we cannot define, will be solved by rules on which none of us agree, based on values that exist in conflict, for a future technology that we do not know how to build, which we could never fully understand, must be provably perfect to prevent unpredictable and untestable scenarios for failure, of a machine whose entire purpose is to outsmart all of us and think of all possibilities that we did not.”

The full elaboration I wrote up here - https://www.mindprison.cc/p/ai-alignment-why-solving-it-is-i...