9 Comments
User's avatar
Lonica Smith's avatar

It’s hard to see what good an “intelligence explosion” would do in the world if it does not relate to raising the intelligence of any people, but simply raising the intelligence of machines.

Does anyone really want a world filled with increasingly smart machines and increasingly dumb people? Only those that think they would benefit because they control the smart machines, and through them, the dumb people. After all, people bereft of critical thinking and learning skills are much easier to control.

State of Play's avatar

OpenAI now requires documented risk assessments and safety cases before reinforcement-learning training runs, and each of three senior leaders can veto a run. It adopted the rule only after a 20 September run in which the automatic stop failed and staff halted training by hand 2.5 hours later.

That's oversight built after an incident, not ahead of one, and on my read it stays internal: I've seen nothing in the rule that gives outsiders visibility into when the veto gets used or why. The paper's first policy avenue, visibility into automation of AI R&D, is this gap. If an intelligence explosion is the risk, the near-term test isn't whether labs have internal brakes, it's whether anyone outside the lab can see them being pulled.

Adi Gaskell's avatar

Most of the authors in the paper are associated with frontier AI labs.

Adam Templeman's avatar

yay, civilization might occur finally

User's avatar
Comment deleted
Oct 4
Comment deleted
Adam Templeman's avatar

lol imagine, humans finally forced to recognize where the evil has always come from (incoherence, irrational certainty)

Adam Templeman's avatar

albeit, i disagree the extraction machine is fine. but intelligence sounds cool

Adam Templeman's avatar

the land of 🫐 and 🥜? haha.. wanna visit out there actually. dad's in oregon.