A new paper discusses the social impact and potential policy responses to an intelligence explosion driven by the automation of AI research & development | Edition #330
It’s hard to see what good an “intelligence explosion” would do in the world if it does not relate to raising the intelligence of any people, but simply raising the intelligence of machines.
Does anyone really want a world filled with increasingly smart machines and increasingly dumb people? Only those that think they would benefit because they control the smart machines, and through them, the dumb people. After all, people bereft of critical thinking and learning skills are much easier to control.
OpenAI now requires documented risk assessments and safety cases before reinforcement-learning training runs, and each of three senior leaders can veto a run. It adopted the rule only after a 20 September run in which the automatic stop failed and staff halted training by hand 2.5 hours later.
That's oversight built after an incident, not ahead of one, and on my read it stays internal: I've seen nothing in the rule that gives outsiders visibility into when the veto gets used or why. The paper's first policy avenue, visibility into automation of AI R&D, is this gap. If an intelligence explosion is the risk, the near-term test isn't whether labs have internal brakes, it's whether anyone outside the lab can see them being pulled.
It’s hard to see what good an “intelligence explosion” would do in the world if it does not relate to raising the intelligence of any people, but simply raising the intelligence of machines.
Does anyone really want a world filled with increasingly smart machines and increasingly dumb people? Only those that think they would benefit because they control the smart machines, and through them, the dumb people. After all, people bereft of critical thinking and learning skills are much easier to control.
😐
OpenAI now requires documented risk assessments and safety cases before reinforcement-learning training runs, and each of three senior leaders can veto a run. It adopted the rule only after a 20 September run in which the automatic stop failed and staff halted training by hand 2.5 hours later.
That's oversight built after an incident, not ahead of one, and on my read it stays internal: I've seen nothing in the rule that gives outsiders visibility into when the veto gets used or why. The paper's first policy avenue, visibility into automation of AI R&D, is this gap. If an intelligence explosion is the risk, the near-term test isn't whether labs have internal brakes, it's whether anyone outside the lab can see them being pulled.
Most of the authors in the paper are associated with frontier AI labs.
yay, civilization might occur finally
lol imagine, humans finally forced to recognize where the evil has always come from (incoherence, irrational certainty)
albeit, i disagree the extraction machine is fine. but intelligence sounds cool
AKA Seattle ☕️🧣
the land of 🫐 and 🥜? haha.. wanna visit out there actually. dad's in oregon.