In the last week, OpenAI announced a change to its structure: Instead of our current complex capped-profit structure—which made sense when it looked like there might be one dominant AGI effort but doesn’t in a world of many great AGI companies—we are moving to a normal capital structure where everyone has stock.

One dominant AGI effort? What were they thinking? I guess the possibility that at some point a sort of AI-chain reaction would take place, where they would amass enough word vectors and transformers that suddenly the rocks that we’d tricked into doing maths would morph into rocks that can think. That the thinking rocks would out-think our limited meaty-thinking brains, out trade us on the stock market, out design us in weaponry, out think us in combat…

And that got my mind thinking to perhaps the last major, world-altering level technology breakthrough. A breakthrough in fundamental physics so large that everything in the world can be dated to either “before” or “after” we unleashed the beast: splitting the atom, nuclear power and the nuclear bomb.

It would be foolish of me to try and downplay the significance of the Manhattan project, so I won’t. But we should also re-evaluate the wild speculation of the time around what the nuclear age would mean for humanity.

No more wars! In theory, the atomic bomb made the outcome of war so unthinkable that it wouldn’t be worth bothering. The concept of Mutually Assured Destruction kept us safe. Rejoice, no more war, everlasting peace….. except…. That’s not really how the last 80 years have turned out. Even nuclear armed neighbours aren’t discouraged from having a pop at each other.

Power too cheap to meter! Even if you discount sloppy design, sloppy operations and very sloppy accidents as the cost of the early learning curve for nuclear power, we’re still not exactly in a good place. The UK is currently spending insane amounts of money to build a nuclear reactor that is only “commercial viable” by extorting UK bill payers for decades to come.

Fission will solve this! Apparently self-sustaining nuclear fission is just around the corner, and this really will make energy too cheap to meter… if only we can overcome the pesky challenge of containing a small star and somehow safely harness it’s energy

On this last point, there is another parallel. We already know how to safely contain a star and harness its energy. It’s simple: get a massive star, position it in space so its own gravity contains it in a massive ball shape, stand a suitable distance away from it and use a material to capture the emerging photos and convert that into electricity…. Yes, I’m talking about solar. That tech from calculators in the 70s. And it makes cheaper power than building big nuclear reactors, even in the averagely non-sunny UK. Likewise, a lot of problems that can be “solved” with AI can be more easily and accurately solved with decidedly non-AI solutions. But hey, that’s not sexy.

So… what’s the conclusion? Ignore the hype? Ignore the warnings? I don’t think so. I’m still pretty worried about nuclear war and nuclear power accidents. But like nuclear, I think humanity might get more of the shitty end than we imagined and utopia will remain as far away as ever. Not dystopia, not utopia, just a general disappointment.