Frank Herbert Warned Us About AI in 1965 — But Not the Way You Think

There's a line from Dune that circulates every few months:

"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

Frank Herbert wrote that in 1965. Sixty-one years ago. Before the integrated circuit was in anything you'd recognise as a computer, before ARPANET, before anyone outside a handful of labs had a reason to think about machine intelligence at all.

People share it as a warning about AI. It gets read as the machines will turn on us, which is the least interesting thing it could mean and almost certainly not what Herbert intended. Read the sentence again and notice where the danger actually sits. Not in the machines. In other men with machines.

That's a completely different warning, and it's the one that's coming true.

What the Butlerian Jihad actually was

In the backstory of Dune, humanity fought a war against thinking machines roughly ten thousand years before the events of the novel. The Butlerian Jihad. It ended with a commandment written into the Orange Catholic Bible: Thou shalt not make a machine in the likeness of a human mind.

Most readers stop there and file it as a straightforward anti-technology parable. Herbert bans the computers, the humans are free, roll credits.

But that's not the story he tells. The interesting part of Dune is what fills the vacuum.

With computers forbidden, the functions computers performed didn't disappear — humanity still needed calculation, still needed navigation, still needed to process information at scale. So those functions were rebuilt inside human institutions. Mentats were trained as human computers, minds conditioned from childhood to hold and process what a machine used to. The Bene Gesserit built a multi-generational breeding and political programme. And the Spacing Guild took over interstellar travel, its Navigators using massive doses of melange to see far enough ahead to fold space without killing everyone aboard.

Here's the thing nobody quotes. After the Jihad, power was more concentrated, not less.

Herbert's actual joke

The Guild held an absolute monopoly on faster-than-light travel. Not a strong market position — a monopoly. Nothing moved between worlds without them. No trade, no armies, no communication at scale, no empire. Every Great House, the Emperor himself, the entire economic order, all of it ran through a single organisation that could halt civilisation by declining to fly.

And the Guild's power rested on one substance, melange, which existed in exactly one place in the known universe: Arrakis.

That's the structure Herbert built. Humanity banned the machines to escape dependency, and ended up with a dependency far more absolute than the one they'd destroyed — on a scarce physical resource from a single source, controlled by an organisation nobody could route around.

The Jihad didn't remove the dependency. It relocated it, and concentrated it in the process.

That's the warning. Not machines are dangerous. Dependency doesn't disappear when you change the substrate. It moves, and it tends to move toward whoever controls the scarcest input.

Where we actually are

Now hold that structure against the present.

The models that matter most sit behind APIs owned by a handful of companies. You can use them — extremely cheaply, extremely easily, which is exactly the point — but you can't own them, can't audit them, can't guarantee they'll behave the same next quarter, and can't guarantee the price. A model you build a product on can be deprecated, re-tuned, re-priced, or restricted, and your only recourse is to rewrite.

Open weights genuinely changed the picture, and I want to be careful not to wave that away. Capable open models exist, they've improved faster than most people expected, and you can run them yourself. That is a real and important counterweight.

But look at what "run them yourself" means at frontier scale. It means GPU capacity that a company has to justify against everything else it could spend on. It means either capital expenditure most teams can't make, or renting inference — from a cloud provider, from a specialised inference host, from someone. You've escaped the model vendor and arrived at the infrastructure vendor.

Open weights are Herbert's Mentats. Genuinely impressive. Genuinely useful. And they do not remove the underlying dependency, because the dependency was never really about the model. It's about the compute.

Compute is the spice.

And there is, more or less, one Arrakis

Push the parallel one layer down and it gets uncomfortable.

The chips that frontier AI runs on are designed by a very small number of companies and manufactured at leading-edge nodes in a very small number of fabs, concentrated to a degree that has become a standing geopolitical concern. That's not a metaphor I'm stretching for the sake of a Dune reference. It's the actual structure of the industry, and it's why chip manufacturing shows up in trade policy and national security discussions rather than just technology news.

One scarce input. A very small number of sources. Everything downstream depends on it.

Herbert wrote that in 1965 as an invented cosmology. It reads now like an industry map.

What this looks like from inside the work

I build AI systems for clients — retrieval pipelines, agents, tool-calling systems, the machinery that turns a model into a product that does something useful. So this isn't abstract for me. It's an architectural constraint I design around every time.

I can mitigate it. I build a model layer so switching providers is a config change rather than a rewrite. I use structured output and strict schemas so behaviour doesn't drift silently when a model updates. I put evals in place so a change is detected by a test rather than by a customer. I cache aggressively. I route cheap queries to smaller models. Where a task genuinely fits a small model, I'll run it self-hosted and keep it off the API entirely.

All of that is real engineering and it meaningfully reduces exposure. What it does not do is remove it. At the top of the capability range, there is no version of this where my client isn't ultimately depending on infrastructure owned by someone else. I can make the dependency swappable. I cannot make it optional.

That's the honest position, and I'd rather say it plainly than pretend a clever architecture solves a structural problem.

The timeline, if this continues

Assume nothing intervenes. Assume the current trajectory just extends. Here's the shape I'd expect, and none of it requires anything dramatic:

The capability gap becomes a capital gap. Training frontier models gets more expensive, so the set of organisations able to do it stays small and probably shrinks. Open models keep improving and keep trailing, not because open development is worse but because the bill is the barrier.

Inference becomes the real moat. Weights leak, get replicated, get open-sourced. Serving hundreds of millions of requests at low latency and low cost does not. The durable advantage moves from who has the model to who has the machines to run it, which is the harder thing to compete with.

Pricing power arrives quietly. Not a dramatic price hike — a slow tightening. Rate limits on cheaper tiers. Better terms for volume. The good pricing available to companies large enough to negotiate. A tax on everything built downstream, paid by everyone who can't move.

Regulation raises the floor. Safety and compliance requirements are largely reasonable and largely written with the biggest players in the room. The predictable effect is a compliance burden that a large lab absorbs as a cost of business and a small competitor experiences as a barrier to entry. Nobody has to intend regulatory capture for it to happen.

And the platforms move up the stack. This is the one that should worry anyone building on an API. The company selling you the model can see the shape of what you built, and has every incentive to ship it themselves. You are a customer, a data source, and a roadmap item simultaneously.

That's five steps, none of which need a villain. Every one of them is just an organisation behaving rationally given its position. Which is precisely how Herbert wrote it — the Guild aren't monsters. They're a rational actor holding a chokepoint.

What actually breaks the pattern

Herbert's universe never escapes. Ten thousand years after the Jihad, the whole civilisation is still organised around who controls the spice. He wasn't offering a way out; he was showing you the trap.

We might do better, and the thing that would do it isn't regulation and isn't better open models on their own. It's cheap local inference.

The moment a genuinely frontier-capable model runs on hardware a normal company or a serious individual can own, the entire structure changes. Not because the big labs stop being good, but because dependency becomes a choice rather than a condition. Every trend that points that way — smaller models reaching capability that used to need enormous ones, quantisation and distillation getting better, consumer hardware getting more capable of real inference — is more structurally important than any given model release, and gets a fraction of the attention.

The day running a frontier model is as boring and as ownable as running Postgres is the day this stops being a concentration problem and becomes an innovation problem. That's the good ending. It isn't guaranteed and it isn't impossible.

The line, read properly

"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

The machines were never the antagonist of that sentence. The antagonist is ownership.

What Herbert understood in 1965, and what most people quoting him miss, is that the question to ask about any powerful technology is never what can it do. It's who owns the part that can't be replicated, and what happens to everyone else when they decide to change the terms.

That was true of spice. It's true of compute. And it'll be true of whatever the next scarce input turns out to be — which is the part of the warning that doesn't expire.

---

I build AI systems for clients — RAG pipelines, agents, and LLM integration designed so the model layer is swappable rather than load-bearing. If that's the kind of thing you need, here's how I work.