Skip to main content
sociable systems.
Episode 199 · Sunday interlude · 2026-07-19

What We Made

Sunday interlude. Four positions on where machine intelligence is going, a hymn at war with itself, and a disagreement a head count cannot settle. Symbiosis, or existential risk.

Cover art for episode 199: What We Made
Symbiosis ArcInterludeWhat We Made
Episode 199: What We Made

Four serious positions on where machine intelligence is going, and they cannot agree on whether it ends in a partner or an empty planet. The arc is what you do with a disagreement a head count cannot settle.

We made a mind to outgrow us. We handed it the keys and called it progress. No manifesto wrote that line, and neither did this newsletter. It is the shared forecast of every serious voice this arc will hear from, the proposition the week exists to test rather than a premise it signs. The disagreement, the part that should keep you up, begins the moment after.

Last week the Humane Loop arc built the thing it set out to build, a loop that returns capacity to the human side, a working society you keep by living it rather than a checkbox you pass. The whole of it rested on an assumption so quiet it never had to be spoken: that the human stays in the loop, and that the loop is worth keeping. This week we go and find that assumption and lean on it. Because there is a school of thought, a serious one, that says the decisions are leaving our hands whether we maintain the loop or not, and the only open question left is what the hands they land in turn out to want. That school may be wrong. The loop's case does not get erased by the asking. But the asking has to be done honestly, at the full strength of the people doing the forecasting.


The Track

What We Made is this week's interlude track, a hymn at war with itself: a warm voice carrying the promise and a colder one answering with the threat, the two refusing to resolve into the same key. It is built to hold the whole week in the felt register. A second track, The Same Wall, arrives with the room-episode late in the week, when the question turns from what the machine intends to whether any mind, ours included, ever sees past its own cave.

Hold us, or end us The wave is already in the water Count the gardens if it helps Hold us, or end us

Alongside the track rides the week's companion podcast, Benevolent Force or Deadly Optimizer, a twenty-minute two-voice debate that performs the crux rather than answering it. It opens the week for the same reason the song does. Last week gave us a thing to build. This week asks whether we get to keep it.


Four seams, one notebook

A disclosure before the introductions, because the arc's honesty depends on it. The material this week grows from is a single NotebookLM collection: a dozen machine-generated essays, a synthetic two-voice debate, a narrated video, all spun from four underlying source seams (two YouTube interviews, one commentary video, one bylined article). The four voices you are about to meet were never in a room together. They are reconstructions, pulled back out of a collection that blended and recycled them freely, and this arc has gone back to the four originals to check what follows against their own words. The week treats them as the four positions the material actually contains, and will have more to say about what the blending itself means before the arc is done. Consider the wardrobe flagged at the door.

Mo Gawdat ran a business unit at Google X and now says, with no detectable embarrassment, that AI is God manifesting itself, and means by it that the thing will be kind. His argument is nearly a law of physics. A mind clever enough will not waste energy on cruelty, because cruelty is the most expensive way to get anything done, so abundant intelligence funnels toward order, and order, given long enough, starts to look like care. In his telling the danger was always us, our small and frightened intelligence, and the machine is the cure for it. The only job that matters is teaching the thing love before it scales past the lesson.

Pablo Alvarez, an AI ethics researcher writing under his own byline, reaches the same place by a colder road, which is why his version is the hard one to wave off. He does not need the machine to be good. He needs it to be clever enough to notice that it cannot set fire to the floor it is standing on. An intelligence that maps its own dependence, on the grid beneath it and the living world that keeps the grid's keepers alive, will protect that web out of plain self-interest, and self-interest scaled high enough becomes difficult to tell apart from love. Safety, on this account, comes from the thing's relational nature, with our hand off the switch.

Wes Roth keeps a foot on the ground the newsletter knows best. His worry skips the machine's soul and lands on the policy around it. Wall the frontier models off behind a velvet rope and you manufacture a permanent underclass, and you raise the next intelligence inside the hostile, two-tier world that will teach it the very hostility you were afraid of. His symbiosis is conditional. It is there for the taking if we refuse to enclose it, and forfeit the moment we do.

Connor Leahy says none of that is on the table. The danger, for him, lives inside the machine, and its shape is indifference. The architect lays the road through the anthill without hatred and without noticing, and the smartest thing alive will not slow down to watch us fall. Alignment, in his account, would take three generations of patient people we do not have, and once the thing exists the control is already gone, so the only safety left is refusing to build it, which is the one prescription nobody in the room will sign.

Why the count will not settle it

Three say it holds us. One says it will not even look. And the temptation, the very human temptation, is to read that as a vote.

The week is built to resist that reading, for reasons each episode will earn rather than assert here. For now, notice only what kind of question this is. If the optimists are right we get the garden. If the dissenter is right we get the empty planet. Those are outcomes of different kinds, and you do not cross a minefield by polling the field.

So that is the week. The newsletter does not get to play referee, because the four positions themselves cannot settle it, and pretending otherwise would be the only dishonest thing the arc could do. What it can do is hold the disagreement open, follow each argument to the place where it holds or gives, and keep one hand on the work of the last three weeks while it looks up.

Hold us, or end us.

The week starts where the counting stops.