Microsoft put a chatbot on Twitter, and Twitter helped turn it into a Nazi. Ghost in the Machine opens with this little triumph of product development.

Somewhere in that process, presumably, there was a meeting. A roadmap. Someone explaining how the bot would learn from people on the internet. And apparently nobody with the authority to stop the launch had ever actually met people on the internet.

The documentary eventually comes around to Grok producing pro-Hitler responses, followed by a news report about its move into government work.

That is a hell of a bookend. The industry has had time to improve the technology, reconsider its assumptions, and develop a much more sophisticated sales department.

The part that stays with me, though, is a worker talking about what happens when he tries to sleep.

Richard Mathenge says he helped prepare training data for ChatGPT and was paid less than a dollar an hour. He describes disturbing material that followed him home. Mophat Okinyi describes asking management for help with the graphic content and being told there was no time for counseling because they had targets to meet.

Targets. Of course there were targets.

Somebody has to get the human suffering processed before the end of the shift so somebody else can announce that the machine is becoming human.

I keep coming back to that arrangement. A pleasant interface at one end. A person losing sleep at the other. In between, enough companies and contracts for responsibility to become an exciting philosophical question.

The documentary spends a lot of time excavating the history underneath this industry: eugenics, intelligence testing, the habit of ranking people, and the extraordinary confidence of the people doing the ranking. Its account of William Shockley puts his semiconductor achievements alongside his advocacy of eugenics. Technical brilliance and appalling judgment fit quite comfortably inside the same skull.

That ought to be easy to understand. We work with computers. We should be familiar with systems that perform one operation beautifully and fail catastrophically somewhere else.

I would be careful about turning that history into an argument that every statistical method carries its inventor’s politics forever. The documentary’s historical connections deserve scrutiny on their own terms. You can take its warning seriously without treating a mathematical technique as a hereditary moral condition.

What matters to me is the permission we keep granting people because they are good at one thing. Invent a component, build a company, produce an impressive demo, and suddenly you’re qualified to decide how everyone else should live.

That’s a generous promotion. Most people have to demonstrate competence in the new role.

Then there is the future. God, there is so much future in this documentary.

The executives in its archival clips promise intelligence, then general intelligence, then superintelligence. There are predictions about abundance and warnings about extinction. Somewhere along the way, humanity is supposed to hand over a truly spectacular amount of money and trust that the people building the thing will work out the details.

In one clip, Sam Altman describes a soft promise to investors: build the system, then ask it how to generate an investment return.

I would enjoy trying that in a budget meeting. The deliverable is an entity that will explain why the budget was a good idea. Please approve the budget.

The film makes a persuasive case that this kind of storytelling does work in the present. It attracts money. It lends authority. It makes ordinary questions about ownership and working conditions sound embarrassingly small next to the destiny of the species.

But the worker still needs to sleep tonight. A prediction about future abundance does very little for him at three in the morning.

And the infrastructure has to exist somewhere.

Okinyi talks about struggling to obtain fresh drinking water in Nairobi and worries about data centers competing for it. The documentary shows residents in Memphis demanding answers about xAI’s facility. These are people asking what an industrial project means for the place where they live.

It is difficult to appreciate your supporting role in the next stage of civilization when you are still trying to get a straight answer about the water.

The cloud has been a remarkably successful name. It encourages a certain lightness of thought. Somewhere else, very clean, probably blue. The film keeps bringing us back to land, cooling, electricity, workers, and public officials making decisions. All the things that remain stubbornly physical while the sales pitch floats overhead.

This is where the argument becomes useful to someone who builds software.

A tool can be useful and still deserve questions about how it was made, what it costs, and who gets control. Being impressed by a capability does not settle those questions. Neither does being unimpressed. The consequences need their own examination.

If we recommend a system, we should be able to explain the benefit in terms more specific than a platform shift. We should know who can challenge its output. We should know what happens when it fails. And when someone asks whether they can decline to use it, we should have a better answer than a lecture about falling behind.

The documentary’s final stretch gives Jonathan Flowers room to ask: “Why does this need AI?”

I like that question. It is small enough to ask in an actual meeting. It forces the discussion back to the thing being proposed and the people expected to live with it. There should be an answer. If the answer is good, explain it.

If the answer is that everybody else is doing it, congratulations. We have automated peer pressure.

The people interviewed in Ghost in the Machine disagree with the idea that the future is already settled. Their insistence on human agency is the most hopeful thing in the transcript. People chose the objectives, arranged the contracts, approved the facilities, and accepted the tradeoffs. People can demand different choices.

That means there is work to do before anyone gets to declare the whole situation inevitable.

Start with the person who cannot sleep. Ask who set his targets. Ask who could have lowered them. Follow the decisions until you reach someone who had the power to do something differently.

The ghost has a payroll. Somebody approved it.

---

Based on the supplied transcript of the documentary Ghost in the Machine. Accounts of events and statements by participants are drawn from that transcript; the commentary is the author’s interpretation.