Chapter 12

Updated 2026-07-02

"Worldmaking as we know it always starts from worlds already on hand; the making is a remaking." — Nelson Goodman, Ways of Worldmaking (1978)

---

In the fall of 2016, two reporters at ProPublica open Facebook's advertising portal — the same self-service tool a dentist uses to promote a teeth-whitening special — and start building an ad. Julia Angwin and Terry Parris Jr. aren't selling anything. They're there to test a menu.

The ad they compose sits in the platform's housing categories, aimed at people the system believes are house hunting. The portal walks them through their targeting options, cheerful and frictionless, and under Demographics they find the one they came for: Ethnic Affinity. It holds a short list of labels — African American (US), Asian American (US), Hispanic (US) — and beside the box for choosing which audiences will see an ad, a box for choosing which won't. The reporters set their housing ad to exclude all three, and click through to checkout.¹

Nobody in those excluded audiences ever told the platform that's who they were. Facebook doesn't ask users their race; it assigned the labels itself, inferred from years of clicks, likes, and pauses — a model's guess about each person, filed beside their name where they'd have to go digging to find it. Even the category's name was chosen with care: not ethnicity but ethnic affinity, not who you are but what your behavior suggests you lean toward. The distinction mattered, because since 1968 the Fair Housing Act has made it illegal to advertise housing in a way that indicates a preference by race — a law written in the era of newspaper classifieds, when there was a person at the ad desk to take the order or refuse it.

The ad goes into the review queue. About fifteen minutes later it comes back approved — no question raised, no human, as far as the reporters can tell, anywhere in the loop. When they show the result to John Relman, a civil-rights lawyer with decades of housing-discrimination cases behind him, he calls it about as blatant a violation of the Fair Housing Act as one can find. The platform said it would fix the problem, then didn't, then said it would fix it again.²

The labels the reporters switched off are the same kind the woman in the scene above found buried in her settings menu: built from behavior rather than declaration, sitting where the person they describe can't easily see them, adjusting somebody's opportunities without ever once asking whether they're correct. And they answer an open question — what happens when the loop people used to run on each other, census board and household, cataloger and tagger, each side visibly revising the category in public, turns into something nobody runs, in the sense of anyone holding a pen and making a decision, but that keeps running anyway, on everyone, continuously? Nothing about the mechanism breaks. The category still shapes the person it's attached to. There is simply no one left at the ad desk to take the order, refuse it, or argue with.

This is where the looping effect, introduced a chapter ago, tips into something larger than itself. The looping effect said that categories and the people inside them move together — offer a box, and some people grow into it. But it still assumed a category was describing something, however imperfectly, and then reshaping what it described. The stronger claim — the one everything from here on rests on — is that the description was never separable from the shaping to begin with. The philosopher Nelson Goodman put the reversal as plainly as it can be put: "We can have words without a world, but no world without words or other symbols."³ Philosophers call this the constitutive turn — not because categories distort a world that exists independently and could in principle be seen clearly without them, but because there is no version of "the world, as experienced and acted on" that isn't already carved up by some category or other. The census didn't count a Native American population that had been sitting there in its true size the whole time, waiting to be measured correctly. It helped make the population it went on to count, the way a fishing net helps make "the fish that got away" real by giving that phrase somewhere to attach. Chapters eight and eleven already showed the mild version of this. What follows are two more, each stranger than the last, run by people with a lot more at stake than an accurate headcount.

In 2003, the physician and journalist Ray Moynihan published an essay in the British Medical Journal documenting something close to a category being manufactured on purpose.⁴ Pharmaceutical companies with a drug in search of a market, he argued, had spent the preceding years funding conferences, sponsoring "key opinion leaders," and paying for prevalence surveys on a condition called female sexual dysfunction — in that order. The surveys came after the marketing need, not before it, and the size of the resulting number depended heavily on how the underlying questions were worded and who was asked. This is the census's MENA story again, but run in reverse and for a different reason: instead of a government finally building a box because fifty million people had already been writing themselves in for decades, an industry built the box first and then went looking — successfully — for people willing to recognize themselves inside it. A decade later, the manual the whole exercise was aimed at, the DSM, revised and partly merged the categories in question, which is its own quiet admission that the earlier edges had been drawn less by nature than by whoever had the clearest financial interest in where they fell.⁵

Not every version of this needs a government or a drug company behind it, and not every version needs to matter this much. In 2008, three researchers ran a study that shrinks the same finding down to a supermarket rack. Shoppers offered a table of magazines sorted into categories — arts, home, and so on — reported liking their selection more, and feeling more confident about it, than shoppers offered the exact same magazines laid out with no categories at all.⁶ The categories in the study weren't especially good ones; they didn't help anyone find a particular magazine faster, and in some versions of the experiment they were close to meaningless. It didn't matter. The mere presence of categorical structure — the fact of there being boxes at all, regardless of what was in them or how well they were drawn — made people feel their choice meant more. Researchers now call this the mere category effect, and it is the constitutive turn's smallest, cheapest, most universal proof: a category doesn't have to describe anything well, or even at all, to change how the world feels once it's there.

Put a supermarket rack, a census form, and a pharmaceutical marketing plan next to each other, and the common thread is that a category was never just a box waiting to receive an already-finished world. It's an argument, dressed up as a container. Frank Herbert wrote a line worth borrowing here, spoken by a character explaining why words are never as innocent as they sound: "If you believe certain words, you believe their hidden arguments. When you believe something is right or wrong, true or false, you believe the assumptions in the words which express the arguments."⁷ A category is a compressed argument of just that kind — this is a fruit, this is a vegetable, this population exists, this feeling is a disorder — and the argument travels with the label whether or not anyone examines it. One line, overheard at a professional conference and never quite improved on since, makes the same point about what happens when a category crosses a border: "If you realize that categorization is essentially a framing activity, then taxonomy translation, as opposed to localization, is an imperialist activity."⁸ Translating someone else's category isn't neutral description in a new language — it's exporting an argument built inside one culture's assumptions and presenting it, in the new language, as if it had simply been discovered rather than made.

The reporters' fifteen minutes aside, none of this required a machine. A federal statistical agency, a drug company, a conference speaker, and three consumer-behavior researchers managed the whole demonstration using nothing but people, decades apart, with no coordination between them. What changes next isn't the mechanism — the mechanism is the one a four-year-old used on a pile of Halloween candy — piles that existed only because she needed them. What changes is who's doing the sorting, and how often, and at what scale. A government redraws a census form once a decade. A pharmaceutical company runs a marketing campaign over a few years. The system that produced the label in a mid-thirties woman's ad settings runs continuously, on everyone with an account, adjusting its categories thousands of times a second, and answering to nobody in the room when it gets one wrong. The framers, from here on, are mostly machines.

---