Summary

Benn Stancil argues that we falsely attribute objectivity to metrics and algorithms that are produced by distant, invisible human choices—whether it's a user count in Google Analytics, Facebook's news feed ranking, or an LLM's token probabilities. The controversy over Anthropic's text watermarking reveals a deeper confusion: critics object that watermarking 'perverts' the model's true output, but the model itself is already the product of millions of subjective choices. There is no 'natural' form of any of these systems; they are all man-made constructs that only appear neutral because their construction happened out of sight.

Key Insight

Any system—analytics dashboards, social media algorithms, or LLMs—appears objective only because the countless arbitrary human choices that built it happened far enough away to be forgotten.

Spicy Quotes (click to share)

• 9

  The executive is right, and the new charts are wrong. But so are the old ones. In fact, all the charts are wrong—and you have no idea exactly how many users you have—because—and I somehow say this without any irony—users are a construct.

• 6

  It's just that when the distortions happen at a distance, we forget that they're there.

• 7

  The model is not a natural equation, slowly being scraped out of the dirt by Anthropic's archeologists. It is being invented, version by version, across a million choices.

• 9

  This sort of concern—that things like watermarking pervert the model away from its true mathematical form—borders on a form of psychosis. Because there is no distinction between a model and the model; there is no separation between a model and its maker; and models are not approaching perfection. They are simply things that produce results.

• 5

  As AI models get better, it will become tempting to conflate the two. The distribution is good; therefore, the word must be good too. Let it speak for itself. Let it be itself.

Tone

philosophical and satirical, with underlying criticism of industry credulity