What We Wish Algorithms Could Predict
In a recent AI Literacy workshop, I asked the prompt: *What do you wish an algorithm could predict?* As the answers moved from the practical towards bigger, harder questions, I arrived at a realization I haven’t been able to shake since.
Algorithms cannot predict anything subjective.
That sounds obvious when you say it like that. But the implications aren’t obvious at all, and I think they’re worth sitting with.
Prediction, in the algorithmic sense, requires a target -- some outcome the system can be trained against. Did the customer churn or not? Was the transaction fraudulent or not? Did the borrower default or not? These work as prediction tasks because there’s a definitive answer to check against. The algorithm learns by comparing what it guessed to what actually happened, and adjusting.
Now ask: did this person flourish?
There’s no ground truth. There is no dataset where someone recorded, objectively and beyond dispute, whether a given life constituted flourishing. You can’t train a model on it because there’s nothing to train it toward. It’s not that we lack enough data or that the models aren’t sophisticated enough. The problem is more basic than that: a prediction requires a right answer, and subjective questions don’t have one. They have interpretations, frameworks, arguments, traditions -- but not a verifiable outcome you can score a model against.
This isn’t a gap that better engineering closes. It’s a category error. Asking an algorithm to predict something subjective is like asking a scale to measure a color. The instrument doesn’t fail at the task -- the task doesn’t coherently exist for that instrument.
And yet we ask these questions constantly. We ask algorithms to predict “risk,” “quality,” “success,” “fit,” “potential” -- words that feel concrete until you try to pin them to a single measurable definition. Each of these smuggles in subjectivity while wearing the uniform of objectivity. The algorithm returns a number, and numbers feel like facts.
Here’s where it gets political, and I mean that in the most literal sense.
Because algorithms need a target variable to function, someone has to provide one. When the concept is subjective, that means someone has to *manufacture* a target -- choose indicators, set thresholds, decide what gets measured and what gets left out. Someone has to translate “flourishing” into a column in a spreadsheet.
That translation is where politics enters, quietly and permanently.
Say you’re building a system to predict community wellbeing. You need to operationalize “well-being.” So you pick: median income, life expectancy, crime rates, and educational attainment. Reasonable choices. But now you’ve made a series of decisions about what wellbeing *is*, and those decisions carry consequences. A community rich in social cohesion but low in median income scores poorly. A place with high earnings but deep isolation scores well. The model didn’t decide that. A person did when they chose the indicators. The model inherited the worldview and produced a numerical output.
Whoever defines the target variable has, in effect, legislated what the concept means. And the algorithm launders that choice — it takes a decision made by a person with a particular perspective and presents the result as if it were discovered rather than decided. The output looks like a measurement. It’s actually policy.