Should we try to eliminate bias? Perhaps in the systems we build, and in our own lives? When we see bias as the source of injustice, or as a tendency to turn something right into something wrong, well, who wouldn’t want to get rid of that?
But there’s more to bias than injustice. Bias is not only about being wrong. Bias is how our brains do most of their work. It’s essential to our survival because most of the time, bias is all about taking necessary action, as quickly and effortlessly as possible.
Let’s start with the “how our brains work” part. We have two kinds of intelligence which Daniel Kahneman1 refers to as System 1 and System 2. More informally, I like to refer to them as do or die and reason why.
Do or die responses are epitomized by neural nets, the kind you find in insects and neural net AIs. They are wired to perform knee-jerk survival reflexes. Literally, knee-jerk circuits are neural nets in your body that act without rational thought. These neural nets can be extremely competent when fully adapted to a stable environment, without reliance on reason.
Our reflexive actions are based on the labels we attach: friend, enemy, or food. The human capacity for attaching labels has its roots as far back as ant pheromones. It hasn’t changed much even in the modern business world:
In our day-to-day lives, we pack our kids off to school, we grab a quick lunch, and we find our way home at the end of the day. To survive all this, we make decisions in every waking moment. We reflexively choose the good lunch, and correct our navigation along the good route. The labels we rely on are a form of bias that predisposes us to action, while consuming as little as possible of our precious and limited capacity for rational thought.
Without labels, we would suffer the terrible fate of drowning in indecision. We can no more shed these biases than we can shed our lungs. We need cognitive shortcuts in order to act decisively, which is to say, to survive.
Accurate bias means we can act with trust — we greet friends, fight enemies, and eat food, without indecision or catastrophe. And we live without having to spend our expensive reasoning resources on the norms of daily life.
So what is the purpose of reason why intelligence? Biases are essential for survival, but they can only take us so far. Any highly adapted neural net will fail on inputs it wasn’t trained for, and fail spectacularly. The ant that smells the food label on a poison trap will eat as much as it can.
Reason why intelligence is the override circuit on do or die circuits. It’s the slow analysis that can overcome the edge cases where bias causes failure.
Reason why intelligence is not a replacement for bias, it’s an extension. Reason offers many new tricks beyond the reach of neural nets, but for now we’ll focus on this one: reason can reclassify a few labels that seemed perfectly good at first, when you chose them quickly and decisively, but on closer inspection they don’t seem right.
With reason, you can deduce that some seeming friends are actually enemies, and some seeming food is actually poison. But at the end of the day, “reason why” intelligence is overhead, and is utterly inadequate for daily survival. Reasoning is very slow and very, very expensive. We don’t have the capacity to do much of it, and it’s exhausting.
There comes a time when even the most rational individuals or societies must put their pencils down. After deliberation, when it’s time to make our choices and act, in that moment reasoning has ceased. Our decisions become the labels that guide our subsequent (and reflexive) actions. In deliberation we do not and cannot eliminate bias or labels. Rather, with deliberation, we seek to replace poor labels for better ones. We proceed to act, necessarily doing so with whatever labels and biases we have constructed for ourselves.
So like cholesterol, I suppose, there’s good bias and bad bias. The good kind helps us act quickly and decisively, without disrupting our thoughts. The bad kind is a kind of misplaced trust. The problem lies not in our capacity for trust, but in how we come to place it where we do.
I think the question we face urgently should not be how to eliminate bias, but how to pursue truthful and accurate bias so we can get on with our lives. How do we accurately label friends, enemies and food in a way that we can trust the labels once they’ve been placed?
Kahneman (2011). Thinking Fast And Slow. Farrar, Straus and Giroux. ↩