Can we build an AI that is conscious? Where does our own consciousness come from? How can evolution produce a conscious brain, and perhaps more interestingly, why would evolution lead to consciousness at all?
What possible evolutionary advantage would there be to being aware of yourself? After all, evolution gives rise to fangs, claws, and feathers, but only insofar as they give an advantage. If they’re not helping much, a species doesn’t keep these gifts for long.
Before there were brains, living things had to find ways to seek food and avoid predators. Even one-celled organisms can have sensitive patches, parts of their outer membranes that react to food or light. When the sensitive part is triggered, the membrane flinches. It contracts or bulges, moving the creature closer to safety and nutrients, or away from danger.
Eventually as these patches got more sophisticated, plants developed phototaxis, as we see in sunflowers which follow the sun’s daily motion. In animals, these patches evolved into nervous systems.
As nervous systems continued to evolve, they invented selective appetite. They got wired into recognizers, trained through trial and error into associating sensory inputs with actionable outputs. An animal looking at a berry bush will come to recognize the red berries as poisonous, the green berries as unripe, and the blue ones as delicious. At a glance, a predator can usually tell an adult prey from a child, and decide which one is better to attack. There’s a clear evolutionary advantage to associating observations with decisions.
With berries, all you need to know is if they’re good to eat. After one sniff or one look, you know just what to do. The recognition is reflexive and instant.
The response is also vivid, compelling, and urgent. To a starving creature, the discovery of a meal is no academic exercise. Appetite drives urgency, as do the other biological imperatives of avoiding predators or seeking a mate. Our instincts compel us, and the vividness of these compulsions is necessary to our survival.
But neural networks didn’t stop evolving there. Eventually, the cues are no longer simple, easily observable assessments of color, size, or smell. The next step is to model behavior and relationships. This is where appetite transitions to reason. It’s one thing to tell an adult from a child. It’s quite another to look at a group and tell which adult is the parent of which child.
As cats evolve to chase mice, and mice evolve to avoid cats, each is driven to figure out the mental states of the other. Among social animals, the advantages go to those who can model what the others in the group are up to. Evolution favors the one who anticipates the other’s behavior.
Whether it’s hunting, finding a mate, or getting along with others, brains have some advantages over brawn. Whatever behavior you pursue, overwhelming force may work for some but it’s crude and rather expensive to rely on. Brute force ignores what others may be planning, which can result in its failure. Sophistication means using strategies - making plans, coming up with hypothetical alternatives, feinting and luring, and using reason to find a way.
But reason isn’t reflexive and it isn’t instantaneous. It requires a different kind of model from a neural net model. Unlike neural net models, a representational model can be reasoned about. A neural net cannot do something that’s never been done before, and it can’t predict if a new idea will work or not. What-if scenarios can be explored, and conclusions can be drawn.
And now we see how consciousness becomes part of the brain’s function. Once you develop a model of others, it’s not a big leap to develop a model of self. It’s the same mechanism of behavioral representation and reasoning, turned on one’s self. The analysis that starts asking “what are they thinking?” becomes “hey, what am I thinking?”
In short, consciousness is the natural evolution of the ability to analyze what others are up to. It started off as a trick to make hunters better at hunting, but eventually it gave us the ability to reflect on what we’re doing and why we’re doing it.
The thing about consciousness is that it feels vivid. This vividness of the experience is not sophisticated or transcendent. It didn’t arrive late in the evolution of our brain, after we developed reason and reflection. It arose much earlier, accompanying the invention of appetite and instinctive drive.
When will our AIs gain consciousness? The answer is clear: our AIs will be conscious when we start tasking them to reflect on their models of self, and start to use reason (and not just pattern recognition) about what they are doing and why they’re doing it.