Professor emerita, George Mason University; visiting scholar, Sloan Center on Aging and Work, Boston College; author, Composing a Further Life: The Age of Active Wisdom

It is a great boon when computers perform operations we fully understand, and do so faster and more accurately than humans can, but not a boon when we use them in situations that aren’t fully understood. We cannot expect them to make aesthetic judgments or show compassion or imagination, for these are capacities that remain mysterious in human beings.

Machines that think are likely to be used to make decisions on the basis of the operations they’re ostensibly able to perform. For instance, we now frequently see letters, manuscripts, or (most commonly) student papers in which corrections proposed by spell-check have been allowed to stand without review: The writer meant mod, but the program decided he meant mad. How tempting to leave the decision to the machine. I referred in an e-mail to a plan to meet with someone in Santa Fe on my way to an event in Texas, using the word rendezvous, and the computer married me off by announcing that the trip was to “render vows.” Can a computer be programmed to support “family values”? Any values at all? We now have drones that, aimed in a given direction, are able to choose their targets on arrival, with an unfortunate tendency to attack wedding parties as conviviality comes to appear sinister. We can surely program machines to prescribe drugs and medical procedures, but it seems unlikely that machines will do better than people in following the injunction to do no harm.

The effort to build machines that can think is certain to make us aware of aspects of thought that aren’t yet fully understood. For example, just as the design of computers led to a new awareness of the importance of redundancy in communication, in deciding how much to rely on probabilities we’ll become more aware of how much ethnic profiling based on statistics enters into human judgments. How many more decisions will follow the logic of “Everyone does it, so it must be OK,” or “I’m just one person—what I do doesn’t make a difference”?

Will those aspects of thought that cannot easily be programmed be valued more, or less? Will humor and awe, kindness and grace, be increasingly sidelined, or will their value be recognized in new ways? Will we be better or worse off if wishful thinking is eliminated and, perhaps along with it, hope?