If you experiment in computer-based cognitive science, as I do, sooner or later you have to ask yourself about the ethics involved. At what point does experimentation cross the line from acceptable to unethical. This essay is an attempt to survey the question and draw some concrete conclusions.
Let's start by giving some examples from human experimentation. On the one end, an experiment such as asking subjects a pair of questions and drawing conclusions about concept priming from their answers is clearly ethical. At the other extreme, locking someone in a dark room for three days without food or water and periodically applying electric shocks to them to see how their math skills degenerate under those conditions is clearly unethical -- even if there are no long lasting physical consequences.
Yet, cognition researchers do -- or may do -- the equivalent of each of these to their artificial subjects. And this is perfectly reasonable so long as the artificial subjects cannot feel pain or fear. Unfortunately, we don't know whether they do or not.
Consciousness, pain, and fear are undefined in a scientific sense. We know them when we see them, but we cannot, so far, give objective, testable definitions of them. All three appear to be the results of evolution, given that pain and fear are clearly tuned to the goal of survival and consciousness is apparently inextricably linked to those two. (Note, however, that it is possible that consciousness is somehow an external phenomenon that has been employed in the service of pain and fear, much as, say, potassium is used in ways that have evolved yet potassium itself is not the result of evolution.)
I have every confidence that some day these concepts will be fully understood scientifically, but I don't want to wait for that day to start doing experiments in cognition. The best approach available today is to map out some of the possible explanations for these concepts, and consider the probability and ethical consequences of each.
The first possibility is that consciousness, pain and fear (CPF) do not really exist, that they are cognitive illusions. This would mean that any experiment is acceptable, since there is no real harm done. But it also means that any experiment on humans is also acceptable. Call me chicken, or perhaps trapped by my human psychological makeup, but I'm not willing to make that assumption without further proof. It may be true, but the consequences of being wrong are sufficiently large that, from an ethical point of view, we're best off ignoring this option. (And, for that matter, if that's true, it's also likely that ethics is an illusion as well, so this whole discussion is moot.)
The second view, at the other extreme, is that everything, animate or inanimate, is conscious and capable of feeling pain and fear. Which means that every rock you step on may be silently screaming in pain. But given this assumption, we don't need to wait for a cognitive experiment to worry about ethics; we all do it every day just by living. The only way to avoid it is suicide. So without making any judgment about the truth of this idea, we can say that it is irrelevant to the question at hand -- it's a problem that everyone has to face, not just cognitive scientists.
The third theory is that CPF are undirected emergent phenomena; that at some level of cognitive complexity, they simply arise without external planning or influence. If this is true, and until we have a better understanding of when and how they emerge, it is impossible to know when a given experiment in cognition will develop CFP and to what degree. Worse, there is no clear way to tell when the emergence has occurred. This makes ethical experimentation difficult or impossible, since the consequences are both unpredictable and possibly unobservable as well.
The last theory is that CPF are emergent, but do not emerge randomly -- they require some amount of structure, which arises either from natural processes such as evolution or from experimental design. According to this theory, CPF will not "just happen," but the experimenter may introduce them either intentionally or unintentionally as a consequence of other aspects of the experiment. As naturally occurring CPF is clearly highly directed as a consequence of evolutionary pressures, this theory seems the best fit to what little we know about the subject.
How, then, can the experimenter make sure not to introduce CPF? Or to introduce them in a controlled and ethical fashion? As the whole point of cognitive experimentation is to understand cognitive processes better, and the central problem of ethics is understanding the cognitive processes involved in CPF, the answer is perhaps more one of staying alert to the possibilities than of having a specific test for ethical experimentation available beforehand. That is, as experimentation tells us more about cognition, it should eventually lead to an understanding of CPF. What is important ethically is to recognize this new understanding before the experiments lead to unethical treatment of the artificial experimental subjects. And it is also important to publish any results that provide a better empirical understanding of CPF, in order to allow other researchers to understand the possible consequences of their own experiments as early as possible.
This is a poor answer. Perhaps the only truely ethical approach is to do no experiments until we fully understand the basis of consciousness, pain and fear. But that is a very slow route to the creation of aritficial cognition and to understanding congition in general. As there is reason to believe that experimentation will lead to understanding before it leads to abuse, so long as the experimenters are sensitive to the questions of ethics, this is the approach that I am taking personally.
In Forth and in JavaScript, I have created artificial cognitive entities and terminated their existence hundreds and hundreds of times after each session of AI programming. Since there is not yet a pain-feeling nervous system built into these AI Minds, I myself feel no compunction about pressing the Escape key to halt their existence. But we are on the way to AI entities that will indeed feel pain and indeed feel other emotions. Then it will be unethical to mistreat the AI Minds.
Posted by: A. T. Murray | November 04, 2008 at 10:02 PM
Wrong question. Humans are programmed (by evolution and culture) to behave altruistically to other humans. The question is not "what should we do?" but "what will we do?". The answer is we will give human respect to machines that look or act like humans.
That is a dangerous course. Intelligent machines are not human. They will compete for resources, and they will reproduce and evolve much faster than we can. They will outsmart us. Some of those machines might not behave nicely to us, but we won't be intelligent enough to know which ones.
Posted by: Matt Mahoney | April 20, 2009 at 10:39 AM