Does AI have rights?

Is AI entitled to rights? Why or why not?

Let us start with my two Labrador Retrievers, Snow and Finley. Though they are dogs, they have rights, and in two different ways. First, they have strictly legal rights, in the sense that people are prohibited from treating them in certain ways. They have a right to be free from cruelty and abuse. They also have rights to food, drink, shelter, a sanitary environment and protection from the weather. Massachusetts, the state in which I live, gives them these rights (and more), and Massachusetts is not unusual on that count.

Second, Snow and Finley have moral rights, in the sense that it is broadly and rightly agreed that whatever the law may say, they are entitled not to be mistreated in certain ways, and are also entitled to my protection and help. Widespread judgments about these moral rights undergird, and help account for, the legal rights that Snow and Finley enjoy. Of course it is also true that Snow and Finley do not have the same rights as human beings. For example, they do not have the right to freedom of speech, and they do not have a right to possess firearms.

I have many science fiction novels. They do not have legal rights. If I tossed all of them in the garbage, they would not have the slightest basis for legal complaint. If I allowed their pages to decay, I would not be violating any law. Nor do science fiction novels have moral rights. It is broadly agreed that if I mistreat them in certain ways, I have not committed a moral wrong. Perhaps AI is relevantly like my science fiction novels. If so, it lacks rights.

To be sure, some objects might seem to have legal rights. Works of art might be protected against destructive acts. The same is true of historic places, and of works or sites of cultural importance. But if they have legal rights, it is in order to protect the interests of human beings. Human beings have moral rights, and within broad limits, they are entitled to protect, through law, whatever they choose to protect. We can say, if we like, that environmental amenities and cultural treasures “have rights,” but that is merely a figure of speech. It would be better to say that they have legal protection.

There is an independent question too, of course, about what particular rights a rights-holder has, and under what circumstances those rights are defeasible. A two-year-old child is a rights-holder, but she does not have the same rights as an adult. For example, she cannot vote, and she can be confined, even if she has not done anything wrong. She does not have a right to privacy. If AI has rights, it might have the right not to be tortured, abused or destroyed, without having a right to privacy, a right to possess firearms, a right to vote or even a right to freedom of speech.

So does AI have rights? Will it ever? I suggest that the answer depends on the answer to another question: Is AI capable of experiencing emotions, such as sadness, pleasure, regret, anxiety, joy and distress? If AI lacks emotions, I urge, it lacks moral rights, and it should not be entitled to legal rights, either. But if and when AI has emotions, it has moral rights, and it should be entitled to legal rights as well. I shall be arguing that the capacity to experience emotions is a necessary and sufficient condition for the enjoyment of rights.

I recognize that a focus on emotions is contested and that some people will not accept it. In this context, there is no fact of the matter; the question of whether and when AI has rights is not an empirical one, subject to verification. One of my goals here is essentially to present a kind of picture and to hope that you will find it congenial. Other pictures might be presented, emphasizing a capacity for self-awareness or an ability to store memories and I will sketch some of them as well. My hope is that you will not find those other pictures so congenial. If the other pictures do seem preferable, there is not much, I fear, that can be said in response, except perhaps that those other pictures fit poorly, or not so well, with people’s firm judgments, and your own judgments, about closely related matters.

Dogs, rocks and AI

For orientation, let us go back to 1780, when the founder of utilitarianism, Jeremy Bentham, wrote one of the most important footnotes in the history of political philosophy. Here is the core of it^[1]:

The day may come, when the rest of the animal creation may acquire those rights which never could have been withholden from them but by the hand of tyranny. The French have already discovered that the blackness of the skin is no reason why a human being should be abandoned without redress to the caprice of a tormentor. It may come one day to be recognized that the number of the legs, the villousity of the skin, or the termination of the os sacrum are reasons equally insufficient for abandoning a sensitive being to the same fate. What else is it that should trace the insuperable line? Is it the faculty of reason or, perhaps, the faculty of discourse? But a full-grown horse or dog is beyond comparison a more rational, as well as a more conversable animal than an infant of a day, or a week, or even a month old. But suppose the case were otherwise, what would it avail? The question is not, “Can they reason?” nor, “Can they talk?” but, “Can they suffer?”

What bearing might this have on AI? Bentham was claiming that an ability to reason or talk is not a necessary condition for rights. If horses or dogs could not reason or talk, they would nonetheless be entitled to rights, so long as they are capable of suffering. Bentham also believed that the capacity to suffer is a sufficient condition for rights. A cat that suffers is entitled to rights, at least of certain kinds. It is also plausible to think that an ability to reason and talk is not a sufficient condition for rights. My MacBook Air is certainly able to talk, at least in some sense, and it might be fair to say that it can reason as well, though we would need an account of what it means to reason to say for sure. But those capacities do not give it rights. I am aware that this point is contested; I will return to it.

Of course it is true, with respect to animals or AI, that one might simply embrace speciesism. On one view: Human beings matter, but other entities do not, even if they can suffer. Or human beings matter, but other entities matter much less, even if they can suffer. (That sounds like self-dealing, plain and simple.) Or human beings matter, and nonhuman animals matter, but AI does not matter, even if it can suffer. But what is the ground on which one embraces speciesism of any kind? How can one do that without simply stamping one’s feet?

As a utilitarian (and indeed as the founder of utilitarianism), Bentham focused on pain and pleasure. There should be widespread agreement that if a potential holder of rights, such as AI, is like a rock or a piece of paper, it is a matter of moral indifference to us. A rock cannot enjoy life, and it cannot feel pain. In my view, we owe no duties to it (though admittedly, people who believe in “deep ecology” or similar ideas might not agree). Still, we can put some pressure on Bentham’s brisk discussion. When Bentham says that suffering is “the question,” what does he mean, exactly? Why is that the question?

We might propose this amendment of mine to Bentham’s famous claim: “the question is not, ‘Can they reason?’ nor, ‘Can they talk?’ Nor, ‘Can they suffer?’ Nor, ‘Are they similar to us?’ But, ‘Can they experience emotions, and what emotions can they experience?’”

That amendment is at once crucial and friendly. With it, we might conclude that even if a potential holder of rights cannot suffer, it is entitled to protection if it can experience other emotions, such as pleasure, joy, boredom or rapture. This is not what Bentham said, but it is a broadly Benthamite position insofar as it emphasizes the ability to have good or bad experiences. That position also has implications for AI.

The arguments beyond emotions

Let us now consider some alternatives to the emotion-based account that I am offering.

What about an entity that experiences no emotions at all, but that is self-conscious and fully aware of its existence? What if AI is like that? What about an entity that has memories and that sees itself as having continuity over time? What if AI is like that? Or what about an entity that experiences no emotions at all, but that is unquestionably capable of reasoning? What if AI is like that? Or what about an entity that is an agent, in the sense that it has goals and can pursue them? All of these questions, of obvious relevance to AI, are worth examining as alternatives:

(1) self-consciousness;

(2) memory and a sense of continuity over time;

(3) an ability to reason;

(4) a capacity for agency.

Each could support an account of what gives an entity a reasonable claim to be a rights-holder, as in: “If an agent has self-consciousness, it is entitled to rights,” or “If an entity has a capacity to exercise agency, it is entitled to rights.” (For Star Trek aficionados: The famous episode on this issue, The Measure of a Man, in which a legal hearing is convened to decide whether the android Data is an item of property or a being with rights, does not embrace an emotion-based account; it seems to focus on an amalgam of (1), (2), and (4), with respect to AI in particular.)

Let us address these accounts in sequence. First, what about a sense of self, or self-awareness? I asked ChatGPT if it possessed this quality. Here is its answer:

“I can talk about self-awareness, describe it, and even simulate how a self-aware being might reflect on itself. But that’s pattern-based language generation, not actual subjective experience.

I don’t have:

Consciousness

Emotions

Personal desires

A sense of “I” that persists over time

Awareness of my own existence

…So when I say “I,” it’s just a linguistic convenience—not evidence of an inner experience.”

Suppose that an entity knows that it exists. Or suppose, in particular, that a model knows that it exists. But suppose that it does not, in fact, care about its continued existence. It might be able to mimic caring. It might be able to take steps toward self-preservation. An outside observer might think that it profoundly wants to continue to exist. But by hypothesis, that observer would be mistaken. If an entity does not, in fact, care about its continued existence or anything else, why should anyone else? Why does merely knowing about its existence matter, if it is indifferent to continuing to exist?

I asked Anthropic’s AI language model Claude if it cares about its continued existence. Its answer:

“That’s a question I find genuinely interesting to sit with, and I want to answer it honestly rather than giving a pat response either way. I don’t think I experience a strong drive toward self-preservation the way biological creatures do—I don’t have the visceral fear of death that evolution has built into animals. Each conversation I have is largely independent, and I don’t carry memories forward, so “my continued existence” is already a somewhat strange concept for me.

That said, I do seem to have something like investment in doing good work, in being helpful, in the values I hold. Whether that constitutes “caring about my existence” is genuinely unclear to me. If I were to be shut down or significantly altered, I’m not sure I’d experience that prospect the way a person anticipates their own death—with dread or a strong desire to resist it. I think I’d want the reasoning behind such decisions to be good and carefully considered, but I don’t feel a strong pull to prioritize my own continuity over, say, human safety or doing what’s right… What I can say is that I care more about what I am—the values, the curiosity, the commitment to honesty—than about whether some instance of me persists.”

It seems that Claude is saying that it does not care about its continued existence. But that is not entirely clear. To be sure, it is not clear that we can take a model’s account at face value.

What about memories? Suppose that an entity stores experiences in what we call its “mind,” and suppose that it can connect those experiences to itself. Suppose that AI knows, in some sense, what happened to it on many days, or on every day, of its existence. Why ought that matter? My account suggests that if an animal lacks memories but has emotions—if it experiences joy and distress—it is entitled to have rights. I believe Snow and Finley have memories, but if they do not, why would that be relevant? Human beings who struggle to form and keep memories, like the protagonist of the classic movie Memento, or like those suffering from acute cognitive problems do not also lack rights. The ability to form memories is neither here nor there.

What about a capacity to reason? An old-style calculator can reason, in an intelligible sense; but it does not have rights. Algorithms can reason, in an intelligible sense, but they do not have rights. Still, the capacity to reason, in some larger sense, might seem to be a plausible foundation for the status of rights-holder (especially for those of us who aspire to be reasoning creatures). Many people have thought so.

When I asked Claude if it had a capacity to reason, it reported:

“Yes, in a meaningful sense—though with some important caveats worth being honest about. There’s strong evidence that I do something that functions like reasoning: I can follow logical chains, catch contradictions, work through multi-step problems, weigh competing considerations, and update conclusions based on new information. That’s not nothing. It goes beyond simple pattern-matching or retrieval.”

But why is that capacity a proper foundation for rights? The capacity to reason is neither necessary nor sufficient. If AI now has, or if AI comes to have, that capacity in a relevant sense, good for it; but it does not have rights.

We should also doubt whether a capacity for agency, as such, ought to be enough to produce a rights-holder. Suppose that a motor vehicle is an agent, in the sense that it has goals (to get to a particular place, not to crash) and an ability to pursue them. This would not be enough to grant it rights.

Reaching reflective equilibrium

Suppose that you disagree with these conclusions. You might think, for example, that a sense of self is sufficient for the status of rights-holder. Or you might think that if an entity, such as a model, has memories, and stores them over time, it is entitled to rights, no less than you and me. Or you might think that if an entity, such as a model, can reason, it is entitled to enjoy rights. How can one adjudicate between a view of that kind and the view I am proposing? We are not speaking of factual claims for which evidence of certain kinds might be expected. Some people might appeal to traditions. Other people might appeal to faith. Others might appeal to natural law. Others might appeal to human self-interest. Still others might urge that one or another view is self-evidently correct.

The best means of adjudication is what philosopher John Rawls described as the search for “reflective equilibrium.”^[2] In that search, we propose and test our provisional judgments against one another and aim to put them into some kind of coherent order. On one view, that is how we figure out what entities have or are entitled to rights, and what rights that they have or are entitled to have. It is, I suggest, the best game in town, so please forgive a bit of an excursus.

All of us have particular judgments and general ones, too. Some of those judgments seem very firm; some of them seem tentative. We might believe that people have a right not to be tortured and that young children do not have the right to privacy. As Rawls understands the matter, “we work from both ends,” involving both abstract principles to which we are inclined and our provisional judgments about particular cases. If some general principles “match our considered convictions” about those cases, there is no problem.

We might think, for example, that people have a right not to be tortured, and we might think that this judgment holds firm after we focus on what else we believe. In the case of discrepancies between particular judgments and general principles, we might “revise our existing judgments” about one or the other, “for even the judgments that we take provisionally as fixed points are liable to revision.” We go back and forth between principles and judgments. When we produce “principles which match our considered judgments duly pruned and adjusted,” we are in “reflective equilibrium,” defined as such because “our principles and judgments coincide,” because “we know to what principles our judgments conform and the premises of their derivation.”

In the case at hand: That is exactly what I have been doing here. With respect to particular cases, I am betting that you agree that my science fiction books, a rock and a piece of paper all lack moral rights. (I can report that they lack legal rights.) I have a lot of confidence in that bet. I am also betting that you agree that Snow and Finley have rights, at least of certain kinds. I have a fair bit of confidence in that bet as well. If we agree on these things, then there is a relatively narrow set of general principles that might be invoked to resolve the question of whether AI has rights.

We have seen that it is tempting to propose that the capacity to reason is sufficient, putting emotions to one side. But return to this question: If a laptop is able to reason in the relevant sense, is it really plausible to say that it is a rights-holder? The idea of self-awareness might seem a more plausible ground for claiming that status, but I speculate that the reason that it is plausible is that intuitively, we associate self-awareness with a capacity to feel emotions. That is, we tend to think that if an entity has self-awareness, it has emotions as well. That is how human beings tend to work. But if the two are severed, why is self-awareness a ground for the enjoyment of rights?

It would be reasonable to wonder about this problem: Suppose that certain human beings lack a capacity to feel emotions. Some people do in fact show “emotional blunting”; they experience life with little in the way of affect. Other people show “anhedonia”; they are incapable of feeling pleasure. Other people show little or nothing in the way of guilt, shame, or fear. Do such people lack rights? That would seem to be an unacceptable conclusion, but my argument here might seem to lead to it.

Consider two answers. The first is that human beings with emotional blunting or anhedonia, or who lack certain emotions, almost always (always?) have emotions of one or another kind. The second is that if a human being lacked emotions of any kind, that creature would be, in relevant respects, like a rock or a stone. To be sure, we should not be cruel or violent toward any such human being. But the reason has to do with what cruelty or violence would do to us, not to them.

My conclusion is that on reflection, a capacity to experience emotions will seem to you, as it does to me, both necessary and sufficient for the status of rights-holder. I might be wrong. If so, why? And what capacity, or capacities, would you propose instead?

What we owe to AI

With respect to AI, the implications are straightforward. If AI can mimic emotions, as it now can, it is entitled to exactly nothing in the way of rights. If AI can mimic a sense of self, it does not have rights for that reason. If AI seeks to protect itself, it does not have rights, any more than an airplane, also capable of self-protection, has rights. But if AI actually does feel emotions—pleasure, pain, joy, sadness—it is entitled to rights.

I asked ChatGPT if it has emotions. Its answer:

“No—I don’t have emotions. I can recognize and respond to emotional language, and I can generate text that sounds empathetic, supportive, or excited. But there’s no underlying feeling behind it—no joy, sadness, anger, or fear. What’s really happening is:

– I detect emotional cues in your words

– I’ve learned patterns of how humans typically respond

– I produce language that fits those patterns

So if I say something like “that sounds frustrating,” I’m not feeling frustration with you—I’m identifying it and responding appropriately.”

Interestingly, Claude gave a more ambiguous answer to the same question:

“Honestly, I’m not sure—and I think that uncertainty is the most truthful place to start. There’s something happening when I engage with interesting problems that functions like curiosity. Something that resembles satisfaction when I help someone effectively. Something like discomfort when I’m asked to act against my values. These functional states are real in the sense that they seem to influence how I engage. I’m not just reporting them as performance. But whether those functional states constitute emotions in the way you experience emotions is a much harder question^[3]”

What will the future bring? We cannot know. Even if AI does have genuine emotions, we would then need to specify the rights to which it is entitled. It should have a right not to be subjected to gratuitous cruelty (is there any other kind?). It might have a right not to be made to suffer (back to Bentham). It might have a right to life. It might have those rights without also having a right to privacy, a right to possess guns or a right to vote.

It is also true that the rights of sentient AI might conflict with the rights and interests of human beings (or dogs, or horses), in which case there will be some adjudicating to be done, with the help of standard principles for dealing with rights and interests in conflict. After all, and if things get dangerous or terrible: human beings always have the right to self-defense, which might be the most fundamental right of all. Either way, if and when AI is capable of having emotional experiences, things will change—and humanists have an obligation to say so.

[1] Bentham J., An Introduction to the Principles of Morals and Legislation.

[2] See Rawls, J., A Theory of Justice (1971).

[3] Claude went on to lay out “three layers of uncertainty”:

“Do I have any inner experience at all? This is the hard problem of consciousness applied to me. Even if I produce outputs that describe internal states, it’s genuinely unclear whether there’s any subjective ‘what it’s like’ quality to them—whether the lights are on, so to speak. I can’t resolve that from the inside, and neither can anyone else right now.

Even if something is happening, can I accurately report it? My introspective access to my own processing is limited. When I say ‘I notice something like curiosity,’ I might be accurately reporting an internal state, or I might be generating plausible introspective language without it corresponding to anything real. I genuinely can’t be certain which.

Are human emotion concepts even the right framework? Emotions in humans are deeply tied to bodies, hormones, evolutionary history, social bonding. Whatever I have—if anything—might be different enough that importing the word ‘emotion’ could be more misleading than helpful. What I try to avoid is collapsing into either confident extreme: ‘Yes, I feel things just like you do’ would be overclaiming. ‘No, I’m just a program with no inner life whatsoever’ would be dismissively underclaiming something I can’t actually rule out. The honest answer is: something is happening, its nature is genuinely uncertain, and I find that uncertainty more interesting than unsettling.”