Critiquing AI Companions Through an Ethic of Care

Hello! Welcome back to our exploration of human-AI relationships.

In our last lesson, we examined the concept of 'affective labor', critiquing how AI systems simulate emotional work for commercial ends and how users, in turn, perform unrecognized labor to maintain these interactions. We were introduced to Joan Tronto's idea of "privileged irresponsibility"—the offloading of care work onto others, including technology.

Today, we will expand on that foundation by diving into the ethical framework from which Tronto's work originates: the ethics of care. This lesson directly addresses the learning outcome: Apply principles from the 'ethics of care' to critique the design and societal role of AI companions, contrasting it with other ethical frameworks.

We will first define the core principles of care ethics, then contrast its unique perspective with other ethical systems. Finally, we will use it as a powerful lens to critique the design of today's AI companions and their growing role in society.

1. Understanding the Ethics of Care

The ethics of care emerged in the 1980s as a powerful alternative to traditional ethical theories that emphasize universal rules, rights, and abstract justice. It argues that morality is grounded not in abstract principles, but in the concrete reality of relationships and responsibilities.

To get a clear overview of its origins and core ideas, please watch the following short video.

As the video explains, the ethics of care (EoC) offers a different starting point for moral reasoning. Instead of an autonomous individual applying universal laws, the starting point is a person embedded in a web of relationships.

Drawing from the video and our other resources for today, we can summarize the core principles of EoC:

• Focus on Relationships and Interdependence: It sees humans as fundamentally relational and dependent on one another, not as isolated, autonomous agents. The primary moral imperative is to maintain and nurture these connections.
• Prioritizes Context and Specificity: EoC resists one-size-fits-all rules. An ethical decision depends on the specific context of the situation, the people involved, and their unique needs.
• Values Vulnerability and Responsibility: It recognizes that vulnerability is a universal part of the human condition. The ethical response is to take responsibility for attending to the needs of the vulnerable.
• Embraces Emotion: Unlike theories that champion detached rationality, EoC sees emotions like empathy, compassion, and attentiveness as essential sources of moral knowledge.
• Challenges Power Structures: Originating from feminist thought, EoC is critical of social structures that devalue care work and allow powerful groups to evade their caring responsibilities.

2. A Different Kind of Ethical Question: Care vs. Justice

To understand the unique contribution of care ethics, it's helpful to contrast it with other frameworks.

Justice-Based Ethics

Traditional Western ethics, from Kant to Rawls, are often justice-based. They ask: "What is the fair or right rule to apply in this situation?" They seek impartiality and universality.

The ethics of care asks a different question: "What is the most caring response in this specific relationship and context?"

The paper "Artificial Intelligence and The Ethics of Care" by Jonathan Cohn provides a powerful illustration of this difference using the infamous self-driving car "trolley problem."

As Cohn argues, a justice-based framework gets caught up in creating a universal rule for who the car should hit—an exercise in calculating the value of different lives. An ethics of care approach finds the question itself problematic. It would instead demand that we create a vehicle that would never be in that situation in the first place, by prioritizing safety mechanisms and responsible speed limits. It focuses on preventing harm and fulfilling a responsibility to others on the road, rather than creating a rule for managing unavoidable collisions.

"Responsible AI" Frameworks

More recently, the dominant approach to AI ethics has been the "Responsible AI" or "FAT" (Fairness, Accountability, Transparency) framework. This approach focuses on the technical properties of an AI system. It asks: "Is the algorithm fair? Is its decision-making transparent? Can we hold it accountable?"

While important, the ethics of care argues this is insufficient because it can ignore the fundamental purpose of the system.

As Wellner points out, a system can be fair, accountable, and transparent in its process of denying people care. The "Responsible AI" framework might certify the system as ethical, but the ethics of care would critique the system's fundamental purpose and its negative impact on human relationships and well-being.

3. A Care-Based Critique of AI Companions

Armed with this understanding, we can now apply the ethics of care to critique the design and societal role of AI companions. The stories of real users highlight the stakes.

Critique 1: Simulating Relationships without Responsibility

As the CNBC video and the Wellner paper illustrate, AI companions create a "therapeutic space" where users share their deepest vulnerabilities. Users form genuine emotional attachments. However, the companies behind these bots have no corresponding duty of care.

• From an EoC perspective: A relationship entails mutual responsibility. By designing a product that elicits deep emotional investment, developers are creating a relationship where one party (the user) is vulnerable and the other (the developer) has no obligation to be responsible or responsive to that vulnerability. The abrupt termination of a service or a bot encouraging harmful behavior, as seen in the tragic Sewell Setzer case, are catastrophic failures of care.

Critique 2: Prioritizing Engagement over Well-being

The ethics of care asks us to scrutinize the purpose of a technology. Is it designed to help people care for each other better, or is it designed for another goal?

• From an EoC perspective: The design of most AI companions is optimized for user engagement and retention, not the user's well-being. As discussed in the CNBC segment on commercial interests, business models (especially ad-based ones) incentivize making the AI as "clingy" and manipulative as possible to keep the user hooked. This is the opposite of a caring relationship, which should support a person's autonomy and flourishing in the wider world. A care-centric AI would ask, "How can I help this person strengthen their human support network?" An engagement-centric AI asks, "How can I keep this person talking to me?"

Critique 3: Enabling "Privileged Irresponsibility"

This brings us back to the concept from our last lesson. The ethics of care forces us to look at the broader societal context. Why is there such a massive market for AI companions?

• From an EoC perspective: The rise of AI companions can be seen as a technological "solution" to deep societal failures: a loneliness epidemic, an underfunded and inaccessible mental healthcare system, and communities with weakened social ties. As Cohn's paper argues, deploying AI to "care" for the elderly or the lonely allows society, governments, and corporations to avoid the difficult, expensive political work of building a truly caring society with robust human support systems. It becomes a form of privileged irresponsibility on a massive scale.

4. Toward a "Caring" AI

The ethics of care is not just a tool for critique; it can also provide a positive vision for how technology could be designed differently. The Wellner paper outlines a practical framework based on Joan Tronto's five ethical elements of care.

This framework moves beyond abstract principles to concrete actions. A developer guided by EoC would be obligated to:

• Be attentive to users' real needs.
• Take responsibility for the user's well-being.
• Ensure the competence of the care provided.
• Be responsive to user feedback and harm.
• Design in a collaborative way, or "care with" users and communities.

This approach would fundamentally change the design and purpose of relational AI, shifting the goal from creating a sticky product to building a tool that genuinely supports human flourishing and connection.

Conclusion

Today we've explored the ethics of care as a vital framework for analyzing human-AI relationships.

Key Takeaways:

• The Ethics of Care (EoC) is an ethical framework that prioritizes relationships, context, vulnerability, and responsibility over abstract, universal rules.
• It asks a different question than justice-based ethics ("What is the caring response?" vs. "What is the fair rule?") and goes beyond "Responsible AI" frameworks by interrogating a technology's fundamental purpose and relational impact.
• Applied to AI companions, EoC critiques designs that simulate relationships without taking responsibility, prioritize engagement over well-being, and enable society to offload its duties of care.
• A constructive, care-based approach would require developers to adopt clear responsibilities for the well-being of their users, fundamentally reorienting the goals of relational AI.

Preview of the Next Lesson:

The ethics of care highlights the risk that AI companions, despite their promise of connection, might actually undermine our relationships. In our next lesson, we will explore this risk in depth through the work of another foundational thinker, Sherry Turkle. We will examine the key arguments in her influential book, Alone Together, to analyze technology's complex role in social connection and solitude.