Perceiving AI: Agency and Experience

Hello! Welcome back to our course on the theory and research behind human-AI relationships.

In our last lesson, we explored the "uncanny valley," focusing on our immediate, visceral reactions to an AI's physical appearance. We saw how a nearly-but-not-quite-human look can provoke a feeling of unease.

Today, we shift from these gut reactions to our more cognitive interpretations. When you interact with an AI, how do you decide what's "going on in there"? Do you see it as a simple tool, a thinking machine, a feeling entity, or something else entirely? This process is called mind perception.

Our learning outcome for this lesson is to: Apply Gray and Wegner's theory of mind perception (agency and experience) to differentiate how users perceive various AI systems (e.g., a chatbot vs. a robot arm).

We'll start by understanding why perceiving a mind is so important, then break down the two key dimensions of this perception, and finally, apply this framework to understand our varied relationships with different kinds of AI.

1. The Importance of Perceiving a Mind

Before we get into the theory, let's establish a core idea: our experience of the world changes dramatically when we believe a "mind" is behind an event. The act of perceiving a mind is not an objective science; it's a subjective interpretation, but one with profound consequences for our emotions and moral judgments.

To get a feel for this, let's watch a short talk by psychologist Kurt Gray, one of the leading researchers on this topic.

As Kurt Gray explains, you can never be 100% certain that another person has a mind, but you assume they do. This assumption is fundamental. When you perceive that an action was driven by a mind with intentions—like someone deliberately pressing all the elevator buttons—your reaction is completely different than if the same outcome happened by accident. This principle is the foundation for understanding how we relate to AI.

2. The Two Dimensions of Mind: Agency and Experience

So, what exactly are we looking for when we perceive a "mind"? According to the theory developed by Kurt Gray and his colleague Daniel Wegner, we intuitively assess entities along two fundamental dimensions: Agency and Experience.

Let's get a formal definition of these terms from a research paper that applies this theory to robots.

As the paper outlines:

• Agency: This is the capacity to do. It includes abilities like planning, acting, exercising self-control, and having intentions. Think of it as the "thinking" and "acting" component of a mind.
• Experience: This is the capacity to feel. It includes the ability to feel emotions like joy and anger, and sensations like pain and pleasure. Think of it as the "feeling" and "experiencing" component.

These two dimensions are independent. An entity can be high on one and low on the other. We can map different entities onto a 2D space to see how this works:

• An adult human is the classic example of high agency and high experience. We can plan complex tasks and we feel a rich spectrum of emotions.
• A dog is perceived as having high experience (it clearly feels joy and pain) but lower agency than a human (it acts more on instinct than complex, long-term plans).
• A sophisticated robot might be seen as having high agency (it can perform complex calculations and actions) but little to no experience (we don't believe it feels anything).

This framework is crucial because, as Gray's research suggests, we tend to assign moral rights to beings with Experience (we feel it's wrong to cause them pain) and moral responsibility to beings with Agency (we hold them accountable for their actions).

3. Applying Mind Perception to AI

This theory becomes incredibly useful when we want to understand why we treat different AI systems so differently. Let's apply the agency/experience framework to a few common examples.

• A Factory Robot Arm: We perceive this as having high agency (it performs its task with precision and control) but virtually zero experience. We don't worry about its feelings, and if it makes a mistake, we blame the programmer or the company, not the arm itself.

• A Therapeutic Robot (like Paro the seal): This is the opposite. It has very low agency (it can't do much besides wriggle and make sounds). However, it is explicitly designed to give the impression of high experience. It coos when petted and yelps if handled roughly, making us feel it can experience pleasure and distress. We treat it gently, as a being with moral rights.

• A Voice Assistant (Siri/Alexa): We perceive this as having moderate-to-high agency (it can access information, control devices, and follow complex commands). However, it's generally perceived as having very low experience. Its voice is a tool, and we don't typically believe it has feelings or a personality. It's a highly capable agent, but not an experiential being.

• An AI Companion (Replika): This is perhaps the most complex case. It has some agency (it can hold a conversation, remember facts, and initiate topics). But its main design goal is to simulate high experience. It uses emotional language, "shares" its feelings, and responds with empathy. Users often form bonds with it precisely because they start to perceive it as a being with experience.

Mind Perception in Modern AI Research

This isn't just a theoretical exercise. Researchers actively use the agency/experience framework to study how people interact with and judge AI. Let's look at a recent study that shows how manipulating mind perception changes how we assign blame to an AI.

As you read, you saw that the researchers manipulated mind perception in a straightforward way: they gave the AI human-like characteristics.

• In Study 2, the AI was given a name ("Tay"), an age, a gender, hobbies ("enjoys reading and hiking"), and even a pet.
• In Study 3, the AI ("Tony") was described as a "fun-loving guy who likes to play guitar" and explicitly said to have "situational awareness and can make intentional decisions."

These descriptions are designed to boost the perception of both agency (intentional decisions) and experience (fun-loving, having hobbies). And the results of the study were clear: when people perceived more of a mind in the AI, they were significantly more likely to blame the AI itself for a moral transgression, rather than just blaming the company or the programmers.

This directly connects back to Kurt Gray's elevator example: perceiving a mind (with agency and experience) is the key that unlocks moral judgment.

Conclusion

Today we moved beyond surface-level appearances to the cognitive act of interpreting an AI's inner world. Gray and Wegner's theory of mind perception gives us a powerful yet simple tool for this analysis.

Here are the key takeaways:

• Mind perception is the subjective process of attributing mental capacities to an entity, which fundamentally shapes our emotional and moral responses to it.
• We perceive minds along two key dimensions: Agency (the capacity to do, plan, and intend) and Experience (the capacity to feel emotions and sensations).
• Different AI systems are perceived at different points on the agency/experience map. A robot arm is a high-agency/low-experience tool, while a therapeutic robot is a low-agency/high-experience companion.
• Design choices, such as giving an AI a name, personality, and hobbies, can increase our perception of its mind, which in turn influences how we assign trust, blame, and moral status to it.

In our last lesson, we looked at an AI's appearance. Today, we looked at how we perceive its mind. In our next lesson, we will connect these ideas to the user interface itself. We will explore how specific design features, known as affordances, in a chatbot or other AI system guide our interactions and set our expectations for the kind of relationship we can have with it.