Mapping Automation: Human-Agent Collaboration

Recommended resources

The freely available resources this lesson will direct you to.

Introduction

Welcome to our final lesson in the "LLM Foundations for Agent Builders" module. In our previous lesson, we established the crucial architectural distinction between using an LLM as a predictable tool within a fixed workflow and as an autonomous agent that directs its own actions. We saw that the difference lies not in the model itself, but in the system's locus of control.

Having drawn that line in the sand, we now face the essential design question for any client engagement: how much autonomy is appropriate? The answer is rarely all or nothing. The real world of business process automation exists on a continuum.

Today's learning outcome is to map business tasks along an automation spectrum from fully manual to fully agentic, identifying where agents add value and where humans should stay in the loop. This lesson provides the mental models to analyze a business problem, assess the risks and rewards of automation, and make a deliberate, defensible design choice about the level of autonomy your system will have. This is a cornerstone skill for scoping and designing any agentic AI service.

The Automation Spectrum: Beyond a Binary Choice

The distinction between a tool and an agent is a useful starting point, but a more nuanced framework is needed for practical system design. Autonomy is not a switch you flip, but a dial you turn. The optimal setting depends on the task's complexity, the cost of failure, and the value of human expertise.

We'll explore two complementary frameworks to build out this spectrum. The first is user-centric, focusing on the role of the human in the loop. The second is capability-centric, focusing on the underlying technology.

A User-Centric Framework: The Five Levels of Autonomy

A powerful way to reason about autonomy is to define it by the user's role in the process. This approach correctly frames autonomy as a design choice that directly shapes the user experience and control structure.

Levels of Autonomy for AI Agents

To build our primary framework, we will use the model proposed by researchers at the University of Washington. It defines five levels of agent autonomy based on the user's role. Please read the sections that introduce and detail this framework.

Please read sections 1, 2, and 3 of the paper. You can find them by scrolling to the headers '1. Introduction', '2.2 Agency and autonomy', and '3. Five Levels of Autonomy for AI Agents'. Focus on: The distinction between 'agency' and 'autonomy'. Given your background, this formal separation will be a useful conceptual tool. The definition of each of the five levels and the corresponding user role: Operator, Collaborator, Consultant, Approver, and Observer.

The five levels presented in the paper provide a clear spectrum for design:

Levels of Autonomy for AI Agents — This framework, from Feng et al., illustrates the spectrum of agent autonomy by defining the user's role at each stage. As the agent's autonomy increases from L1 to L5, the user's role shifts from active operator to passive observer.

Let's summarize the user roles, which directly map to the "human-in-the-loop" requirement:

L1: User as Operator. The human does the primary work; the agent is a "copilot" that assists on demand. This is the classic "LLM as a tool" paradigm.
L2: User as Collaborator. The human and agent work in parallel, delegating tasks to each other. Communication is frequent.
L3: User as Consultant. The agent takes the lead on planning and execution but consults the human for feedback and high-level guidance.
L4: User as Approver. The agent operates mostly autonomously, only stopping for approval on pre-defined critical actions or when it hits a blocker.
L5: User as Observer. The agent is fully autonomous. The human role is limited to monitoring and an emergency stop.

This user-centric model is invaluable for client discussions. It shifts the conversation from abstract "AI" to concrete questions about control, oversight, and responsibility.

A Capability-Centric Framework

Complementing the user-centric view, a capability-centric framework helps connect these levels of autonomy to the underlying technology.

The 5 levels of AI agents

This short video by Pascal Bornet provides a business-oriented view of five levels of agent capabilities, drawing a useful analogy to automotive automation.

Watch the entire video (it's less than 4 minutes). Pay attention to how each level is associated with specific technologies, such as RPA, NLP, and LLMs.

Bornet's levels map technology to autonomy:

Level 1: Rule-Based Automation (e.g., RPA - Robotic Process Automation). Deterministic, no intelligence.
Level 2: Intelligent Automation (e.g., RPA + NLP/ML). Handles structured data with some intelligence, but within rigid parameters.
Level 3: Agentic Workflows (LLM-based). Can plan and adapt within a predefined domain but requires human intervention for novel scenarios.
Level 4: Semi-Autonomous Agents. Can learn from outcomes and adapt strategies within their domain.
Level 5: Fully Autonomous Agents. Theoretical; can generalize across domains.

Synthesizing the Frameworks for Design

By combining these two frameworks, we get a powerful tool for analyzing business problems. The user-centric levels help define how a human should interact with the system, while the capability-centric levels help determine what technology is needed to achieve that interaction.

User Role (Feng et al.)	Autonomy Level	Typical Tasks & Justification	Capability Level (Bornet)	Architectural Pattern
Operator	L1: Low	Creative work, high-stakes decisions, skill development. Human directs, AI assists.	L2: Intelligent Automation	LLM as a tool (Workflow)
Collaborator	L2: Low-Medium	Complex problem-solving requiring both human intuition and AI speed. Human and AI delegate to each other.	L3: Agentic Workflows	Hybrid; Agent loop + HITL
Consultant	L3: Medium	Multi-step processes with subjective decision points. AI directs, human guides.	L3: Agentic Workflows	ReAct-style Agent Loop
Approver	L4: Medium-High	High-volume tasks with identifiable, high-risk checkpoints. AI directs, human validates.	L4: Semi-Autonomous Agents	Orchestrator-Worker with Gates
Observer	L5: High	Fully specified, low-risk, or sandboxed tasks. AI directs, human monitors.	L4/L5: (Semi-)Autonomous	Multi-Agent Systems

This table serves as a design heuristic. When you analyze a business task, you can ask: "What role should the human play here?" Your answer places you on the spectrum and points toward the appropriate architecture and technology stack.

Practical Application: Deconstructing a Hybrid System

In the real world, a single business process often contains tasks that fall at different points on the automation spectrum. A well-designed system is a hybrid, applying the minimum necessary autonomy for each step.

Let's analyze a concrete example of a loan application process.

How AI Agents and Decision Agents Combine Rules & ML in Automation

This video from IBM Technology walks through a complex loan application workflow automated with different types of agents. It's a perfect case study for seeing a hybrid system in action.

Watch the following segments, focusing on the specific role and technology of each agent described: Chat & RAG Agent (01:24 - 05:58): Answering policy questions. Workflow Agent (06:34 - 08:21): Managing the multi-step application. Decision Agent (08:38 - 09:35): Checking eligibility. Ingestion Agent (11:29 - 13:17): Extracting data from a document. Human-in-the-Loop (14:49 - 17:36): A human rep using companion/explainer agents.

Analysis Exercise

After watching the segments, consider the different components of the loan application system. Where does each one fall on our five-level automation spectrum (from User as Operator to User as Observer)?

Click here to see an analysis.

Decision Agent (Eligibility Check): This agent uses a business rules engine. It's fully deterministic and executes a predefined policy. This falls outside the L1-L5 agent spectrum and is better classified as Level 1: Rule-Based Automation. It's a pure tool, not an agent. The system is the operator.
Workflow Agent (Loan Application): This agent manages the state of a multi-step process. While it orchestrates other components, its logic is predefined (likely using a BPMN model). This is another example of a Level 1/2 system. The user is a Collaborator, providing data at each step of the predefined flow.
RAG Agent (Policy Questions) & Chat Agent: These agents use an LLM to understand natural language and synthesize information from documents. They are designed to interact with a user. This fits perfectly into Level 2: User as Collaborator or Level 3: User as Consultant, where the agent assists the user in finding information.
Ingestion Agent (Document Processing): This agent takes an unstructured document (a brochure) and extracts structured data. While it uses a powerful LLM, it's performing a single, well-defined task. The system that calls it is the operator, making this a powerful tool used in a Level 1 fashion.
Human Rep with Companion/Explainer Agents: This is a textbook example of Level 2: User as Collaborator. The human call center representative is in charge, but they are supported by a Companion Agent (to find information) and an Explainer Agent (to translate rule-engine logic into natural language). The agents are powerful tools that augment, rather than replace, the human expert.

This case study demonstrates the core principle of effective system design: it's not about maximizing autonomy everywhere, but about applying the right level of autonomy to each part of the problem.

Conclusion

This lesson provided a structured approach to moving from the binary tool/agent distinction to a nuanced automation spectrum. This is the critical bridge between understanding the technology and designing valuable business solutions.

Key Takeaways:

Autonomy is a Design Choice: The level of agent autonomy is not an inherent property of the model but a deliberate design decision you make based on risk, complexity, and the need for human oversight.
Use a User-Centric Framework: The five levels of autonomy (Operator, Collaborator, Consultant, Approver, Observer) provide a clear language for defining the human's role in an agentic system. This is an essential tool for scoping projects and managing client expectations.
Match Capabilities to Roles: Different levels of autonomy are enabled by different technologies, from simple rule-based engines to complex, multi-agent architectures.
Design Hybrid Systems: Most real-world business processes are not monolithic. The most robust and valuable solutions are often hybrids, using different levels of automation for different sub-tasks, always keeping the human in the loop where their judgment is most valuable.

Preview of the Next Module:

You have now completed the foundational module on LLMs for agent builders. You understand the core technology, its failure modes, the architectural patterns of workflows vs. agents, and how to place business tasks on an automation spectrum.

In our next module, "Mental Models for Agentic Systems," we will begin to design these systems at a deeper level. The first lesson will introduce the fundamental perception-reasoning-action-memory framework, which serves as the universal blueprint for analyzing and constructing any agentic system.