Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 1

AI Agent's Core Loop: Tools & Memory

Recommended resources

Hello! Welcome to your first lesson on understanding AI agents.

Since you're already an active user of OpenClaw and Hermes, you have a solid intuitive grasp of what they can do. Our goal in this course is to peel back the layers and examine the mechanics "under the hood." This will not only help you maximize their utility but also give you the framework to diagnose why they sometimes behave in unexpected ways.

Today, we'll start with the most fundamental concept: the agent's core action loop. We'll break down how an agent "thinks," how it uses tools to "act," and how it uses memory to learn and maintain context.

The Agentic Loop: From Knowing to Doing

To begin, let's establish the fundamental difference between a standard chatbot and an AI agent. A chatbot knows things; an agent does things. This capability arises from a core process often called the "agentic loop."

The following video from IBM Technology provides a great high-level overview of this concept.

What is OpenClaw? Inside AI Agents, LLMs and the Agentic Loop

Watch the first four minutes of this video, "What is OpenClaw? Inside AI Agents, LLMs and the Agentic Loop".

Pay close attention to the distinction between "knowing" and "doing" from the start. Then, focus on the explanation of the agentic loop, which the presenter calls the "ReAct Pattern" (Reason, Act, Observe). This loop is the fundamental heartbeat of any agent.

The video introduces the Reason -> Act -> Observe cycle. This is the engine that drives the agent. Now, let's break down the specific steps that an agent like Hermes or OpenClaw takes to execute this loop. The process is systematic, much like a network protocol handling a request.

The "Hermes Agent Deep Dive" guide provides a concise, step-by-step description of this process.

🔮 Hermes Agent 🤖 — Deep Dive & Build-Your-Own Guide 📘

This section outlines the six key steps in the agent's operational cycle.

In the article, find the section titled "4. The Agent Loop (the heart of everything)". Read the numbered list from steps 1 to 6. This is the technical blueprint of the "Reason-Act-Observe" cycle we just saw.

As you can see, the loop is a structured process:

  1. It receives input.
  2. It assembles context (the "system prompt"). This is a critical step we'll revisit when we discuss memory.
  3. It calls the LLM for reasoning (the "think" step).
  4. It parses the LLM's response. If the LLM wants to use a tool, the agent dispatches it (the "act" step).
  5. It appends the tool's result to the context and loops back to the LLM.
  6. Once the task is complete, it delivers the final response and persists the conversation.

This entire loop repeats until the initial task is resolved. Now let's look closer at two of the most important components: tools and memory.

Tool Use: The "Act" Component

Tools are what connect the agent's reasoning to the real world. They are extensions that allow the agent to read files, browse the web, run terminal commands, or interact with any API. Your background in infrastructure gives you a good perspective on how critical well-defined interfaces and execution environments are.

The Hermes guide explains how its tool system is architected for safety and scalability.

🔮 Hermes Agent 🤖 — Deep Dive & Build-Your-Own Guide 📘

This section explains how tools are registered, grouped, and executed in different environments.

Please read section "6. Tools System" in the guide. Focus on these three concepts: Toolsets, which are logical groupings of tools. Execution environments. The idea of swapping backends (local, docker, ssh) to change the blast radius of a tool should be familiar from infrastructure management. Agent-level tools. Note how tools like skill_manage are special because they modify the agent's internal state, not the external world. This is the key to self-improvement.

This structured approach to tools, particularly the separation of execution environments, is crucial for running agents safely. Just as you wouldn't run an unknown binary directly on a production server, agents use sandboxing (like Docker or remote SSH environments) to contain the potential impact of their actions.

Persistent Memory: The Agent's Brain

An agent's ability to maintain context and learn over time is what makes it truly powerful. However, memory is also the most complex and misunderstood part of agent architecture, and its limitations are often why agents "do unexpected things."

There isn't one single "memory." Instead, it's a layered system with different levels of permanence, much like the memory hierarchy in a computer system (e.g., registers, cache, RAM, disk).

This diagram from the "OpenClaw Memory Masterclass" illustrates this concept perfectly.

This diagram shows the four layers of agent memory, from the temporary LLM context window to the permanent bootstrap files and retrieval index.

Let's dive into what these layers mean. The "OpenClaw Memory Masterclass" is an excellent resource written by one of the OpenClaw maintainers.

OpenClaw Memory Masterclass: The complete guide to agent memory that survives • VelvetShark

These sections explain the different types of memory and the critical process of "compaction" that manages the limited context window.

In the article, first read the section "The four layers". It provides a clear breakdown of the four distinct layers of memory and their durability. Next, read about the "The compaction lifecycle" and look at the associated diagram. Pay close attention to the difference between the good path and the bad path. Understanding this is key to avoiding unexpected context loss during long conversations.

This diagram contrasts the "good path" where memory is safely flushed before compaction with the "bad path" of overflow recovery, which leads to maximum data loss.

The key takeaway is that the LLM's context window is a temporary, fixed-size workspace. Everything—the system prompt, your instructions, tool outputs, and conversation history—competes for this limited space.

When this window fills up, compaction occurs. This is a process where older parts of the conversation are summarized to save space. This process is "lossy" by design. Important details, instructions, or constraints given early in a conversation can be lost in the summary. This is often the root cause when an agent seems to "forget" what it was told to do.

The most durable forms of memory are the bootstrap files (like MEMORY.md or AGENTS.md) and the retrieval index. These are loaded from disk and are not subject to the same compaction as the conversational history. The most important principle, as the article states, is: "if it's not written to a file, it doesn't exist."

The Closed Learning Loop

When you combine the agent loop, tool use, and persistent memory, you get the agent's most powerful feature: the ability to learn and improve.

An agent can follow its core loop to solve a problem. Along the way, it can realize the solution is reusable. It then uses an "agent-level tool" (like skill_manage) to write a summary of that solution—a new "skill"—into its own persistent memory (skills/ directory). The next time it faces a similar problem, it loads this skill as part of its initial context, making it faster and more effective.

The Hermes Deep Dive guide puts it best:

A Hermes-style agent is a loop that fills its own filing cabinet: it reads from skills/ and MEMORY.md to do its job, and writes back to those same files when it learns something.

This "closed learning loop" is what allows the agent to get more capable the more you use it.

Hermes Agent: The Self-Improving AI That Learns You

This short clip explains the self-improvement loop in Hermes.

Watch the segment from "Now, let's look at this self-improvement loop". It visualizes how the agent creates and updates skills based on its experiences.

Conclusion

We've covered the foundational architecture of an AI agent. Let's summarize the key points:

  • Core Action Loop: Agents operate on a Reason-Act-Observe cycle, repeatedly thinking (calling an LLM), acting (using tools), and observing the results until a task is complete.
  • Tool Use: Tools are the agent's hands, allowing it to interact with systems beyond text generation. Managing their execution environment is critical for safety and control.
  • Persistent Memory: Memory is a multi-layered system. The LLM's context window is temporary and finite. Long-term, reliable memory depends on writing information to persistent files (MEMORY.md, skills), which survive the "compaction" process that trims conversation history.
  • Learning Loop: The magic happens when the agent uses its own tools to write what it has learned into its own persistent memory, creating a self-improving system.

Understanding this architecture is the first step toward mastering agents. When an agent behaves unexpectedly, you can now trace its actions back to this loop. Did it fail in reasoning? Did a tool produce an error? Or, most commonly, did it "forget" something because of context window limitations and memory compaction?

In our next lesson, we will focus on the "Reason" part of the loop and analyze the trade-offs of using different LLM backends, such as cloud models from Anthropic and OpenAI versus local models. The choice of LLM has significant implications for the agent's capabilities, cost, and privacy.