Code-as-Policy: Generating Executable Plans

Why Code? The Expressiveness of Programs

A traditional motion planner might output a trajectory as a list of coordinates: [(x1, y1), (x2, y2), ...]. This is effective but rigid. It cannot easily adapt to unexpected events or represent complex, conditional logic.

The "Code-as-Policy" paradigm takes a different approach. Instead of generating a fixed path, the planner—powered by an LLM—generates an entire program that accomplishes the goal. Why is this better? Because LLMs trained on billions of lines of code have a deep, implicit understanding of programming constructs.

• Logic and Conditionals (if/else): The plan can include checks like "if the light is red, stop."
• Loops (for/while): The plan can handle repetitive tasks, like "pick up all the blue blocks."
• Abstraction and Reusability: The LLM can generate functions that can be reused for similar tasks in the future.

Let's begin by reading a concise explanation of this concept and its advantages.

As the article illustrates, a plan expressed as code is inherently more generalizable. The sort_blocks function can handle any number of blocks, a feat that would be cumbersome for a planner that must specify every single action in a linear sequence.

The image below provides another excellent visualization of this process. The LLM receives a high-level command and generates a Python program that uses provided perception and control APIs to execute the task. Notice how it even generates helper functions like is_empty and stack_objects that were not explicitly part of the API, showcasing its powerful reasoning capabilities.

Application to Autonomous Driving: The LaMPilot Framework

This concept is not limited to robot arms in controlled environments. It is being actively applied to the complex domain of autonomous driving. The LaMPilot framework is a prime example of this.

In this framework, the LLM acts as a high-level strategic planner. It doesn't generate low-level steering and throttle commands. Instead, it writes "Language Model Programs" (LMPs)—Python functions—that call upon a library of established driving algorithms, which we call functional primitives.

Let's dive into the research paper that introduced this framework.

The overtake_using_right_lane example from the paper is a perfect illustration of Code-as-Policy in action.

• Strategic, Not Tactical: The LLM decides what to do (check for a safe opening, change lanes, monitor distance) but not the micro-level how (the precise steering angle or throttle input).
• Adaptive Logic: The policy uses while True: loops to wait for conditions to be met (e.g., if is_safe_enter(...)). This makes the plan robust to dynamic changes in the environment.
• Bridging the Gap: The yield autopilot() command is crucial. It hands control back to a traditional, fast, low-level controller to execute the next step of the high-level plan. This elegantly solves the problem of LLM latency, as the slow, deliberative LLM is only used for high-level strategy, not real-time vehicle control.

This concept can also be extended to coordinate multiple agents, as shown in the SMART-LLM framework below, where different code-based policies are generated and assigned to different robots.

Implementation: Giving Tools to an Agent

So, how do we practically enable an LLM to "call" these functions? The mechanism is known as tool use. We define a set of functions (our "tools" or "APIs") and make the LLM aware of them, typically through the context prompt. The LLM can then decide which tool to use, with what arguments, to accomplish a task.

Frameworks like LangChain make this process straightforward. A Python function can be exposed to an LLM by simply adding a @tool decorator.

This video demonstrates a real robotics project using this exact concept, explaining how an LLM can be given tools to control a simulated robot.

The core idea is simple yet powerful: any Python function, whether it publishes a ROS message to move a robot, calls a web API, or saves a file, can be wrapped as a tool for an LLM agent to use. The agent decomposes a user's high-level request into a sequence of tool calls to achieve the final goal.

Here is a minimal, self-contained Python snippet using LangChain that encapsulates this entire process:

from langchain.tools import tool
from langchain.agents import initialize_agent, AgentType
from langchain.chat_models import ChatOpenAI




# 1. Define Robot Tools (Functional Primitives)
@tool
def move_robot(x: int, y: int) -> str:
    """Moves the robot to the specified (x, y) coordinates."""



    # In a real system, this would call a navigation stack.
    print(f"EXECUTING: Move to ({x}, {y})")
    return f"Robot successfully moved to {x}, {y}"

@tool
def grab_object(object_name: str) -> str:
    """Attempts to grab an object seen by the camera."""



    # In a real system, this would call a grasping controller.
    print(f"EXECUTING: Grab '{object_name}'")
    return f"Successfully grabbed {object_name}"




# 2. Initialize the Planner (LLM) and Tools
llm = ChatOpenAI(model="gpt-4", temperature=0)
tools = [move_robot, grab_object]




# 3. Create the Agent
agent = initialize_agent(
    tools, 
    llm, 
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, 
    verbose=True # Shows the LLM's thought process
)




# 4. Run a high-level command
# The agent will generate and execute "code" (tool calls) to solve this.
agent.run("Go to the kitchen at location (10, 20) and grab the apple.")

When you run this code, the agent's verbose=True output would show the LLM's "thoughts" as it decides which tool to call next, demonstrating the "Code-as-Policy" reasoning loop in action.

Conclusion

In this lesson, we have defined the Code-as-Policy paradigm, a powerful method for integrating LLM reasoning into control systems. By having the planner generate executable code instead of low-level control signals, we can create plans that are more expressive, adaptive, and generalizable.

Key Takeaways:

• Code-as-Policy: An approach where an AI planner, typically an LLM, outputs a program (e.g., Python code) as its policy.
• Expressiveness: Code can naturally represent complex logic, including loops and conditional statements (if/else), which are difficult to encode in traditional trajectory-based plans.
• Mechanism: The LLM is provided with a set of APIs or functional primitives (tools) that it can call. It accomplishes high-level tasks by composing these primitives into a coherent program.
• Strategic Planning: In robotics and autonomous driving, this paradigm is used for high-level strategic planning (the what and when), while fast, traditional controllers handle the low-level, real-time execution (the how).

Preview of the Next Lesson:
Code-as-Policy represents one type of "generative planning." In our next lesson, which kicks off the "Generative Planning for Autonomous Driving" module, we will broaden our perspective. We will formally compare the traditional modular AD pipeline (perception, prediction, planning) with the emerging paradigm of unified generative planning, situating approaches like Code-as-Policy within this larger context and setting the stage for our deep dive into end-to-end models like DAP.