Parallelize Agent Tasks

Introduction

In our last lesson, we successfully built an execution engine for our orchestrator that can manage sequential tasks, correctly handling dependencies by running one task after another. While essential for workflows with strict ordering, this sequential approach becomes a significant bottleneck when tasks are independent. For example, if a user asks for a company's latest financial filings and recent news headlines, there's no reason to wait for one to finish before starting the other.

This lesson tackles that inefficiency head-on. We will upgrade our orchestrator to support the parallel execution of independent agent tasks. You'll learn the core patterns for parallelism in agentic systems and refactor our executePlan function to leverage modern asynchronous capabilities in TypeScript. By the end, our orchestrator will be significantly faster and more efficient, capable of handling complex requests by dispatching multiple agents simultaneously.

Why Parallel Execution Matters

In any complex system, performance is key. Sequential processing, where Task B must wait for Task A to complete, is inherently slow if the tasks are unrelated. Parallel processing allows us to execute multiple independent tasks concurrently, drastically reducing the total time to completion. The total execution time is determined by the slowest task in the parallel set, not the sum of all task durations.

This concept is central to the Orchestrator-Worker pattern, where a main orchestrator agent dispatches subtasks to specialized worker agents.

The article "Building Multi-Agent Research Systems" provides a compelling argument for this approach, particularly for research-oriented tasks that span multiple domains.

As the article highlights, running three specialists that each take 5 seconds results in a total wait time of just 5 seconds in a parallel model, compared to 15 seconds in a sequential one.

Patterns for Parallelism

There are two primary ways we can leverage parallelism in our agentic system, both supported by the Vercel AI SDK.

1. Parallel Tool Calls within a Single Agent

Modern LLMs are capable of identifying multiple, independent actions that can be taken in response to a single prompt. When using the Vercel AI SDK, the model can generate an array of tool calls in its response, which the SDK will then execute. This is a form of parallelism that happens within a single agent's turn.

A great example is asking for the weather in two different cities. The model understands these are two independent requests for the same tool.

This pattern is useful for agents that need to gather multiple pieces of information from their available tools before proceeding.

2. Parallel Agent Execution via an Orchestrator

This is the pattern we will focus on for our executePlan function. Here, the orchestrator itself initiates multiple, independent tasks (which may be handled by different agents) and runs them concurrently. This is achieved at the application level using standard asynchronous programming features.

The fundamental JavaScript/TypeScript tool for this is Promise.all. This function takes an array of promises and returns a single promise that resolves when all of the input promises have resolved. This is a perfect fit for our use case.

The official Vercel AI SDK documentation highlights this as a key agentic pattern.

Implementing Parallel Execution in executePlan

Now, let's refactor our executePlan function. In the previous lesson, our main loop iterated through the task list and executed the first available task it found before starting the loop over. To enable parallelism, we will modify it to find all executable tasks in a single pass and then run them concurrently using Promise.all.

This new logic still respects the dependency graph. It processes the plan in "layers"—all tasks with no dependencies run first in parallel. Once they complete, the next layer of tasks whose dependencies have just been met becomes available to run in the next parallel batch.

Here is the updated implementation:

// Assume 'plan', 'agentRegistry', and 'Subtask' types are defined as before.

async function executePlan(plan: { plan: Subtask[] }) {
  const taskResults = new Map<string, any>();
  const allTaskIds = new Set(plan.plan.map(t => t.id));
  const executedTaskIds = new Set<string>();

  // Failsafe to prevent infinite loops on cyclic dependencies
  const maxIterations = plan.plan.length; 

  for (let i = 0; i < maxIterations && executedTaskIds.size < allTaskIds.size; i++) {
    // 1. Identify all tasks that are ready to run in this iteration.
    // A task is ready if it hasn't been executed yet AND all its dependencies have been met.
    const executableTasks = plan.plan.filter(task => 
      !executedTaskIds.has(task.id) &&
      task.dependencies.every(depId => executedTaskIds.has(depId))
    );

    // If no tasks can be executed, but we are not done, there's a problem.
    if (executableTasks.length === 0) {
      const remainingTaskIds = [...allTaskIds].filter(id => !executedTaskIds.has(id));
      console.error("Deadlock detected. Remaining tasks with unmet dependencies:", remainingTaskIds);
      throw new Error("Could not execute all tasks. Check for cyclic or missing dependencies.");
    }

    console.log(`Executing batch of ${executableTasks.length} parallel tasks...`);

    // 2. Create a promise for each executable task.
    const taskPromises = executableTasks.map(async (task) => {
      const agentFunction = agentRegistry.get(task.agent);
      if (!agentFunction) {
        throw new Error(`Agent "${task.agent}" for task "${task.id}" not found.`);
      }

      // Gather context from the results of dependency tasks.
      const context = task.dependencies.reduce((acc, depId) => {
        acc[depId] = taskResults.get(depId);
        return acc;
      }, {} as Record<string, any>);

      const result = await agentFunction(task.task, context);
      
      return { id: task.id, result };
    });

    // 3. Execute all tasks in the current batch concurrently.
    const results = await Promise.all(taskPromises);

    // 4. Process the results of the batch.
    for (const { id, result } of results) {
      console.log(`Task ${id} completed.`);
      taskResults.set(id, result);
      executedTaskIds.add(id);
    }
  }

  if (executedTaskIds.size !== allTaskIds.size) {
    console.error("Execution finished, but not all tasks were completed.");
  }
  
  return Object.fromEntries(taskResults);
}

With this change, our orchestrator is now much more powerful. Given a plan, it will execute independent tasks in parallel batches, layer by layer, leading to significantly faster response times for complex queries.

Conclusion

In this lesson, we have dramatically improved the performance and efficiency of our agentic system. By moving beyond a purely sequential model, our orchestrator can now intelligently execute independent tasks in parallel.

Key Takeaways:

• Parallelism is Key for Performance: Running independent tasks concurrently is crucial for building a responsive and scalable agentic system.
• Parallelism Patterns:
  1. Parallel Tool Calls: An LLM can generate multiple tool calls in a single response for an agent to execute.
  2. Parallel Agent Execution: An orchestrator can use Promise.all to run multiple independent agents or tasks simultaneously.
• Layered Execution: Our updated executePlan function now processes the task dependency graph in layers, executing all possible tasks within a layer in parallel before moving to the next.

Our orchestrator is now capable of managing both sequential and parallel execution. As these workflows become more intricate, the simple Map we use for taskResults might become limiting. In our next lesson, we will explore more robust strategies to maintain and propagate shared state across orchestrated agent calls, ensuring that context is handled cleanly and consistently throughout the entire execution process.