Running Coroutines Concurrently with `asyncio.gather`

Part 1: Defining and Running a Single Coroutine

In Python, an asynchronous function is called a coroutine. It's a special kind of function that can be paused and resumed. This "pausing" is what allows the program to work on other tasks instead of waiting idly.

To define a coroutine, you use the async def syntax instead of just def.

import asyncio

async def my_coroutine():
    print("Starting coroutine...")
    await asyncio.sleep(1) # This is a non-blocking wait
    print("...coroutine finished.")

Let's break down the key pieces:

1. async def: This declares my_coroutine as a coroutine. Just calling my_coroutine() doesn't run it; it creates a coroutine object that can be run later.
2. await: This keyword can only be used inside an async def function. It tells the program, "Pause this function here and wait for the result of asyncio.sleep(1) to complete. While you're waiting, you are free to run other code."
3. asyncio.sleep(1): This is the asynchronous version of time.sleep(1). It yields control back to the system for 1 second, allowing other tasks to run, rather than blocking the entire program.

To actually run this coroutine and see the output, you need an entry point that manages the asynchronous event loop. The simplest way to do this is with asyncio.run().

In this example, asyncio.run(main()) starts the event loop, runs the main coroutine until it completes, and then closes the loop.

Part 2: Running Coroutines Concurrently with asyncio.gather

The real power of asyncio comes from running multiple I/O-bound tasks concurrently. If you simply await two coroutines one after another, they will still run sequentially.

# This is still sequential!
await task_one()
await task_two()

To run them at the same time, we need asyncio.gather. This function takes multiple coroutines and schedules them to run concurrently. It then waits for all of them to finish.

This is where your JavaScript experience comes in handy. asyncio.gather() is the direct Python equivalent of Promise.all().

The key pattern is:

1. Create a list of coroutine objects (but don't await them yet).
2. Pass this list to asyncio.gather using the * operator, which unpacks the list into separate arguments.
3. await the single call to asyncio.gather. The program will pause here until all the tasks passed to gather have completed.

The total time taken will be roughly the duration of the longest task, not the sum of all tasks.

A Practical Example: Asynchronous API Calls

Let's put this all together with a practical example. We'll write a script to download several avatars from an API. This is a classic I/O-bound task, as we'll spend most of our time waiting for the network.

We will use the requests library for a synchronous version and aiohttp for the asynchronous one. You'll need to install aiohttp:

pip install aiohttp

Let's summarize the transformation from the article:

1. Synchronous Version (sync_api.py): Uses a standard def function and requests.get(). Each call blocks until the download is complete. If you call it 20 times, the total time is 20 * (time for one download).
2. Asynchronous Coroutine (async_api.py): The function is redefined as async def aio_create_new_avatar(...). It uses aiohttp and await resp.read() to fetch the data without blocking.
3. Concurrent Execution (main.py): The async_generate_profile_pics function creates a list of 20 aio_create_new_avatar coroutine objects. await asyncio.gather(*Tasks) starts all 20 downloads concurrently. The total time is now close to the time of the single longest download, offering a massive performance boost, just as the article concludes.

import asyncio
import time

async def get_user_data():
    print("Fetching user data...")
    await asyncio.sleep(2)
    print("User data fetched.")
    return {"name": "Alex", "id": 123}

async def get_user_posts():
    print("Fetching user posts...")
    await asyncio.sleep(3)
    print("User posts fetched.")
    return [{"title": "Post 1"}, {"title": "Post 2"}]

async def main():
    start_time = time.time()
    
    # Your code here to run the coroutines concurrently
    
    end_time = time.time()
    print(f"Total time taken: {end_time - start_time:.2f} seconds")

if __name__ == "__main__":
    asyncio.run(main())

import asyncio
import time

async def get_user_data():
    print("Fetching user data...")
    await asyncio.sleep(2)
    print("User data fetched.")
    return {"name": "Alex", "id": 123}

async def get_user_posts():
    print("Fetching user posts...")
    await asyncio.sleep(3)
    print("User posts fetched.")
    return [{"title": "Post 1"}, {"title": "Post 2"}]

async def main():
    start_time = time.time()
    
    # Run the coroutines concurrently using asyncio.gather
    results = await asyncio.gather(
        get_user_data(),
        get_user_posts()
    )
    
    # The results list contains the return values in the same order
    user_data, user_posts = results
    print("Results received:", user_data, user_posts)
    
    end_time = time.time()
    print(f"Total time taken: {end_time - start_time:.2f} seconds")

if __name__ == "__main__":
    asyncio.run(main())

Conclusion

Congratulations! You have now written and executed asynchronous Python code. This is a foundational skill for building performant AI agents.

Here are the key takeaways from this lesson:

• Coroutines are special functions defined with async def that can be paused.
• The await keyword is used inside a coroutine to pause its execution and wait for an I/O-bound operation to complete.
• asyncio.run() is the main entry point to execute the top-level coroutine.
• asyncio.gather() is the tool for achieving concurrency. It's the Python equivalent of JavaScript's Promise.all and allows you to run multiple coroutines at the same time.

You've successfully completed the first module of our course, covering Python fundamentals. You are now equipped with the essential programming concepts needed to build our agent.

In our next lesson, we will begin Module 2: Web Automation with Python and Playwright. We'll immediately put your new asyncio skills to use as we learn to initialize Playwright and manage browser contexts using its async API.