Choosing Sync vs. Async for I/O-Bound Tasks

1. The Foundation: Concurrency vs. Parallelism

Before we look at code, it's crucial to understand two fundamental concepts: concurrency and parallelism. They both sound like "doing things at the same time," but they are different in a way that deeply impacts web application design.

The official FastAPI documentation offers a fantastic analogy involving burgers to explain this.

To summarize the key ideas from that reading:

• Concurrency (async): One worker (e.g., a cashier) juggles multiple tasks. When one task is waiting for something (like a burger being cooked), the worker switches to another task (taking the next order). This is perfect for I/O-bound operations, where the program spends most of its time waiting for a network, a database, or a file.
• Parallelism: Multiple workers (e.g., several cooks) work on different tasks simultaneously. This is ideal for CPU-bound operations, where the program is doing intense calculations (like cleaning a house or, more relevant to your goals, processing an image or running a complex ML algorithm).

Web APIs are overwhelmingly I/O-bound. They wait for incoming requests, query databases, and call other services. This is why asynchronous frameworks like FastAPI, which are built for concurrency, are so powerful.

2. How FastAPI Manages async def and def

FastAPI is brilliant because it handles both synchronous (def) and asynchronous (async def) route handlers gracefully. But it does so in very different ways. Understanding this is the key to using them correctly.

• async def functions are run directly by the main event loop.
• def functions are run in a separate thread pool.

Let's see what this means in practice. The following short video provides a perfect demonstration of the behavior of each type of function.

As the video demonstrated, using a blocking I/O call (time.sleep) inside an async def function is the worst-case scenario. It freezes the entire event loop, preventing the server from handling any other requests.

This diagram provides a simple visual for the asynchronous path:

The key mechanism is that await in an async def function signals to the event loop, "I'm going to wait for something; you can go do other work now." In contrast, a def function is handed off to a separate thread, so even if it blocks, it only blocks that thread, not the main event loop.

3. Choosing the Right Handler for the Job

Now we can establish clear rules for when to use async def versus def. The choice depends entirely on the type of work being done inside your endpoint.

Here is a summary of the rules, with examples relevant to your goal of serving ML models.

Use async def when:

Your function needs to await an I/O-bound operation. This is the primary use case.

• Calling external APIs:

  import httpx
  from fastapi import FastAPI
  
  app = FastAPI()
  
  @app.get("/users/{user_id}")
  async def get_user_data(user_id: int):
  
  
  
      # httpx is an async-compatible HTTP client
      async with httpx.AsyncClient() as client:
  
  
  
          # We 'await' the network call, allowing the event loop to work on other requests.
          response = await client.get(f"https://api.example.com/users/{user_id}")
      return response.json()
  

• Querying a database with an async driver: We did this in Module 5 when we set up SQLAlchemy with aiosqlite. Every database call was awaited.
• Reading/writing files asynchronously (e.g., with the aiofiles library).

Use def when:

Your function performs CPU-bound work or uses a library that is synchronous (blocking).

• Running an ML model inference: Most ML libraries like scikit-learn and PyTorch have synchronous, CPU-intensive predict() methods. Using def allows FastAPI to run this heavy computation in a separate thread, so it doesn't block the event loop from handling other requests.

  from fastapi import FastAPI
  
  
  
  # Assume 'model' is a loaded scikit-learn model
  
  app = FastAPI()
  
  @app.post("/predict")
  def predict(features: list):
  
  
  
      # model.predict() is a synchronous, CPU-bound operation.
      # FastAPI runs this function in a thread pool, protecting the event loop.
      prediction = model.predict([features])
      return {"prediction": prediction.tolist()}
  

• Using a blocking library: If you must use a library that doesn't support async/await, like the popular requests library for HTTP calls.

  import requests # This is a synchronous library
  from fastapi import FastAPI
  
  app = FastAPI()
  
  @app.get("/sync-data")
  def get_sync_data():
  
  
  
      # This call blocks, but it only blocks the worker thread, not the main event loop.
      response = requests.get("https://api.example.com/data")
      return response.json()
  

You can freely mix async def and def endpoints in the same application. FastAPI is smart enough to do the right thing with each.

Conclusion

You've now explored one of the most powerful and nuanced features of FastAPI. Understanding when and why to use synchronous versus asynchronous route handlers is what separates a functional API from a high-performance one.

Key Takeaways:

• Concurrency is about efficiently handling waiting (I/O-bound), while Parallelism is about doing multiple things at once (CPU-bound).
• FastAPI runs async def routes on the main event loop, making them perfect for non-blocking I/O operations where you can use await.
• FastAPI runs def routes in a separate thread pool, which is ideal for long-running CPU-bound tasks (like ML inference) or when using synchronous libraries.
• The worst mistake is to perform a blocking operation (like time.sleep or using a sync library) inside an async def route, as this freezes the entire server.

In our next lesson, we will shift our focus to ensuring our application's correctness and reliability. We will learn how to set up a testing environment with pytest and FastAPI's TestClient, enabling us to write automated tests for our endpoints.