Handling Network Errors with Try-Except

The Foundation: Python's try...except Blocks

Before we dive into network-specific errors, let's quickly review the fundamental error handling mechanism in Python: the try...except block. You might already be familiar with this, but it's crucial to have the flow clear in your mind.

The basic idea is:

1. You place code that might cause an error (an "exception") inside the try block.
2. If an exception occurs, the rest of the try block is skipped, and Python jumps to the except block that matches the type of error.
3. The code inside the except block is executed, allowing you to log the error, retry the operation, or exit gracefully.

Common Exceptions in the requests Library

While you could use a generic except Exception:, it's much better practice to catch specific exceptions. The requests library provides a hierarchy of custom exceptions that give you fine-grained control over error handling.

Let's explore the most common ones you'll encounter. This article provides clear, practical examples of each.

To summarize the key exceptions:

• requests.exceptions.HTTPError: Raised for unsuccessful status codes (4xx client errors and 5xx server errors). This is triggered by calling response.raise_for_status().
• requests.exceptions.ConnectionError: Raised for network-related problems, like a DNS failure or a refused connection. This happens when you can't even establish a connection with the server.
• requests.exceptions.Timeout: Raised when a request takes longer than the specified timeout value to complete.
• requests.exceptions.RequestException: This is the base exception from which all other requests exceptions inherit. Catching this will catch any error originating from the requests library.

Handling Specific Error Types

Let's build a robust request function by handling these errors one by one.

1. Handling Bad HTTP Status Codes (HTTPError)

In our last lesson, we briefly used response.raise_for_status(). This is the primary way to handle HTTP errors. It checks the response's status code and, if it's an error code (like 404 Not Found or 500 Internal Server Error), it raises an HTTPError.

import requests

# This URL does not exist, so it will return a 404 error.
url = "https://api.github.com/this-is-not-a-real-endpoint"

try:
    response = requests.get(url)
    print("Request was successful (this won't be printed).")
    
    # This line will raise an HTTPError because the status code will be 404.
    response.raise_for_status() 

except requests.exceptions.HTTPError as err:
    print(f"An HTTP error occurred: {err}")
    # Sometimes the server provides useful error details in the response body
    if err.response:
        print(f"Response Body: {err.response.text}")

As you can see, catching the HTTPError allows us to inspect the error object and even print the response body from the server, which is invaluable for debugging. This technique is highlighted in a StackOverflow answer. In the answer by user "tsh", they show how to access e.response.text.

2. Handling Timeouts

Your agent can't afford to wait forever for an API to respond. The timeout parameter is essential for preventing your script from hanging. It can be a single number (for both connect and read timeouts) or a tuple (connect_timeout, read_timeout).

If the server doesn't respond within the specified time, requests raises a Timeout exception.

import requests

# Use a site that is known to be slow, or set an impossibly short timeout.
url = "https://httpbin.org/delay/5" # This endpoint waits 5 seconds before responding.

try:
    print("Making a request with a 2-second timeout...")
    # This request will fail because the timeout (2s) is less than the delay (5s).
    response = requests.get(url, timeout=2)
    
except requests.exceptions.Timeout as err:
    print(f"The request timed out: {err}")

For your agent, you could decide to retry a timed-out request a few times before giving up.

3. Handling Connection Errors

This error happens when you can't even talk to the server. This could be because your internet is down, a firewall is blocking the connection, or the domain name doesn't exist.

import requests

# This is not a real domain name.
url = "http://thissitedoesnotexistandneverwill.com"

try:
    print(f"Attempting to connect to {url}...")
    response = requests.get(url, timeout=5)

except requests.exceptions.ConnectionError as err:
    print(f"A connection error occurred: {err}")

Structuring Your except Blocks: Specific to General

When you use multiple except blocks, Python checks them in order. It executes the first one that matches the exception. Because HTTPError, Timeout, and ConnectionError are all more specific types of RequestException, you must place them before the general RequestException block.

If you put the general except requests.exceptions.RequestException: first, it would catch all requests-related errors, and your more specific blocks for Timeout or HTTPError would never be reached.

The following resource provides an excellent, concise explanation of this ordering.

Putting It All Together: A Robust Request Function

Now, let's combine everything into a single, reliable structure. This is a pattern you can reuse throughout your agent's code whenever you need to make an API call.

import requests

def make_robust_request(url):
    """
    Makes a GET request to a URL with robust error handling.
    Returns the response JSON if successful, otherwise None.
    """
    try:
        response = requests.get(url, timeout=10)
        
        # Raise an exception for bad status codes (4xx or 5xx)
        response.raise_for_status()
        
        # If we get here, the request was successful
        print("Request successful!")
        return response.json()

    except requests.exceptions.HTTPError as errh:
        print(f"Http Error: {errh}")
    except requests.exceptions.ConnectionError as errc:
        print(f"Error Connecting: {errc}")
    except requests.exceptions.Timeout as errt:
        print(f"Timeout Error: {errt}")
    except requests.exceptions.RequestException as err:
        # This is the most general exception, catching anything else
        print(f"An unknown error occurred: {err}")
    
    return None # Return None if any error occurred

# --- Let's test our function ---
print("--- Test 1: Successful Request ---")
make_robust_request("https://jsonplaceholder.typicode.com/todos/1")

print("\n--- Test 2: HTTP Error (404) ---")
make_robust_request("https://jsonplaceholder.typicode.com/todos/9999")

print("\n--- Test 3: Connection Error ---")
make_robust_request("http://thisisnotarealurl.xyz")

print("\n--- Test 4: Timeout Error ---")
# Using a very short timeout to force the error
try:
    requests.get("https://jsonplaceholder.typicode.com/todos/1", timeout=0.001)
except requests.exceptions.Timeout:
     print("Timeout Error: Request timed out as expected.")

This structure ensures you can identify and react to specific problems while still having a safety net for any unexpected issues.

import requests

def make_robust_post_request(url, payload):
    """
    Makes a POST request to a URL with a JSON payload and robust error handling.
    Returns the response JSON if successful, otherwise None.
    """
    try:
        # Use requests.post and pass the payload to the 'json' parameter
        response = requests.post(url, json=payload, timeout=10)
        
        # Raise an exception for bad status codes (e.g., 400, 500)
        response.raise_for_status()
        
        # Check for a 201 Created status, which is the typical success for a POST
        if response.status_code == 201:
            print("Post created successfully!")
            return response.json()
        else:
            print(f"Request was successful, but status was {response.status_code}")
            return response.json()

    except requests.exceptions.HTTPError as errh:
        print(f"Http Error: {errh}")
    except requests.exceptions.ConnectionError as errc:
        print(f"Error Connecting: {errc}")
    except requests.exceptions.Timeout as errt:
        print(f"Timeout Error: {errt}")
    except requests.exceptions.RequestException as err:
        print(f"An unknown error occurred: {err}")
    
    return None

# --- Testing the function ---
sample_payload = {
    "title": "My Robust Post",
    "body": "This was sent using a function with error handling.",
    "userId": 1,
}

post_url = "https://jsonplaceholder.typicode.com/posts"

print("--- Testing robust POST request ---")
result = make_robust_post_request(post_url, sample_payload)

if result:
    print("\n--- Server Response ---")
    import json
    print(json.dumps(result, indent=2))

Conclusion

Congratulations! You've moved beyond simple requests and have learned how to build resilient code that can withstand the unreliability of the internet. This is a non-negotiable skill for creating any application that communicates over a network, and it's especially critical for an autonomous agent that needs to operate without constant human supervision.

Here are the key takeaways:

• Always wrap network requests in try...except blocks.
• Catch specific exceptions from the requests library like HTTPError, ConnectionError, and Timeout to handle different failure scenarios appropriately.
• Use response.raise_for_status() to turn bad HTTP status codes into HTTPError exceptions.
• Always include a timeout on your requests to prevent your program from hanging.
• Order your except blocks from most specific to most general to ensure correct error handling.

In our next lesson, we will move on to a new topic: Define and use Python classes to structure agent components. Now that we have these foundational skills—handling data, making requests, and managing errors—we need a way to organize this logic into clean, reusable, and maintainable building blocks. Classes are the perfect tool for this, and they will form the skeleton of our AI agent.