Calling REST APIs in Python: Handling Responses and Errors

Hello! Your script can already distinguish a missing file from invalid input and record a useful log event. Today the input comes from another program instead of a local file. You’ll call a REST API from Python and make four outcomes unmistakable: a usable response, an HTTP error, a timeout, and invalid response JSON.

Allow about 40 minutes. You’ll first learn the request–response pattern, then run a small API and client together on your Windows computer. This needs neither an AWS account nor an API key.

What changes when the data comes from an API?

A REST API exposes URLs, often called endpoints, that a client can call. A GET request asks for data; the server replies with a status code and, often, a body. For your research-and-decision project, a future search provider might return evidence in a JSON body. Before your workflow uses that evidence, it must establish both that the request succeeded and that the body is usable.

Watch the selected portions of Tech With Tim’s Master Python Requests In 15 Minutes. Call Any API. The video uses publicly accessible examples; you will use a local one afterward so your results do not depend on an outside service.

The response object gives you two separate pieces of information. response.status_code says what happened at the HTTP level: 200 generally means success, while 404 means the requested resource was not found and 503 indicates a service-side problem. response.json() attempts to decode the body into Python data. A body can be valid JSON even when the status is 503, and a 200 response can contain invalid JSON. Neither check replaces the other.

Put the checks in the right order

Requests provides response.raise_for_status() to raise an HTTPError for a 4xx or 5xx response. Call it before decoding the body. Otherwise, a JSON error document returned with 503 could be mistaken for the evidence your program requested.

Also pass a timeout on every call. Without one, your program may wait indefinitely. A timeout is different from an HTTP error: if waiting expires before a usable response arrives, you may have no status code to report.

Read the relevant parts of the Requests documentation before writing the client:

One useful sequence for a JSON endpoint is:

1. Send the request with a timeout.
2. Check the HTTP status with raise_for_status().
3. Decode the body with json().
4. Check that the decoded data has the shape your application needs.

The fourth step matters because [] is valid JSON, but it cannot supply a "title" field. It is a different problem from an invalid JSON body such as not JSON. Likewise, a successful 204 No Content response has no body to decode; a client expecting JSON must account for that endpoint’s contract rather than assuming every successful status carries data.

Build a repeatable local API call

In the VS Code PowerShell terminal, go to your research-decision-api folder. Install Requests into the virtual environment from the earlier lessons and update your recorded dependencies:

.\.venv\Scripts\python.exe -m pip install requests
.\.venv\Scripts\python.exe -m pip freeze > requirements.txt

Create api_client_demo.py beside json_demo.py. This file starts a tiny mock API on your own computer, calls it five times, and shuts it down. 127.0.0.1 means the client is connecting to the same computer. The operating system selects an available port, so you do not have to configure one.

import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Optional

import requests

from json_demo import logger


class MockAPI(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/slow":
            time.sleep(0.8)

        routes = {
            "/ok": (200, b'{"title":"Sample evidence"}'),
            "/http-error": (503, b'{"error":"service unavailable"}'),
            "/slow": (200, b'{"title":"Late evidence"}'),
            "/bad-json": (200, b'not JSON'),
            "/wrong-shape": (200, b'[]'),
        }
        status, body = routes.get(
            self.path, (404, b'{"error":"not found"}')
        )

        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()

        try:
            self.wfile.write(body)
        except (BrokenPipeError, ConnectionResetError):
            # The timed-out client may have closed its connection.
            pass

    def log_message(self, format_string, *args):
        # Keep the mock server's access messages out of our client logs.
        pass


def fetch_title(url: str, case: str) -> Optional[str]:
    try:
        response = requests.get(url, timeout=(1, 0.2))
        response.raise_for_status()
        data = response.json()
    except requests.exceptions.HTTPError as exc:
        status = (
            exc.response.status_code
            if exc.response is not None
            else None
        )
        logger.error(
            "api_http_error",
            extra={"fields": {"case": case, "status": status}},
        )
        return None
    except requests.exceptions.Timeout:
        logger.error("api_timeout", extra={"fields": {"case": case}})
        return None
    except requests.exceptions.JSONDecodeError:
        logger.error("api_invalid_json", extra={"fields": {"case": case}})
        return None
    except requests.exceptions.ConnectionError:
        logger.error("api_connection_failed", extra={"fields": {"case": case}})
        return None

    if not isinstance(data, dict) or not isinstance(data.get("title"), str):
        logger.error("api_bad_shape", extra={"fields": {"case": case}})
        return None

    logger.info(
        "api_success",
        extra={"fields": {"case": case, "status": response.status_code}},
    )
    return data["title"]


def main() -> None:
    server = ThreadingHTTPServer(("127.0.0.1", 0), MockAPI)
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()

    base_url = f"http://127.0.0.1:{server.server_port}"
    try:
        for case in (
            "/ok",
            "/http-error",
            "/slow",
            "/bad-json",
            "/wrong-shape",
        ):
            title = fetch_title(base_url + case, case)
            print(f"{case}: {title!r}")
    finally:
        server.shutdown()
        server.server_close()


if __name__ == "__main__":
    main()

Run it:

.\.venv\Scripts\python.exe .\api_client_demo.py

You should see an api_success log for /ok, then distinct error events for the other four paths. The printed result is 'Sample evidence' for /ok and None for each failed case. Log lines and printed lines may appear in a different order because logging and print use different output streams.

The logger import reuses the structured JSON logger in your earlier json_demo.py. Its guarded if __name__ == "__main__" block means importing it does not rerun the file-processing script.

Read the failures as an operator would

Follow one call through fetch_title. requests.get() either returns a response or raises a connection-related exception or timeout. If it returns a 503, raise_for_status() raises HTTPError, so the client records api_http_error with status 503 and never treats the error document as evidence. The /slow handler waits longer than the client’s read timeout, so the client records api_timeout without inventing a status code.

For /bad-json, the status check succeeds, but response.json() raises requests.exceptions.JSONDecodeError. For /wrong-shape, decoding succeeds and produces a Python list; the final shape check rejects it. Keeping these cases separate helps you decide whether to investigate a provider outage, a slow connection, malformed output, or a changed response contract.

The tuple timeout=(1, 0.2) specifies a connection timeout and a read timeout, in seconds. It is not a guaranteed total deadline for the whole operation. The short read timeout makes the local demonstration quick; a real provider call needs a timeout chosen for its expected latency and your application’s requirements.

Notice what the handlers do not log: the full URL, response body, credentials, or raw exception text. Those can contain queries or sensitive provider data. The event name, case, and available status are enough to identify these controlled failures. The function returns None for a failed call, so code using it must check for None rather than presenting an old or missing title as a fresh success. As in the previous lesson, unexpected programming errors are not relabeled as ordinary provider failures.

Where this leaves your project

You can now make a Python GET request with Requests and distinguish HTTP status failures, timeouts, invalid JSON, and valid JSON with the wrong shape. You also have a local, repeatable way to see each path without depending on a live provider.

Next, you’ll use Git to track and inspect the changes to this growing project. Later, the same request-handling pattern will help you integrate a real search API without mistaking an error response for research evidence.