"Making API Requests with `requests`"

The requests Library: Your Gateway to the Web

Python's standard library includes modules for handling HTTP, but the third-party requests library has become the de facto standard for its simplicity and elegance. It abstracts away much of the complexity, allowing you to interact with web services in a clean, human-readable way.

Since requests isn't part of the standard library, it needs to be installed in your virtual environment. You can do this with pip:
pip install requests

Once installed, you can import it into your scripts with a simple import requests.

Making Your First GET Request

The most common type of HTTP request is GET, used to retrieve data from a specific URL or API endpoint. With the requests library, this is a one-line command: requests.get().

When you make a request, the server sends back a response, which requests encapsulates in a Response object. This object contains everything you need: the data, the status code, the headers, and more.

Let's explore the fundamental workflow: making a request and inspecting the response.

As the reading highlights, checking the status code is the first step to verify if your request was successful. A code of 200 means "OK," while codes in the 4xx range (like 404 Not Found) indicate a client-side error.

A more robust way to handle errors is to use the response.raise_for_status() method. This method will do nothing if the request was successful (status code 2xx), but it will raise an HTTPError exception if the status code was a 4xx or 5xx error. This is a great way to integrate API calls into the try...except blocks we learned about previously.

import requests
from requests.exceptions import HTTPError

url = "https://api.github.com/invalid-endpoint"

try:
    response = requests.get(url)
    # Raise an exception for bad status codes (4xx or 5xx)
    response.raise_for_status() 
except HTTPError as http_err:
    print(f"HTTP error occurred: {http_err}") # e.g., 404 Client Error
except Exception as err:
    print(f"Other error occurred: {err}")
else:
    print("Success!")

Processing the JSON Response

Once you've confirmed a successful request, you need to access the data in the response body. The Response object gives you a few ways to do this, but the most important one for our purposes connects directly to our last lesson.

So, while response.text gives you the raw JSON string (which you could parse with json.loads()), the response.json() method is a convenient shortcut that does the work for you. After calling it, you have a standard Python dictionary or list that you can work with.

This image shows the entire process in a Python IDE:

A simple script showing import requests, a requests.get() call to the GitHub API, and printing the status code and JSON content from the successful response.

The following short video provides a great end-to-end demonstration of this entire workflow, from installing requests to parsing the data.

Customizing Your Requests

Often, you'll need to provide more information to an API, for instance, to search for something specific or to authenticate yourself.

Query Parameters

Many APIs use query parameters in the URL to filter or specify data. For example: https://api.example.com/search?q=python&sort=popular. While you could build this string yourself, requests provides a cleaner way using the params argument.

Using the params dictionary is less error-prone and makes your code more readable than manually concatenating strings.

POST Requests and Sending Data

While GET is for retrieving data, POST is for sending data to a server to create or update a resource. This is what you'll use when you send your prompt to the Claude API.

With POST requests, the data is sent in the request body, not the URL. Your experience with web development has likely exposed you to form data and JSON payloads. The requests library handles both elegantly. For our purposes, sending JSON is the most relevant.

You can use requests.post() and pass your Python dictionary directly to the json argument. The library will automatically serialize your dictionary to a JSON string and set the appropriate Content-Type header to application/json.

This example shows a requests.post() call. The output confirms the method was POST, the status was 200, and the Content-Type of the response was application/json.

Here is a simple example of posting JSON data to httpbin.org, a service designed for testing requests:

import requests

url = "https://httpbin.org/post"
payload = {
    "username": "learner123",
    "action": "submit_query",
    "query_details": {
        "topic": "Python requests",
        "length": "concise"
    }
}

try:
    # Use the `json` parameter to send the payload
    response = requests.post(url, json=payload)
    response.raise_for_status()

    # The response from httpbin includes the data we sent
    response_data = response.json()
    print("Data received by server:")
    print(response_data['json'])

except requests.exceptions.HTTPError as err:
    print(f"HTTP error occurred: {err}")

Conclusion

You've now added a vital tool to your Python toolkit. By combining your knowledge of JSON from the last lesson with the requests library, you can now communicate with the vast majority of web APIs.

Key Takeaways:

• The requests library is the standard for making HTTP requests in Python.
• Use requests.get() to retrieve data and requests.post() to send data.
• The Response object returned by a request contains the status_code, headers, and the response body.
• Check for success by inspecting response.status_code or by using the response object in a conditional (if response:).
• Use response.raise_for_status() within a try...except block for robust error handling.
• The response.json() method is the most direct way to parse a JSON response into a Python dictionary.
• For GET requests, pass a dictionary to the params argument to add query parameters.
• For POST requests, pass a dictionary to the json argument to send a JSON payload.

In our next and final lesson of this module, we will put all these foundational skills together. We'll perform our first Project Milestone: initializing the project repository with a standard Python structure, including a virtual environment, a .gitignore file, and a requirements.txt file to manage dependencies like requests. This will set up the professional workspace for building our grasp.study clone.