Interacting with LLM APIs

Welcome to the third module of our course! In the previous module, we explored the theoretical underpinnings of how Large Language Models are trained and aligned. We covered everything from pre-training on vast datasets to the nuances of scaling laws and emergent abilities.

Now, it's time to transition from theory to practice. This module, "Interacting with LLMs: APIs and Prompting," is all about getting your hands on the keyboard and making these models work for you.

Today's lesson directly addresses our first practical learning outcome: Use an LLM API (e.g., OpenAI) to send prompts and process responses programmatically. We will walk through every step, from getting your credentials to writing and running a Python script that communicates with an LLM. This foundational skill is the gateway to building every application that follows, from RAG systems to AI agents.

1. The API: Your Programmatic Gateway to LLMs

As a software engineer, you're already familiar with APIs (Application Programming Interfaces) as contracts that allow different software systems to communicate. LLM APIs are no different. They provide a structured way for your applications to send requests (prompts) to a powerful, remotely-hosted model and receive its generated responses (completions).

This request-response cycle is the fundamental building block of any LLM-powered application.

2. Setup and Authentication: Your Keys to the Kingdom

Before you can make your first API call, you need to set up your account and get an API key. This key authenticates your requests and links your usage to your account for billing.

Step 1: Get Your OpenAI API Key

You'll need to create an OpenAI account, add a payment method, and generate a secret API key. While many introductory tutorials use free credits, setting up a payment method is a crucial step for building real applications. OpenAI's APIs are very affordable for development and experimentation—the code in this lesson will likely cost you less than a few cents.

Step 2: Secure Your API Key

Never hardcode your API key directly in your source code. This is a major security risk, especially if you use version control like Git. The standard best practice is to store the key as an environment variable.

Your Python script can then read this variable from your environment at runtime. This keeps your credentials separate from your code.

For project-specific development, it's also common to use a .env file to manage environment variables. We'll explore this approach in the code example below.

Step 3: Set Up Your Python Environment

Finally, you need to install the official OpenAI Python library. Open your terminal and run the following command:

pip install openai

Optionally, if you prefer using .env files for managing your API key within a project directory, you can also install python-dotenv:

pip install python-dotenv

3. Making Your First API Call

With the setup complete, let's write the code. Create a new Python file named first_call.py.

Here is the complete script to send a prompt to OpenAI and print the response.

import os
from openai import OpenAI




# The OpenAI library automatically looks for the OPENAI_API_KEY 
# environment variable.
# If you set it up correctly, you don't need to pass the api_key parameter.
client = OpenAI()




# This is the core of the API call
response = client.chat.completions.create(
  model="gpt-4o-mini", # Specifies which model to use
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the difference between an API and an SDK?"}
  ]
)




# Extracting the content from the response object
assistant_reply = response.choices[0].message.content

print(assistant_reply)

Now, run the script from your terminal: python first_call.py. You should see the LLM's explanation appear on your screen!

Understanding the Code

Let's break down the key parts of that client.chat.completions.create() call.

• model: This string tells the API which model you want to use. "gpt-4o-mini" is a great choice for general tasks—it's fast, capable, and very cost-effective. You can easily swap this out for other models like "gpt-4o".

• messages: This is the most important parameter. It's a list of dictionaries, where each dictionary represents one part of the conversation. Each message has a role and content.

  • role: "system": The system message sets the context and instructions for the AI's behavior. It's like giving the AI its job description before the conversation starts.
  • role: "user": This is the input from the end-user—the prompt you want the AI to respond to.
  • role: "assistant": This role is used to include previous responses from the AI, which is how you build conversational memory. We'll explore this in a later lesson.

• The Response Object: The API doesn't just return a simple string. It returns a complex JSON object containing the model's reply along with other metadata. The actual text is nested inside. The path response.choices[0].message.content is how you access the primary response text.

4. Controlling Model Output

You can customize the model's output by adding more parameters to your API call. Two of the most important are temperature and max_tokens.

Let's see this in action.

Here's how you would add these to your code:

response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "system", "content": "You are a creative poet."},
    {"role": "user", "content": "Write a short, four-line poem about code."}
  ],
  temperature=0.8,  # Higher temperature for more creativity
  max_tokens=60     # Limit the response to 60 tokens
)

Experiment by changing these values in your script and see how the output changes. Getting a feel for these parameters is a key part of learning to control LLMs.

Conclusion

Congratulations! You have successfully moved from theory to practice by making your first programmatic call to a large language model. This is the fundamental skill upon which almost all modern AI applications are built.

Key Takeaways:

• LLM APIs allow your applications to interact with models via a standard request-response pattern.
• Securely managing your API key using environment variables is a critical security practice.
• The openai Python library provides a straightforward interface for making API calls.
• The messages parameter, with its system and user roles, is the primary way to structure your prompts.
• The API returns a JSON object, and you need to parse it to extract the model's generated text (response.choices[0].message.content).
• Parameters like temperature and max_tokens give you essential control over the creativity and length of the model's output.

Preview of the Next Lesson:

Now that you know how to send a prompt, our next lesson will focus on what to send. We will dive into the world of prompt engineering, covering fundamental techniques like zero-shot, few-shot, and chain-of-thought prompting to elicit more accurate and complex behavior from the model.