Managing LLM Costs: Tokens and Dollars

The Unit of Cost: What Exactly is a Token?

Everything in the world of LLM pricing revolves around tokens. But what are they? An LLM doesn't read text as words or characters; it reads it as a sequence of tokens. A token can be a whole word, a part of a word (like ing), a punctuation mark, or even a space.

Different models use different "tokenizers" to split text, which is why the same prompt can result in a different number of tokens—and a different cost—depending on the provider. Since you're building in TypeScript, the best way to develop an intuition for this is to see it in code.

The video makes it clear that tokens are the fundamental unit. As a developer, you need to be aware of what kind of text is "token-heavy."

From Tokens to Dollars: Calculating the Cost

Now that you understand what tokens are, let's connect them to your bill. Every modern LLM provider's API response includes a usage object that tells you exactly how many tokens were processed.

The basic cost formula is:

Note that prices are almost always quoted "per million tokens" (or sometimes per 1K tokens). The most important part of this formula is the cost asymmetry: output tokens are typically 3-5 times more expensive than input tokens. This is because generating output requires significantly more computation than simply reading the input.

In your Node.js code, after a non-streaming API call (or after a stream is complete and you have the final response object), you would access this like so:

// Example using OpenAI's SDK
import OpenAI from 'openai';

const openai = new OpenAI();

async function main() {
  const completion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Tell me a short story.' }],
    model: 'gpt-4o',
  });

  // Extract usage data
  const usage = completion.usage;
  if (usage) {
    console.log(`Input tokens: ${usage.prompt_tokens}`);
    console.log(`Output tokens: ${usage.completion_tokens}`);
    console.log(`Total tokens: ${usage.total_tokens}`);

    // Example cost calculation (prices are for illustration)
    const inputPricePerMillion = 3.00; // $ per 1M tokens
    const outputPricePerMillion = 15.00; // $ per 1M tokens

    const inputCost = (usage.prompt_tokens / 1000000) * inputPricePerMillion;
    const outputCost = (usage.completion_tokens / 1000000) * outputPricePerMillion;
    const totalCost = inputCost + outputCost;

    console.log(`Total cost: $${totalCost.toFixed(6)}`);
  }
}

main();

Below the Water Line: Hidden Costs

The simple calculation above is the "tip of the iceberg." To truly control costs for production agents, you must understand the factors lurking below the surface.

A comprehensive article from "Oh That Data Girl" dives deep into these nuances. While the code is in Python, the concepts are universal and directly applicable to the APIs you're using.

Putting It All Together: Practical Monitoring

Understanding cost is one thing; tracking it is another. For any production service, you need a system to log every API call and its associated cost. This allows you to monitor expenses, identify costly workflows, and bill clients accurately.

Your experience with n8n is a huge advantage here. You can build a robust monitoring system with tools you already know.

This n8n workflow is a powerful and immediately usable pattern. You can adapt it to log data to a PostgreSQL database or any other system as your needs grow. The principle is the same: log every call, capture the usage data, and calculate the cost.

Conclusion

This lesson concludes our foundational module. You now have a comprehensive understanding of the core mechanics of LLMs, from token generation and streaming to the critical economics of their operation.

Here are the key takeaways on cost management:

• Tokens are the currency: All costs are derived from the number of input and output tokens.
• Cost is asymmetrical: Output tokens are significantly more expensive than input tokens.
• Look below the surface: The true cost is influenced by hidden factors like different tokenizers, "thinking" tokens, and caching.
• Monitoring is non-negotiable: You cannot control what you cannot measure. Setting up a logging system from day one is essential for any commercial AI application.

You've built a solid foundation. You know how LLMs work, how to compare them, how to control their output, how to stream it efficiently, and how to manage the cost.

Now, we are ready to move from mechanics to artistry. In the next module, Advanced Prompt Engineering for Agents, we'll begin to explore how to craft the instructions that make an LLM a capable and reliable agent. Our first lesson will focus on the most important prompt of all: the system prompt, which defines your agent's role, capabilities, and personality.