Introduction to LLMs: Capabilities and Limitations

What is a Large Language Model?

At its core, an LLM is a type of AI model designed to understand and generate human-like text. You can think of it as a highly sophisticated autocomplete system. Its fundamental task is to predict the next word in a sequence of text.

To get a concrete sense of what an LLM is, from the perspective of a software engineer, let's watch the first few minutes of a talk by Andrej Karpathy, a prominent AI researcher. He explains that a powerful, 70-billion-parameter LLM is essentially just two files: a very large file of numbers (the parameters or weights) and a relatively small piece of code to run them.

So, an LLM takes a sequence of words as input and, based on the patterns it has learned, calculates the probabilities for what the next word might be. By repeatedly picking the next word and feeding the new sequence back into itself, it can generate entire sentences, paragraphs, or more.

For a quick, animated overview of this process, the following video from 3Blue1Brown is excellent.

How Are LLMs Created? The Two-Stage Training Process

As you just saw, creating an LLM involves a massive training process, which happens in two main stages.

Stage 1: Pre-training (Building Knowledge)

This is where the model learns about language, grammar, facts, reasoning styles, and countless other patterns.

• Data: The model is trained on an enormous dataset, essentially a huge chunk of the public internet (think Wikipedia, books, articles, code repositories, etc.). For a model like GPT-3, this amounts to hundreds of billions of words.
• Objective: The model's only goal during this stage is to get good at next-word prediction.
• Outcome: The result is a base model. As Karpathy puts it, this process is like creating a "lossy compression of the internet." The model isn't just memorizing text; it's creating an internal representation of the knowledge contained within it. These base models are not yet helpful assistants; they are more like "internet document dreamers."

This pre-training process is computationally immense and expensive, costing millions of dollars and requiring thousands of specialized processors (GPUs) running for weeks.

Stage 2: Fine-Tuning (Teaching a Skill)

A base model knows a lot, but it doesn't know how to be a helpful assistant. The fine-tuning stage aligns the model with this goal.

• Data: Instead of the raw internet, the model is now trained on a much smaller, curated, high-quality dataset. This dataset consists of example conversations, often in a question-and-answer format, crafted by humans to demonstrate desired behavior (e.g., being helpful, harmless, and following instructions).
• Objective: The training objective is the same (next-word prediction), but the data teaches the model a specific conversational style.
• Outcome: The result is an assistant model, like the one you interact with in ChatGPT. It leverages the vast knowledge from pre-training but applies it in a helpful, conversational format.

This fine-tuning process is much cheaper and faster, allowing companies to iterate and improve their assistant models regularly. Karpathy provides a great overview of these two stages.

What Can LLMs Do? Capabilities

The simple process of next-word prediction leads to a surprising range of emergent capabilities.

As the image shows, LLMs are excellent at:

1. Content Generation and Summarization: They can draft emails, write articles, create marketing copy, and summarize long documents.
2. Semantic Search and Classification: They understand the meaning behind words, allowing them to perform searches based on concepts, not just keywords. They can also classify text (e.g., sentiment analysis).
3. Code Generation and Debugging: They can write functions, translate code between languages, explain code snippets, and help identify bugs. This is a powerful application for software engineers.
4. Conversational Interfaces and Tool Use: This is the core of their use in agents. LLMs can act as a reasoning engine that understands a user's intent and decides to use external tools—like a calculator, a search engine, or an API—to accomplish a task.

This "tool use" capability is a major leap forward and is what separates a simple chatbot from a true agent. Let's see a powerful demonstration of this.

What Can't LLMs Do? Limitations and Risks

Understanding the limitations of LLMs is critical for using them effectively and safely. Their power comes from statistical pattern matching, not from true understanding or consciousness.

1. Hallucinations (Factual Inaccuracy): Because LLMs are designed to generate plausible-sounding text, they can confidently invent facts, sources, or details. This is often called "hallucination." They are not knowledge databases and cannot be relied upon for factual accuracy without verification.

2. Lack of True Reasoning: LLMs mimic reasoning patterns they've seen in their training data. They don't possess genuine logic or causal understanding. They may struggle with complex, multi-step reasoning, especially in novel situations that deviate from patterns they've seen before. For a deeper explanation of this distinction, the article "The Strengths and Limitations of Large Language Models" is a good resource.

3. Limited Context Window: LLMs can only consider a certain amount of text (the "context window") at one time. For very long conversations or documents, they will forget what was said earlier. We will explore techniques to mitigate this in Module 3.

4. Security Vulnerabilities: As LLMs are integrated into applications, they introduce new attack vectors. This is especially relevant for agent development.

   • Jailbreaking: Tricking a model into bypassing its safety rules, often through clever role-playing prompts.
   • Prompt Injection: This is a serious vulnerability where an attacker hides malicious instructions within a piece of text (like a webpage or a document) that the LLM processes. The LLM may then follow the attacker's instructions instead of the user's, potentially leading to data leaks or other malicious actions.

The following segments from Karpathy's talk provide some striking and important examples of these security risks.

A Glimpse Inside the Black Box

You might be wondering how an LLM internally represents language to achieve these capabilities. While we will cover this in-depth in Module 10, here's a quick, high-level conceptual overview.

LLMs don't see words as strings of characters. Instead, they convert them into numerical lists called word vectors or embeddings. Each vector represents a word's position in a high-dimensional "meaning space." Words with similar meanings have similar vectors. This allows the model to reason about relationships between words.

If you're curious, the following article explains these concepts very clearly without requiring deep mathematical knowledge.

Conclusion

In this lesson, we've taken a high-level tour of Large Language Models. Here are the key takeaways:

• What they are: LLMs are AI models that excel at predicting the next word in a sequence. They are trained in two stages: pre-training on vast internet text to gain knowledge, and fine-tuning on curated examples to become helpful assistants.
• Their capabilities: Their core text-prediction ability enables a wide range of applications, including content generation, semantic understanding, code writing, and, most importantly for us, acting as a reasoning engine to use external tools.
• Their limitations: They are not infallible. They can hallucinate incorrect information, they lack true reasoning, and they introduce new security risks like prompt injection that developers must be aware of.

Understanding this balance of power and peril is the first step to becoming a capable AI agent builder.

In our next lesson, we will move from concept to practice. You'll set up your development environment and use the OpenAI API to send your first prompts to an LLM and get responses back. This will be our first hands-on step toward building an agent.