Understanding AI: Capabilities and Limitations

Introduction

In our last lesson, we examined the mechanics of text generation, seeing how Large Language Models (LLMs) operate as sophisticated token predictors. You learned that their ability to produce complex text stems from a simple, autoregressive loop: predicting the next most probable token based on the preceding sequence. This probabilistic foundation is the key to both their power and their fallibility.

Today, we will explore the direct consequences of this design. Your objective is to identify the critical capabilities and limitations of current AI models. We will focus on three fundamental constraints that you must understand to effectively build with and work alongside AI: hallucinations, bias, and the finite context window. Mastering these concepts is non-negotiable for any developer aiming to create robust, reliable AI-powered applications.

The Double-Edged Sword of Probabilistic Prediction

The token-prediction engine we discussed is designed to optimize for one thing: plausibility. The model learns patterns from its training data and uses them to generate sequences that look like human-written text. It is not designed to verify truth, adhere to facts, or possess a true understanding of the world. This distinction is the root cause of many of its most prominent limitations.

The image below provides a useful mental model. The most visible problem, hallucinations, is merely the tip of an iceberg. The deeper, structural causes lie in the quality of the training data, the model's inherent lack of real-world context, and the way we interact with it.

Let's dissect these core limitations one by one.

1. Hallucinations: When Plausibility Isn't Truth

An AI hallucination is when a model generates text that is factually incorrect, nonsensical, or disconnected from the provided context, yet presents it with complete confidence. It’s not a bug, but an emergent property of a system optimizing for statistical coherence over factual accuracy.

The IBM Technology video "Why Large Language Models Hallucinate" offers a concise overview of what hallucinations are and their primary causes.

As the video explains, the core reasons for hallucinations are deeply tied to the model's construction. An article from MIT Sloan provides a clear, structured explanation.

For you as a developer, this means you can never blindly trust the output of an LLM, especially for factual information, code snippets, or API documentation references. Always assume verification is required. Later in this course, when we discuss integrating LLM APIs, we'll cover mitigation strategies like Retrieval-Augmented Generation (RAG), which helps ground the model's responses in a trusted body of data.

2. Bias: A Distorted Reflection

AI models are trained on vast datasets scraped from the internet—a corpus that contains not only the sum of human knowledge but also the full spectrum of its biases, stereotypes, and prejudices. Since the model's job is to learn statistical patterns, it inevitably learns and often amplifies these societal biases.

The same MIT article we just referenced provides a powerful overview of this problem.

The findings in the article are critical. When a system is shown to be less accurate for darker-skinned females, or when it generates stereotypical images for job titles, it's not acting with intent. It is simply reflecting the statistical imbalances and historical biases present in its training data.

As a developer building AI-powered features, you must be vigilant about this. An AI feature that recommends job candidates, generates user profile pictures, or moderates content could easily cause significant harm if its inherent biases are not accounted for and actively mitigated through careful testing, fine-tuning, and the implementation of guardrails.

3. The Context Window: A Finite and Imperfect "Memory"

One of the most important technical limitations to understand is the context window. This is the fixed amount of text (measured in tokens) that a model can process at one time. Everything within this window—your prompt, the preceding conversation, any documents you provided—is visible to the model as it generates the next token.

A common misconception is that the context window is a form of memory. It is not. The model does not "remember" your conversation. Rather, the application you're using (like a web UI) continuously re-sends the entire conversation history with every new turn, up to the context window's limit.

The following article is one of the best developer-focused explanations of how context windows truly work.

This "rereading" process has a significant computational cost. The attention mechanism, which allows the model to weigh the importance of different tokens in the context, has a complexity that scales quadratically with the length of the input sequence ($O(n^2)$). Your computer science background will tell you immediately why this is a major bottleneck.

Even within the context window, the model's "attention" is not perfect. Models exhibit a "lost in the middle" problem, where they pay more attention to the beginning and end of a long context and tend to forget information buried in the middle. Furthermore, as the context grows, the attention gets "diluted," making it harder for the model to focus on any single instruction.

When the conversation exceeds the context window, the application managing the conversation must resort to strategies like:

• Truncation: Simply cutting off the oldest messages.
• Summarization: Using the LLM to summarize the beginning of the conversation.

Both of these are lossy and can cause the model to forget critical instructions or context, a behavior you need to anticipate when designing long-running interactions.

Conclusion

In this lesson, we transitioned from the how of LLM text generation to the why of its failures and limitations. Understanding these constraints is just as important as knowing the model's capabilities.

Here are your key takeaways:

• Hallucinations are a feature, not a bug: They are a natural consequence of a model designed to generate plausible, statistically likely sequences rather than factually accurate statements.
• Bias is a reflection: LLMs learn and often amplify the biases present in their training data. This makes them a mirror of societal patterns, both good and bad.
• Context windows are not memory: They are a finite processing buffer. The quadratic scaling of the attention mechanism imposes a hard computational limit, and information can be "lost in the middle" even within this buffer.

You have now completed the first module on AI foundations. You understand the relationships between AI, ML, and Deep Learning; how neural networks learn; the different types of machine learning; the supervised training process; how LLMs generate text; and their fundamental limitations.

This theoretical groundwork is essential for the next phase. In our next lesson, we will begin Module 2, "Applied AI," where you will learn to use this knowledge to your advantage. We will start with a highly practical application: using AI-powered code assistants like GitHub Copilot to accelerate your daily development work, all while keeping the limitations we've discussed today firmly in mind.