Introduction to Machine Learning Paradigms

In our previous lesson, we established a formal definition for machine learning, framing it as a paradigm shift where programs are not explicitly written but are instead inferred from data. We explored this idea through the lens of "Software 2.0," where the goal is to find an optimal model within a vast space of possibilities. Now, we will build upon this foundation by categorizing the broad field of machine learning.

While the core idea of learning from data is universal, the specific nature of the task and the type of data available lead to fundamentally different approaches. This lesson dissects machine learning into its three canonical paradigms: supervised, unsupervised, and reinforcement learning. Understanding the distinctions between these three is the first and most critical step in framing any real-world problem in machine learning terms. By the end of this lesson, you will be able to distinguish between these paradigms, identify the type of problem each is suited for, and cite illustrative examples for each.

A Map of the Machine Learning World

At the highest level of abstraction, machine learning algorithms can be grouped based on the kind of "experience" (E) they learn from, as per Tom Mitchell's definition. Does the experience consist of labeled examples provided by a "supervisor"? Does it consist of raw, unlabeled data where the algorithm must find structure on its own? Or is the experience an interactive process of trial and error guided by environmental feedback? These questions give rise to the three major paradigms.

This image provides a useful map, showing the three categories and some of their real-world applications.

We will now examine each of these branches in detail.

1. Supervised Learning: Learning with a Teacher

Supervised learning is the most common and conceptually straightforward paradigm. The name derives from the idea that a "supervisor" or "teacher" provides the algorithm with a curated dataset containing the correct answers.

Formally, in supervised learning, we are given a dataset of $n$ examples, $(x^{(1)},y^{(1)}),(x^{(2)},y^{(2)}),...,(x^{(n)},y^{(n)})$, where each $x^{(i)}$ is an input feature vector and each $y^{(i)}$ is the corresponding known output or "label." The goal is to learn a function $h:X\to Y$ that can accurately predict the output $y$ for a new, unseen input $x$.

Dr. Andrew Ng's Stanford lecture provides a clear and intuitive introduction to this concept.

As Dr. Ng explained, supervised learning encompasses two primary task types:

• Regression: The target variable $y$ is a continuous, real-valued number. The goal is to predict a quantity.
  • Example: Predicting the price of a house based on its size, location, and number of bedrooms. The input $x$ is a vector of house features, and the output $y$ is the price.
• Classification: The target variable $y$ is a discrete category from a finite set of labels. The goal is to assign a class.
  • Example: Classifying an email as "spam" or "not spam." The input $x$ could be a vector representing the words in the email, and the output $y$ is one of two classes. The tumor example from the video is another classic case.

The key takeaway is that supervised learning is fundamentally about predictive modeling, leveraging labeled data to learn a mapping that generalizes to new, unlabeled data.

2. Unsupervised Learning: Finding Structure on Your Own

What if we don't have the "correct answers"? In many real-world scenarios, obtaining labeled data is expensive, time-consuming, or simply impossible. This is where unsupervised learning comes in.

In unsupervised learning, we are given a dataset consisting only of input data $x^{(1)},x^{(2)},...,x^{(n)}$, with no corresponding labels $y$. The goal is not to predict an output, but to discover interesting patterns, relationships, or structures inherent in the data itself. It is about descriptive modeling.

The primary types of unsupervised learning include:

• Clustering: The task of grouping a set of objects such that objects in the same group (or cluster) are more similar to each other than to those in other groups. This is what the Google News and market segmentation examples from the video illustrate.
• Representation Learning & Dimensionality Reduction: The process of transforming data into a more informative and usually lower-dimensional representation. This is useful for data visualization, compression, and as a pre-processing step for other ML tasks. Principal Component Analysis (PCA) and autoencoders are canonical examples.
• Association Rule Learning: Discovering interesting relations between variables in large databases. The classic example is "market basket analysis," which might find that customers who buy diapers also tend to buy beer.

The following resource provides a clear textual overview of both supervised and unsupervised learning, with helpful diagrams.

An excellent way to contrast supervised and unsupervised learning is to see how they approach the same dataset. This MIT lecture clip does exactly that using an intuitive example.

3. Reinforcement Learning: Learning from Consequences

The third major paradigm, reinforcement learning (RL), is fundamentally different from the other two. It is not about learning from a static dataset but about learning to make optimal decisions through interaction with an environment.

In the RL framework, an agent exists in an environment. At each step, the agent observes the environment's state, takes an action, and receives a reward (or penalty) and a new state as a consequence. The goal of the agent is to learn a policy—a mapping from states to actions—that maximizes its total cumulative reward over time.

This paradigm is about learning through trial and error, much like how we train a pet.

The defining characteristics of RL are:

• No Supervisor: There are no "correct" action labels. The only feedback is a scalar reward signal.
• Delayed Gratification: An action might not be immediately rewarded or punished. The agent must learn to link actions to their delayed consequences, a problem known as credit assignment.
• Sequential Decision-Making: The agent's choice of action affects the next state it sees, and thus its future opportunities for reward. The agent must learn a sequence of actions.

Classic applications of RL include:

• Game Playing: Mastering games like Go (AlphaGo) and chess where the goal is to win (the ultimate reward).
• Robotics: Training a robot to walk, grasp objects, or fly a helicopter, where the reward can be based on staying upright, successfully picking up an item, or following a trajectory.
• System Optimization: Managing resources in a data center or optimizing a chemical reaction.

Emerging Paradigms

While supervised, unsupervised, and reinforcement learning are the three classical pillars, the field is continuously evolving. Two other important categories, which blur the lines between the main three, are crucial in modern AI.

• Semi-supervised Learning: This is a hybrid approach that uses a small amount of labeled data along with a large amount of unlabeled data. The core idea is to use the labeled data to help infer labels for the unlabeled data, effectively amplifying the small labeled set.
• Self-supervised Learning: This is a powerful technique where labels are generated automatically from the input data itself. For example, in natural language processing, a model can be trained to predict a masked word in a sentence. The sentence itself provides the "supervision." This approach is the engine behind modern large language models like GPT, as it allows them to learn from vast amounts of unlabeled text on the internet.

Summary and Comparison

The choice of paradigm is dictated by the problem you are trying to solve and the data you have. The following flowchart provides a clear visual summary of the different inputs and outputs for each of the three main paradigms.

Here is a summary table for quick reference:

Conclusion

In this lesson, we have partitioned the vast field of machine learning into its primary operational paradigms. You can now distinguish between supervised, unsupervised, and reinforcement learning based on their goals, data requirements, and fundamental approaches.

Here are the key takeaways:

• Supervised Learning uses labeled data to train predictive models for either classification (discrete outputs) or regression (continuous outputs).
• Unsupervised Learning works with unlabeled data to discover hidden structures, such as clusters of similar data points or lower-dimensional representations.
• Reinforcement Learning involves an agent learning to make a sequence of decisions in an environment through trial and error, guided by a reward signal.
• Hybrid Paradigms like semi-supervised and self-supervised learning are powerful modern techniques that leverage both labeled and unlabeled data, with self-supervision being a key driver of recent advances in AI.

We have now defined what machine learning is and mapped out its main territories. Our course will now proceed with a deep dive into the most common paradigm: supervised learning. In the next lesson, we will begin to unpack the components of a supervised learning model, defining the concepts of hypothesis spaces, model parameters, and hyperparameters, which are the essential building blocks for constructing and training these models.