Types of Machine Learning: A Comparative Analysis

What is Machine Learning?

Before we categorize different types of learning, let's first establish what we mean by "learning" in a computational context. Machine learning, at its core, is about creating systems that can improve their performance on a task by processing data or gaining experience, without being explicitly programmed for every scenario.

To formalize this, please watch this introductory segment from a classic Stanford machine learning lecture by Andrew Ng. He presents two influential definitions.

Tom Mitchell's definition is particularly useful: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E."

Throughout this course, we will see that the key difference between the major types of machine learning boils down to the nature of the "experience" (E) the algorithm is given.

The Three Paradigms of Machine Learning

Almost all machine learning problems can be grouped into one of three main categories: Supervised Learning, Unsupervised Learning, and Reinforcement Learning. The distinction is based on the type of data we have and the feedback mechanism used during the learning process.

This flowchart provides a high-level visual summary of their inputs, processes, and outputs. We will use this as our guide as we explore each paradigm in detail.

This flowchart illustrates the fundamental differences in the data, learning process, and output for supervised, unsupervised, and reinforcement learning.

1. Supervised Learning: Learning with an Answer Key

Supervised learning is the most common and well-established type of machine learning. The "supervision" comes from the data itself: for every data point in our training set, we have an input and a corresponding correct output, or "label". The goal is to learn a general rule that maps inputs to outputs. Mathematically, we are trying to learn a function, $f$, such that $y\approx f(x)$, where $x$ is the input and $y$ is the known output label.

This is analogous to system identification in engineering, where you might model a system's transfer function by observing its behavior for a known set of input signals and their corresponding output signals.

Andrew Ng provides an excellent explanation with clear examples.

To summarize, the key tasks in supervised learning are:

• Regression: The target variable $y$ is a continuous number.
  • Example: Predicting the future price of a stock based on historical data.
  • Example: Estimating the energy output of a solar panel based on weather conditions like temperature and cloud cover.
• Classification: The target variable $y$ belongs to a finite set of discrete categories.
  • Example: Classifying an email as "spam" or "not spam".
  • Example: Identifying defects in a manufactured electronic component from an image (e.g., "good", "short", "open").

To see how this works in a more complex application, watch this short clip on an early self-driving car project.

2. Unsupervised Learning: Finding Hidden Structure

In contrast to supervised learning, unsupervised learning algorithms work with data that has no labels. We are given only the inputs $x$, and the goal is to discover interesting patterns, structures, or relationships within the data itself. This is like being given a pile of mixed documents and being asked to sort them into coherent topics without knowing what the topics are beforehand.

Let's return to Andrew Ng's lecture for his explanation of this paradigm.

The cocktail party problem is a great example of a task known as source separation. Given multiple microphone recordings where each one captures a mix of different speakers, the goal is to isolate the individual speech signals. From a signal processing perspective, this is a classic blind source separation problem, often solved with techniques like Independent Component Analysis (ICA), which is an unsupervised learning algorithm.

The main tasks in unsupervised learning include:

• Clustering: Grouping data points such that points in the same group (or cluster) are more similar to each other than to those in other groups.
  • Example: Segmenting customers into different marketing groups based on their purchasing behavior.
• Dimensionality Reduction: Reducing the number of variables under consideration to find a lower-dimensional representation that still captures the essence of the data.
  • Example: Compressing high-dimensional financial data into a few key factors that drive market movements.
• Association Rule Learning: Discovering rules that describe large portions of your data, such as "customers who buy product A also tend to buy product B".
  • Example: Market basket analysis in a supermarket to optimize product placement.

For a slightly different and very structured perspective on the distinction between supervised and unsupervised learning, the following video is highly recommended.

3. Reinforcement Learning: Learning from Trial and Error

Reinforcement Learning (RL) is a completely different beast. It's not about learning from a static dataset. Instead, it’s about an agent that learns to behave in an environment by performing actions and seeing the results. The only feedback the agent gets is a numerical reward signal—a simple score indicating how good or bad its last action was. The goal of the agent is to learn a strategy, or policy, to maximize its total cumulative reward over time.

This paradigm is very similar to control theory. You have an agent (a controller) that interacts with an environment (a plant or system) by taking actions (control signals). It observes the new state of the environment and receives a reward (which could be related to minimizing an error signal). The objective is to learn an optimal control policy.

Andrew Ng's analogy to training a pet is perhaps the most intuitive way to understand this.

Key components of Reinforcement Learning:

• Agent: The learner or decision-maker.
• Environment: The world the agent interacts with.
• Action: A choice the agent can make.
• State: The current situation or configuration of the environment.
• Reward: The feedback signal from the environment.
• Policy: The agent's strategy for choosing actions based on the current state.

Examples:

• Training a computer program to play a game like chess or Go, where the reward is +1 for a win and -1 for a loss.
• Optimizing the energy consumption of a data center by having an agent control the cooling systems.
• Developing a robotic arm that learns to pick up and assemble objects through trial and error.

Putting It All Together

We've covered the three main paradigms. This diagram provides a useful summary, mapping each learning type to its common sub-problems and applications.

A hierarchical overview of the main types of machine learning, their associated tasks (like classification or clustering), and real-world examples.

It's also worth noting that the lines between these categories can sometimes blur. There are hybrid approaches like semi-supervised learning (where you have a lot of unlabeled data and a small amount of labeled data) that are becoming increasingly important. For now, however, a solid understanding of these three core areas is the most critical foundation you can build.

Conclusion

In this lesson, we've established the fundamental classification of machine learning problems. The key takeaway is the distinction based on the type of data and feedback involved:

• Supervised Learning: Learns from data that is fully labeled ($(x,y)$). The goal is to predict the label for new, unseen data.
• Unsupervised Learning: Learns from unlabeled data ($x$). The goal is to find hidden patterns or intrinsic structures within the data.
• Reinforcement Learning: An agent learns by interacting with an environment. The feedback is a numerical reward signal, and the goal is to learn a policy that maximizes long-term rewards.

In our next lesson, we will zoom in on the most common paradigm, supervised learning. We'll begin to explore the practicalities of building a model, starting with a crucial question: how do we properly split our data to train a model and reliably evaluate its performance? This will lead us to the concepts of training, validation, and test sets.