Hello! Welcome to your second lesson. In our previous session, we established the core distinction between supervised and unsupervised learning. We saw that supervised learning is like having a "teacher" who provides labeled examples, allowing an algorithm to learn a predictive function.
Today, we'll build directly on that idea. Your learning outcome is to define Concept Learning and understand its role as a theoretical foundation for supervised learning. We will formalize the process of "learning from examples" by introducing a structured framework. This lesson will equip you with the precise terminology and conceptual model needed to understand how many supervised learning algorithms, which we'll cover later, actually work.
1. The Core Idea: Learning a "Concept"
In everyday life, we learn concepts all the time. For example, you have a concept of what constitutes a "car." It doesn't matter if it's a sports car, an SUV, or a vintage model you've never seen before; you can identify it as a car because you've learned the general concept from various examples.
Machine learning tries to replicate this process. Concept Learning is the task of inferring a general rule or function from specific training examples. This "general rule" is the concept.
To make this concrete for an exam, we formally define a concept as a boolean-valued function. This function takes an object (described by its attributes) and returns true (if it's an instance of the concept) or false (if it is not).
Let's begin by reading a short piece that introduces this idea.
Concept Learning In Machine Learning
This article from Applied AI Course provides a straightforward introduction to concept learning.
Please read the introductory paragraphs and the section titled "What is Concept Learning?". Focus on the definition of concept learning as learning a general rule from specific instances.
From your perspective in finance, a relevant concept might be "fraudulent transaction." The attributes could be the transaction amount, location, time of day, and the customer's typical spending history. The concept learning task would be to learn a function that takes these attributes and correctly labels a transaction as fraudulent (true) or not fraudulent (false).
2. The Formal Framework of Concept Learning
To understand how a machine learns a concept, we need a formal framework. This framework breaks down the problem into several key components. The diagram below illustrates the overall process.
Let's define each of these components, which are essential terms for your exam:
- Instance (X): This represents the set of all possible objects or data points. In the diagram,
Xis the input space. For our "fraud" example, an instance is a single transaction. - Target Concept (t): This is the true, unknown function that we are trying to learn. It perfectly classifies every instance in
X. We never actually see the target concept; we only have samples from it. In the diagram, this is the "Unknown Target Function." - Training Examples: These are a subset of instances that have been labeled by the target concept. This is our "labeled data" from the previous lesson. For example, a set of historical transactions, each marked as fraudulent or legitimate.
- Hypothesis (h): A hypothesis is a candidate function that we believe might be the target concept.
- Hypothesis Space (H): This is the set of all possible hypotheses that the learning algorithm is allowed to consider. For example, the hypothesis space could be all possible linear equations or, as we'll see later, all possible decision trees.
- Learning Algorithm (A): The algorithm's job is to search through the hypothesis space
Hto find the single "best" hypothesis that is most consistent with the training examples. This "best" hypothesis is our Final Hypothesis.
The goal of concept learning is to find a hypothesis h in H such that h(x) is as close as possible to the true target concept t(x) for all instances x.
To see these definitions in action, let's explore a classic example.
Concept Learning and General to Specific Ordering-Part(1) – MLDawn Academy
This article from MLDawn Academy walks through a simple and clear example of learning the concept "days to play golf."
Please read the section An Example for Concept Learning. As you read, try to map the terms from the article to the ones we just defined: The "target concept" is "Days on which James will play golf." The table shows the "training examples." A single row is an "instance." A "hypothesis" is defined as a vector of constraints on the attributes.
3. Concept Learning as a Search Problem
As the reading suggests, concept learning is fundamentally a search problem. The learning algorithm searches through the hypothesis space to find a hypothesis that best fits the training data.
This framing is central to understanding machine learning. The choice of algorithm and hypothesis space determines how this search is conducted.
Concept Learning Concept Space Hypothesis Space Distinct Hypothesis Space Machine Learning Mahesh
This video from Mahesh Huddar formally defines concept learning as a search task.
Please watch from the beginning up to the formal definition. Pay close attention to how he phrases the task: "searching through the predefined space of potential solutions... for the hypothesis that best fits the training examples."
The size of this search space can be vast. For a problem with even a few binary attributes, the number of possible concepts can be enormous. This is why we can't just check every single hypothesis; we need intelligent algorithms to search this space efficiently.
For example, a video from the channel Trouble-Free explains that for a problem with d binary attributes, there are 2^d possible instances but 2^(2^d) possible concepts. This exponential growth illustrates why the search is non-trivial.
4. The Foundation for Supervised Learning
So, how does this relate to supervised learning? Concept learning is the theoretical model that formalizes the task of supervised learning.
- In our last lesson, we said supervised learning finds a mapping
y = f(x)from labeled data. In the language of concept learning,fis the target concept, and the labeled data are the training examples. - The algorithm we choose (e.g., for linear regression, SVM, or a decision tree) defines the hypothesis space
H. - The "training" process is the algorithm's search through
Hto find the final hypothesishthat best approximatesf.
The critical assumption that allows us to trust our final hypothesis on new, unseen data is called the Inductive Learning Hypothesis.
Concept Learning and General to Specific Ordering-Part(1) – MLDawn Academy
Let's return to the MLDawn Academy article to understand the fundamental assumption that makes this all work.
Please read the short section titled "The Inductive Learning Hypothesis."
The Inductive Learning Hypothesis states that the best hypothesis for future data is the one that best fits our current training data. This is the cornerstone of supervised machine learning. It's a leap of faith, but it's a leap that allows a model to generalize from the specific examples it has seen to the broader world of unseen data.
Conclusion
In this lesson, we have placed our understanding of supervised learning onto a firm theoretical foundation. We have moved from the intuitive idea of "learning from examples" to a formal model of searching for a function.
Here are the key takeaways for your exam:
- Concept Learning is the task of inferring a general, boolean-valued function (a concept) from a set of labeled training examples.
- It is formally modeled as a search problem where a learning algorithm searches a hypothesis space (H) for a hypothesis (h) that best approximates the unknown target concept (t).
- The Inductive Learning Hypothesis is the fundamental assumption that a hypothesis that performs well on the training data will also generalize well to new, unseen data.
- This framework provides the theoretical basis for many supervised learning algorithms.
In our next lesson, we will look at our first practical example of a concept learning algorithm: the Decision Tree. You'll see how a decision tree provides a specific hypothesis space and a method for searching it to find a predictive model.