Building ML Class Hierarchies with Dunder Methods

1. The Core Idea: Classes and Objects

At its heart, OOP is about organizing software by bundling related data and the functions that operate on that data into self-contained units called objects. A class is the blueprint or template for creating these objects.

Think of a class as a cookie cutter and objects as the cookies. The cutter defines the shape (the data structure and available operations), and you can use it to create many individual cookies (objects), each with its own specific properties (e.g., some are burnt, some have extra chocolate chips).

In the context of machine learning, a LinearRegression class could be a blueprint. An object model = LinearRegression() would be a specific instance of that model, with its own learned coefficients after being trained on a particular dataset.

Sebastian Raschka's video provides an excellent introduction to these core concepts. Pay close attention to his cookie-cutter analogy and his explanation of the self keyword, which is how an object refers to itself from within its class methods.

2. Defining a Class: Attributes and Methods

A class definition consists of two main components:

• Attributes: Variables that store the data associated with an object. For an ML model, this could include hyperparameters set at creation (like n_neighbors in KNN) or learned parameters found during training (like the weights in a linear model).
• Methods: Functions defined inside a class that operate on the object's attributes. The canonical examples in ML are fit() and predict().

Dunder Methods: Python's Special Hooks

Python classes include special methods with names that start and end with double underscores, often called "dunder" methods. These aren't meant to be called directly; instead, Python calls them for you when you perform certain actions.

The most common dunder method is __init__, the class constructor. It's automatically called when you create a new object, and it's the perfect place to initialize the object's attributes.

Let's return to the video by Sebastian Raschka to see how __init__ is used to create a new object and how other methods can access or modify its attributes.

3. Building Your First ML Estimator

Now, let's apply these concepts to build a class that mimics the structure of a Scikit-learn estimator. This design pattern is ubiquitous in the Python ML ecosystem. It provides a consistent interface for all models, which makes it easy to swap them out and build reusable pipelines.

The core interface consists of:

1. Initialization: Creating an instance of the model class, typically setting hyperparameters.
2. Fitting: Calling a fit(X, y) method to train the model on data.
3. Predicting: Calling a predict(X) method to generate predictions on new data.

The following article from Towards Data Science provides a step-by-step guide to building a custom linear regression estimator.

The Scikit-Learn Convention for Fitted Attributes

When building Scikit-learn-compatible estimators, there's an important convention: attributes that are learned from the data during the fit method should have a trailing underscore. For example, self.coef_ or self.classes_.

This convention serves two purposes:

1. It clearly distinguishes between hyperparameters set during initialization (e.g., self.n_neighbors) and parameters learned during training.
2. It provides a simple way to check if a model has been fitted by checking for the existence of these attributes.

Sebastian Raschka explains this convention clearly in his implementation of a KNN classifier.

4. Code Reusability with Inheritance

Imagine you have several ML model classes. They might share common functionality, like methods for loading data, scoring, or plotting results. Instead of copying and pasting this code into every class, you can use inheritance.

Inheritance allows you to define a parent class (or base class) that contains common logic, and then create child classes that "inherit" all of the parent's methods and attributes. The child classes can then add their own specific functionality or override the parent's methods if needed.

The following article provides a clear, simple example of inheritance using a Person and Worker class before applying the concept to an ML problem.

Inheritance in Scikit-Learn: Base Classes and Mixins

The Scikit-learn library makes extensive use of inheritance to ensure all its estimators are consistent and interoperable. If you want to build a custom model that works seamlessly with Scikit-learn tools like GridSearchCV or Pipeline, you should inherit from the provided base classes.

The key base classes are:

• BaseEstimator: The fundamental base class for all estimators. It provides essential methods like get_params() and set_params().
• ClassifierMixin: A "mixin" class for classification estimators. It provides helpful methods, most notably the .score() method, which calculates the mean accuracy.
• RegressorMixin: A mixin for regression estimators, which provides a .score() method that calculates the R² score.

The term "mixin" refers to a class that provides methods to other classes via multiple inheritance but is not intended to be instantiated on its own.

The video below demonstrates exactly how to use these base classes to create a custom model.

5. A Complete Example: Custom Nearest Centroid Classifier

Now, let's tie everything together and examine a complete, practical implementation of a custom classifier. The following video walks through the creation of a NearestCentroidClassifier. This model works by finding the "centroid" (the mean vector) for each class in the training data. To make a prediction, it finds the closest centroid to the new data point and returns that centroid's class.

This is a perfect example because the logic is easy to understand, but it requires all the components we've discussed:

• An __init__ method to prepare attributes.
• A fit method to learn the centroids from data (these will be our centroids_).
• A predict method to calculate distances and return the closest class.

A More Rigorous Approach: Abstract Base Classes

For completeness, it's worth knowing about a more formal way to define a base class: using Python's abc module to create an Abstract Base Class (ABC).

In the article "Leverage Python Inheritance in ML projects" (LINK), the author defines a BaseMLModel using class BaseMLModel(ABC):. They then use the @abstractmethod decorator on the preprocess_train_data method. This decorator ensures that any child class must implement its own version of that method. If a child class is instantiated without implementing it, Python will raise a TypeError. This is a powerful software engineering pattern for enforcing a consistent API across a family of classes.

Conclusion

In this lesson, we've taken a deep dive into Object-Oriented Programming and its application in machine learning. You've learned how to structure your code into reusable and logical units, a skill that is paramount for any AI research or engineering role.

Key takeaways from today's lesson include:

• Classes and Objects: Classes are blueprints that bundle data (attributes) and functionality (methods). Objects are the instances created from these blueprints.
• Dunder Methods: Special methods like __init__ allow your custom objects to integrate with Python's core syntax and behaviors.
• The Estimator API: The fit/predict pattern is the standard interface for ML models in Python, promoting consistency and interoperability.
• Inheritance: This powerful OOP principle allows you to create hierarchies of classes, reuse common code, and extend functionality. We saw how Scikit-learn uses this with BaseEstimator and mixin classes to build its entire ecosystem.

So far, we've built our models using basic Python loops and data structures. While this is great for understanding the logic, it's not efficient for the large-scale numerical computations that underpin machine learning. In our next lesson, we will begin our journey into NumPy, the fundamental library for numerical computing in Python. You'll learn how to perform fast, vectorized array operations—the bedrock of virtually every ML algorithm you will encounter.