Training Gaussian Naive Bayes with scikit-learn

From Counts to Curves: Gaussian Naive Bayes

In our last discussion, we used text classification as an example, where features are discrete (e.g., word counts). But what if our features are continuous, like temperature, age, or sensor readings? We can't simply count their occurrences.

This is where Gaussian Naive Bayes comes in. It handles continuous features by making an additional assumption:

For each class, the distribution of each continuous feature is assumed to be Gaussian (i.e., a normal distribution or "bell curve").

Under this assumption, "training" the model doesn't involve counting frequencies. Instead, the algorithm calculates the mean ($\mu$) and variance (${\sigma }^2$) of each feature for each class from the training data. When a new data point comes in, the model uses the Gaussian probability density function to calculate the likelihood $P(\text{Feature}|\text{Class})$ for each feature.

The image below illustrates this perfectly. For the feature 'PSA level', the model learns two separate Gaussian distributions: one for the 'Healthy' class and one for the 'Cancer' class, each with its own mean and standard deviation.

With this intuition in mind, let's see how to implement it.

Training a Classifier with Scikit-learn

We will now walk through the process of training a GaussianNB classifier on the classic Titanic dataset. The goal is to predict survival based on features like passenger class, sex, and age.

The following video provides a clear, practical demonstration of the entire workflow. Since you're already familiar with libraries like Pandas and Scikit-learn, focus on how the specific GaussianNB model is integrated into the familiar machine learning pipeline.

Step 1: Data Preparation

First, the data must be loaded and prepared for the model. This involves dropping irrelevant columns, converting categorical text data into numerical format (one-hot encoding), and handling any missing values.

Step 2: Model Training

Once the data is clean and numeric, the next steps are to split it into training and testing sets, import the GaussianNB model, and train it. The beauty of scikit-learn is how consistent this process is across different algorithms.

The key lines of code for training are remarkably concise:

# 1. Import the model
from sklearn.naive_bayes import GaussianNB




# 2. Instantiate the model
model = GaussianNB()




# 3. Train the model on the training data
model.fit(X_train, y_train)

All the underlying work of calculating the means and variances for each feature and class happens automatically within the .fit() method.

Step 3: Evaluation and Prediction

With a trained model, we can now evaluate its performance on the unseen test data and use it to make predictions.

Using .predict_proba() is a great way to see the model's "confidence" in its prediction, which ties directly back to the posterior probabilities we discussed in the previous lesson.

A Complete Example for Reference

To bring everything together, here is a complete, self-contained example of training a GaussianNB classifier on a synthetic dataset. You can use this as a quick reference for the entire process.

This example is a great template for applying GaussianNB to any classification problem with continuous features.

Conclusion

Congratulations! You have successfully bridged the gap from theory to practice. In this lesson, we focused on applying the Naive Bayes algorithm to a dataset with continuous features.

Key Takeaways:

• Gaussian Naive Bayes is the version of the algorithm used for continuous features (e.g., age, measurements).
• It operates by assuming that features for each class follow a Gaussian (normal) distribution.
• "Training" involves calculating the mean and variance of each feature for each class.
• Implementing Gaussian Naive Bayes in scikit-learn is a straightforward process: import GaussianNB, instantiate it, and use the .fit() and .predict() methods.

Preview of the Next Lesson

We've now explored our first probabilistic classifier. Next, we will pivot to an entirely different and highly intuitive family of models: tree-based methods. In the upcoming lesson, you will learn how a single Decision Tree makes predictions by learning a series of simple, hierarchical rules from the data.