CNN Training: Loss & Accuracy Monitoring

Introduction

Welcome back! In our previous lesson, you successfully designed the "blueprint" for a Convolutional Neural Network by defining its architecture in PyTorch. You learned how to stack nn.Conv2d and nn.MaxPool2d layers and, critically, how to calculate the tensor dimensions to connect them to the final nn.Linear classifier.

So far, our network is just a structure with random, uninitialized weights. It doesn't "know" anything. Today, we will bring it to life. This lesson focuses on the core process of training your CNN on a standard image dataset and monitoring its performance.

We will cover the complete workflow:

1. Loading and preparing a dataset (CIFAR-10).
2. Defining a loss function and an optimizer.
3. Implementing the training loop to update the model's weights.
4. Evaluating the trained model's accuracy on unseen data.

This entire process is often called the training loop, a fundamental concept in deep learning.

1. Preparing for Training: Data, Loss, and Optimizer

Before we can start training, we need three key components:

1. Data: A labeled dataset to learn from. We'll use CIFAR-10, a popular benchmark dataset of 32x32 color images in 10 classes (airplanes, cars, birds, etc.). torchvision makes it incredibly easy to load.
2. Loss Function: A way to measure how wrong the model's predictions are. For multi-class classification, nn.CrossEntropyLoss is the standard choice. It combines a softmax activation and the negative log-likelihood loss, making it perfect for our needs.
3. Optimizer: An algorithm that updates the model's weights based on the calculated loss. We will use Stochastic Gradient Descent (SGD) with momentum, a classic and effective choice.

The following reading from the official PyTorch tutorials demonstrates how to set up these three components in code.

To recap, your setup will include:

• A transform pipeline to convert images to normalized tensors.
• A trainset and testset created using torchvision.datasets.
• A trainloader and testloader created using torch.utils.data.DataLoader to serve data in batches.
• An instance of nn.CrossEntropyLoss for your criterion.
• An instance of torch.optim.SGD for your optimizer, passing in the model's parameters (net.parameters()) and a learning rate.

2. The Training Loop: Teaching the Network

The training loop is where the learning happens. It consists of two nested loops:

• An outer loop that iterates over the entire dataset multiple times. Each full pass over the dataset is called an epoch.
• An inner loop that iterates over the DataLoader, processing one batch of data at a time.

For each batch, we perform five core steps:

1. Get the inputs and labels from the DataLoader.
2. Zero the gradients: optimizer.zero_grad().
3. Forward pass: Feed the inputs to the network to get outputs.
4. Compute loss: Compare the network's outputs with the true labels using the criterion.
5. Backward pass & Optimize: Calculate gradients with loss.backward() and update the weights with optimizer.step().

A common point of confusion is Step 2: optimizer.zero_grad(). Why is this necessary? In PyTorch, gradients are accumulated (i.e., added up) on subsequent backward passes. If we didn't clear them, the new gradients would be added to the gradients from the previous batch, leading to incorrect weight updates.

The following video provides an exceptionally clear explanation and visual demonstration of this concept.

Now that you understand the "why" behind each step, let's see the complete training loop in code.

3. Monitoring Loss and Measuring Accuracy

As you saw, printing the running_loss gives us an idea of whether the network is learning. If the loss is consistently decreasing, the training is on the right track.

However, loss is an indirect measure. We ultimately care about accuracy: how many images does the model classify correctly? To measure this, we evaluate the model on the testloader, which contains data the model has never seen during training. This gives us an honest assessment of its generalization ability.

The evaluation process involves:

1. Setting the model to evaluation mode with net.eval(). This deactivates layers like dropout or batch normalization that behave differently during training and inference.
2. Looping through the test data within a with torch.no_grad(): block. This tells PyTorch not to calculate gradients, saving memory and computation.
3. For each batch, getting the model's predictions. The model outputs raw scores (logits) for each class. We use torch.max(outputs, 1) to find the index (i.e., the class) with the highest score.
4. Comparing the predicted classes to the ground-truth labels and counting the number of correct predictions.
5. Calculating the final accuracy percentage.

Let's see how this is implemented.

To tie everything together, let's watch a final video that provides a concise, narrated walkthrough of the entire training and evaluation process. This will reinforce the concepts you've just read about.

Conclusion

Congratulations! You have now walked through the entire process of training and evaluating a Convolutional Neural Network. This is a milestone in your journey from theory to practice.

Key Takeaways:

• Training requires a dataset (DataLoader), a loss function (nn.CrossEntropyLoss), and an optimizer (optim.SGD).
• The training loop iterates through the data in epochs and batches.
• Each training step involves a forward pass, loss calculation, backward pass, and weight update.
• optimizer.zero_grad() is essential to prevent gradients from accumulating across batches.
• Monitoring the training loss helps verify that the model is learning.
• Evaluating the model on a separate test set using torch.no_grad() provides a crucial measure of its accuracy and generalization ability.

In this lesson, we focused on a single metric: overall accuracy. In our next lesson, we will explore more detailed ways to evaluate a CNN's performance, including using a confusion matrix and calculating per-class metrics to understand where your model excels and where it struggles.