YOLO: A Conceptual Dive into Object Detection

From Classification to Detection

In a previous module, you fine-tuned a ResNet model for image classification. This task answers the question: "What is in this image?" For example, does this image show a "Good," "Adequate," or "Poor" quality meal?

Object detection takes this a step further. It answers two questions simultaneously: "What is in this image, and where is it?" Instead of a single label for the whole image, the output is a set of bounding boxes, each with an associated class label. For your project, this means drawing a box around the portion of rice and labeling it "rice," another around the dal and labeling it "dal," and so on.

The video below provides an excellent introduction to this concept, showing how object localization extends image classification by adding bounding box coordinates to the model's output.

The YOLO Philosophy: A Single-Shot Approach

Older object detection methods, like the R-CNN family (Regions with CNN features), used a two-stage approach. First, they would propose several potential regions in the image that might contain objects. Second, they would run a separate classifier on each of these regions. This is effective but computationally expensive and slow.

YOLO revolutionized the field by framing object detection as a single regression problem, straight from image pixels to bounding box coordinates and class probabilities. It processes the entire image in a single forward pass through a neural network, which is why it's called a "single-shot detector." This design makes YOLO incredibly fast and suitable for real-time applications.

To understand the core idea, let's explore this article from DigitalOcean.

How YOLO Works: The Grid System

The central innovation of YOLO is how it processes the image. Instead of proposing arbitrary regions, YOLO divides the input image into an $S\times S$ grid. Each grid cell is then responsible for detecting objects whose center point falls within that cell.

For each grid cell, the model predicts a fixed-size tensor containing all the necessary information:

1. Bounding Box Predictions: Each grid cell predicts a number of potential bounding boxes.
2. Confidence/Objectness Score: For each bounding box, it predicts a score that represents the probability that the box actually contains an object.
3. Class Probabilities: For each grid cell, it predicts the probability of the object belonging to each of the possible classes (e.g., rice, dal, vegetable).

This process is explained exceptionally well in the following video and article.

So, if you have a $7\times 7$ grid, and for each cell, you predict 2 bounding boxes and have 3 classes (rice, dal, vegetable), the final output of the network is a large tensor of size $7\times 7\times (2\times 5+3)$. The '5' comes from 4 bounding box coordinates ($x,y,w,h$) plus 1 objectness score. This unified prediction is what makes YOLO so efficient.

Refining Raw Predictions: NMS and IoU

The raw output from the grid contains hundreds of potential bounding boxes. Many of these will be redundant (multiple boxes for the same object) or have very low confidence. Two post-processing steps are crucial to clean this up.

1. Confidence Thresholding: A simple first step is to discard any predicted boxes whose confidence score is below a certain threshold (e.g., 0.25).

2. Non-Maximum Suppression (NMS): After thresholding, you might still have multiple overlapping boxes for the same object. NMS is a greedy algorithm that selects the best box and eliminates the redundant ones. It works as follows:

   • Select the box with the highest confidence score.
   • Calculate its Intersection over Union (IoU) with all other boxes. IoU is a metric from 0 to 1 that measures the extent of overlap between two boxes.
   • Remove any boxes that have a high IoU (e.g., > 0.5) with the selected box.
   • Repeat the process with the remaining boxes.

This ensures that for each object, you are left with only one, high-quality detection.

A Final Detail: Anchor Boxes

There's one last problem: what if a single grid cell contains the centers of multiple objects? Or what if an object has an unusual aspect ratio (e.g., a long, thin breadstick)? The original YOLO had difficulty with this.

The solution is anchor boxes. Instead of having each grid cell predict bounding boxes from scratch, it predicts offsets from a set of predefined default box shapes of various sizes and aspect ratios. A grid cell can be assigned multiple anchor boxes, allowing it to detect multiple objects that are centered there. This makes the model much better at handling objects of different shapes and sizes.

Conclusion

In this lesson, we've unpacked the conceptual framework behind the YOLO object detection algorithm. You've seen how it builds upon classification to provide localization and how its unique single-shot architecture makes it remarkably fast and efficient.

Here are the key takeaways:

• Object detection combines classification ("what") with localization ("where").
• YOLO is a single-shot detector, processing an image in one pass for high speed.
• The core idea is to divide the image into an $S\times S$ grid, with each cell responsible for detecting objects centered within it.
• Each grid cell predicts bounding boxes, an objectness/confidence score, and class probabilities.
• Post-processing steps like confidence thresholding and Non-Maximum Suppression (NMS), which uses Intersection over Union (IoU), are essential to refine the raw predictions into a clean set of detections.
• Anchor boxes help the model detect multiple objects within a single grid cell and handle objects with varied shapes.

You now have the theoretical foundation needed to start working with YOLO. In our next lesson, we will move from concept to code. You will set up the Ultralytics YOLOv8 library and run a powerful, pretrained model on your meal images to see object detection in action.