Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 9

Atrous Convolutions: Multi-Scale Context and Spatial Preservation

Recommended resources

Hello! Let's dive into our next topic.

In our previous lesson, we explored the encoder-decoder architecture of Fully Convolutional Networks (FCNs). We saw how they ingeniously enabled pixel-wise prediction but also highlighted a core tension: the encoder's pooling layers, necessary for creating large receptive fields and abstract features, simultaneously discard the precise spatial information needed for accurate segmentation. Skip connections offer a partial remedy, but the fundamental problem of resolution loss remains.

Today, we will tackle this problem head-on. This lesson covers the learning outcome: Describe the mechanism of atrous (dilated) convolutions in capturing multi-scale context while preserving spatial resolution. You'll learn about the elegant technique that allows a network to expand its "field of view" to understand context, all while avoiding the resolution-destroying effects of pooling. This concept is central to modern segmentation models like DeepLabV3, which you've already worked with.

The Challenge: Resolution vs. Context

Let's begin by recapping the central conflict in dense prediction tasks. A standard CNN, like one used for classification, aggressively downsamples the input. It uses pooling layers or strided convolutions to reduce the feature map's height and width at each stage.

There are two main reasons for this:

  1. Computational Efficiency: Smaller feature maps require fewer computations.
  2. Receptive Field Growth: Each pooling/striding operation makes the subsequent convolutional kernels cover a larger effective area of the original input image, allowing the network to learn more abstract, high-level features.

However, for semantic segmentation, this is a major drawback. We need a dense output, not a single label. The video below explains this problem in the context of the DeepLab model.

MLT __init__ Session #2: DeepLab — Semantic Image Segmentation

This video from the MLT channel introduces the challenges that the DeepLab architecture was designed to solve. The first problem discussed is exactly what we need to focus on.

Watch the segment from the beginning. The speaker clearly explains how typical downsampling operations in CNNs, while useful for classification, result in a loss of feature resolution that is detrimental to segmentation.

The core question then becomes: can we expand the receptive field to capture context without downsampling and losing spatial resolution? The answer is yes, and the tool is atrous convolution.

Atrous Convolution: A Convolution with Holes

The term "atrous" comes from the French phrase à trous, meaning "with holes." This perfectly describes the mechanism. An atrous, or dilated, convolution is a standard convolution where the kernel is applied to the input with defined gaps between its elements.

This is controlled by a new hyperparameter: the dilation rate, denoted by .

  • A dilation rate of is a standard convolution, where the kernel's weights are applied to a contiguous patch of pixels.
  • A dilation rate of means there is a 1-pixel gap between the kernel weights. A 3x3 kernel will now cover a 5x5 area of the input feature map, but it still only uses 9 parameters and 9 multiplications.
  • A dilation rate of means there are gaps between the kernel weights.

The image below provides a great visual intuition for how this works.

This diagram illustrates a 3x3 kernel with dilation rates of 1, 2, and 3. With a rate of 1 (left), it's a standard convolution covering a 3x3 area. With a rate of 2 (center), the same kernel covers a 5x5 area by skipping pixels. With a rate of 3 (right), it covers a 7x7 area. The number of learnable weights (red squares) remains just 9 in all cases.

Let's ground this in a more formal definition.

Dilated (Atrous) Convolutions Explained

This article from apxml.com provides a very clear definition of dilated convolutions and their motivation.

Read the initial section, starting from the introduction down to the 1D mathematical formula. This will give you the core definition and mathematical intuition behind the dilation rate r.

The Power of Dilation: Exponential Receptive Field

The main advantage of this technique is that we can dramatically and efficiently increase the receptive field. By stacking atrous convolutions, we can achieve an exponential growth in the receptive field without any loss of resolution.

The original paper that introduced this for modern segmentation models explains this property very clearly.

[PDF] multi-scale context aggregation by dilated convolutions

Let's consult the source paper by Yu and Koltun that popularized this technique. It provides the core theoretical justification for its effectiveness.

In Section 2, find the paragraph starting with "Our architecture is motivated..." and read it carefully. Pay attention to how the receptive field size is calculated: (2^(i+2) - 1) x (2^(i+2) - 1). This formula demonstrates how stacking layers with exponentially increasing dilation factors (1, 2, 4, ...) causes the receptive field to grow exponentially.

The video we looked at earlier also contains an excellent, hand-drawn visualization of how receptive fields expand. This will help solidify your intuition about why a larger field of view is so important and how atrous convolutions achieve it.

MLT __init__ Session #2: DeepLab — Semantic Image Segmentation

Let's return to the MLT video for a visual walkthrough of the receptive field concept.

Watch the explanation starting from this timestamp. The presenter first clarifies what a receptive field is using standard convolutions, and then shows precisely how a dilated convolution achieves a larger field of view with the same 3x3 kernel.

Visualizing the Architectural Impact

So, what does this look like in the context of a full network? We can replace the standard pooling or strided convolutional layers in the deeper parts of a network with atrous convolutions. This allows the network to keep growing its receptive field while maintaining the spatial resolution of its feature maps.

The image below shows a side-by-side comparison that makes the benefit crystal clear.

This image compares two network architectures. The top (a) shows a standard deep CNN where repeated downsampling (increasing output stride from 4 to 256) drastically reduces the feature map size. The bottom (b) shows a network where, after an initial downsampling to a stride of 16, atrous convolutions with increasing rates are used. This allows the receptive field to expand while keeping the feature map size (and output stride) constant, preserving crucial spatial detail for segmentation.

This image perfectly encapsulates the lesson's main point. The top network loses spatial information, ending with tiny feature maps. The bottom one maintains a much larger feature map throughout its deeper layers, making it far better suited for generating a dense, pixel-wise prediction.

Implementation and Practical Considerations

Given your background, you'll appreciate how straightforward this is to implement in modern deep learning frameworks. It's not a complex new layer type, but simply a parameter of the existing convolution layer.

MLT __init__ Session #2: DeepLab — Semantic Image Segmentation

Seeing this in code helps demystify the concept. This part of the video shows a simple PyTorch implementation.

Watch the code walkthrough from this section. Notice how atrous convolution is enabled simply by setting the dilation parameter in the torch.nn.Conv2d layer. The presenter also highlights a practical detail about adjusting padding to maintain feature map size, which is an important implementation detail.

While powerful, this technique is not without a potential pitfall. Naively stacking atrous convolutions with the same high dilation rate can lead to a "gridding" artifact, where the kernel consistently misses sampling certain pixels.

Dilated (Atrous) Convolutions Explained

Let's return to the apxml.com article to learn about this artifact.

Read the short section on The Gridding Effect. Understanding this issue and its solution—using varied dilation rates in subsequent layers—is key to applying this technique effectively and is a common practice in state-of-the-art models.

Conclusion

In this lesson, we've unpacked the mechanism of atrous (dilated) convolutions and seen how they provide a powerful solution to the context-vs-resolution trade-off in semantic segmentation.

Key Takeaways:

  • Mechanism: Atrous convolution introduces "holes" into a standard convolution kernel, controlled by a dilation rate parameter.
  • Benefit 1: Multi-Scale Context: It allows a kernel to have a larger receptive field without increasing the number of parameters or computations. Stacking these layers leads to an exponential growth of the receptive field.
  • Benefit 2: Preserved Spatial Resolution: This increase in receptive field is achieved without any downsampling operations like pooling or striding, thus preserving the full spatial resolution of the feature maps.
  • Application: This makes atrous convolution a cornerstone of modern semantic segmentation models like DeepLab, which replace pooling layers in their backbone with atrous convolutions to extract dense, context-rich features.

You've already used DeepLabV3, a model that heavily relies on this concept. In our next lesson, we will analyze the architecture of DeepLabV3 itself, focusing specifically on its Atrous Spatial Pyramid Pooling (ASPP) module. ASPP takes the idea of atrous convolution a step further by applying multiple dilation rates in parallel to capture contextual information at several scales simultaneously.