Importance Sampling for Variance Reduction

1. The Core Idea: Sample Where It Matters

The intuition behind importance sampling is simple: if we are trying to estimate an integral by taking random samples, we should concentrate our efforts where the function's value is highest. It's inefficient to waste computation on samples that will contribute little or nothing to the final result.

In path tracing, this means we want to generate rays that are more likely to travel along paths that carry significant light energy.

To get a solid grasp of this principle, let's start with a reading from the Physically Based Rendering book, a cornerstone text in computer graphics.

As the text explains, the ideal (but impossible) scenario would be to sample from a probability density function (PDF) that is directly proportional to the function we're integrating, $f(x)$. If we could do this, the estimator $f(x)/p(x)$ would be a constant, and the variance would be zero! This is a paradox because creating such a PDF would require us to already know the integral's value—the very thing we're trying to estimate.

However, this theoretical ideal gives us our practical goal: find a PDF $p(x)$ that is similar to $f(x)$ and that we can easily sample from.

2. The Mathematical Mechanism

So, how do we technically use a different distribution without calculating the wrong result? The method relies on a simple but powerful mathematical trick. We can change the distribution we sample from, as long as we compensate for that change by weighting our samples appropriately.

This video gives a clear, step-by-step derivation of how this works.

Let's summarize the key steps. We want to estimate the expectation ${\mathbb{E}}_p[f(x)]=\int f(x)p(x)dx$.

1. We introduce a new, "smarter" PDF, $q(x)$, that we want to sample from instead.
2. We rewrite the integral by multiplying and dividing by $q(x)$:
   $\int f(x)p(x)\frac{q(x)}{q(x)}dx$
3. We regroup the terms to form a new expectation, this time with respect to $q(x)$:
   $\int \left(f(x)\frac{p(x)}{q(x)}\right)q(x)dx={\mathbb{E}}_q\left[f(x)\frac{p(x)}{q(x)}\right]$
4. This gives us a new Monte Carlo estimator. Instead of averaging $f(X_i)/p(X_i)$ with samples from $p$, we now average $f(X_i)p(X_i)/q(X_i)$ with samples from $q$.

In our path tracer, we started with uniform hemisphere sampling. In that case, $p(x)$ was a constant $1/(2\pi )$. Our new estimator becomes:
$L_o\approx \frac{1}{N}{\sum }_{i=1}^N\frac{f(X_i)\cdot (1/2\pi )}{q(X_i)}$
Where $f(X_i)$ is the rendering equation integrand and samples $X_i$ are now drawn from our new, smarter PDF, $q(x)$.

The whole point is that if we choose $q(x)$ to be similar to $f(x)$, the term we are averaging will have much less variation, leading to lower variance and a cleaner image.

A word of caution: choosing a bad importance sampling distribution—one that is small where the integrand is large—can actually make the noise worse than uniform sampling. This happens because the few "lucky" samples that hit a high-energy region get weighted by a huge factor ($1/q(x)$), creating extreme fireflies and high variance.

3. Importance Sampling Strategies in Path Tracing

Now let's make this concrete. What functions can we use to build our "smarter" PDF? The rendering equation integrand gives us the main candidates:
$L_o(p,{\omega }_o)={\int }_{\Omega }\underset{\text{Incoming Light}}{\underbrace{L_i(p,{\omega }_i)}}\cdot \underset{\text{BRDF}}{\underbrace{f_r(p,{\omega }_i,{\omega }_o)}}\cdot \underset{\text{Cosine Term}}{\underbrace{\cos ({\theta }_i)}}d{\omega }_i$
We can create importance sampling strategies based on any of these terms:

1. Sampling the Cosine Term: For diffuse surfaces, the $\cos ({\theta }_i)$ term dominates the shape of the reflected light. A good strategy is to generate more sample rays near the surface normal $\mathbf{n}$ and fewer near the horizon. This is called cosine-weighted hemisphere sampling.

2. Sampling the BRDF: For glossy or reflective materials, the BRDF $f_r$ is highly directional. A good strategy is to generate sample rays that are concentrated around the specular reflection or refraction direction.

3. Sampling the Light ($L_i$): In scenes with distinct light sources, the incoming light $L_i$ is zero for most directions, and non-zero only for directions that lead to a light. The most effective strategy here is to explicitly sample directions that point directly at the lights. This is often called Next Event Estimation.

Let's watch a short segment that shows the derivation and effect of the first strategy: cosine-weighted sampling.

This cosine-weighting strategy is a significant improvement over uniform sampling for diffuse scenes, but it highlights a new problem.

4. The Challenge: A Product of Functions

The integrand is a product. What happens if one term is "peaky" (like a small light source) and another is also "peaky" (like a mirror reflection)? Which one should we follow?

• Consider a diffuse floor lit by a small, bright light. Cosine sampling (matching the BRDF and cosine term) is inefficient because it rarely hits the light. A better strategy would be to sample the light directly.
• Consider a mirror-like floor lit by a large, uniform sky dome. Sampling the light source (the sky) is inefficient because the mirror will only reflect a tiny patch of it. A better strategy would be to sample the BRDF (i.e., cast a single ray in the perfect reflection direction).

This reveals a fundamental challenge: no single importance sampling strategy is optimal for all situations. The best strategy depends on the complex interplay between lighting, geometry, and materials.

Modern renderers solve this by using Multiple Importance Sampling (MIS). Instead of choosing just one strategy, they take samples from several (e.g., one sample from the BRDF distribution, one from the light distribution) and then combine the results using a clever weighting heuristic that favors the strategy that was most likely to find that particular light path. This makes the renderer robust, allowing it to efficiently handle scenes with both large and small lights, and both diffuse and specular materials. We will delve into MIS in a later lesson.

Conclusion

You now understand the core principle behind one of the most important concepts in modern rendering. Importance sampling is the answer to the inefficiency of brute-force rendering.

Key Takeaways:

• Importance sampling reduces variance by choosing samples from a Probability Density Function (PDF) that is similar in shape to the function being integrated.
• This concentrates computational effort where the integrand is large, avoiding wasted samples.
• Mathematically, this is achieved by changing the sampling distribution and weighting the result by the ratio of the original PDF to the new PDF.
• In path tracing, common strategies involve creating PDFs based on the BRDF, the cosine term, or the light sources.
• Because the rendering integrand is a product of functions, no single strategy is always optimal. This motivates the use of more advanced techniques like Multiple Importance Sampling (MIS).

Next Steps

We've covered the "what" and "why" of importance sampling. In our next lesson, we'll move on to the "how." We will implement our very first, and arguably most impactful, importance sampling technique: explicit light source sampling. This will allow us to directly target the "fireflies" caused by small, bright lights and achieve a dramatic improvement in image quality.