Quantization Effects on Digital Filters

1. The Landscape of Quantization Effects

When we implement a digital filter on a processor or a specialized chip like an FPGA, we are forced to move from the infinite precision of mathematics to the finite precision of hardware. This constraint introduces several types of errors.

As the report outlines, we can classify these effects into four groups:

1. Coefficient Quantization: The coefficients of our filter (e.g., the multipliers in a difference equation) are designed with ideal values but must be stored using a finite number of bits. This means the filter we build is slightly different from the one we designed.
2. A/D Conversion Error: The initial process of converting an analog signal to a digital one involves quantization. This is an unavoidable source of error at the system's input.
3. Arithmetic Roundoff Error: When we perform arithmetic operations like multiplication, the result often requires more bits than we have available. We must round or truncate the result, introducing a small error.
4. Overflow: The finite number of bits also limits the maximum value a number can represent. If a calculation exceeds this range, it "overflows," causing a large, non-linear error.

To understand these effects, we must first understand the two primary ways numbers are represented in DSP systems: floating-point and fixed-point.

2. Number Representations: Floating-Point vs. Fixed-Point

You are likely familiar with floating-point numbers from general-purpose programming. They are the standard float and double types in languages like C++ or Python.

A floating-point number uses a certain number of bits for a mantissa (which determines precision) and the remaining bits for an exponent (which determines the scale or range). This allows it to represent both very large and very small numbers.

In contrast, many specialized DSP chips and FPGAs use fixed-point arithmetic for reasons of speed, cost, and power efficiency.

As the video illustrates, a fixed-point number representation has a constant step size between adjacent values. It's essentially an integer that is scaled by a fixed amount. The "binary point" (the equivalent of a decimal point) is in a fixed position.

Let's formalize this. A fixed-point number is defined by its word length (total number of bits) and its fraction length (how many of those bits come after the binary point).

For example, a signed 16-bit number with a fraction length of 15 (often denoted Q15) uses 1 bit for the sign and 15 bits for the fractional part. The underlying signed 16-bit integer ranges from -32768 to +32767. To get the real-world value, we multiply by $2^{-15}$. This means the number can represent values from $-32768\times 2^{-15}=-1.0$ up to $32767\times 2^{-15}\approx 0.99997$.

The key takeaway is that in fixed-point, the programmer must choose the allocation of bits between the integer part (which sets the dynamic range) and the fractional part (which sets the precision).

3. Effect on Signal Representation and Arithmetic

The difference between these representations directly impacts the nature of the quantization error that occurs during A/D conversion and arithmetic operations.

Fixed-Point Quantization Error

In a fixed-point system, the gap between any two representable numbers is constant, equal to the value of the Least Significant Bit (LSB), which is $2^{-\text{fraction length}}$. When we round a real value to the nearest representable number, the error is additive.

The error, $e$, is typically modeled as a random variable. If we use rounding-to-nearest, the error is bounded:
$-\frac{1}{2}\text{LSB}\le e\le +\frac{1}{2}\text{LSB}$

This error is independent of the signal's magnitude. A signal with a value of 0.9 and a signal with a value of 0.1 are both quantized with the same absolute error bound.

A very useful rule of thumb in signal processing connects the number of bits to the dynamic range or Signal-to-Noise Ratio (SNR).

The "6 dB per bit" rule states that each additional bit of quantization improves the maximum possible SNR by approximately 6 decibels. For an $N$-bit quantizer, the SNR is roughly $6.02\times N$ dB. This is a fundamental guideline in DSP system design.

Floating-Point Quantization Error

In a floating-point system, the quantization error is more complex. Because the step size between representable numbers depends on the exponent, the absolute error is larger for larger numbers. The error is best described as multiplicative or relative.

When a real value $x$ is represented as a floating-point number fl(x), the result can be modeled as:
$\text{fl}(x)=x(1+ϵ)$
where $ϵ$ is the relative error. The magnitude of $ϵ$ is bounded by the precision of the mantissa (e.g., for a t-bit mantissa, $|ϵ|\le 2^{-t}$).

This means the quantization error is proportional to the signal itself. A large signal will have a large absolute error, while a small signal will have a small absolute error.

4. Effect on Digital Filter Coefficients

The coefficients in a filter's difference equation, like $y[n]=b_{0}x[n]+b_{1}x[n-1]-a_{1}y[n-1]$, are calculated to achieve a specific frequency response. These coefficients must also be quantized, which means the filter we actually implement has slightly different coefficients than our design intended. This alters the filter's pole and zero locations, and thus its frequency response.

This effect is most dramatic for filters that require very precise coefficient values, such as low-frequency, high-Q (narrow-bandwidth) filters.

Your background in control systems and pole-zero analysis provides a powerful way to visualize this. For a given number of bits in a fixed-point system, there is a finite grid of possible values for the coefficients $a_1$ and $a_2$. This, in turn, defines a finite grid of possible pole locations in the z-plane.

This visualization makes it clear why low-frequency, high-Q filters are so sensitive. In that region of the z-plane (close to $z=1$ on the real axis), the density of realizable poles is low, and a tiny change in a coefficient can cause a large jump in pole location, drastically altering the filter's performance. Floating-point representations create a different, non-uniform grid, but the fundamental problem of finite placement options remains.

5. Summary and Comparison

Let's summarize the trade-offs between fixed-point and floating-point for DSP applications.

Ultimately, the choice is an engineering trade-off. Fixed-point offers superior performance and efficiency if you are willing to do the careful numerical analysis required. Floating-point offers convenience and a massive dynamic range at the cost of more complex hardware.

Conclusion

In this lesson, we have seen that moving from theory to practice introduces the critical constraint of finite precision.

Key Takeaways:

• Digital implementation introduces errors from coefficient quantization, arithmetic roundoff, and potential overflow.
• Fixed-point representation uses a fixed binary point, offering uniform absolute precision. Its quantization error is additive and its magnitude is independent of the signal level.
• Floating-point representation uses a mantissa and exponent, offering a vast dynamic range. Its quantization error is multiplicative, meaning the absolute error is proportional to the signal level.
• Quantizing filter coefficients alters the filter's pole-zero locations, leading to errors in the frequency response. This is most severe for low-frequency, high-Q filters.
• The choice between fixed-point and floating-point is a fundamental engineering trade-off between hardware efficiency and ease of development.

Preview of the Next Lesson:

We've characterized the nature of these quantization errors. The next logical step is to analyze their impact. We will learn how to model arithmetic roundoff errors as a source of random noise within the filter structure. This will allow us to derive expressions for the output noise variance and the overall Signal-to-Noise Ratio (SNR) of a digital filter, providing a quantitative way to assess the damage done by finite precision.