STFT for Time-Frequency Analysis

1. The Limitation of the Fourier Transform

The Fourier Transform is a cornerstone of signal processing, telling us what frequencies are present in a signal. However, it does so by averaging over the signal's entire duration. This is perfect for stationary signals, where the statistical properties (like frequency content) do not change over time.

But what about signals like speech, music, or radar chirps? Their frequency content is constantly evolving. These are called non-stationary signals.

For such signals, knowing only what frequencies exist is not enough; we also need to know when they exist. The following video uses a great example—the tones generated by a telephone keypad (DTMF signals)—to illustrate this exact problem.

As the video shows, we need a method that combines the strengths of both domains. This is precisely what the Short-Time Fourier Transform was designed to do.

2. The "Divide and Conquer" Strategy of STFT

The core idea of the STFT is elegantly simple: if a signal is non-stationary globally, we can assume it is "stationary enough" (or quasi-stationary) within a small, localized time window. The STFT formalizes this by breaking the long signal into a sequence of shorter, overlapping segments and then computing the Fourier transform for each segment.

This process generates a series of frequency spectra, each corresponding to a different point in time, allowing us to see how the frequency content evolves.

The following diagram illustrates this process.

This sliding window approach gives us two key parameters:

• Window Length: The duration of the segment being analyzed at each step.
• Hop Length (or Hop Size): The number of samples the window is shifted forward for the next analysis. The difference between the window length and hop length is the overlap.

3. Mathematical Formulation of the STFT

Let's formalize this process mathematically. The discrete-time STFT of a signal $x[m]$ is defined as:
$X[n,k]={\sum }_{m=-\infty }^{\infty }x[m]w[n-m]e^{-j\frac{2\pi }{N}km}$

Let's break this down:

• $x[m]$ is our input signal.
• $w[n-m]$ is a window function (e.g., Hamming, Hann, or a simple rectangular window) of a finite length. The index $n$ represents the position of the window in time; as $n$ changes, the window slides along the signal. The product $x[m]w[n-m]$ isolates a short segment of the signal.
• The rest of the expression is simply the Discrete Fourier Transform (DFT) of this windowed segment, computed for $N$ frequency bins indexed by $k$.

The result, $X[n,k]$, is a two-dimensional function representing the signal's frequency content (indexed by $k$) at a specific time (indexed by $n$).

For a rigorous and clear presentation of this formula, please refer to the following notes.

The Spectrogram

The STFT $X[n,k]$ produces complex numbers, containing both magnitude and phase information. For visualization, we are often most interested in the magnitude. The spectrogram is a visual representation of the magnitude (or magnitude squared) of the STFT. It is typically plotted as an image with:

• Time on the horizontal axis.
• Frequency on the vertical axis.
• Magnitude represented by color or intensity (e.g., bright colors for high energy, dark for low).

The next video segment brings this to life, applying the STFT to the DTMF signal and constructing its spectrogram.

4. The Time-Frequency Resolution Trade-off

The STFT seems like the perfect solution, but it comes with a fundamental compromise, dictated by the choice of the window function's duration. This is known as the time-frequency resolution trade-off.

• A long (wide) window captures many samples. This allows the FFT to distinguish between closely spaced frequencies, providing good frequency resolution. However, because the analysis is averaged over a long duration, any rapid changes within that window are blurred, leading to poor time resolution.

• A short (narrow) window captures few samples. This pinpoints the timing of events accurately, providing good time resolution. However, with fewer samples, the FFT has a harder time separating close frequencies, resulting in poor frequency resolution.

This trade-off is not a flaw in the algorithm but an inherent property of signals, related to the Heisenberg Uncertainty Principle.

The following video provides an excellent discussion of this trade-off, using the terms "wideband" and "narrowband" spectrograms, and demonstrates it with both the DTMF and a speech signal.

5. A Deeper Perspective: The Filter Bank Interpretation

Given your background in signal processing, you'll appreciate that the STFT can be interpreted in ways other than just a "sliding window FFT." The CMU notes you looked at earlier describe two other mathematically equivalent interpretations that connect the STFT directly to filtering concepts.

Let's summarize the three interpretations:

1. Fourier Transform Interpretation (Fig 3.2): This is the view we've used so far. The signal $x[m]$ is multiplied by a sliding window $w[n-m]$, and the DFT of the product is taken.

2. Lowpass Filter Interpretation (Fig 3.3): Here, the signal $x[m]$ is first modulated by a complex exponential $e^{-j{\omega }_{k}m}$. This shifts the frequency content at ${\omega }_k$ down to baseband (DC). This frequency-shifted signal is then passed through a lowpass filter whose impulse response is the window function $w[m]$. The output of the filter at time $n$ gives you the STFT coefficient $X[n,k]$. You do this in parallel for each frequency ${\omega }_k$.

3. Bandpass Filter Interpretation (Fig 3.4): In this view, the STFT is implemented as a bank of bandpass filters. The original signal $x[m]$ is fed into a parallel bank of filters. Each filter, with impulse response $w[m]e^{j{\omega }_{k}m}$, is a bandpass filter centered at frequency ${\omega }_k$. The output of each bandpass filter is then demodulated (shifted back to baseband) to produce the STFT coefficients.

This filter bank interpretation is particularly powerful. It shows that the STFT is fundamentally a way of analyzing a signal through a set of uniformly spaced bandpass filters, where the bandwidth of each filter is determined by the Fourier transform of the window function. The time-frequency trade-off can now be seen as a filter design problem: a narrow passband (good frequency resolution) requires a long impulse response (poor time resolution), and vice versa.

Conclusion

In this lesson, we have moved from one-dimensional analysis (time or frequency) to two-dimensional time-frequency analysis. We established why this is necessary for non-stationary signals and saw how the STFT accomplishes this.

Key Takeaways:

• The standard Fourier Transform is unsuitable for non-stationary signals because it averages frequency information over all time, losing temporal detail.
• The Short-Time Fourier Transform (STFT) provides a time-frequency representation by computing the DFT on a series of short, overlapping segments of a signal.
• The spectrogram, a plot of the magnitude of the STFT, is a powerful tool for visualizing how a signal's frequency content changes over time.
• The STFT is governed by a fundamental time-frequency resolution trade-off: a wide window gives good frequency resolution but poor time resolution, while a narrow window gives good time resolution but poor frequency resolution.
• The STFT can be equivalently interpreted as a bank of uniformly spaced bandpass filters, providing a deep connection to classic filter theory.

Preview of the Next Lesson:

The time-frequency trade-off we introduced today is a manifestation of a deep physical and mathematical constraint known as the Uncertainty Principle. In our next lesson, we will apply the uncertainty principle to explain the time-frequency resolution trade-off in the STFT more formally. We will quantify the relationship between a window's duration and its bandwidth and see how different window shapes (like Gaussian, Hamming, etc.) perform in this context.