Quantization Trade-offs: Accuracy vs. Performance

Recommended resources

The freely available resources this lesson will direct you to.

Welcome back! In our last lesson, you applied 4-bit AWQ quantization to a model and measured a significant performance boost against the FP16 baseline. This demonstrated that quantization works. Now, we'll address a more nuanced and practical question for any LLM Ops engineer: which quantization method should you choose?

Today, we will compare two of the most prominent post-training quantization (PTQ) techniques: Activation-Aware Weight Quantization (AWQ) and Generative Pre-trained Transformer Quantization (GPTQ). By understanding their underlying strategies and comparing their real-world performance and accuracy, you'll be equipped to make informed decisions for your own serving infrastructure.

1. Conceptual Differences: Error Correction vs. Salient Weight Protection

At a high level, both AWQ and GPTQ aim to compress a model to 4-bits with minimal accuracy loss, but they take different philosophical approaches to get there.

To start, let's get a brief overview of the main quantization methods you'll encounter in the wild.

Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)

The video "Which Quantization Method is Right for You?" provides a clear, code-first introduction to several quantization formats.

Focus on these two sections: GPTQ: Note its focus on GPU inference and how we load a pre-quantized model. AWQ: Pay attention to the core idea of focusing on "weights that are more important than others." (You can skip the section on GGUF for now, as it's primarily optimized for CPU inference, whereas our focus is on high-performance GPU serving).

Now, let's dive deeper into the mechanics of how these methods work. While a full mathematical treatment is beyond our scope, understanding the core concepts is crucial for an infrastructure engineer.

Double Inference Speed with AWQ Quantization

This video, "Double Inference Speed with AWQ Quantization," offers one of the clearest conceptual explanations of how these algorithms differ.

Watch the segment from the lecture portion. Don't worry about catching all the math. Instead, focus on grasping the core ideas for each method: Round-to-Nearest (RTN): The simple, "naive" baseline method. AWQ: How it searches for an optimal scaling factor to minimize the error introduced by quantization, based on the input data (activations). GPTQ: How it quantizes weights one by one and iteratively corrects the remaining unquantized weights to compensate for the error just introduced. Data Dependence: A key insight is that both AWQ and GPTQ are data-dependent. They use a small calibration dataset to make better quantization decisions. This is why you'll see pre-quantized models on Hugging Face that were quantized using a specific dataset (like wikitext).

To solidify these concepts, the following article provides a superb written summary of AWQ and GPTQ.

Accelerating LLM inference with post-training weight and activation ...

This AWS blog post gives a concise, professional summary of the two main PTQ algorithms.

Read the section titled "Post-training quantization algorithms", which contains subsections for "Activation aware weights quantization" and "Generative pre-trained transformers quantization (GPTQ)". This will reinforce the concepts from the video.

Here's a summary of the key differences:

Method	Core Strategy	Analogy
GPTQ	Error Compensation: Quantizes weights sequentially, and after each step, it adjusts the remaining full-precision weights to "undo" the error just introduced.	Imagine meticulously balancing a complex mobile sculpture. As you hang each new piece (quantize a weight), you slightly adjust all the others to keep the whole structure perfectly balanced.
AWQ	Salient Weight Protection: Identifies the small fraction of weights (~1%) that have the biggest impact on the output (because they are multiplied by large activation values) and protects them by using a special scaling factor.	Think of it like audio compression. Instead of treating all frequencies equally, you preserve the ones most audible to the human ear and compress the rest more aggressively. AWQ does this for weights.

2. The Accuracy vs. Performance Trade-off

Now that you understand the "how," let's look at the results. As an LLM Ops engineer, you'll constantly navigate the trade-off between model accuracy and serving performance.

Let's look at two charts that visualize this trade-off.

This chart shows the accuracy of different quantization methods on the GSM8K benchmark, a test of grade-school math reasoning. A higher score is better. Notice that AWQ scores higher than GPTQ-Int4, indicating better accuracy preservation on this task. Both are lower than the unquantized BF16 baseline.

This chart compares the performance (Tokens per Second and Latency) of different methods. A higher TPS (blue bar) and lower latency (red line) are better. Here, GPTQ shows a slight performance edge over AWQ.

These charts illustrate the classic dilemma: one method might be more accurate, while another might be faster. Your job is to pick the right balance for your application.

3. Hands-On: Benchmarking GPTQ

In the previous lesson, you benchmarked the FP16 and INT4-AWQ versions of Llama-2-7B-Chat. Now, let's add GPTQ to the comparison. You'll run the vLLM server with a pre-quantized GPTQ model and use the same benchmark script to gather comparable performance data.

Step 1: Start the vLLM Server with a GPTQ Model
We'll use another popular pre-quantized model from the Hugging Face Hub. This command is very similar to the one you used for AWQ.




# Ensure you are in an environment with vllm installed
# vLLM automatically downloads the model
python -m vllm.entrypoints.openai.api_server \
    --model TheBloke/Llama-2-7B-Chat-GPTQ \
    --quantization gptq

Step 2: Re-run Key Benchmarks
To save time, let's run just two of the four workloads from our benchmark suite. These two are representative of common use cases.

Test 1: Short Input, Long Output (Generation)
This workload is sensitive to the decode phase performance, which is memory-bound.

python benchmarks/benchmark_serving.py \
    --backend openai-chat \
    --endpoint "/v1/chat/completions" \
    --model TheBloke/Llama-2-7B-Chat-GPTQ \
    --tokenizer TheBloke/Llama-2-7B-Chat-GPTQ \
    --dataset-name random \
    --random-input-len 256 \
    --random-output-len 1024 \
    --num-prompts 200 \
    --max-concurrency 50 \
    --save-result

Test 2: Long Input, Short Output (RAG/Classification)
This workload is more sensitive to the prefill phase performance, which is compute-bound.

python benchmarks/benchmark_serving.py \
    --backend openai-chat \
    --endpoint "/v1/chat/completions" \
    --model TheBloke/Llama-2-7B-Chat-GPTQ \
    --tokenizer TheBloke/Llama-2-7B-Chat-GPTQ \
    --dataset-name random \
    --random-input-len 2048 \
    --random-output-len 128 \
    --num-prompts 200 \
    --max-concurrency 50 \
    --save-result

4. Analysis: Synthesizing the Results

You now have performance data for three model versions: FP16, INT4-AWQ, and INT4-GPTQ. Compile your results for the two workloads you just tested into a table.

Your results should look something like this:

Model	Workload	Input Len	Output Len	Mean TTFT (ms)	Mean TPOT (ms)	Output TPS (tok/s)
FP16	Short/Long (Generate)	256	1024	From Les. 1	From Les. 1	From Les. 1
INT4-AWQ	Short/Long (Generate)	256	1024	From Les. 2	From Les. 2	From Les. 2
INT4-GPTQ	Short/Long (Generate)	256	1024	New Result	New Result	New Result
FP16	Long/Short (RAG)	2048	128	From Les. 1	From Les. 1	From Les. 1
INT4-AWQ	Long/Short (RAG)	2048	128	From Les. 2	From Les. 2	From Les. 2
INT4-GPTQ	Long/Short (RAG)	2048	128	New Result	New Result	New Result

Now, let's analyze this data alongside professionally run benchmarks that include accuracy metrics.

Best Quantization Settings for vLLM Models Guide - Ventus Servers

This practical guide contains excellent summary tables comparing different quantization settings.

Focus on two tables: "GPTQ vs AWQ for vLLM Performance": This gives a quick throughput and memory comparison. "Benchmarks and Pros-Cons of Top Settings": This is a crucial table. It compares AWQ and GPTQ on perplexity (a common accuracy metric where lower is better), tokens/second, and VRAM, and lists their pros and cons.

Based on your own benchmarks and the tables in the article, you should be able to draw some conclusions:

Performance (Throughput): In your own tests and the resource tables, you'll likely see that GPTQ often has a slight edge in throughput (tok/s). This is generally attributed to the availability of highly optimized custom kernels for GPTQ that are very efficient on certain GPU architectures.
Accuracy (Perplexity): The LINK resource shows that AWQ typically yields better accuracy (lower perplexity). This aligns with the GSM8K chart we saw earlier. AWQ's method of protecting salient weights is often more effective at preserving model quality.
Memory: Both methods achieve the same level of memory reduction for INT4, making the model roughly 1/4th the size of its FP16 counterpart.

This leads to the general rule of thumb for choosing between them:

Choose GPTQ when your primary goal is to maximize raw throughput and you can tolerate a slightly higher accuracy loss.
Choose AWQ when preserving model accuracy is paramount, and you still need a massive performance boost over the baseline.

Conclusion

Congratulations! You've moved beyond simply applying an optimization to critically evaluating and comparing different optimization strategies. This ability to analyze trade-offs is a hallmark of a senior infrastructure engineer.

Here are your key takeaways from this lesson:

AWQ and GPTQ use different philosophies: AWQ protects important weights identified via activations, while GPTQ sequentially corrects for quantization error.
A clear trade-off emerges: GPTQ often provides slightly higher throughput, whereas AWQ tends to preserve accuracy better.
The choice is context-dependent: The best method depends on your application's sensitivity to accuracy vs. its need for maximum request throughput. For a creative writing bot, accuracy might be key (favoring AWQ). For a high-volume summarization service, throughput might be the priority (favoring GPTQ).
Always Benchmark: As you've seen, performance can vary by workload (e.g., prefill-heavy vs. decode-heavy). The only way to know for sure is to measure, as you did today.

In the next module, we'll peel back another layer of the onion and dive into the hardware itself. You'll learn how to interpret a GPU specification sheet and understand how components like CUDA cores, Tensor Cores, and HBM directly influence the performance metrics you've been measuring. This will help you understand why these performance differences exist at a much deeper level.