The ABCs of Attention: Query, Key, and Value

From Static to Contextual Representations

Let's briefly revisit the problem we are trying to solve. The input vector for the word "bank" is the same whether it appears in "a river bank" or "a savings bank." The model needs a mechanism to update this initial representation to reflect the surrounding context. The attention mechanism achieves this by allowing each token to 'look at' other tokens in the sequence and selectively draw information from them.

To set the stage and build intuition for why this dynamic, context-building process is so crucial, the following video provides some excellent examples.

As the video explains, the goal is to update each token's embedding by incorporating information from its neighbors. To do this systematically, the model needs a formal process for tokens to signal their relevance to each other and exchange information.

An Analogy: The "Fuzzy" Key-Value Lookup

Given your computer science background, a useful analogy for the attention mechanism is a key-value store, like a dictionary in Python or a hash map. In a standard dictionary, you use a query (a key) to find an exact match and retrieve its associated value.

# Standard dictionary lookup
my_dictionary = {
    "apple": "a fruit",
    "banana": "another fruit"
}
query = "apple"
value = my_dictionary[query] # Returns "a fruit"

Attention works like a "soft" or "fuzzy" version of this. Instead of an exact match, the query looks for relevance. And instead of retrieving a single value, the final output is a weighted sum of all values, where the weights are determined by how relevant each key is to the query.

Imagine querying our dictionary with "fruit". Neither "apple" nor "banana" is an exact match, but both are highly relevant. A soft-lookup mechanism might decide the query is 60% similar to "apple" and 40% to "banana", and then combine their values accordingly.

The following resource develops this intuition beautifully, starting from a basic dictionary and evolving it towards the logic of attention.

This analogy gives us our three core components: a query that asks a question, keys that are compared against the query, and values that hold the information to be retrieved.

Defining Query, Key, and Value Vectors

Now, let's formalize these roles in the context of a Transformer. For every input token in a sequence, we have its corresponding input vector $x_i$ (the sum of its token embedding and positional encoding). To participate in the attention mechanism, this single vector $x_i$ is projected into three separate vectors, representing the three distinct roles the token will play.

This diagram illustrates the core concept: a single input token embedding (X) is transformed into three distinct vectors—Query, Key, and Value—by multiplying it with three different weight matrices.

1. Query (Q): The query vector, $q_i$, represents the current token in its role as an information seeker. It is a representation of the token that asks the question: "Given my identity and position, which other tokens in this sequence are most relevant to me for building a better understanding of myself?"

2. Key (K): The key vector, $k_i$, represents the token in its role as an information provider's label. It is a representation of the token that can be "seen" by other queries. It essentially says: "This is the information I contain. Compare your query to me to see how relevant I am."

3. Value (V): The value vector, $v_i$, represents the token's actual content or payload. It is the information that the token will contribute if it is attended to. It answers the question: "Once you've determined I'm relevant, this is the rich information I will pass on to you."

These three vectors are generated from the original input vector $x_i$ via three separate, learnable linear transformations. This is accomplished by multiplying $x_i$ by three weight matrices: $W_Q$, $W_K$, and $W_V$.

These weight matrices are parameters that the model learns during training. Their job is to learn the optimal projections to transform the initial input vectors into representations that are best suited for their respective roles in the attention calculation. The following resource provides a formal definition.

The Rationale for Separate Projections

You might wonder: why do we need three separate matrices? Why not just use the input vector $x_i$ for all three roles?

The separation allows the model to create different "subspaces" optimized for different tasks. The Serrano.Academy video explains this distinction very clearly. The space created by the Query and Key matrices is optimized for calculating similarity. It learns to project vectors such that their dot product becomes a meaningful measure of relevance. In contrast, the space created by the Value matrix is optimized for aggregating information. It learns to project vectors into a representation that is most useful for updating other token embeddings and, ultimately, for predicting the next token.

Let's watch a couple of segments that clarify how these matrices are generated and what their distinct purposes are.

By learning these separate projection matrices, the model gains the flexibility to disentangle the task of "determining relevance" from the task of "providing useful information."

Conclusion

In this lesson, we have defined the three fundamental roles that every token plays within the self-attention mechanism. These roles are not inherent to the tokens themselves but are dynamically created through learned linear projections.

Here are the key takeaways:

• Analogy: The attention mechanism can be understood as a "soft" key-value lookup system, where a query finds relevant keys to retrieve a weighted combination of values.
• Query (Q): The "seeker." A representation of a token asking which other tokens are relevant to it.
• Key (K): The "identifier." A representation of a token that can be compared against by other queries to establish relevance.
• Value (V): The "content." A representation of a token that contains the information to be passed to other tokens.
• Projections: The Q, K, and V vectors for each token are generated by multiplying the token's input vector ($x_i$) with three distinct, learned weight matrices ($W_Q,W_K,W_V$). This separation allows the model to learn specialized representations for calculating similarity (Q/K) and for providing information (V).

We have now established the cast of characters. In our next lesson, we will see how they interact. We will take the query and key vectors and use them to compute the first major component of the attention formula: the attention scores, which quantify the relevance between every pair of tokens in the sequence.