9. Attention: Let the Model Look Back Selectively
Understand the breakthrough that removes the single-context bottleneck.
01 Learning objectives
By the end of this module, you should be able to explain the concept in plain language, connect it to a use case, and recognize when it is — and is not — appropriate.
-
Objective 1
Understand queries, keys and values conceptually.
-
Objective 2
Explain attention weights.
-
Objective 3
Understand self-attention.
02 Core idea
Instead of forcing the decoder to rely on one compressed summary, attention lets it compute a weighted combination of relevant source states.
The model learns: which earlier pieces matter most for the current computation?
03 Q, K, V intuition
- Query: what information am I looking for?
- Key: what kind of information does this token contain?
- Value: the information carried by this token.
Scaled dot-product attention:
Attention(Q,K,V) = softmax(QKᵀ / √dₖ)V
04 Self-attention
In self-attention, Q, K and V are derived from the same sequence. Every token can directly interact with other tokens rather than passing information through a long recurrent chain.
05 Pitfalls and nuances
These are the mistakes most likely to create a wrong mental model or a weak implementation.
-
Pitfall 1
Interpreting attention weights as perfect causal explanations.
-
Pitfall 2
Ignoring computational cost as sequence length grows.
06 Checkpoint questions
Try answering these without looking back at the module. If the explanation is clear in your own words, you are ready to continue.
-
Check 1
What does the softmax produce in attention?
-
Check 2
Why can self-attention connect distant words more directly than an RNN?