8. Bidirectional RNNs and Encoder–Decoder Models
Bridge recurrent models to translation and later attention.
01 Learning objectives
By the end of this module, you should be able to explain the concept in plain language, connect it to a use case, and recognize when it is — and is not — appropriate.
-
Objective 1
Understand bidirectional context.
-
Objective 2
Understand encoder-decoder sequence-to-sequence modeling.
-
Objective 3
Recognize the fixed-context bottleneck.
02 Bidirectional RNN
For tasks where the entire input sentence is available, a bidirectional RNN reads left→right and right→left so each token representation can incorporate both preceding and following context.
03 Encoder–decoder
The encoder processes the source sequence into a representation. The decoder generates the target sequence one token at a time.
Classic translation:
English sequence → Encoder → context → Decoder → French sequence
04 The bottleneck
Compressing an entire long sentence into a single fixed context vector makes translation harder as sequences grow. This limitation motivates attention.
05 Pitfalls and nuances
These are the mistakes most likely to create a wrong mental model or a weak implementation.
-
Pitfall 1
Using bidirectionality for strictly causal prediction where future tokens are unavailable.
-
Pitfall 2
Confusing encoder-decoder architecture with Transformer specifically.
06 Checkpoint questions
Try answering these without looking back at the module. If the explanation is clear in your own words, you are ready to continue.
-
Check 1
Why is bidirectional context useful for NER?
-
Check 2
What is the fixed-vector bottleneck?