5. ANN vs RNN: Why Sequence Changes the Architecture
Connect classical fixed-vector modeling to recurrent sequence processing.
01 Learning objectives
By the end of this module, you should be able to explain the concept in plain language, connect it to a use case, and recognize when it is — and is not — appropriate.
-
Objective 1
Explain feed-forward vs recurrent computation.
-
Objective 2
Understand hidden state and timestep.
-
Objective 3
Know when an RNN is more natural than an ANN.
02 Feed-forward ANN
A typical ANN receives the full vector at once:
x → hidden layers → output
For text, that x could be a TF-IDF vector. The model sees a fixed-length representation rather than explicitly walking token-by-token through the sentence.
03 RNN
An RNN receives a sequence:
x₁ → x₂ → x₃ → ... → xₜ
At each step it combines the current token representation with the previous hidden state:
hₜ = f(Wₓ xₜ + Wₕ hₜ₋₁ + b)
04 Use-case suitability
Use ANN/fixed-vector approaches when:
- a strong bag-of-words signal is enough,
- interpretability and simplicity are valuable,
- data is limited,
- latency/cost must be low.
Use sequence-aware neural models when:
- order strongly affects meaning,
- context spans multiple tokens,
- generation or sequence outputs are required.
05 Pitfalls and nuances
These are the mistakes most likely to create a wrong mental model or a weak implementation.
-
Pitfall 1
Saying ANN 'cannot' process text; it can process a fixed numerical representation of text.
-
Pitfall 2
Saying RNN automatically understands unlimited context.
-
Pitfall 3
Ignoring classical baselines.
06 Checkpoint questions
Try answering these without looking back at the module. If the explanation is clear in your own words, you are ready to continue.
-
Check 1
What extra input does an RNN cell receive beyond the current token?
-
Check 2
Why might complaint classification still work extremely well with TF-IDF?