N-gram Word Predictor
Train a real n-gram language model on nursery rhymes, news, or your own text — then watch it predict the next word, just like autocomplete on your phone!
How Language Models Predict Words
The Corpus
A corpus is a collection of text the model learns from. Nursery rhymes give childlike language; news gives formal language.
N-Grams
An n-gram is a sequence of N words. Bigrams = 2 words, trigrams = 3. "the cat sat" has the bigram "the cat" and "cat sat".
Probability
After seeing "the cat", what comes next? Count how often each word follows "the cat" in the corpus to get probabilities.
Prediction
Given the last N-1 words, the model picks the next word by its probability. High prob = common continuation.
Step 1 — Choose Your Training Text
Step 2 — Explore the N-Gram Table
| Context (prefix) | Next Word | Count | Probability |
|---|
📊 Model Stats
Step 3 — Word Prediction Engine
Add words to see how the model uses context to make predictions.
🔢 Prediction Stats
Style Comparison — Same Seed, Different Corpus
🌸 Nursery Rhymes model
📰 News Headlines model
🧚 Fairy Tales model
🔬 Science Facts model
Language Model Badge!
You trained an n-gram model and explored how corpus choice shapes language style!
Optional. Stays on this device only — not sent to WhizzStep.
Key Concepts Mastered
📚 Training Text
The text a language model learns from. The corpus completely determines what the model knows and how it "talks".
🔢 Word Sequences
A sequence of N consecutive words. Bigram = 2 words, trigram = 3. Used to capture context for prediction.
📊 How Likely?
Given context words, what's the probability of each next word? Counted from how often it appears in the corpus.
😵 How Confused?
A measure of how "surprised" a language model is by new text. Lower perplexity = better model!
🪟 Memory Size
How many previous words the model looks at. N=2 uses 1 word context; GPT-4 uses thousands of tokens!
🔄 Handling Unknowns
What if a word combination never appeared in training? Smoothing gives a small probability to unseen n-grams.
About this lab
Learning objective: Explore next-word prediction using N-gram frequency statistics from a small sample text.
What this simplifies: This uses simple frequency counting, not embeddings, semantic understanding or genuine comprehension.
Privacy: No learner input leaves the device.
Teacher prompt: Ask the class why this simulation might mislead someone who takes it too literally.
Reflect: What is one thing this activity showed you that you did not expect?