Markov Babbler
Build a simple text generator from scratch. See how counting word pairs creates a model that can produce surprisingly coherent (and sometimes hilarious) text.
Markov Babbler
Select or paste text to train a simple bigram model, then generate text one word at a time. At each step, you can see the probability distribution over the next possible words.
Pedagogical Goals
- •Demystify text generation by starting with the simplest possible model
- •Show that statistical patterns in language can produce surprisingly coherent output
- •Build intuition for how "more context" improves predictions — the same principle behind LLMs
- •Give students a hands-on sense of what a "language model" actually is before introducing transformers
How It Works
The babbler counts how often each word follows another in a source text (bigram statistics). To generate text, it picks a starting word, then randomly samples the next word weighted by how often it followed in the training data. Students can switch source texts and see how the generated output reflects the style and vocabulary of the training data.
How It Was Built
Built as a client-side React component with no backend. The Markov chain runs entirely in the browser. Source texts are bundled with the page. The visualization shows the probability distribution over next words as a bar chart, making the sampling process visible.