6 lessons
What is a token? Chopping code into words
Why a compiler's first move is treating your code like a sentence to be broken into words, and the difference between a lexeme, a token, and a pattern.
Regular expressions and formal languages, from scratch
What it actually means for a string to "belong to a language," building regular expressions from three simple operations, and why they're exactly what a lexer needs.
Finite automata, from NFA to DFA
The state-machine "board game" that makes regular expression matching actually fast, and the mechanical algorithm that turns a flexible NFA into a fast, deterministic DFA.
DFA minimization, and how a lexer reads input fast
The exam-favorite numerical: shrinking a DFA to the fewest possible states using the table-filling algorithm, worked step by step, plus how real lexers buffer input efficiently.
Building a real lexer in C++, character by character
A complete, working hand-written lexer in C++ that turns real source code into a token stream, built directly on the DFA ideas from the previous lesson.
Lexer edge cases and error recovery
What a lexer should do when it meets characters that don't fit any pattern, and why crashing on the first bad character is the wrong answer.