Context Engineering
How LLMs actually read context, from tokenization and attention math up through retrieval, memory, agents, rigorous evaluation, and the full production tech stack — with C++ hands-on labs throughout, plus a deep Context Designing module for building it all for real.
Written byRohan Sharma
Foundations
4 lessons- 9 min read
What is context engineering, really?
Why the words you feed a language model matter more than the model itself, and what "context" actually means under the hood.
- 9 min read
How LLMs actually read text
Text isn't fed into a model as letters or words — it's converted into a strict sequence of numbers first. Here's exactly how, and why it matters for context engineering.
- 8 min read
The context window, explained properly
What actually limits how much text a model can handle at once, why the limit exists, and the arithmetic every context-engineered app runs before every single request.
- 7 min read
Context engineering vs. prompt engineering
Two terms people use interchangeably that actually describe very different jobs — one writes a good sentence, the other builds the whole pipeline around it.
Foundations Quiz
6 questions
Tokenization & Embeddings
3 lessons- 10 min read
Vectors and embeddings, from absolute zero
How a word becomes a list of numbers that captures its meaning, and why that list is the single most important data structure in modern AI.
- 9 min read
How embeddings are actually learned
The training trick behind word2vec and every modern embedding model — predicting context from words, and why that alone is enough to teach a model meaning.
- 9 min read
Measuring similarity: the actual math
Cosine similarity, dot product, and Euclidean distance — the three ways retrieval systems decide "how close" two pieces of meaning are, worked out by hand and in code.
Tokenization & Embeddings Quiz
6 questions
Attention & the Context Window
4 lessons- 10 min read
Self-attention, explained without the jargon
The core trick that lets a model figure out which words in a sentence matter to which other words — the mechanism that actually reads your context window.
- 10 min read
The full math of attention, worked by hand
Scaled dot-product attention end to end — every formula, why the scaling factor exists, and a complete numerical example you can trace by hand.
- 9 min read
Why context windows cost quadratically (and what the KV cache does about it)
The precise reason doubling your prompt length more than doubles the work, worked out from the attention matrix itself, plus the caching trick that makes generation bearable.
- 9 min read
How a model knows word order at all
Attention alone can't tell "dog bites man" from "man bites dog" — positional encoding is the patch that fixes this, and it's stranger and more elegant than you'd expect.
Attention & the Context Window Quiz
6 questions
Prompt & Context Construction
4 lessons- 8 min read
The anatomy of a well-engineered prompt
Breaking a prompt down into its actual structural parts, and why a well-organized prompt is a context engineering artifact, not just good writing.
- 8 min read
Few-shot prompting and in-context learning
How showing a model two or three examples inside the prompt can change its behavior more than paragraphs of instructions — and why this even works at all.
- 8 min read
Chain-of-thought and giving the model room to think
Why asking a model to "think step by step" measurably improves accuracy, what's mathematically happening when it does, and how reasoning traces are just another form of context.
- 8 min read
Lost in the middle: why context ordering matters
The real, measured research finding that models pay less attention to information buried in the middle of a long context, and what to actually do about it.
Prompt & Context Construction Quiz
6 questions
Retrieval-Augmented Context
5 lessons- 8 min read
What is RAG, and why does it exist?
The open-book exam idea behind retrieval-augmented generation, and why it beats both "just ask the model" and "retrain the model on your data" for most real applications.
- 8 min read
Chunking strategies for retrieval
Why you can't just embed a whole book as one vector, the actual trade-offs in splitting documents, and the strategies real systems use.
- 9 min read
Vector search at scale, ANN, and how HNSW actually works
Why comparing your query against every stored vector doesn't scale, and the clever shortcut approximate nearest neighbor search uses instead.
- 9 min read
Reranking and hybrid search
Why the first retrieval pass often isn't good enough on its own, how BM25 keyword search complements embeddings, and the math behind combining both with reciprocal rank fusion.
- 11 min read
Hands-on: building a complete mini-RAG pipeline in C++
A capstone lesson wiring together chunking, embeddings, cosine similarity, and prompt construction into one working retrieval pipeline you can run end to end.
Retrieval-Augmented Context Quiz
6 questions
Memory & Long Context
4 lessons- 8 min read
Conversation memory patterns
How chatbots appear to "remember" a conversation when the model itself has no memory at all — the actual patterns systems use to fake continuity.
- 8 min read
The math of summarization-based memory
How much do you actually save by summarizing old context, when does it break down, and how to think about the compression ratio precisely.
- 9 min read
How context windows get stretched beyond their training length
RoPE scaling, ALiBi, and sliding/sparse attention — the actual architectural tricks that let models handle sequences far longer than anything they saw during training.
- 8 min read
Vector memory and long-term recall across sessions
Turning "remembering the user" into a retrieval problem — storing past interactions as searchable vectors and pulling back only what's relevant right now.
Memory & Long Context Quiz
6 questions
Advanced Context Engineering
7 lessons- 9 min read
Agentic context management and the ReAct pattern
How an AI agent that plans, calls tools, and observes results manages its own growing context — and why this is a materially harder problem than a simple chatbot.
- 9 min read
Tool calling and structured outputs
How a model actually "does things" instead of just talking — function/tool calling mechanics, JSON schemas, and why structured output is a context engineering concern, not just a formatting nicety.
- 8 min read
Human-in-the-loop and oversight
Why fully autonomous context-engineered systems are often the wrong design goal, and the concrete patterns for keeping a human involved at exactly the right moments.
- 8 min read
Multi-agent context sharing
What happens when several AI agents need to collaborate — the coordination patterns for sharing context between them without duplicating or losing information.
- 8 min read
Context compression techniques beyond summarization
Prompt compression, selective context pruning, and structured formats — practical ways to shrink what you send without losing what matters.
- 8 min read
Context window budgeting as a formal allocation problem
Turning "what fits in the context window" into an explicit, numerical allocation decision across competing needs, with a worked algorithm and code.
- 12 min read
Capstone: a complete context engineering pipeline in C++
Combining retrieval, layered memory, agent scratchpad management, and explicit budgeting from every module into one working end-to-end system.
Advanced Context Engineering Quiz
9 questions
Evaluation & Scoring
6 lessons- 8 min read
Why evaluation is part of context engineering
Every technique in this course so far was a guess about what improves quality — evaluation is how you find out whether that guess was actually right.
- 8 min read
Building a golden evaluation set
The single most valuable asset for evaluating a context-engineered system — where good test cases come from, how many you need, and how to keep the set honest over time.
- 9 min read
Retrieval metrics: precision, recall, MRR, and nDCG
The full family of metrics for scoring whether retrieval actually found the right documents — from simple precision@k up through rank-aware metrics like MRR and nDCG.
- 9 min read
LLM-as-judge and generation scoring
How to score a generated answer's quality when there's no single correct string to match against — faithfulness, relevance, and using a model to judge a model.
- 9 min read
Scoring strategies for rerankers and agent trajectories
Evaluation doesn't stop at plain retrieval and generation — scoring a reranker's actual lift, and judging whether a multi-step agent took a good path, need their own strategies.
- 8 min read
Continuous evaluation in production
Evaluation doesn't stop at launch — regression testing, online A/B testing, and monitoring for silent quality drift keep a context-engineered system honest long after it ships.
Evaluation & Scoring Quiz
8 questions
Context Designing
7 lessons- 8 min read
What is Context Design? Going from theory to production
Introducing Context Design — the systems-architecture discipline of actually building the pipeline this course has taught the theory behind, tech stack and all.
- 9 min read
Designing the context data pipeline
How raw, messy, constantly-changing source content becomes clean, chunked, retrievable knowledge — ingestion, cleaning, chunking at scale, and staying up to date.
- 9 min read
Choosing an embedding model for production
The real trade-offs between OpenAI, Cohere, and open-source embedding models — dimensionality, cost, latency, domain fit, and the MTEB benchmark.
- 9 min read
Choosing a vector database
FAISS, pgvector, Qdrant, Weaviate, Milvus, and Pinecone compared honestly — what each is actually good at, and how to pick for your specific scale and constraints.
- 9 min read
Caching and latency design for context-heavy systems
Prompt caching, prefix caching, embedding caching, and semantic caching — how production systems keep context-heavy LLM applications fast and affordable.
- 9 min read
Guardrails and observability
Defending a production context-engineered system against prompt injection and misuse, and instrumenting it so a bad answer is always debuggable.
- 11 min read
Capstone: designing a production context system end to end
The closing case study for the whole course — walking through every design decision for the internal knowledge assistant used throughout Module 9, tied back to every earlier module.
Context Designing Quiz
7 questions
Assessments
10 quizzesFoundations Quiz
Foundations · 6 questions
Tokenization & Embeddings Quiz
Tokenization & Embeddings · 6 questions
Attention & the Context Window Quiz
Attention & the Context Window · 6 questions
Prompt & Context Construction Quiz
Prompt & Context Construction · 6 questions
Retrieval-Augmented Context Quiz
Retrieval-Augmented Context · 6 questions
Memory & Long Context Quiz
Memory & Long Context · 6 questions
Advanced Context Engineering Quiz
Advanced Context Engineering · 9 questions
Evaluation & Scoring Quiz
Evaluation & Scoring · 8 questions
Context Designing Quiz
Context Designing · 7 questions
Context Engineering Final Assessment
Final assessment · 20 questions