What is context engineering, really?

The world's smartest goldfish

Imagine a friend who has read every book ever written, can solve almost any puzzle you give them, and speaks every language on earth. Now imagine that same friend has zero memory of anything you said more than a few minutes ago, and only knows what's happening right now because you write it down on a small whiteboard and hand it to them each time you ask a question.

That whiteboard is the entire world as far as your friend is concerned. If the fact isn't written on it, it doesn't exist for them. If the whiteboard is messy, cluttered, or missing the one detail that mattered, your brilliant friend will confidently give you a wrong answer.

That whiteboard is context, and deciding what goes on it, in what order, and in what form, is context engineering. A large language model (LLM) like the ones behind ChatGPT, Claude, or Gemini is that brilliant, memory-less friend. It has no memory between separate conversations, no eyes to see your screen, and no idea what you're working on unless you put it on the whiteboard.

Context is everything the model can "see"

When you send a message to an LLM, it doesn't just see your latest question. It sees a stitched-together bundle of text called the context, which can include:

• The system instructions (the model's job description — "you are a helpful coding assistant")
• The conversation history so far
• Any documents, code, or search results you've pasted or retrieved for it
• Your actual question

        ┌───────────────────────────────────────────┐
        │              CONTEXT WINDOW               │
        │                                           │
        │  [system prompt: "You are a tutor..."]    │
        │  [retrieved doc: "Photosynthesis is..."]  │
        │  [chat history: "Q: ... A: ..."]          │
        │  [your new question: "So why is..."]      │
        └───────────────────────────────────────────┘
                            │
                            ▼
                      ┌───────────┐
                      │    LLM    │──▶ generates one token at a time
                      └───────────┘

All of that gets turned into numbers (we'll get to exactly how in the next lessons) and fed into the model in one shot. The model has no separate channel for "important background info" versus "random chat" — it's all just text in a sequence, and the model has to figure out what matters from position and wording alone.

Why not just make the model smarter?

A very reasonable question: if the model is confusing itself, why not just train a better model? Two reasons this doesn't fully solve the problem:

1. The model still can't read your mind. No matter how smart it is, it cannot know your company's internal pricing policy, the contents of a PDF you haven't shown it, or what happened in a meeting yesterday, unless that information is placed into its context.
2. Bigger models don't fix badly organized information. Even a perfect student does worse on an exam if the question paper is scrambled, has irrelevant sections mixed in, and buries the actual question at the bottom of page 40. This is a real, measured effect for LLMs too, and we'll look at the exact experiment (called "lost in the middle") in Module 4.

Context engineering is the discipline of treating "what goes into the model's input" as seriously as engineers treat what goes into a database query or a function call — because to the model, that input is the entire universe.

Context engineering vs. just talking to a chatbot

Casually chatting with a chatbot in a browser tab is context engineering happening automatically and invisibly: the chat app collects your messages, keeps a running history, and sends it all back on every turn. You don't see any of the machinery.

Context engineering as an engineering discipline is what you do when you're building the system that decides:

• What background documents get pulled in (and how)
• How much of the old conversation to keep vs. summarize vs. drop
• What order pieces of information appear in
• How to fit everything inside a hard size limit
• What format (plain text, JSON, markdown, code blocks) makes the model most reliable

This course builds all of that up from first principles — actual math, actual C++ code — so none of it feels like magic by the end.

The one number that runs this whole course: the context window

Every model has a maximum number of "chunks of text" (called tokens — covered in depth in Module 2) it can accept as input at once. This is the context window. If a model's context window is 128,000 tokens, that's the entire whiteboard — system prompt, retrieved documents, chat history, and your question all have to fit inside that one number.

$\text{tokens available for new info} = \text{context window} - \text{system prompt} - \text{chat history} - \text{reserved output space}$

If you've ever seen an AI app say "conversation too long, please start a new chat," this equation is exactly why: the left side hit zero.

A tiny hands-on: counting your own "context budget"

Here's a deliberately simple C++ program that mimics the arithmetic every context-engineered system runs before calling a model. We're not tokenizing yet (that's Module 2) — we're just treating each word as roughly one token, which is a rough but useful approximation for getting the shape of the problem right.

#include <iostream>
#include <string>
#include <sstream>

// Very rough token estimate: count whitespace-separated words.
// Real tokenizers (Module 2) split differently, but this gets the idea across.
int estimateTokens(const std::string& text) {
    std::istringstream stream(text);
    std::string word;
    int count = 0;
    while (stream >> word) {
        count++;
    }
    return count;
}

int main() {
    const int CONTEXT_WINDOW = 8192;      // total tokens the model accepts
    const int RESERVED_FOR_OUTPUT = 1024; // leave room for the model's answer

    std::string systemPrompt = "You are a helpful assistant that answers questions about physics only.";
    std::string chatHistory  = "User: What is gravity? Assistant: Gravity is a force that attracts two bodies with mass toward each other.";
    std::string retrievedDoc = "Newton's law of universal gravitation states that every point mass attracts every other point mass.";

    int used = estimateTokens(systemPrompt) + estimateTokens(chatHistory) + estimateTokens(retrievedDoc);
    int budget = CONTEXT_WINDOW - RESERVED_FOR_OUTPUT - used;

    std::cout << "Tokens used so far: " << used << "\n";
    std::cout << "Tokens left for new user input: " << budget << "\n";

    if (budget <= 0) {
        std::cout << "No room left! Something has to be trimmed or summarized.\n";
    }

    return 0;
}

Common mistakes

1. Assuming the model "remembers" past conversations. Unless a system explicitly re-sends history (or stores and retrieves it), every new conversation starts from zero.
2. Treating context as unlimited. Every model has a hard ceiling. Applications that ignore this eventually crash, truncate silently, or produce garbled answers.
3. Confusing "more context" with "better context." Dumping in every document you have is often worse than picking the three most relevant ones — we'll see the math behind this in the "lost in the middle" lesson.

A few more concepts worth knowing

• Context length vs. context window: often used interchangeably, but "context length" sometimes refers to a specific request's actual token count, while "context window" refers to the model's maximum capacity — the next lesson but one pins this down precisely.
• Multimodal context: modern models increasingly accept images, audio, or video as part of context alongside text, each converted to tokens through its own encoding scheme — the same budget and attention principles from this course still apply, just with a different "alphabet."
• Grounding: a general term for tying a model's output to verifiable external information (documents, tool results, retrieved facts) rather than relying purely on what it memorized during training — most of this course is, in some sense, about grounding well.
• Context rot: an informal term some practitioners use for the gradual quality degradation that can occur as irrelevant, stale, or poorly organized content accumulates in a long-running context — a preview of problems Modules 4 and 6 tackle directly.

Summary

An LLM only knows what's inside its context window at the moment you ask it something — no memory, no eyes, no assumptions. Context engineering is the practice of deliberately deciding what goes into that window, in what form, and in what order, subject to a hard token budget. The next lesson goes one level deeper: how the model actually reads that text at all, turning words into the numbers it can compute with.