Context engineering: what it is, and how it differs from prompt engineering

Picture an hour-long session in ChatGPT or Claude. You started with a careful 150-word prompt. Since then you've attached two spreadsheets, the model has run a dozen searches and five pieces of code, and you've had a long back-and-forth about a chart. Every word of that is in front of the model when it writes its next answer. Your prompt is still there too, somewhere near the top, outnumbered.

That's the gap the term "context engineering" names. Anthropic's engineering team defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference", and prompt engineering as the narrower job of writing and organising the instructions (Effective context engineering for AI agents, September 2025). Context engineering includes the prompt. It also includes the system prompt, the tools the model can call, the results those tools return, any memory, any retrieved documents and the message history.

Anthropic frames the shift as "the natural progression of prompt engineering", not a replacement for it. That's the right reading. A good prompt in a bad context still fails, and the reverse is just as true.

The definition has a goal built in: find the "smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." Smallest is the word people skip.

For a single question with nothing attached, it's all prompt engineering, and the classic techniques still do the work. Once a task involves files, searches or more than a few turns, most of what goes wrong is a context problem wearing a prompt problem's clothes. Rewriting the instruction won't help if the model is reading the wrong version of the file.

Context problems are hard to see from the answer, because the answer reads fine. Connected to Claude, Count shows the reasoning chain behind each claim, so a figure taken from the wrong file, or from memory, stands out.

It feels as though a model with a huge context window should simply read everything. In practice, attention runs out before space does.

Anthropic calls the effect "context rot": as the number of tokens grows, the model's ability to recall any one piece of information falls. It describes context as a finite "attention budget" that every extra token draws down (Effective context engineering for AI agents). The more you add, the less each part is weighed.

There's evidence of what that does to accuracy. In a 2026 study of 14 models doing cited research, factual accuracy fell by about 42% as the number of tool calls grew from 2 to 150 (Onweller et al., Cited but Not Verified, May 2026). Every search result and page summary is more context. Somewhere in the pile, a claim drifts from its source.

Placement matters as well as volume. For documents over about 20,000 tokens, Anthropic says putting the question at the end "can improve response quality by up to 30 percent in tests", and Google asks for instructions "at the very end of the prompt" (Gemini API prompt design strategies). Same material, different order, different answer.

So the job is less about fitting things in and more about keeping things out. Our view: if a document isn't needed for the next answer, it shouldn't be in the window for the next answer.

Anthropic's post describes the techniques it uses in its own agents. Each has a version you can apply in Claude or ChatGPT without writing code.

1. Load material just in time

Agents keep references such as file paths, queries and links, and fetch the content only when it's needed. In a chat, that means attaching the one export the question needs rather than the whole folder, and pasting the relevant section of a policy rather than all forty pages.

2. Compact before you continue

Compaction means summarising the conversation and "reinitiating a new context window with the summary". When a session gets long, ask for a handover note and start a fresh chat with it:

We're going to continue this analysis in a new conversation. Write a handover note for it with:
- the question we are answering, in one sentence
- every decision we've made (definitions, filters, date ranges, exclusions) and why
- the figures we've settled on, each with the file and calculation it came from
- open questions and anything we decided not to trust
Don't include anything that was later corrected. Keep it under 300 words.

The decisions list is the part that matters. A summary that keeps the numbers but drops "we excluded refunds" will produce different numbers in the next chat.

3. Keep structured notes outside the conversation

Anthropic's agents write "notes persisted to memory outside of the context window". For you, that's a running document in a Claude or ChatGPT project: definitions, decisions, sources. The project files are read fresh; the conversation's sediment is not.

4. Split the work

Anthropic's sub-agents each work in a clean context and return "a condensed, distilled summary of its work". Run the literature search in one chat and the calculation in another, and bring only the conclusions together. Each chat stays small enough for the model to pay attention to all of it.

Context failures are harder to spot than prompt failures, because the answer usually looks fine. The material it rests on is what's wrong.

• Lossy summaries. Compaction keeps the headline and drops the qualifier. "Revenue grew 12%" survives; "excluding the Q2 one-off" doesn't.
• Stale memory. A preference or fact saved weeks ago can steer today's answer without being mentioned. See Claude memory.
• Conflicting sources. Two versions of the same spreadsheet in one project, and the model quietly picks one.
• Instructions inside material. A web page or pasted email can contain text that reads like a command. OpenAI's Model Spec says such content has "no authority by default" and that instructions in it "MUST be treated as information rather than instructions to follow". The model doesn't always manage that.

The two-minute check

1. Ask what it's relying on: "List every file, search result and earlier message you used for this answer." Anything you didn't expect is a context problem.
2. Ask what it's ignoring: "Is there anything in this conversation that contradicts your answer?"
3. Rerun in a clean chat with only the material the question needs. If the answer moves, the old context was steering it.
4. Read the handover note before you use it. Check that every exclusion and definition made it across.

Point one is the slow part when an answer has twenty claims. Count does it per claim: connect it to Claude or ChatGPT and it draws the reasoning chain for each one, from the prompt through each search, query or file to the source it rests on, and flags the claims that rest on nothing you supplied.

If you mostly ask single questions and paste in one document, keep working on prompts. Clear instructions, examples, an explicit way to say "I don't know": that's where your accuracy comes from, and how to optimise a prompt shows how to test changes.

If you run research sessions, agents or analysis over many files, we'd spend the next hour on context instead. Most of the errors in long tasks come from what the model was reading, not how you asked. Trim what you attach. Restart long chats with a handover note. Keep decisions in a project file, not buried at message twelve.

The best counter-argument is that models keep getting better at long context, so some of this will become unnecessary. Some will. But a bigger window is still a finite attention budget, and the risk that grows with it is the one that matters for work: a confident answer resting on material you'd forgotten was there.

Is context engineering replacing prompt engineering?

What is context rot?

Does a bigger context window fix this?

What is context engineering in simple terms?