Context Engineering for Production AI, What It Is and How to Do It Well
Vasim Gujrati
Solutions Architect, AI & Platforms, Unico Connect
In this article
- Quick Answer
- Key Takeaways
- What Is Context Engineering?
- Why Did Thoughtworks Move Context Engineering to Adopt?
- Context Engineering Best Practices, the Four Techniques Thoughtworks Says Have Matured
- What Is Progressive Context Disclosure?
- A Context Pipeline for a Production Agent
- How to Measure Whether Your Context Is Working
- Common Context Engineering Mistakes
- Context Engineering, RAG, and Fine Tuning
- How Unico Connect Helps
- Frequently Asked Questions
- Sources
In November 2025 the Thoughtworks Technology Radar placed context engineering in its Assess ring, meaning worth exploring. Five months later, in Volume 34 in April 2026, it moved straight to Adopt, the ring Thoughtworks reserves for practices it believes the industry should be using now. Moving from Assess to Adopt without a stop in Trial is a strong signal, and it reflects what teams shipping AI agents have learned the hard way. The model is rarely the weak point. What the model sees at each step usually is. This guide explains what context engineering is, the four techniques behind it, what it costs and saves, and how to tell whether your context is working. Sources were checked on 22 September 2026.
Quick Answer
Context engineering is the practice of designing everything a model sees at inference time, including instructions, retrieved data, tool definitions, memory, and prior work, so it produces reliable output. Prompt engineering tunes the wording of one prompt, while context engineering assembles the whole context window dynamically at each step. Thoughtworks moved it to Adopt in April 2026 and names four maturing techniques, prompt caching, dynamic retrieval of tools and data, context graphs, and context management with compression and subagents.
Key Takeaways
- Bigger context windows do not remove the problem. Chroma tested 18 models and found performance grows less reliable as input length increases, even on simple tasks.
- Thoughtworks calls the result context rot, and recommends progressive context disclosure, where an agent starts with a lightweight index and pulls in detail only when it becomes relevant.
- Prompt caching turns context design into a cost lever. Cached input costs a tenth of the normal input price on GPT-6 Astra and most Claude models, and a fortieth on Claude Fable 5.1 and Claude Mythos 5.1.
- Loading every tool and MCP server up front wastes context. Dynamic tool selection and tool search keep the working set small.
- Context quality can be measured. Track task success as context grows, cost per completed task, cache hit rate, and how often the agent reasons from stale or wrong context.
What Is Context Engineering?
Thoughtworks defines context engineering as the systematic design and optimization of the information provided to a large language model during inference to reliably produce the desired output. Anthropic, in an engineering guide published in September 2025, puts the goal as finding the smallest possible set of high signal tokens that maximize the likelihood of the outcome you want.
Two posts from June 2025 are often credited with popularizing the term. Tobi Lütke wrote that he preferred context engineering to prompt engineering because it describes the art of providing all the context for the task to be plausibly solvable by the LLM. Andrej Karpathy agreed on 25 June 2025, calling it the delicate art and science of filling the context window with just the right information for the next step.
| Aspect | Prompt engineering | Context engineering |
|---|---|---|
| Unit of work | The wording of one prompt | Everything in the context window at each step |
| When it happens | Written once, then tweaked | Assembled dynamically as the task runs |
| Main inputs | Instructions and examples | Instructions, retrieved data, tool definitions, memory, prior outputs, environment signals |
| Typical failure | An ambiguous or badly worded instruction | Too much, too little, stale, or conflicting information |
| Where it matters most | Single turn tasks | Agents, long running workflows, and retrieval heavy products |
Why Did Thoughtworks Move Context Engineering to Adopt?
Because agents made the old approach break. Thoughtworks writes that as agents tackle more complex tasks, dumping raw data into large context windows leads to context rot and degraded reasoning, and that treating context as a static text box is a fast track to hallucinations.
The research behind that phrase is concrete. In July 2025, Chroma published a technical report evaluating 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, while holding task difficulty constant and changing only input length. Performance did not stay flat as inputs grew. It became increasingly unreliable, and models even performed better on shuffled text than on logically structured text in its needle tests, a sign that structure in long inputs changes how attention behaves. Anthropic describes the same limit as an attention budget, where every new token depletes the model focus a little.
The practical conclusion is that a one million token window is capacity, not a strategy. What goes in still decides the quality of what comes out.
Context Engineering Best Practices, the Four Techniques Thoughtworks Says Have Matured
1. Context setup with prompt caching
Put stable content, such as system instructions, tool definitions, and reference material, at the start of the prompt in a fixed order, and cache it. Thoughtworks notes this reduces cost and improves time to first token. The savings are large enough to shape architecture.
Prices are in dollars per million tokens.
| Model | Normal input | Cached input | Cache write |
|---|---|---|---|
| OpenAI GPT-6 Astra, prompts up to 272K tokens | $10.00 | $1.00 | $12.50 |
| OpenAI GPT-6 Astra, prompts over 272K tokens | $20.00 | $2.00 | $25.00 |
| Claude Fable 5.1 | $10.00 | $0.25 | $12.50 for 5 min, $20.00 for 1 hour |
| Claude Opus 5 | $5.00 | $0.50 | $6.25 for 5 min, $10.00 for 1 hour |
Context size also has a price step of its own. OpenAI charges 2x the input and cache rates and 1.5x the output rate for the whole request once a GPT-6 Astra prompt passes 272K input tokens. Claude Fable 5.1 and Claude Mythos 5.1 charge 0.025x the base input price for cache hits, against 0.1x on every other Claude model. Our breakdown of GPT-6 Astra pricing and API changes covers the rest of the rate card.
The caching design rule follows directly. Anything that changes per request, such as the user message, retrieved passages, or timestamps, belongs after the stable prefix, or it breaks the cache for everything behind it. OpenAI made Prompt Cache Diagnostics generally available on 8 September 2026, which shows why a request missed the cache.
2. Dynamic retrieval of data and tools
Classic retrieval augmented generation fetches documents before the model starts. Context engineering extends the idea to tools. Thoughtworks describes selecting tools and loading only the necessary MCP servers to avoid unnecessary context expansion, and OpenAI built tool search into its Agents API, in public beta since 10 September 2026, so tool definitions load only when relevant. Anthropic calls the broader pattern just in time retrieval, where an agent keeps lightweight references such as file paths, stored queries, or links, and loads the underlying data with tools when it needs it.
The cost of skipping this is measurable. Anthropic documents that a typical multiserver setup with GitHub, Slack, Sentry, Grafana, and Splunk can consume about 55,000 tokens in tool definitions before Claude does any work. Its tool search tool typically cuts that by over 85 percent by loading only the 3 to 5 tools a request needs, and Claude picks the right tool less reliably once more than 30 to 50 tools are available. Tool results need the same discipline, and our guide to running MCP in production AI agents shows how to trim oversized MCP responses before they reach the model.
3. Context graphs for institutional reasoning
This is the newest of the four, added in April 2026, and Thoughtworks also lists it as a separate technique in its Assess ring. A context graph models decisions, policies, exceptions, precedents, evidence, and outcomes as connected nodes, capturing why something happened rather than only what happened. Thoughtworks gives the example of an agent handling a discount exception that cannot tell whether it reflects standing policy or a one time override. A context graph surfaces that provenance, and unlike GraphRAG built from static documents, it keeps temporal validity on every edge so superseded facts are invalidated rather than overwritten.
4. Context management for long running work
Long tasks outgrow any window. The mature responses are compaction, where earlier context is summarized as the window fills, structured note taking that persists memory outside the window, and subagents that explore in their own context and return only a condensed result. Anthropic describes subagents that may use tens of thousands of tokens exploring but return summaries of around 1,000 to 2,000 tokens, and OpenAI compacts context automatically in its Agents API. Whether a task should be split across agents at all is covered in our comparison of single agents and multi agent systems. Long running work also needs to survive crashes and restarts, which our guide to what breaks when AI agents run in production covers.
What Is Progressive Context Disclosure?
Progressive context disclosure, in the Trial ring since April 2026, is the technique that ties the others together. Instead of front loading every instruction and reference an agent might need, you give it a lightweight discovery step. It sees an index of what is available, selects what is relevant to the request, and only then loads the detailed instructions or data.
Thoughtworks points out that this is how many agentic coding tools handle Agent Skills, by deciding which skills apply before loading their full instructions, and it warns against the common trap of bloating instructions with endless do and do not rules, which can degrade performance. Our notes on using Claude Code skills in real projects show the pattern in practice. A practical implementation has three parts.
- An index. Short descriptions of each skill, document set, or tool group, small enough to always include.
- A selection step. The agent, or a lightweight router, chooses what the current request needs.
- A loader. Only the chosen instructions, documents, or tools enter the context, and they leave again when the step is done.
A Context Pipeline for a Production Agent
A production agent usually ends up with the same layers, assembled in the same order on every call.
- Stable prefix. Role, rules, output format, and tool definitions that rarely change, in a fixed order so they cache.
- Task framing. The user request and the goal of the current step.
- Retrieved evidence. Only the passages, records, or query results relevant to this step, each with its source.
- Working memory. Notes and decisions from earlier steps, summarized rather than replayed.
- Tool results. Trimmed to the fields the next step needs, not the full payload.
- Guardrails. Checks that stop the step when evidence is missing, rather than letting the model guess.
The discipline is to decide, for every layer, what earns its place in the window at this step and what can be fetched later.
How to Measure Whether Your Context Is Working
Context engineering is measurable, and it should be treated like any other performance work.
| Metric | What it tells you |
|---|---|
| Task success rate against context length | Whether quality drops as inputs grow, the signature of context rot |
| Cost per completed task | The real cost, across every model call, retry, and subagent |
| Cache hit rate | Whether the stable prefix is actually stable |
| Retrieval precision | How much of what you load is used in the answer |
| Stale or conflicting context incidents | How often the agent reasons from superseded information |
| Tokens per step | Whether the working set stays lean as the task runs |
Build an evaluation set of real tasks, run it whenever prompts, tools, retrieval, or models change, and compare these numbers before and after. Our guide to AI observability for LLMs in production covers how to collect them.
Common Context Engineering Mistakes
- Dumping whole documents. Loading full files when a few passages would do increases cost and invites context rot.
- Rule sprawl. Long lists of do and do not instructions, added after every incident, that eventually contradict each other.
- Loading every tool. Registering every MCP server and function on every call, when most steps use two or three.
- Breaking the cache. Putting timestamps, user names, or retrieved text at the start of the prompt, ahead of stable content.
- Memory without provenance. Summaries that lose where a fact came from and when it was true.
- No evaluation. Changing prompts and retrieval without a fixed set of tasks to prove the change helped.
Context Engineering, RAG, and Fine Tuning
These are complementary rather than competing. Retrieval augmented generation is one context engineering technique, focused on fetching documents. Fine tuning changes the model itself and suits stable behavior, format, or style, while context engineering changes what the model sees and suits facts, tools, and state that change often. Most production systems use context engineering first and fine tuning only where it earns its cost. Our comparisons of RAG and fine tuning and of fine tuning and prompt engineering go deeper on that choice.
How Unico Connect Helps
We design and build production AI agents and retrieval systems, and context engineering is where most of that work goes. Our AI development services cover retrieval pipelines, prompt caching strategy, tool design, and evaluation, and our agentic AI development team builds long running agents with compaction, subagents, and approval steps. If you need engineers inside your own team, you can hire AI engineers who build these pipelines. If you are choosing an agent runtime, our guide to the OpenAI Agents API explains what the managed harness now handles for you, or you can talk to our team.
Frequently Asked Questions
What is context engineering in AI?
Context engineering is the design and optimization of everything a model sees at inference time, including instructions, retrieved data, tool definitions, memory, and prior outputs, so it reliably produces the output you want. It treats the context window as a design surface that is assembled dynamically at each step, rather than a single prompt written once.
How is context engineering different from prompt engineering?
Prompt engineering focuses on the wording of a prompt. Context engineering covers the whole configuration of context, including which documents, tools, memories, and prior results are loaded, in what order, and when they are removed. Prompt engineering is one part of it.
What is context rot?
Context rot is the decline in model reliability as the amount of input grows. Chroma measured it across 18 models in July 2025 and found performance became increasingly unreliable with longer inputs, even when the task stayed equally hard, and Thoughtworks uses the term to explain why dumping raw data into large context windows degrades reasoning.
Where is context engineering on the Thoughtworks Technology Radar?
In the Adopt ring, since Volume 34 in April 2026. It entered the Radar in the Assess ring in November 2025. Two related techniques sit alongside it, progressive context disclosure in Trial and context graphs in Assess.
How much can prompt caching save?
Cached input costs a tenth of the normal input price on GPT-6 Astra, 1 dollar instead of 10 per million tokens, and on most Claude models. On Claude Fable 5.1 and Claude Mythos 5.1, cache hits cost 25 cents instead of 10 dollars per million tokens, a fortieth of the base price. Writing to the cache costs more than normal input, so the savings depend on how often a stable prefix is reused.
Does a larger context window make context engineering unnecessary?
No. A larger window adds capacity, but research on context rot shows reliability still falls as inputs grow, and every extra token adds cost. Deciding what belongs in the window at each step still determines quality.
What is the difference between context engineering and harness engineering?
A harness is everything in an AI agent except the model itself. Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, writing on martinfowler.com on 2 April 2026, sums this up as Agent = Model + Harness, and she calls engineering a user harness for a coding agent a specific form of context engineering. In practice the two overlap. Harness engineering shapes the whole system around the model, including the guides and feedback controls that steer it, while context engineering decides what the model sees at each step. OpenAI describes its Agents API, in public beta since 10 September 2026, as a managed Codex harness that handles session orchestration, context compaction, and recovery.
Is RAG the same as context engineering?
RAG is one context engineering technique. It covers fetching documents, while context engineering also covers tool selection, memory, compaction, subagents, caching, and the order in which everything is assembled.
Sources
The Radar positions and descriptions come from the Thoughtworks Technology Radar entries for context engineering, progressive context disclosure, and context graphs. The harness definition comes from Birgitta Böckeler, writing about harness engineering on martinfowler.com, and the early uses of the term from the Karpathy post and the Lütke post. The context rot findings come from the Chroma technical report Context Rot, the attention budget and subagent figures from Anthropic on effective context engineering for AI agents, and the tool definition figures from the Anthropic tool search documentation. Caching prices come from the Anthropic prompt caching documentation, the OpenAI pricing page, and the GPT-6 Astra model page. All were read on 22 September 2026.
