Context Engineering for Production AI, What It Is and How to Do It Well
Vasim Gujrati
Solutions Architect, AI & Platforms, Unico Connect
In this article
- Quick Answer
- Key Takeaways
- What Is Context Engineering?
- Why Did Thoughtworks Move Context Engineering to Adopt?
- Context Engineering Best Practices, the Four Techniques Thoughtworks Says Have Matured
- What Is Progressive Context Disclosure?
- A Context Pipeline for a Production Agent
- How to Measure Whether Your Context Is Working
- Common Context Engineering Mistakes
- Context Engineering, RAG, and Fine Tuning
- How Unico Connect Helps
- Frequently Asked Questions
- Sources
In November 2025 the Thoughtworks Technology Radar placed context engineering in its Assess ring, meaning worth exploring. Five months later, in Volume 34 in April 2026, it moved straight to Adopt, the ring Thoughtworks reserves for practices it believes the industry should be using now. Skipping Trial on the way from Assess to Adopt is a strong signal. It matches what teams shipping AI agents have learned the hard way. The weak point is usually what the model sees at each step, and only rarely the model itself. Below we cover what context engineering is, the four techniques behind it, what it costs and saves, and how to tell whether your context is working. Sources were checked on 22 September 2026.
Quick Answer
Context engineering is the practice of designing everything a model sees at inference time, including instructions, retrieved data, tool definitions, memory, and prior work, so it produces reliable output. Prompt engineering tunes the wording of one prompt, while context engineering assembles the whole context window dynamically at each step. Thoughtworks moved it to Adopt in April 2026 and names four techniques that have matured, namely prompt caching, dynamic retrieval of tools and data, context graphs, and context management with compression and subagents.
Key Takeaways
- Do not count on a bigger context window to fix quality. Chroma tested 18 models and found performance grows less reliable as input length increases, even on simple tasks.
- Thoughtworks calls the result context rot, and recommends progressive context disclosure, where an agent starts with a lightweight index and pulls in detail only when it becomes relevant.
- Put the stable part of your prompt first so it can be cached. Cached input costs a tenth of the normal input price on GPT-6 Astra and most Claude models, and a fortieth on Claude Fable 5.1 and Claude Mythos 5.1.
- Use dynamic tool selection or tool search to keep the working set small, since loading every tool and MCP server up front wastes context.
- Track task success as context grows, cost per completed task, cache hit rate, and how often the agent reasons from stale or wrong context, so you can prove whether a context change helped.
What Is Context Engineering?
Thoughtworks defines context engineering as the systematic design and optimization of the information provided to a large language model during inference to reliably produce the desired output. Anthropic, in an engineering guide published in September 2025, puts the goal as finding the smallest possible set of high signal tokens that maximize the likelihood of the outcome you want.
Two posts from June 2025 are often credited with popularizing the term. Tobi Lütke wrote that he preferred context engineering to prompt engineering because it describes the art of providing all the context for the task to be plausibly solvable by the LLM. Andrej Karpathy agreed on 25 June 2025, calling it the delicate art and science of filling the context window with just the right information for the next step.
| Aspect | Prompt engineering | Context engineering |
|---|---|---|
| Unit of work | The wording of one prompt | Everything in the context window at each step |
| When it happens | Written once, then tweaked | Assembled dynamically as the task runs |
| Main inputs | Instructions and examples | Instructions, retrieved data, tool definitions, memory, prior outputs, environment signals |
| Typical failure | An ambiguous or badly worded instruction | Too much, too little, stale, or conflicting information |
| Where it matters most | Single turn tasks | Agents, long running workflows, and retrieval heavy products |
Why Did Thoughtworks Move Context Engineering to Adopt?
Agents broke the old approach. Thoughtworks writes that as agents tackle more complex tasks, dumping raw data into large context windows leads to context rot and degraded reasoning, and that treating context as a static text box is a fast track to hallucinations.
The research behind that phrase comes from Chroma, which in July 2025 published a technical report evaluating 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, while holding task difficulty constant and changing only input length. Performance became increasingly unreliable as inputs grew, and in the needle tests the models even did better on shuffled text than on logically structured text, a sign that structure in long inputs changes how attention behaves. Anthropic describes the same limit as an attention budget, where every new token depletes the focus of the model a little. So a one million token window gives you more room, but what you put in it still decides the quality of what comes out.
Context Engineering Best Practices, the Four Techniques Thoughtworks Says Have Matured
1. Context setup with prompt caching
Put stable content, such as system instructions, tool definitions, and reference material, at the start of the prompt in a fixed order, and cache it. Thoughtworks notes this reduces cost and improves time to first token. The savings are large enough to shape the architecture of an agent.
Prices are in dollars per million tokens.
| Model | Normal input | Cached input | Cache write |
|---|---|---|---|
| OpenAI GPT-6 Astra, prompts up to 272K tokens | $10.00 | $1.00 | $12.50 |
| OpenAI GPT-6 Astra, prompts over 272K tokens | $20.00 | $2.00 | $25.00 |
| Claude Fable 5.1 | $10.00 | $0.25 | $12.50 for 5 min, $20.00 for 1 hour |
| Claude Opus 5 | $5.00 | $0.50 | $6.25 for 5 min, $10.00 for 1 hour |
Context size carries a price step of its own. OpenAI charges 2x the input and cache rates and 1.5x the output rate for the whole request once a GPT-6 Astra prompt passes 272K input tokens. Claude Fable 5.1 and Claude Mythos 5.1 charge 0.025x the base input price for cache hits, against 0.1x on most other Claude models. The rest of the rate card is in our breakdown of GPT-6 Astra pricing and API changes.
Anything that changes per request, such as the user message, retrieved passages, or timestamps, belongs after the stable prefix, or it breaks the cache for everything behind it. OpenAI made Prompt Cache Diagnostics generally available on 8 September 2026, and it shows why a request missed the cache.
2. Dynamic retrieval of data and tools
Classic retrieval augmented generation fetches documents before the model starts, and context engineering extends the same idea to tools. Thoughtworks describes selecting tools and loading only the necessary MCP servers to avoid unnecessary context expansion, and OpenAI built tool search into its Agents API, in public beta since 10 September 2026, so tool definitions load only when relevant. Anthropic calls the broader pattern just in time retrieval, where an agent keeps lightweight references such as file paths, stored queries, or links, and loads the underlying data with tools when it needs it.
Skipping this has a measurable cost. Anthropic documents that a typical multiserver setup with GitHub, Slack, Sentry, Grafana, and Splunk can consume about 55,000 tokens in tool definitions before Claude does any work. Its tool search tool typically cuts that by over 85 percent by loading only the 3 to 5 tools a request needs, and Claude picks the right tool less reliably once more than 30 to 50 tools are available. Tool results need the same care. Our post on running MCP in production AI agents shows how to trim oversized MCP responses before they reach the model.
3. Context graphs for institutional reasoning
Context graphs are the newest of the four, added in April 2026, and Thoughtworks also lists them as a separate technique in its Assess ring. A context graph models decisions, policies, exceptions, precedents, evidence, and outcomes as connected nodes, so it records why something happened as well as what happened. Thoughtworks gives the example of an agent handling a discount exception that cannot tell whether it reflects standing policy or a one time override. A context graph surfaces that provenance, and unlike GraphRAG built from static documents, it keeps temporal validity on every edge so superseded facts are invalidated rather than overwritten.
4. Context management for long running work
Long tasks outgrow any window. The responses that have matured are compaction, where earlier context is summarized as the window fills, structured note taking that keeps memory outside the window, and subagents that explore in their own context and hand back only a condensed result. Anthropic describes subagents that may use tens of thousands of tokens exploring but return summaries of around 1,000 to 2,000 tokens, and OpenAI compacts context automatically in its Agents API. Before you split a task across agents at all, read our comparison of single agents and multi agent systems. Long running work also has to survive crashes and restarts, the subject of our guide to what breaks when AI agents run in production.
What Is Progressive Context Disclosure?
Progressive context disclosure, in the Trial ring since April 2026, is the technique that ties the others together. Instead of front loading every instruction and reference an agent might need, you give it a lightweight discovery step. It sees an index of what is available, selects what is relevant to the request, and only then loads the detailed instructions or data.
Thoughtworks points out that this is how many agentic coding tools handle Agent Skills, by deciding which skills apply before loading their full instructions, and it warns against the common trap of bloating instructions with endless do and do not rules, which can degrade performance. You can see the pattern at work in our notes on using Claude Code skills in real projects. To build it yourself you need three parts.
- An index. Short descriptions of each skill, document set, or tool group, small enough to always include.
- A selection step. The agent, or a lightweight router, chooses what the current request needs.
- A loader. Only the chosen instructions, documents, or tools enter the context, and they leave again when the step is done.
A Context Pipeline for a Production Agent
A production agent usually ends up with the same layers, assembled in the same order on every call.
- Stable prefix. Role, rules, output format, and tool definitions that rarely change, in a fixed order so they cache.
- Task framing. The user request and the goal of the current step.
- Retrieved evidence. Only the passages, records, or query results relevant to this step, each with its source.
- Working memory. Notes and decisions from earlier steps, summarized rather than replayed.
- Tool results. Trimmed to only the fields the next step needs.
- Guardrails. Checks that stop the step when evidence is missing, so the model does not guess.
For every layer, decide what earns its place in the window at this step and what can be fetched later.
How to Measure Whether Your Context Is Working
Treat context engineering like any other performance work and measure it.
| Metric | What it tells you |
|---|---|
| Task success rate against context length | Whether quality drops as inputs grow, the signature of context rot |
| Cost per completed task | The real cost, across every model call, retry, and subagent |
| Cache hit rate | Whether the stable prefix is actually stable |
| Retrieval precision | How much of what you load is used in the answer |
| Stale or conflicting context incidents | How often the agent reasons from superseded information |
| Tokens per step | Whether the working set stays lean as the task runs |
Build an evaluation set of real tasks, rerun it whenever prompts, tools, retrieval, or models change, and compare these numbers before and after. To collect them, start with our guide to AI observability for LLMs in production.
Common Context Engineering Mistakes
- Loading full documents when a few passages would do, which increases cost and invites context rot.
- A new do or do not instruction added after every incident, until the rules start to contradict each other.
- Registering every MCP server and function on every call, when most steps use two or three.
- Putting timestamps, user names, or retrieved text at the start of the prompt, ahead of the stable content, which breaks the cache.
- Memory summaries that lose where a fact came from and when it was true.
- Changing prompts and retrieval with no fixed set of tasks to prove the change helped.
Context Engineering, RAG, and Fine Tuning
The three work together. Retrieval augmented generation is one context engineering technique, focused on fetching documents. Fine tuning changes the model itself and suits stable behavior, format, or style, while context engineering changes what the model sees and suits facts, tools, and state that change often. Most production systems use context engineering first and fine tuning only where it earns its cost. For more on that choice, see our comparisons of RAG and fine tuning and of fine tuning and prompt engineering.
How Unico Connect Helps
We design and build production AI agents and retrieval systems, and context engineering is where most of that work goes. Our AI development services cover retrieval pipelines, prompt caching strategy, tool design, and evaluation, and our agentic AI development team builds long running agents with compaction, subagents, and approval steps. To add engineers inside your own team, you can hire AI engineers who build these pipelines. If you are choosing an agent runtime, our guide to the OpenAI Agents API explains what the managed harness now handles for you, or you can talk to our team.
Frequently Asked Questions
What is context engineering in AI?
Context engineering is the design and optimization of everything a model sees at inference time, including instructions, retrieved data, tool definitions, memory, and prior outputs, so it reliably produces the output you want. It treats the context window as a design surface that is assembled dynamically at each step of a task, instead of a single prompt written once.
How is context engineering different from prompt engineering?
Prompt engineering focuses on the wording of a prompt, while context engineering covers the whole configuration of context, including which documents, tools, memories, and prior results are loaded, in what order, and when they are removed. That makes prompt engineering one part of context engineering.
What is context rot?
Context rot is the decline in model reliability as the amount of input grows. Chroma measured it across 18 models in July 2025 and found performance became increasingly unreliable with longer inputs, even when the task stayed equally hard, and Thoughtworks uses the term to explain why dumping raw data into large context windows degrades reasoning.
Where is context engineering on the Thoughtworks Technology Radar?
In the Adopt ring, since Volume 34 in April 2026. It entered the Radar in the Assess ring in November 2025. Progressive context disclosure (in Trial) and context graphs (in Assess) sit alongside it as related techniques.
How much can prompt caching save?
Cached input costs a tenth of the normal input price on most Claude models and on GPT-6 Astra, where it is 1 dollar instead of 10 per million tokens. On Claude Fable 5.1 and Claude Mythos 5.1, cache hits cost 25 cents instead of 10 dollars per million tokens, a fortieth of the base price. Writing to the cache costs more than normal input, so the savings depend on how often a stable prefix is reused.
Does a larger context window make context engineering unnecessary?
No. A larger window adds capacity, but research on context rot shows reliability still falls as inputs grow, and every extra token adds cost. Deciding what belongs in the window at each step still determines quality.
What is the difference between context engineering and harness engineering?
A harness is everything in an AI agent except the model itself. Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, writing on martinfowler.com on 2 April 2026, sums this up as Agent = Model + Harness, and she calls engineering a user harness for a coding agent a specific form of context engineering. In practice the two overlap. Harness engineering shapes the whole system around the model, including the guides and feedback controls that steer it, while context engineering decides what the model sees at each step. OpenAI describes its Agents API, in public beta since 10 September 2026, as a managed Codex harness that handles session orchestration, context compaction, and recovery.
Is RAG the same as context engineering?
RAG is one context engineering technique. It covers fetching documents, while context engineering also covers tool selection, memory, compaction, subagents, caching, and the order in which everything is assembled.
Sources
The Radar positions and descriptions come from the Thoughtworks Technology Radar entries for context engineering, progressive context disclosure, and context graphs. The harness definition comes from Birgitta Böckeler, writing about harness engineering on martinfowler.com, and the early uses of the term from the Karpathy post and the Lütke post. The context rot findings come from the Chroma technical report Context Rot, the attention budget and subagent figures from Anthropic on effective context engineering for AI agents, and the tool definition figures from the Anthropic tool search documentation. Caching prices come from the Anthropic prompt caching documentation, the OpenAI pricing page, and the GPT-6 Astra model page. All were read on 22 September 2026.
