Context Engineering: The 2026 Replacement for Prompt Engineering

Key takeaways:

• The bottleneck moved. Two years ago the quality difference between systems came from prompt wording; today it comes from context assembly — what gets included, in what order, at what length, and with what caching strategy.
• Every context window is finite. Even at 1M+ tokens, a real system burns through the budget fast: system prompt, retrieved docs, history, tool outputs, and examples all compete for the same space.
• Long context does not replace retrieval. Brute-force packing helps for small corpora; retrieval wins as soon as relevance density drops. Middle-of-context information is reliably the weakest zone — place important content at the edges.
• Prompt caching changes the economics. A long, stable system prompt is cheap per call when it is cached and expensive when it is not. Designing for cache hits is an architectural decision, not a micro-optimization.
• Context rot is real and observable. As context grows, accuracy degrades — not to zero, but enough to matter. The fix is the same as the cause: fewer tokens, better ordered, with deliberate summarization.

Prompt engineering asked: "how do I phrase this?" Context engineering asks: "what should the model actually see?" That is a different job, and in 2026 it is the job that determines whether your LLM product works.

This guide walks through the definition, why the term changed, how to budget context across the five inputs that compose it, how caching and long windows reshape the economics, where context rot hides, and how static versus dynamic assembly fits different systems. It pre-links to a cluster of deep-dive posts on each sub-topic.

Definition and Origin

Context engineering is the practice of assembling the full set of tokens a model attends over for a given turn — not just the instruction, but the system prompt, retrieved documents, conversation or agent history, cached chunks, memory, tool outputs, and few-shot examples. The object of optimization is the assembled bundle, not the phrasing. See the context engineering glossary entry for the short-form definition.

The term gained traction once three things stopped being exotic. First, prompt caching became a first-class feature on the major APIs, which meant the economics of long, stable prefixes flipped from expensive to cheap-per-call. Second, context windows stretched to over a million tokens on recent Claude and Gemini models, which turned "what should we include?" into a real question rather than a cramped tradeoff. Third, agentic systems made context assembly dynamic: every step of an agent loop rebuilds context from retrieval, memory, and tool results. See the agentic AI glossary entry.

Context engineering is not the same as prompt engineering, RAG, or memory systems — it is the superset. Prompt engineering is writing the instruction well. RAG is one way of supplying retrieved text. Memory systems handle what to remember across turns. Context engineering is the discipline of deciding which of these to use, in what proportion, in what order, and at what cost. Where prompt engineering optimizes what you say, context engineering optimizes what the model sees — and in modern systems those are very different optimization surfaces. For a side-by-side on the framing, see context engineering vs. prompt engineering.

The shift is not a rebranding. Prompt engineering has not vanished — writing clear instructions, well-chosen few-shot examples, and well-scoped system prompts still matters. It has been subsumed. A 2026 prompt engineer who ignores caching, retrieval formatting, and history management is leaving most of the quality on the table.

Why the Term Changed in 2026

Three forces combined to make "prompt" too narrow a frame.

Caching went mainstream. Both OpenAI and Anthropic now ship prompt caching on their APIs, though the mechanics differ. OpenAI caches automatically when prefixes repeat; Anthropic uses explicit cache breakpoints. Either way, the implication is the same: the cost of a long, stable system prompt drops dramatically once it is cached. That flipped the economics of an entire class of design decisions — suddenly it is cheap to front-load structured context that used to feel wasteful on every call. See the prompt caching glossary entry and the prompt caching guide.

Context windows grew past 1M tokens. Long context is not new, but the scale is. Recent Claude and Gemini models support well over a million input tokens on their long-context configurations. That does not mean you should pack a million tokens into every call — it means the constraint is no longer "cramp it in" but "decide what belongs." The question changed shape.

Agents made context dynamic. A chat prompt is usually static: you type it, the model answers. An agent rebuilds its context on every step — reading files, calling tools, summarizing progress, retrieving more information. Assembly is the loop. See the tool use glossary entry and our companion pillar on prompting AI coding agents for what dynamic assembly looks like in practice.

The combined effect: the bottleneck moved from instruction phrasing to context composition. Which is why the community started using a different word.

Prompt engineering vs. context engineering

Both disciplines still matter. You cannot rescue a bad context assembly with clever wording, and you cannot rescue ambiguous instructions with more retrieval. Context engineering sets the playing field; prompt engineering is what you do on it.

The Context Budget

Every context window is finite — even when "finite" is a million tokens. More importantly, every token you add costs something: dollars for input tokens (discounted or free when cached), latency while the model reads them, and a measurable hit to attention quality as the window fills. See our token economics guide for the cost side of this in detail.

The framing that helps: treat your context window as a budget you allocate across five inputs.

• System prompt — identity, rules, style, tools, reference material. Stable, usually cached.
• Retrieved context — documents or snippets fetched for this specific turn.
• Conversation history / memory — prior turns, or a compressed summary of them.
• Tool outputs — the results of function calls made during the turn.
• Few-shot examples — demonstrations of input-output pairs.

Then the user's actual request sits on top. That is the whole picture.

A realistic allocation for a 128k-token production assistant might look like: 8-12k system prompt (cached), 20-40k retrieved context, 10-30k history, 2-10k tool outputs per step, and 1-3k few-shot examples. The user turn might be 500 tokens. You have not touched half the window and you are already making tradeoffs. See context window management strategies for the per-budget-line decisions.

The discipline is allocating the budget deliberately instead of letting retrieval or history blow past their allotment by accident. When retrieval returns 60 chunks and you paste all of them, you have silently charged history and examples for the overflow. When history replay grows to 80k tokens, you have pushed retrieval out. Keep a running tally per request; treat budget overruns the way you would treat a database query that suddenly returns 10x more rows.

Context Assembly: The Five Inputs

Each input has its own patterns, its own failure modes, and its own best practices. What follows is a short field guide for each, with links to the deep-dive posts.

System prompt

The system prompt is the most persistent piece of context. It is where you put identity ("you are a legal research assistant"), rules ("always cite sources"), stable reference material (domain vocabulary, style constraints), and tool or schema definitions. Because it is the same on every turn, caching favors it — you pay the input cost once per cache lifetime, then near-zero per call.

That changes design guidance. Pre-caching, short system prompts were cheaper and therefore favored. Post-caching, a longer, more thorough system prompt is often the right call because it is cached. The question becomes: what belongs in stable context, and what belongs in the per-turn user prompt? See system prompt vs. user prompt context for the decision rubric.

A reasonable system-prompt skeleton for a cached production prompt:

# Identity and role
You are [role] at [org]. Your job is [one sentence].

# Operating rules
- [Rule 1]
- [Rule 2]
- Never [guardrail]

# Stable reference material
[Domain vocabulary, style guide, product facts that never change in this session]

# Tool and schema definitions
[Tool names, signatures, when to call each]

# Output format
[Exact structure you want every response to follow]

Nothing task-specific lives here. The stability is the point — it is what makes the prompt cacheable.

Retrieved context (RAG)

RAG supplies retrieved text chunks into the prompt for a specific turn. The quality of retrieval dominates here — if the top-k chunks are irrelevant, no amount of prompt tuning recovers. Equally important is format: the same chunks formatted badly yield worse downstream reasoning than chunks formatted well.

A bad retrieval-snippet format:

Here is some information you might find useful:

The widget returns a 400 when... Also, according to our records, widgets were introduced in Q2 and... The retry policy is... See also: the section on idempotency, which explains...

A good retrieval-snippet format:

<retrieved_chunk source="api-docs/widgets.md" score="0.91">
The `POST /widgets` endpoint returns 400 when `name` is missing or longer than 128 characters.
</retrieved_chunk>

<retrieved_chunk source="changelog/2024-q2.md" score="0.83">
Widgets were introduced in Q2 2024. See the migration guide for v1 compatibility notes.
</retrieved_chunk>

<retrieved_chunk source="reliability/retry.md" score="0.78">
The default retry policy is 3 attempts with exponential backoff (100ms, 200ms, 400ms). Only 5xx and network errors are retried.
</retrieved_chunk>

The second format is easier for the model to cite, to distinguish chunks from each other, and to ignore irrelevant ones. See retrieval-augmented prompting patterns for a deeper treatment, including ordering, deduplication, and citation styles.

Placement matters too. Put the most-relevant chunks at the start or end of the retrieval block — the middle of a long block of text is the weakest attention zone, which the next two sections explore.

Conversation history and memory

For chat apps, the question "how much history do I include?" shows up every turn. The naive answer — replay everything — works until history gets long enough that it dominates the budget, drowns retrieval, and degrades attention.

A better policy: keep the last two or three turns verbatim, and summarize everything older into a compact memory. The summary is cheap; the model keeps contextual awareness without being swamped. See AI memory systems for the patterns — rolling summaries, structured memory, semantic retrieval over history, and hybrids.

Agents complicate this further. An agent's "history" is not just turns; it is a trace of thoughts, actions, and observations. Keeping all of it verbatim is almost always wrong. Compressing it intelligently — by pruning failed branches, summarizing reads, and preserving the last few steps — is almost always right.

Tool results

When an agent calls a tool, the tool's output becomes context for the next model step. How that output is formatted determines whether the model can use it reliably.

Bad tool output:

Tool returned: OK, 200, data is here: Alice Smith, 34, engineer, joined 2019. Next: Bob Jones, 28, designer, joined 2022. ...

Good tool output:

<tool_result name="search_users" status="ok">
[
  {"name": "Alice Smith", "age": 34, "role": "engineer", "joined": "2019"},
  {"name": "Bob Jones",  "age": 28, "role": "designer", "joined": "2022"}
]
</tool_result>

A structured, clearly-delimited result lets the model distinguish "what I asked for" from "what I got back," and lets downstream reasoning cite fields by name. See tool use prompting patterns and the tool use glossary entry for the conventions.

A related point: tool errors need the same treatment. An error returned as "failed: timeout" is much worse than an error returned as a structured object the model can reason about — the difference shows up in whether the agent retries intelligently or flails.

Few-shot examples

Few-shot prompting is still a strong quality lever when the task has a consistent shape. Context engineering tightens the focus: which examples, in what order, at what count.

Three rules cover most of it.

• Diversity over quantity. Three varied, high-quality examples usually beat eight near-duplicates. The model learns the shape of the task from variation.
• Match the input distribution. Pick examples that look like the real inputs you expect at inference, including edge cases.
• Order for recency bias. Models often lean on the example closest to the user's instruction. Put your strongest, most representative example last.

See few-shot example selection for the details — dynamic vs. static selection, retrieval-based example banks, and when to skip examples entirely.

Caching Strategies

Prompt caching changes the cost function in a way that reshapes design decisions. The headline mechanism: for long inputs that have a stable prefix, the provider keeps an internal representation of that prefix and reuses it on subsequent calls, charging a lower rate (or nothing) for the cached portion.

The two major approaches:

• Automatic prefix caching (OpenAI). Repeated prefixes across calls are cached transparently. You benefit by keeping the front of your prompt stable.
• Explicit cache breakpoints (Anthropic). You mark where the cached prefix ends. Everything up to that point is eligible for cache reuse.

Either way, the design rule is the same: do not rotate the front of the prompt. Put stable content first — system prompt, tool definitions, canonical reference material — and variable content at the end. If you rotate a timestamp, a user ID, or a retrieved snippet into the cached region, every "hit" becomes a miss.

# Cache-friendly prompt structure

[CACHED - system prompt, identity, rules]            <-- stable
[CACHED - tool and schema definitions]               <-- stable
[CACHED - reference docs, glossary, style guide]     <-- stable
---- cache breakpoint ----
[UNCACHED - retrieved chunks for this query]         <-- variable
[UNCACHED - recent conversation turns]               <-- variable
[UNCACHED - user's actual instruction]               <-- variable

Other practical considerations:

• TTL. Cached prefixes expire. Exact lifetimes vary by provider and can change; design assuming tens of minutes for passive caching, not hours or days.
• Minimum cache sizes. Very short prefixes often are not worth caching. Check current provider minimums before optimizing heavily.
• Cache granularity. Anthropic lets you set multiple breakpoints, which helps when you have layered stability — ultra-stable system prompt + fairly-stable retrieved docs + volatile user turn.
• Cross-tenant safety. Do not cache prefixes that mix tenant A's data with tenant B's prompt. Partition caches by tenant where relevant.

For a side-by-side on the two major approaches, see Claude vs. OpenAI prompt caching. For how prompt caching differs from semantic caching (caching whole answers by meaning), see semantic caching vs. prompt caching. The short version: prompt caching reuses input representations; semantic caching reuses whole outputs when a new query is sufficiently similar to an old one.

When caching pays off:

Long-Context Strategies

A million-token context window is a tool, not a solution. Three facts determine when it helps.

Attention is not uniform. In long inputs, models attend more strongly to the beginning and end of the context than to the middle. The effect is often called "lost in the middle" and it is reliably observable. Our needle-in-a-haystack prompting guide walks through how to probe it for your specific model. The practical implication: if you have one critical piece of information in a 500k-token input, do not put it in the middle.

Density matters. A 1M-token window packed with relevant material outperforms a 1M-token window diluted with mostly-irrelevant text. Relevance density — the fraction of tokens that actually inform the answer — is the hidden variable. Retrieval wins at low density; long context wins at high density.

Retrieval and long context are complementary, not competing. Many real systems use retrieval to narrow a huge corpus down to the top few thousand relevant tokens, then fit those into a long-context call for reasoning. The split is: retrieval for relevance, long context for synthesis. See long-context prompting and context window management strategies for the patterns.

When to reach for long context over retrieval:

• Corpus is small enough to fit.
• Most of the corpus is plausibly relevant to any given query.
• Reasoning requires cross-document synthesis, not single-document lookup.
• You do not need to update the corpus at query time.

When retrieval wins:

• Corpus is large, and most of it is irrelevant per query.
• Freshness matters (new documents added frequently).
• You want to cite specific sources.
• Cost at scale matters more than the simplicity of "just stuff it in."

Context Rot and How to Detect It

Context rot is the observed degradation in model accuracy as context grows longer, denser, or more cluttered. It is not a binary cliff — it is a gradual decline, and you can measure it on a per-model basis. See the context rot problem explained for the causes and current thinking.

Signals you are hitting rot:

• The model starts ignoring instructions it was following in shorter contexts.
• Retrieval chunks at the middle of the block get cited less than chunks at the edges.
• Contradictions between retrieved chunks produce confident but wrong answers.
• History-aware behavior (remembering something from earlier) degrades past a certain history length.
• "I don't know" answers increase on questions whose answer is demonstrably present in context.

The mitigations are structural, not phrasing-based.

• Compression. Summarize older history, dedupe near-duplicate retrieval chunks, strip irrelevant metadata. See context compression techniques.
• Hierarchical loading. Put the most specific, most relevant material closest to the user's instruction. See hierarchical context loading.
• Better retrieval. The cheapest way to shrink context without losing signal is to retrieve fewer, better chunks. A 10-chunk retrieval with 9 relevant hits beats a 30-chunk retrieval with 10 relevant hits and 20 distractors.
• Context-length budgeting. Set a per-input cap (retrieval ≤ X tokens, history ≤ Y tokens) and enforce it in the assembly code, not just in prose.

Rot is worse on some models than others, and it moves with every model release. The only defensible answer is to measure it on your own eval set, not to trust marketing benchmarks.

Dynamic vs. Static Context Assembly

Two broad shapes show up across production systems.

Static assembly. The context for a given task is templated. A customer-support assistant always includes the same system prompt, a deterministic set of account facts for the user, and the last N turns of conversation. Nothing is retrieved dynamically per turn. This is the common shape for chat apps, templated Q&A, and simple workflows.

Static assembly is easy to reason about. It caches beautifully — the whole prefix is usually stable within a session. The downside is that it cannot adapt to the specifics of a query: if the user asks something the static template did not anticipate, the model is under-supplied.

Dynamic assembly. The context is rebuilt per turn from retrieval, memory, and tool outputs. This is the shape of agents and retrieval-heavy assistants. The system prompt is fixed; everything after it is assembled on demand. See dynamic context assembly patterns for the common designs.

Dynamic assembly is more capable but harder to debug. It caches partially — the stable prefix is cacheable, the rest is not. Its failure modes are distinct: bad retrieval, bad memory summarization, bad tool-output formatting, and bad step-to-step pruning all compound.

Most real systems are hybrids. A customer-support agent might use a static system prompt plus dynamic retrieval of the user's tickets plus a dynamic memory of the current conversation. An AI coding agent — see our companion pillar on prompting coding agents for the full picture — goes even further: every step's context is rebuilt from the agent's plan, file reads, and tool outputs. Dynamic context is how agents work, and the underlying loop is usually a variant of ReAct — reason, act, observe, reassemble.

A rule of thumb for choosing:

• If the task is known and repeatable, lean static. Cache the whole prefix, keep the template tight.
• If the task is open-ended or the information needed depends on the query, lean dynamic. Invest in retrieval quality and assembly plumbing.
• If you can separate the stable layer from the dynamic layer, do so explicitly — the stable layer becomes the cache target, the dynamic layer becomes the thing you tune. This is also the idea behind hierarchical context loading.

Token Economics

Context engineering is, unavoidably, an economics exercise. The rules are not intuitive, because caching and input/output pricing asymmetries tilt the surface. See our token economics guide for the full treatment; the core points are worth naming here.

Longer prompts can be cheaper. A 20k-token system prompt called ten thousand times is far cheaper when it is cached than a 2k-token prompt called ten thousand times without caching — if the cached rate is low enough. "Shorter is cheaper" is only true uncached.

Input and output tokens are not symmetric. Output tokens are typically more expensive than input tokens, often by a meaningful multiple. That means trimming verbose outputs is often a bigger win than trimming input. "Answer in one sentence" saves more than "be concise" in the system prompt would suggest.

Latency scales with input length. Even when cached, very long inputs take longer to process than short ones. Latency and cost are separate axes; both belong in the budget.

Per-call vs. per-session economics. A feature that uses 5x the tokens per call but needs one tenth the retries is cheaper in practice. Measure total cost per successful task, not per API call.

When longer prompts save money:

• Stable prefixes are cached.
• Longer context eliminates retries.
• Richer context reduces hallucinations that would cost additional calls to fix.

When longer prompts cost money:

• Prefixes are uncached or rotate often.
• Added context is noise, not signal.
• Output length grows in proportion to input length.

Hierarchical Context Loading

Hierarchical loading is a structured way to order context from general to specific. The idea is simple: the model pays the most attention to what is closest to the instruction, so the most specific, most relevant material should live there. Everything more general sits further up. See hierarchical context loading for the patterns.

A typical hierarchy:

• System identity and rules — who the model is, what it may never do.
• Domain vocabulary and stable reference — things that are always true in this system.
• Session-specific context — the user's account, current workspace, memory of prior turns.
• Query-specific retrieval — documents fetched for this exact question.
• Few-shot examples (if used) — demonstrations tuned to the query pattern.
• The user's instruction — the thing the model has to do right now.

Why it helps: attention is not uniform. Recency bias is a real force — the model leans on what is near the instruction. Hierarchical loading uses that bias on purpose, placing the highest-signal material where it will be weighted most. It also composes with caching — the top of the hierarchy is the most stable, which is exactly what you want caching.

A good test for a hierarchy: if you delete the most-specific layer, does quality drop noticeably? If yes, the layer is earning its spot. If no, it is noise and should go.

Model-Specific Notes

Claude, GPT, and Gemini all reward context engineering, but they do not reward the same things to the same degree. The model-by-model differences shift with every release, so the responsible thing is to speak generally rather than invent numbers.

Claude. Tends to follow structured, explicit instructions closely, including long system prompts and XML-style delimiters. Extended thinking modes on recent Claude configurations pair naturally with context-heavy tasks — the model has more budget to reason over a large assembled input. See extended thinking prompts for Claude. Prompt caching uses explicit cache breakpoints, which rewards designs that separate stable and dynamic layers.

GPT. Benefits from explicit role assignment, system/user separation, and clearly scoped instructions. Automatic prefix caching means the key design move is keeping the front of prompts stable across calls. Reasoning-capable GPT variants internalize step-by-step thinking, which changes the few-shot and CoT math — explicit reasoning prompts add less than they did on older models.

Gemini. Long-context handling is a focus of the family; the very-long-context configurations are a natural fit for stuffing entire codebases or document sets into one call. As with every model, measure where its attention fades in your own evals — published benchmarks are a starting point, not a substitute.

What not to do: build a system that only works on one provider's quirks. The context-engineering principles — budget, ordering, caching the stable layer, retrieving the relevant layer, watching for rot — are portable. The specific knobs (cache breakpoint syntax, delimiter conventions, thinking budgets) are not. A good system separates the two layers so that switching models is a configuration change, not a rewrite. See context engineering vs. prompt engineering for more on the portable-skill argument.

FAQ

What is context engineering?

Context engineering is the discipline of assembling everything a model sees for a given turn — system prompt, retrieved documents, conversation history, memory, tool outputs, and few-shot examples. Where prompt engineering optimizes the wording of a single instruction, context engineering optimizes the full bundle of tokens the model attends over. It becomes the dominant quality lever once you have agents, retrieval, long contexts, or prompt caching in the system.

Is prompt engineering dead?

No — prompt engineering is now a sub-skill inside context engineering. You still need to write clear instructions, good few-shot examples, and well-scoped system prompts. What changed is that wording alone no longer explains most quality differences between systems. Two teams with identical prompts can ship very different experiences depending on how they assemble retrieved context, manage history, cache prefixes, and format tool outputs.

How is context engineering different from RAG?

RAG is one input among five — it supplies retrieved documents. Context engineering is the broader discipline of deciding which inputs to include at all (system prompt, retrieval, memory, tools, examples), in what order, at what length, and with what caching strategy. You can do context engineering without RAG, and you can do bad context engineering with an excellent RAG system.

Do I need to care about prompt caching?

If your prompts have a stable prefix that gets reused — a long system prompt, tool definitions, reference documents — caching can cut input token cost dramatically and improve latency. If each request is short and unique, caching is less relevant. In 2026, most production LLM pipelines have enough repetition that caching is worth designing for from day one.

When should I use a 1M token context window versus retrieval?

Use long context when the corpus is small enough to fit and the model needs to reason across most of it — a single codebase, a legal contract, a research paper set. Use retrieval when the corpus is large, most of it is irrelevant to any given query, or freshness matters. Long context is simpler; retrieval is more scalable. Many real systems combine the two.

What is context rot?

Context rot is the observed degradation in model accuracy as context grows longer and denser. Even with 1M token windows, models do not weight every token equally — middle-of-context information is often underused, contradictions accumulate, and relevance signals get diluted. The mitigation is the same as the cause: fewer tokens, better ordered, with the most important material at the edges.

Should the system prompt be long or short?

Long enough to cover identity, style, constraints, and stable context, short enough not to drown the user's actual request. With caching, longer stable system prompts become cheaper per call, which pushes the sweet spot up. Without caching, long system prompts tax every request. The honest answer is: put stable, reusable content in the system prompt, and everything task-specific in the user turn.

How do I pick few-shot examples?

Pick examples that match the input distribution you expect at inference time and cover the edge cases you care about. Diversity matters more than quantity — three varied examples usually outperform eight near-duplicates. Order matters too: models often lean on the examples closest to the final instruction, so put your strongest demonstration last.

Does context engineering apply to chat apps or only agents?

Both. Agents make it more visible because context gets rebuilt dynamically on every step, but chat apps face the same problems: when does the history get summarized, what goes in the system prompt, how are retrieved documents injected. Any LLM product whose context is non-trivial is doing context engineering, whether the team calls it that or not.

How do I test whether my context is good?

Run your model on a fixed eval set and vary one input at a time — retrieval top-k, system prompt length, example count, cache breakpoint placement — and watch quality and cost move. Pair that with needle-in-a-haystack style probes to confirm the model can actually retrieve from positions you expect it to. If you cannot measure context changes, you cannot improve them.

Context Engineering Best Practices

A compact checklist for day-to-day use. The summary post on context engineering best practices for 2026 goes deeper.

• Design the system prompt to be cached — stable front, no timestamps or per-call variables.
• Budget the context window across the five inputs and track it per request.
• Format retrieved chunks with explicit delimiters, sources, and scores.
• Put the most-relevant material at the start or end of long blocks, not the middle.
• Summarize history deliberately; do not let replay grow unbounded.
• Return tool outputs in structured, parseable shapes with consistent field names.
• Pick few-shot examples for diversity; order them so the strongest is last.
• Measure context changes on a fixed eval set before shipping them.
• Treat static and dynamic layers separately — cache the static, tune the dynamic.
• Watch for rot. Accuracy on long inputs is an empirical question, not an assumption.

Prompt engineering taught the field to respect phrasing. Context engineering asks the harder question: given everything a model could see, what should it see? Answer that well and the phrasing will mostly take care of itself.