What is context engineering?

Context engineering is the practice of deciding what information a model has in front of it at the moment it answers. Not how the question is phrased — that is prompt engineering — but what evidence, instructions, examples and history occupy the limited space the model can read, and where each piece sits.

The shift in name is not marketing. When models were small and tools were rare, the lever that changed output quality most was wording: ask for step-by-step reasoning, give two examples, specify a format. Those techniques still work, but they are now a small effect next to the question of whether the renewal date was in the request at all. A perfectly worded prompt over missing evidence produces a confident guess.

What the job consists of

Selection. Which records, documents and prior turns are relevant to this question. Everything you include competes for attention with everything else, so including more is not reliably better.

Exclusion. The harder half. A model given six thousand tokens of company background and one relevant invoice is doing a worse job of finding the invoice than one given the invoice. Irrelevant context does not sit politely in the corner; it dilutes.

Ordering. Where a block sits changes both what the model weights and what you pay. Instructions and stable material belong at the front, the question at the end. Put anything that varies per request near the top and you invalidate prompt caching for everything behind it.

Sourcing. Whether context arrives pre-fetched in the request, or the model asks for it through tools when it finds it needs it. Pre-fetching is predictable and often wasteful; tool calls are responsive and harder to reason about. Most working systems do both.

Compression. Summaries, rolling conversation windows, structured extracts instead of raw documents. Every one of these trades fidelity for room, and the trade is only sound if you know which details were safe to lose.

Why it is mostly an architecture problem

Teams usually arrive at context engineering through a cost shock or a quality complaint, and treat it as a tuning exercise: trim the system prompt, cap the history, summarise more aggressively. That helps once and then stops helping, because the real constraint is structural.

If the only way your system can find out about a customer is to search a pile of text for passages that resemble the question, then the context you assemble will always be approximate, and it will always be assembled from scratch. No amount of ordering fixes a retrieval step that cannot tell that two records describe the same company. That is the argument for resolving identity in a knowledge graph rather than hoping similarity stands in for it.

Likewise, if nothing the model learns survives the conversation, every session starts from zero and pays to be told the same things again. Durable memory is a context-engineering decision disguised as a storage decision: it is what makes the stable part of the prompt actually stable.

Signs it needs attention

• The same background is re-sent on every turn, and nobody can say which parts are load-bearing.
• Answers get worse as conversations get longer, rather than better.
• Cache hit rates are low and nobody knows why.
• Removing a section of the system prompt changes nothing measurable, and so nobody dares remove it.
• The model answers questions it had no evidence for, in the same tone as questions it did.

Where it stops

Context engineering cannot supply facts that are not in any system, settle a disagreement between two records, or make a model admit uncertainty it does not feel. Those need, in order: a connector, contradiction detection, and a product willing to return nothing.

What it can do is make sure the evidence that exists is in the room — which is, in practice, the difference between most AI pilots that work and most that do not.