Context Engineering, Minus the Hype

"Prompt engineer" had a short career. The title arrived around 2023, attracted a genre of listicle about magic phrases, and has largely been replaced by something less catchy and considerably more useful: context engineering.

The rebrand is easy to dismiss as vocabulary churn. It isn't. It reflects a finding that teams building real systems kept arriving at independently — roughly 80% of output quality comes from what is in the context window, not how the request is worded. A well-crafted prompt over a badly assembled context still fails. A mediocre prompt over a well-assembled context usually works.

That single observation moves the work from writing to engineering, which is why around 89% of teams report planning to invest in context-management infrastructure over the next year.

The distinction that matters

These are not competing disciplines. One contains the other.

• Prompt engineering is interaction design. Output format, reasoning strategy, tone, task decomposition, how the model should behave when uncertain. It is about how the model communicates.
• Context engineering is knowledge infrastructure. What the model can see: retrieved documents, tool results, prior state, entity definitions, access policy, data lineage. It is about what the model knows.

Put concretely: no prompt can fix a system that retrieved the wrong document. And a system that retrieved the right document usually produces something reasonable even if the instruction was clumsy. Effort should follow leverage, and for most production systems the leverage is entirely on the retrieval side.

Treat context as infrastructure, not as a prompt file in the repo.

Four operations on context

The useful framing — formalised by the agent frameworks and now fairly standard — splits context management into four operations. Most systems need all four.

Write: keep state outside the window

Anything that must persist across steps belongs in durable storage, not in an ever-growing transcript. Scratchpads, intermediate results, decisions already made. The context window is working memory; it should be reconstructed each step from something more permanent, not accumulated.

Select: retrieve what this step needs

This is where most quality problems live. It is also where most teams stop after a first pass — chunk the documents, embed them, take the top five by cosine similarity, ship it. That baseline leaves a lot on the table. Hybrid search combining keyword and semantic matching, reranking the candidate set, filtering by metadata and permission before scoring, chunking along document structure rather than fixed character counts: each of these routinely produces a larger improvement than any prompt change.

Compress: summarise finished work

Long-running processes need a way to fold completed steps into summaries. The judgement call is what survives compression — a summary that drops the one constraint that mattered is worse than no summary, because it looks complete. Compress explicitly, and keep decisions and constraints even when discarding the reasoning that produced them.

Isolate: separate concerns into separate contexts

One agent holding everything performs worse than several holding one thing each. This is partly instruction clarity and mostly attention: an agent whose context contains only what its task needs has nothing competing for relevance. It is the strongest practical argument for multi-agent designs, and it is a memory argument, not an architectural fashion.

What this looks like in a real system

A system with proper context engineering has a piece most prototypes lack: a context assembly pipeline. It is ordinary code, versioned and tested, that runs before every model call and decides what goes in.

A reasonable pipeline does roughly this:

• Resolve identity and permissions — establish what this user is allowed to see, first, so nothing downstream can leak past it.
• Retrieve candidates — hybrid search across the permitted corpus.
• Rerank and trim — order by relevance to the actual question and cut to a budget.
• Load definitions — the entity and metric definitions this domain requires, so "active customer" means the same thing every time.
• Assemble and record — build the final context and log exactly what went in.

That last step is the one teams skip and later wish they hadn't. When an answer is wrong, the first question is what the model was looking at. If you logged the assembled context you can answer it in a minute. If you didn't, you are guessing at a non-reproducible run.

Context is where governance actually happens

Access control in an AI system is not enforced at the answer. It is enforced at retrieval. If a document reaches the context window, it can influence the output regardless of any instruction telling the model to keep it confidential — instructions are not a security boundary.

This has a clarifying effect on architecture. Permission filtering has to happen before scoring, not as a post-filter on results, and the retrieval layer becomes the component your security review should focus on. It is also where data residency, retention and audit obligations land. Get the context pipeline right and most AI governance questions have a concrete place to be answered.

Where prompts still earn their keep

None of this makes prompting irrelevant — it makes it a smaller, better-defined job. Prompts are still the right tool for specifying output structure, setting the behaviour on uncertainty (abstain, escalate, ask), fixing tone and register, and decomposing a task into steps.

The rule of thumb: if the model doesn't know something, that's a context problem. If it knows and expresses it badly, that's a prompt problem. Most reported "prompt problems" turn out to be the former.

The unglamorous conclusion

Context engineering is data engineering with a language model attached to the end. Curation, retrieval quality, access control, lineage, observability — the discipline that decides whether a system is trustworthy has almost nothing to do with model selection and almost everything to do with what you feed it.

Which is good news, because it is work your team already knows how to do.

We build retrieval and context pipelines for production AI systems — with the permission model, evaluation and logging that make the answers defensible. Start a conversation.