Context engineering: what actually decides how your AI agent responds
Context engineering decides what an AI agent sees before it answers: instructions, tools, memory, and retrieved knowledge. See the five key ingredients.
On this page
Two agents run the same prompt, the same model, the same task. One answers correctly. The other invents a price that doesn’t exist, or forgets a rule the company changed last week. The difference is almost never the prompt. It’s what each one actually had on hand when it answered: which chunk of the knowledge base got pulled in, whether the last conversation was remembered, whether the right tool was even loaded. That has a name: context engineering.
What context engineering is
Context engineering is the practice of deciding, for every single response, exactly what goes into the agent’s context window: instructions, tools, retrieved documents, memory, and recent history. It isn’t writing a better instruction. It’s assembling the right package of information, at the right size, at the right moment.
The scope is different from prompt engineering. Prompt engineering answers one question: how do you phrase the instruction? Context engineering answers a bigger one: what does the model need to see for that instruction to actually work? A perfect prompt with the wrong context still gets the answer wrong. A simple prompt with the right context usually gets it right.
The five ingredients that make up an agent’s context
Context isn’t one thing. It’s five separate layers, and every agent has to decide how much of each to load for a given response.
| Ingredient | What it is | Risk if missing |
|---|---|---|
| Instructions (system prompt) | The agent’s role, tone, and fixed rules | An agent with no personality and no boundaries |
| Available tools | What the agent can actually do: query a CRM, open a ticket, calculate | Promises it can’t deliver on |
| Knowledge base | Documents, policies, and FAQs retrieved on the fly | Generic or outdated answers |
| Memory | Facts about that specific person, carried across past conversations | Customer has to repeat themselves every time |
| Conversation history | What’s been said in this session so far | Agent forgets what the user just told it |
A good agent doesn’t weigh all five layers equally on every turn. It decides, question by question, how much of each is worth loading. That decision is the real work.
Why bad context breaks more agents than a weak prompt
The prompt gets reviewed by the team that builds the agent, calmly, before launch. Context shows up, or fails to, only at answer time, with the customer already on the other end. That makes the failure more expensive and much harder to catch in testing.
Three common ways it goes wrong:
One-size-fits-all context. The same policy block gets stuffed into every response, even when 90% of it has nothing to do with the actual question. The model spends attention, and tokens, on irrelevant text, and the answer gets vaguer, not sharper.
Polluted context. An outdated document, an old ticket that’s already closed, a piece of conversation unrelated to the current problem. Every irrelevant fragment competes for attention with the one piece that actually matters for that answer.
Missing context. The customer’s question needs a specific fact (current price, order status, a policy that changed last week) and the agent doesn’t have access to it at that moment. So it fills the gap with something plausible-sounding. That’s a hallucination born from a context failure, not a bad model.
How to assemble the right context for each answer
The practical rule: for every response, ask what needs to be there for it to be correct. Not what would be nice to have.
- Selective retrieval, not a data dump. Pull the specific chunks of the knowledge base relevant to that exact question, not the whole manual. Good RAG filters before it delivers, not after.
- Summarize, don’t accumulate. A long conversation doesn’t need to be replayed verbatim. A summary of what’s already been said, plus the literal recent turns, covers the same ground with far less noise.
- Prioritize when space runs out. Not everything fits in the window. What decides the answer (customer data, a company rule) comes before what only enriches it (old history, niche context).
- Fix the source, not just the prompt. If the agent got it wrong because a policy changed, the fix is updating the knowledge base. Rewriting the instruction to “remind” the agent every time doesn’t scale.
Context is also a cost
Every chunk that goes into the window costs tokens, and tokens cost money on every single run. Dumping the entire document “just to be safe” doesn’t make the agent more reliable. It makes it more expensive and, in practice, less precise: the model has to separate signal from noise inside text that never needed to be there.
The math changes on a multi-model platform, where you can run the retrieval step on a cheap model and save the expensive one for generating the final answer. That kind of choice, made run by run, is what separates an AI operation that’s expensive from one that scales.
Assembling the right context gets easier when the platform already handles multimodal input with built-in token savings and lets you switch models per task. SquadOS runs 30 models from 15 providers, with up to 95% token savings and BYOK to bring your own key.
