Context engineering is the practice of giving an LLM exactly the information it needs to give a relevant answer to your question, and no more than that. A large language model gets no background on who you are, what you are trying to achieve or which organisation you work for, unless you tell it yourself. The quality of your context largely determines the quality of the answer. This article covers how to build up context, why context quality matters more than context size, and which mistakes to avoid in practice.
What is context in an LLM?
In an LLM, context is all the text you send along with your question: background information, examples, documents, earlier messages in the conversation and any instructions about tone or format. The model uses that text to work out what your question means and to generate a fitting answer. Without context, the model falls back on generic assumptions, and that shows in the answer.
A simple example: the word ‘model’ means something different in a conversation about machine learning than in one about photography. The surrounding text steers the interpretation. For professionals it works exactly the same way, only at a higher level. A question about ‘a good strategy for our new campaign’ produces a different answer depending on whether the model knows you sell B2B SaaS to HR managers or run a local physiotherapy practice.
Context consists of several layers that complement each other. The direct question is one layer, but around it you have the audience, the sector, earlier work that serves as a reference, the tone you want, and hard constraints such as word count or layout. The more clearly you separate those layers, the better the model responds to them.
How does the context window work?
The context window of an LLM is the amount of text the model can process at once, measured in tokens. A token is a piece of text of three to four letters on average; a ten-word sentence in a language such as Dutch comes to roughly fifteen to twenty tokens. Current models such as Claude Sonnet 5.5 and Claude Opus 5.5 work with context windows of one million tokens, which is plenty for long documents or entire conversations.
Keep in mind that the context window is not memory. A chat model does not remember earlier conversations by itself; it starts afresh with every new conversation. Chat apps such as Claude (on all plans, including Free) and ChatGPT do have a memory feature that passes information from earlier conversations to a new chat as extra context. In an ongoing chat session, the whole conversation history is sent again so the model has the full thread. As that conversation grows, the window gradually fills up, even if you add nothing new.
With API use this is even more explicit. In the Python integrations I build with the Claude API, every request has to include all the relevant context, including earlier exchanges that matter for the current question. That forces choices about what to include and what to leave out. The same discipline helps in ordinary use through the chat interface too.
Why context quality matters more than context size
A large context window is not a licence to throw everything in. A common misconception is that more context always leads to better answers. In practice you often see the opposite: the more noise you add, the harder it is for the model to pick out the relevant signals.
Research from Stanford, UC Berkeley and Samaya AI has shown that LLMs process information at the beginning and end of the context best. Whatever sits in the middle of a long context is used less effectively in the answer. This is known as the ‘lost in the middle’ problem and shows up across several model families. The practical consequence: if you include a long document and ask a specific question, key information in the middle of that document can get overlooked, even though the literal text is in the window.
The lesson for daily use: put your most important instructions and core information at the beginning or end of your prompt, and keep the context between those points as short as possible. That sounds easier than it is. Once you have a long document or a large dataset to hand, you feel the urge to ‘just send everything to be on the safe side’. That reflex rarely produces better answers.
A second reason to keep context tight is cost control in API work. A Python script that sends a prompt full of unnecessary tokens hundreds of times a day runs up costs faster than you expect.
How to structure your context in four blocks
A workable structure for context consists of four blocks that you provide in this order. First the role or frame: who the model is talking to and what it is being used for. Then the background information: who the audience is, what the goal is, which constraints apply. Next the actual task: exactly what the model needs to do. And finally the output format you want: length, structure, tone and, optionally, an example.
Put the core information at the beginning or end of the prompt.
This order is not a law, but it works in practice because it narrows the model’s focus step by step. First you place it in the right situation, then you give it the building blocks, then the assignment, then the packaging. It is the same order in which a colleague reads a briefing.
For longer tasks, it pays to build up context in phases rather than dumping everything into one prompt. One technique that helps here is prompt chaining: you have the model summarise or analyse first and use that output as input for the next step. For tasks where reasoning matters, chain-of-thought prompting works well: you explicitly ask the model to work through the problem step by step before it reaches an answer.
When is RAG a better choice than a large prompt?
Sometimes your relevant information is too big for a single prompt: an entire knowledge base, product documentation running to thousands of pages, or an archive of customer contacts. In situations like that, retrieval-augmented generation (RAG) becomes worth considering. With RAG you build a system that first retrieves the most relevant chunks from a large dataset and passes that selection to the LLM as context. So the model does not get everything, but it does get the right fragments.
This works well when the underlying information changes regularly (knowledge bases, documentation, intranet) or when the total dataset is too large to load. It is not a solution for everything. If you are doing a one-off analysis of a hundred-page document, you do not need a RAG system; that fits easily in the context window of a current model.
RAG requires technical infrastructure: a vector search engine, an embedding model, and logic that decides which fragments are retrieved. For most knowledge workers, building RAG is not the first step. Getting better at what you can do in an ordinary prompt pays off sooner. Only when you hit the limits (more information than fits in one window, or information that changes too often to send along manually) does RAG become a serious option.
Common context engineering mistakes
In practice, giving an LLM good context often goes wrong on a few recurring points. The most common mistake is being too vague. A prompt such as ‘write something about our new service’ gives the model no frame, no audience and no format. The answer is predictably generic. The fix is not to write more, but to write more specifically: who will read this, what action do you want, which tone suits your brand.
A second mistake is mixing context and instructions. When background information and the task sit in one long paragraph, the model has to guess which part describes and which part asks. Separate the two. A ‘context’ block followed by a ‘task’ block works better than running prose where everything is mixed together.
A third mistake is sending all available information ‘to be on the safe side’. More information in one prompt often makes the output less consistent rather than better. An approach that gives each step exactly the information that step needs and nothing more usually produces more stable results.
A fourth mistake is leaving irrelevant context from earlier steps in place. In longer chat sessions, everything you asked earlier stays in the window. If you switch to a new topic halfway through, you drag the old context along with you. Sometimes starting a new conversation is more effective than carrying on in the same session.
Frequently asked questions about context engineering
Giving an LLM context touches on concepts you come across more and more often in AI providers’ documentation.
What is the difference between context engineering and prompt engineering?
The difference between context engineering and prompt engineering lies in scope: prompt engineering is about writing instructions, context engineering is about all the information the model receives during a task. In September 2025, Anthropic described context engineering as curating and maintaining the optimal set of tokens, including information outside the prompt itself. According to Anthropic, it is an ongoing process, whereas writing a prompt is a discrete task.
What is a system prompt?
A system prompt is an instruction given to a language model before a conversation starts, separate from the questions you ask afterwards. Anthropic recommends using it to set the model’s role, because that steers behaviour and tone; according to the documentation, even a single sentence makes a difference. In chat apps you set up something similar with fixed instructions, such as custom instructions in ChatGPT.
Can you save fixed context so you do not have to repeat it?
Yes, you can save fixed context in a project, so you do not have to provide background and documents again with every question. In Claude, each project has its own knowledge base with files and its own project instructions, and according to Anthropic this also works with a free account. On paid plans, Claude automatically switches to RAG when a project holds a lot of knowledge, which lets it hold up to ten times as much.
Context engineering as part of a repeatable way of working
Giving an LLM context is not a standalone skill but part of a broader way of working. If you invent every prompt from scratch, you get a different result every time. If you build fixed templates for recurring tasks and keep improving them based on the output, you build up a way of working that produces predictable results. That is exactly the difference between knowing a few prompts and using AI professionally.
In my own way of working, I moved from individual prompts to workflows when I saw errors ending up in production. A well-structured context with clear constraints prevents a large share of those errors. You catch the rest with checkpoints afterwards: read-back sessions, final editing and, in API work, automated validation of the output. Building context well is an individual skill, but as soon as a team uses the same structure for similar tasks, it becomes a shared way of working instead of a set of personal habits.
The ChatGPT Course covers this approach systematically and builds a way of working you can apply straight away in your own work. LearnLLM was set up to teach professionals repeatable ways of working with AI, rather than quick tricks that soon wear off.
