Loading module…
Loading module…
Module 1 of 6
Prompt engineering is widely taught as a bag of tricks — magic phrases, secret words, templates that supposedly unlock hidden power. This is the wrong way to learn it, and it produces people who can recite prompts but cannot invent them, who are helpless the moment a trick stops working or a new model behaves differently. This course teaches prompting the other way: from the mechanism up. Every effective prompting technique works because of how a language model actually processes a prompt, and once you understand the processing, the techniques stop being tricks to memorize and become obvious consequences you could have derived yourself. This first module builds that understanding of the mechanism, so that everything after it — the techniques, the reasoning prompts, the structured outputs — lands as "of course, given how it works" rather than "another thing to memorize." The reward is enormous: you will be able to prompt any model, including future ones you have never seen, because you understand the thing you are prompting rather than a list of spells that worked on one model on one day.
If you have taken the foundational AI course, this module extends its treatment of language models directly into the practical question of how to direct them; if you have not, everything you need is built here from the ground up. Either way, the discipline is the same: understand the machine, and the skill of directing it follows. We begin with what a prompt actually is to a language model, which is not what most people imagine, and which explains nearly everything about why prompting works the way it does.
The first and most important correction to make is about what a prompt fundamentally is to a language model, because almost everyone imagines it wrongly, and the wrong picture makes prompting mysterious while the right picture makes it clear. Most people imagine a prompt as a command — an instruction you give to an intelligent agent that understands what you want and tries to obey. This picture is wrong, and holding it will mislead you at every turn. The correct picture is that a prompt is context — text that the model uses to generate a plausible continuation.
Recall (or learn) the core mechanism: a language model does exactly one thing, which is to predict what text should come next given the text so far. It was trained on vast quantities of human writing to become extremely good at this single task of continuation. When you give it a prompt, you are not issuing a command that it comprehends and executes; you are giving it text, and it generates the text that is most plausible to come next given what you provided. This is the entire operation, and everything about prompting follows from it. When you type "Write a poem about the ocean," the model does not "understand that you want a poem and decide to write one." It processes that text and generates the continuation that is statistically most plausible to follow it — and because, in its training data, text like "Write a poem about the ocean" was overwhelmingly followed by an actual poem about the ocean, the plausible continuation is a poem about the ocean. It looks like the model obeyed a command. What actually happened is that it continued the text in the way the patterns in its training made most plausible.
This distinction — context versus command — is not pedantic; it is the master key to all of prompting, and internalizing it changes how you write every prompt. If a prompt is a command, then prompting is about phrasing your orders clearly and hoping the model obeys, and when it fails you are stuck, because you gave a clear order and it disobeyed. But if a prompt is context that shapes a plausible continuation, then prompting is about providing the context that makes your desired output the most plausible continuation — which is a completely different and far more powerful way to think. Your job is not to command but to construct a textual situation in which what you want is the natural thing to come next. When a prompt fails, you do not conclude the model disobeyed; you conclude that the context you provided made some other continuation more plausible than the one you wanted, and you fix it by changing the context. This reframing turns every prompting problem into a solvable question — "what context would make my desired output the plausible continuation?" — rather than a mysterious failure of obedience.
Consider how this explains a common frustration. You ask a model a question and it gives a rambling, hedged, mediocre answer, and you think "why won't it just answer well?" The command picture leaves you stuck. The context picture explains it immediately: the text you provided made a rambling hedged answer the plausible continuation, probably because your prompt resembled the kind of context that, in the training data, was followed by rambling hedged text. To fix it, you change the context so that a sharp, direct, expert answer becomes the plausible continuation instead — perhaps by framing the situation as an expert responding concisely, which resembles training text that was followed by sharp expert answers. You are not commanding the model to be better; you are constructing a context in which being better is the natural continuation. This is the whole game, and it is why understanding "prompt as context" is the foundation everything else rests on. Every technique in this course is, at bottom, a way of shaping context so that your desired output becomes the most plausible thing to come next — and once you see that, the techniques become obvious rather than magical.
Key Insight: A prompt is not a command the model understands and obeys — it is context the model uses to generate a plausible continuation. The model does one thing: predict what text plausibly comes next given the text so far. This reframes all of prompting: your job is not to phrase orders clearly and hope for obedience, but to construct a textual situation in which your desired output is the natural thing to come next. When a prompt fails, the context made some other continuation more plausible — so you fix the context, not the command. This is the master key everything else follows from.
To prompt well you need to know what the model actually perceives when it reads your prompt, and it is not words as you understand them — it is tokens, and this seemingly technical detail has real practical consequences that explain several otherwise-baffling model behaviors. Understanding tokens is understanding the model's actual sensory world, which is different from yours in ways that matter.
A language model does not process text as characters or as whole words; it processes it as tokens, which are chunks of text — often whole common words, but frequently word-fragments for longer or rarer words, and sometimes individual characters or punctuation. The word "the" is one token; the word "unbelievable" might be split into several tokens like "un," "believ," "able." The model's entire perception of your prompt is a sequence of these tokens, and its entire output is generated one token at a time — it predicts the most plausible next token, appends it, predicts the next, and so on, building its response token by token. This is why, if you watch a model generate text, it appears one chunk at a time: you are watching the token-by-token generation directly. The model never sees your prompt as the meaningful whole you experience; it sees a sequence of tokens and generates a continuing sequence of tokens, and the meaning is something that emerges from the patterns, not something the model perceives directly.
This tokenization has concrete practical consequences that explain real behaviors and inform real prompting decisions. First, it explains why models are notoriously bad at tasks involving individual letters — counting the letters in a word, spelling manipulations, character-level operations. The model sees "strawberry" as a small number of tokens, not as a sequence of individual letters, so asking it to count the r's is asking it to work against the grain of its perception, operating on units (letters) it does not directly see. When a model miscounts letters, it is not stupid; it is perceiving in tokens and being asked about characters, which are below its resolution. Knowing this, you avoid relying on models for character-level tasks, or you structure the task to work with tokens rather than against them. Second, tokenization has cost and length consequences: models process and are often priced by tokens, and they have a limited token capacity (the context window, Section 4), so understanding roughly that text translates to tokens at a certain rate helps you reason about cost and length, though the specifics vary by model and change over time. Third, tokenization is why models handle different languages and formats with varying efficiency — text that tokenizes into more tokens costs more capacity and may be handled less fluently, which occasionally matters for prompting in less common languages or unusual formats.
The deeper practical lesson is to think in terms of what the model perceives, not what you perceive, because the gap between the two is the source of many prompting surprises. You perceive meaning, sentences, and letters; the model perceives a sequence of tokens and generates a continuation based on learned patterns over tokens. Most of the time this gap does not matter, because the patterns over tokens capture meaning well enough that the model behaves as if it understood your words. But at the edges — character-level tasks, unusual formatting, precise counting, anything operating below the token — the gap surfaces as failure, and understanding tokens lets you predict and avoid those failures rather than being surprised by them. You do not need to think about tokens constantly; most prompting is about meaning and context (Section 1). But you need to know that tokens are the model's actual medium, so that when a task lives below the token — letters, exact counts, character manipulation — you recognize you are asking the model to do something against its perceptual grain, and you adjust. The model's sensory world is made of tokens, and knowing that is knowing where its perception is sharp and where it is blind.
Key Insight: The model perceives your prompt not as words or letters but as tokens — chunks of text, often whole words but frequently fragments — and generates its output one token at a time. This is the model's actual sensory world, different from yours. It explains real behaviors: models are bad at letter-counting and character-level tasks because they see tokens, not letters, so those tasks run against their perceptual grain. Think in terms of what the model perceives, not what you do — most of the time the gap doesn't matter, but at the edges (character operations, exact counts) it surfaces as failure you can now predict and avoid.
A behavior that confuses and frustrates people learning to prompt is that the same prompt can produce different outputs on different tries — sometimes a good answer, sometimes a poor one, from identical input. This seems like unreliability or randomness for its own sake, but it is a direct and understandable consequence of how the model selects each token, and understanding it changes how you interpret and use model outputs.
Recall that the model, at each step, does not simply pick a single next token — it computes a probability for every possible next token, producing a whole distribution: after some text, it might assign 40% to one token, 25% to another, 15% to a third, and small probabilities across many others. The question is then how it selects from this distribution, and the answer is that it usually does not just take the single highest-probability token every time; instead it samples from the distribution, with a degree of randomness controlled by a setting often called temperature. At low temperature, the model heavily favors the highest-probability tokens, making it nearly deterministic and repetitive — it takes the safe, most-likely path. At higher temperature, it gives lower-probability tokens more chance of being selected, introducing variety and surprise. Because there is randomness in the selection, the same prompt run twice can take different paths — a different token selected early leads to a different continuation, which leads to a different response — and this is why identical prompts yield varying outputs. The variation is not a bug or malfunction; it is the deliberate randomness of the sampling process, and it is often desirable (it is what lets a model produce varied creative options rather than the same response every time).
This has several important practical implications for how you prompt and how you interpret results. First, a single output is a sample, not the model's definitive answer — it is one draw from a distribution of possible responses, and another draw might be better or worse. This means that if you get a poor response, regenerating may well produce a better one without any change to your prompt, simply because the next sample takes a better path; trying a prompt once and judging it on that single output can mislead you, because you saw one sample, not the range. It also means that a single good output does not prove your prompt reliably produces good outputs — you saw one lucky sample, and the next might disappoint. To judge a prompt's quality, especially for something you will use repeatedly, you look at several outputs, not one, because the prompt defines a distribution of responses and you want to know the distribution, not one point from it. Second, the temperature setting, where you can control it, is a real lever: for tasks where you want consistency, reliability, and the safest answer (factual tasks, precise instructions), lower temperature helps by making the model take the most-likely path; for tasks where you want variety, creativity, and exploration (brainstorming, creative writing, generating options), higher temperature helps by loosening the selection. Matching temperature to the task is a genuine and often-overlooked prompting technique that follows directly from understanding the sampling mechanism.
The deeper lesson connects back to what a language model fundamentally is. The randomness in generation reflects that the model is not retrieving a fixed correct answer from a store of knowledge — it is generating a plausible continuation, and there are many plausible continuations, so it samples among them. This is why consistency, which we take for granted in a person who knows something (ask them twice, get the same answer), is not guaranteed by the mechanism: the model is not consulting a settled belief but sampling from a distribution of plausible text, and different samples differ. Understanding this protects you from two errors: over-trusting a single good output (it was one sample, not a guarantee) and giving up after a single poor output (the next sample might be fine). It also reframes what your prompt does: your prompt shapes the distribution of likely continuations — a good prompt makes good continuations more probable across the distribution, so that more of your samples are good — rather than deterministically producing one output. You are not aiming a rifle at a single target; you are shaping a distribution so that most of what it produces is what you want. That is a subtler and more accurate picture of what prompting achieves, and it follows directly from understanding that the model samples from a distribution rather than retrieving an answer.
Key Insight: The same prompt gives different answers because the model samples each token from a probability distribution with controllable randomness (temperature), not by retrieving one fixed answer. So a single output is one sample, not the definitive answer — regenerating a poor response may yield a better one, and judging a prompt requires looking at several outputs, not one. Lower temperature for consistency and factual tasks, higher for variety and creativity. Your prompt shapes the whole distribution of likely continuations — making good ones more probable across the distribution — rather than aiming at a single target.
Every language model has a hard limit on how much text it can consider at once, called the context window, and this limit governs a whole class of model behaviors and prompting decisions in ways that are essential to understand. The context window is, in effect, the model's working memory — everything it can "see" and take into account at a given moment — and knowing its nature and its edges is central to prompting well, especially for anything involving long inputs or extended conversations.
The context window is the total amount of text — measured in tokens (Section 2) — that the model can process at once, encompassing both your prompt and the response it generates. Everything within the window, the model can attend to and use; everything outside it simply does not exist for the model. This has a stark and important consequence for conversations: in an extended back-and-forth, the whole conversation so far is fed to the model as context each turn, but once the conversation grows longer than the context window, the earliest parts fall outside it and the model can no longer see them — which is why a model can seem to "forget" what you said at the start of a long conversation. It has not forgotten in any human sense; those early words have simply scrolled out of the window and are no longer part of the context it processes, so they cannot influence its generation. Understanding this explains the behavior precisely and tells you how to handle it: if something from early in a long conversation matters, you may need to restate it, bringing it back into the current window, because the model can only use what is currently in the window, not what has scrolled out of it.
The context window governs several practical realities. First, there is a limit to how much you can give the model at once — a document too large to fit in the window cannot be processed whole, and you must either use a model with a larger window or break the input into pieces the window can hold. Knowing the rough capacity of the model you are using tells you what inputs are feasible. Second, what you put in the window is what the model reasons over, which is the foundation of a crucial prompting principle: to have the model work with specific information, put that information in the prompt, in the window, rather than assuming the model knows or remembers it. If you want the model to answer about a document, give it the document (in the window); if you want it to account for specific facts, state them in the prompt; if you want it to remember something from earlier, ensure it is still in the window. The model's reasoning is bounded by its window, so filling the window with the right context is much of the skill of prompting — a point that connects directly to Section 1's "prompt as context," because the context window is literally the space in which that context lives. Third, there can be subtleties in how well the model uses different parts of a large window — information at certain positions may be attended to more or less reliably than others — so for long contexts, the placement of critical information within the prompt can matter, and important instructions or information are often best placed where the model attends to them most reliably, which for many models means not buried in the middle of a very long context.
The unifying way to hold this is that the context window is the model's working memory for the task at hand, and prompting well means managing that working memory deliberately. You decide what goes into the window — the instructions, the relevant information, the examples, the conversation history that matters — and you recognize that anything not in the window cannot be used, no matter how relevant, because it is outside the model's perception for this generation. This makes context management a core prompting discipline: for a simple short prompt it is trivial (everything fits easily), but for complex tasks involving long documents, extended conversations, or lots of relevant information, deliberately managing what occupies the limited window — including what to leave out to make room for what matters, and where to place the most important content — becomes a real skill that determines success. Many prompting failures on complex tasks trace to context-window mismanagement: relevant information left out of the window, important instructions buried where the model attends poorly, or a conversation grown so long that critical early context has scrolled away. Understanding the window as bounded working memory, and managing it deliberately, prevents these failures and is foundational to prompting anything beyond the simplest tasks.
Key Insight: The context window is the model's working memory — the total text (in tokens) it can consider at once, covering both your prompt and its response — and everything outside it simply does not exist for the model. This explains why models "forget" the start of long conversations (that text scrolled out of the window) and bounds how much you can give at once. The core principle: the model reasons only over what is in the window, so put the information you want used in the prompt, manage what occupies the limited space deliberately (including what to leave out and where to place critical content), because anything outside the window cannot be used no matter how relevant.
A distinction that determines the reliability of everything a model tells you, and that governs how you should prompt for different purposes, is the difference between knowledge the model absorbed during training and knowledge you provide in the prompt. These two sources of "knowledge" have completely different reliability, and confusing them is the source of many prompting failures and much misplaced trust. Understanding the distinction is essential to prompting for anything where accuracy matters.
The first kind is trained-in knowledge — the patterns the model absorbed from its vast training data, which lets it answer questions and discuss topics from what it "learned" during training. When you ask a model a question without providing source material, it answers from this trained-in knowledge, generating a plausible continuation based on the patterns in its training. The critical property of this knowledge, established in the foundational course, is that it is fundamentally unreliable for specifics: the model has no model of truth, only patterns of plausible text, so it generates plausible-sounding answers that may be accurate (when the topic was well-represented in training, so plausible and true coincide) or may be confident fabrications (for specifics, obscure topics, or anything where plausible diverges from true). Trained-in knowledge is also frozen at training time — the model does not know about events after its training, and its knowledge of any topic is as of when it was trained. So when you prompt a model to answer from its trained-in knowledge, you are getting plausible continuations from frozen patterns with no guarantee of truth, which is fine for some purposes (brainstorming, general orientation, well-established facts) and dangerous for others (specific facts, recent information, anything you will rely on).
The second kind is provided knowledge — information you put directly into the prompt, in the context window, for the model to work with. When you paste a document and ask the model about it, or state facts in your prompt and ask the model to use them, the model is working from provided knowledge, and this is fundamentally more reliable, because the model is processing actual information in front of it rather than generating from frozen patterns. The answer is grounded in the source you provided, which you can verify against that source, and it is current (as current as what you provided) rather than frozen at training time. This is why a crucial prompting principle for accuracy is to provide the knowledge rather than relying on trained-in knowledge whenever accuracy matters: instead of asking "what does the law say about X?" (trained-in, unreliable), you provide the relevant legal text and ask "what does this text say about X?" (provided, grounded and verifiable). Shifting from trained-in to provided knowledge, by putting the relevant information in the prompt, is one of the highest-leverage moves in all of prompting for accuracy, because it changes the operation from unreliable generation-from-memory to reliable working-from-source.
The practical discipline that follows is to know, for every prompt, which kind of knowledge the model is using, and to prefer provided knowledge whenever accuracy matters. When you prompt without providing source material, you are getting trained-in knowledge — treat its specifics as unverified, useful for orientation and ideation but not to be trusted for facts without checking. When you prompt with source material in the context, you are getting provided knowledge — more reliable, grounded, verifiable against the source you gave. And you can deliberately choose which mode you are in: for tasks where the model's broad trained-in knowledge is enough (general explanation, brainstorming, well-established concepts), rely on it while holding its specifics lightly; for tasks where accuracy is essential (specific facts, current information, anything you will act on), provide the knowledge in the prompt and have the model work from it rather than from its memory. This choice — trained-in versus provided — is a fundamental prompting decision that determines the reliability of your output, and making it deliberately, rather than unconsciously always relying on trained-in knowledge, is what separates prompting that produces trustworthy results from prompting that produces confident-sounding unreliable ones. Much of the skill of prompting for real work is recognizing when you need provided knowledge and supplying it, rather than defaulting to the model's frozen, unreliable-for-specifics trained-in patterns.
Key Insight: A model has two kinds of knowledge with completely different reliability: trained-in knowledge (patterns absorbed during training — used when you ask without providing sources; unreliable for specifics because it's plausible-continuation from frozen patterns with no model of truth) and provided knowledge (information you put in the prompt for the model to work with — grounded, verifiable, current). The highest-leverage move for accuracy is to shift from trained-in to provided: instead of "what does the law say?" (unreliable memory), provide the text and ask "what does this say?" (reliable source-work). Know which kind every prompt uses, and prefer provided knowledge whenever accuracy matters.
We close by assembling the pieces into a single coherent mental model of what happens when you prompt a language model — because holding the whole picture at once is what turns the individual facts into the intuition that lets you prompt well and invent your own techniques. This mental model is the foundation the entire rest of the course builds on, and getting it clear now is what makes everything after it obvious.
Here is the complete picture. You provide a prompt, which the model perceives as a sequence of tokens (Section 2) occupying its context window (Section 4) — its bounded working memory for this task. The model then generates a response one token at a time, at each step computing a probability distribution over possible next tokens and sampling from it with some randomness (Section 3), each generated token becoming part of the context for generating the next. The generation is fundamentally continuation: the model produces the text most plausible to follow what is in its context, based on the patterns it absorbed in training (Section 1). The "knowledge" informing this continuation comes either from those trained-in patterns (unreliable for specifics) or from information you provided in the context (grounded and reliable) (Section 5). And the whole process is bounded by the context window: only what is in the window influences the generation. That is the complete mechanism of what happens when you prompt, and every prompting technique in this course is a way of working with this mechanism to make your desired output the plausible continuation.
From this picture, the master principle of all prompting follows directly and can now be stated in full: prompting is the art of constructing context so that your desired output is the most plausible continuation, across the distribution of possible continuations, using knowledge you provide where accuracy matters, within the bounded working memory of the context window. Every technique you will learn is an application of this principle. Providing examples works because examples in the context make outputs matching them the plausible continuation. Assigning a role works because role-framing resembles training text that was followed by the kind of output that role produces. Asking for step-by-step reasoning works because it makes a reasoned continuation plausible rather than a jump-to-conclusion one. Structuring your request works because clear structure in the context makes structured output the plausible continuation. Providing source material works because it shifts the knowledge from unreliable trained-in patterns to reliable provided information. None of these are tricks; all of them are ways of shaping the context so that the plausible continuation is what you want — which is the only thing prompting ever does, because continuation-from-context is the only thing the model ever does.
This mental model is what makes you a prompt engineer rather than a prompt memorizer, and the distinction matters enormously for your capability. The memorizer collects prompts and templates that worked, applies them by rote, and is helpless when they fail or when a new model behaves differently, because they never understood why the prompts worked. The engineer understands the mechanism, so for any task they can reason from first principles: "what context would make my desired output the plausible continuation? what knowledge do I need to provide? what would make the good output more probable across the distribution? what must be in the window?" — and construct a prompt to achieve it, inventing the technique the situation needs rather than reaching for a memorized template. When a prompt fails, the engineer diagnoses it through the mechanism ("the context made some other continuation more plausible; let me change the context"), while the memorizer is stuck. And when models change, as they constantly do, the engineer's understanding transfers — because all models work by this same continuation-from-context mechanism — while the memorizer's specific templates may not. This is why the whole course is built mechanism-first: not because the mechanism is interesting in itself (though it is), but because understanding it is what makes you able to prompt anything, invent techniques, diagnose failures, and adapt to new models — a durable capability rather than a perishable list of spells. Hold this mental model firmly, and everything that follows will be not a set of tricks to memorize but a set of obvious consequences you understand and could have derived yourself. That understanding is the goal, and it begins here, with a clear picture of what actually happens when you prompt.
Key Insight: The complete mental model: you provide a prompt (perceived as tokens in the bounded context window), and the model generates a continuation one token at a time by sampling from a probability distribution, producing the text most plausible to follow the context, informed by either unreliable trained-in patterns or reliable provided knowledge. From this follows the master principle: prompting is constructing context so your desired output is the most plausible continuation across the distribution, using provided knowledge where accuracy matters, within the context window. Every technique is an application of this. Understanding it makes you a prompt engineer who can invent techniques and adapt to any model — not a memorizer helpless when templates fail.
You now understand what actually happens when you prompt a language model, which is the foundation everything else builds on. You know that a prompt is context, not command — text that shapes a plausible continuation, not an order to be obeyed. You know the model perceives tokens, not words or letters, and where that gap surfaces as failure. You know why the same prompt gives different answers — sampling from a distribution with controllable randomness — and what that means for interpreting and judging outputs. You know the context window is bounded working memory that governs what the model can use. You know the crucial difference between unreliable trained-in knowledge and reliable provided knowledge. And you hold the complete mental model — continuation from context — from which the master principle of all prompting follows: construct context so your desired output is the plausible continuation.
The next module, Basic Prompting Techniques, takes this mechanism and derives the foundational techniques from it — specificity, providing examples, clear instructions, and the other core moves that dramatically improve output. Because you now understand the mechanism, each technique will land as an obvious consequence: you will see why specificity works (it narrows the plausible continuation toward your target), why examples work (they make matching outputs the plausible continuation), why clear structure works (it shapes the continuation's form). You will not be memorizing techniques; you will be recognizing the natural implications of the mechanism you now understand. That is the payoff of building the foundation first, and it compounds through every module that follows, all the way to the advanced techniques — each one, at bottom, a way of shaping context so that what you want becomes the plausible thing to come next.