Prompt Engineering From First Principles: What It Covers

Why the term needs unpacking

Prompt engineering has become one of those phrases that gets used so widely it has nearly stopped meaning anything specific. To some people it is a synonym for typing questions into a chat window with more patience than usual. To others it is a discipline with its own folklore of magic phrases, where adding the words 'think step by step' or threatening the model with a bad performance review is treated as serious methodology. Neither extreme is useful if you actually have to build something that works reliably.

Underneath the hype, there is a real engineering problem. A large language model does not have access to your intentions, your constraints, or the implicit context you are carrying around in your head. All it has is the text you give it, plus whatever it learned during training. Prompt engineering, stripped of the mysticism, is the practice of constructing that text so the model's output distribution lines up with what you actually need. That is it. It is specification work, not incantation.

I think the confusion arises because the same surface activity, writing a block of text before an answer appears, can sit at wildly different levels of rigour. Someone tweaking a single prompt until an example looks good is doing something qualitatively different from someone designing a prompt template, testing it against a held-out set of cases, and measuring failure modes. Both get called prompt engineering. Only one of them generalises.

The three things a prompt is actually doing

It helps to break a prompt down into distinct jobs rather than treating it as one undifferentiated string. The first job is task specification: telling the model what output format you want, what the success criteria are, and what it should do when it is unsure. The second job is context provisioning: giving the model the facts, examples, or constraints it cannot be expected to know or infer on its own. The third job is behavioural framing: nudging the model towards a style, persona, or reasoning pattern that tends to correlate with better answers for that task.

Consider a concrete case: you want a model to extract structured data from customer support emails, pulling out the product name, the issue category, and the urgency, formatted as JSON. The specification part of the prompt states the exact schema and field names, and says what to put in a field when the information is missing, perhaps a null rather than a guess. The context part might include two or three worked examples of emails and their correct JSON output, chosen to cover edge cases like multi-product complaints. The framing part might instruct the model to act as a careful classifier rather than a conversational assistant, which in practice changes how verbose and hedging its answers are.

If you ran an evaluation on five hundred real support emails, you would typically find that schema violations, like producing a string where you specified a number, drop sharply once the specification is explicit and includes a worked example, often from double digits in percentage terms down to a handful of cases. That drop is not magic. It is the direct, explainable result of narrowing the space of plausible continuations the model considers. A vague prompt leaves that space wide open; a precise one collapses it towards the structure you actually want.

This is also where the folklore version of prompt engineering falls apart. Phrases that supposedly unlock better reasoning are not doing anything structurally different from a well-specified instruction; where they appear to help, it is usually because they implicitly add a small amount of task decomposition or context that the original prompt was missing. The fix is rarely the incantation itself, it is making explicit something the prompt had left implicit.

Few-shot examples, formatting, and the evaluation problem

A large part of practical prompt engineering is choosing and arranging examples, often called few-shot demonstrations. The intuition is straightforward: showing the model two or three instances of input paired with correctly formatted output anchors its behaviour far more reliably than describing the format in prose. The subtlety is that the choice of examples matters enormously, and changing their order, their count, or which edge cases they cover can shift output quality in ways that are easy to miss if you are only eyeballing a handful of outputs.

This is where leakage-style thinking becomes essential even outside formal machine learning pipelines. If your few-shot examples are drawn from the same distribution as your test cases, or worse, overlap with them, you will convince yourself a prompt works far better than it does. I would treat prompt development the same way I would treat model development: hold out a representative sample of real cases you never look at while iterating, and only use it at the end to check that your prompt's measured performance was not just overfitting to the three examples you kept staring at.

The other discipline worth borrowing from evaluation practice is defining success criteria before you start iterating, not after. For the support-email example, that might mean specifying in advance that a correct output must have a valid JSON schema, the right issue category from a fixed list, and an urgency label that matches a human annotator's judgement at least a stated proportion of the time. Without that upfront definition, it is very easy to keep adjusting the prompt until a handful of favourite examples look right, while quietly degrading performance on the cases you are not actively watching.

What this means in practice

Treat prompt engineering as applied specification and evaluation, not as a search for secret phrases. The actual skill is in being precise about what you want, in choosing representative examples rather than convenient ones, and in measuring outcomes on a held-out set rather than trusting your impression from a few trial runs. A prompt that was tuned against three examples and never checked against anything else is not an engineered artefact, it is an anecdote.

If you want a quick practical checklist: write the specification down explicitly, including the format and the failure behaviour; choose examples that cover the awkward cases rather than the easy ones; separate your development cases from your evaluation cases; and define what counts as correct before you start iterating. None of this requires any special vocabulary or trick phrasing. It requires the same discipline that makes any other piece of engineering reliable, and that discipline is the part of prompt engineering actually worth the name.