The Prompt Engineering Skill Set: What Transfers

Why this distinction matters

Every few months a new model ships and a chunk of the internet's prompt advice quietly stops working. Phrases that reliably nudged one model towards structured output do nothing on the next, or worse, make it verbose and evasive. If you have spent time collecting these incantations, it is worth asking honestly how much of that time built a durable skill and how much built trivia about a specific checkpoint.

I think this matters because teams often hire or train for prompt engineering as if it were one coherent competency, when in practice it is a bundle of at least three different things: knowledge of a particular model's quirks, general skill at specifying tasks precisely, and skill at evaluating whether an output is actually good. Only the last two travel well. The first is closer to knowing a specific piece of software than to a durable engineering skill, and it depreciates on a timescale of months.

Separating these matters practically because it changes what you invest in. If you train juniors purely on phrasing tricks for the model your company currently licenses, that investment evaporates the day procurement switches vendors. If you train them on decomposition, constraint specification and evaluation design, that knowledge survives the switch and often improves it, because those skills are exactly what let you diagnose why the new model behaves differently.

What does not transfer

The most obviously non-transferable layer is model-specific incantation: telling a model it is an expert, promising it a reward, or using a particular emoji as a delimiter because someone found it worked on a leaderboard once. These behaviours are artefacts of how a specific model was fine-tuned and aligned, not properties of language or reasoning. A trick that improved accuracy on one model by three percentage points can do nothing, or actively hurt, on the next version from the same vendor, because the fine-tuning data and reinforcement signal changed.

Token-level superstition is a related trap: exact phrasing of system messages, specific formatting of few-shot examples, or magic words that supposedly unlock a hidden mode. Some of this reflects real mechanics, such as models being sensitive to example ordering, but the specific sensitivity pattern is rarely stable across model families. I have seen a delimiter choice that mattered enormously for one provider's model make no measurable difference for another, tested on the same fifty held-out prompts with the same scoring rubric.

Version-specific workarounds for known weaknesses are the third category. If a model struggles with a particular arithmetic pattern or a particular date format, people build elaborate scaffolding around that specific failure mode. That scaffolding is dead weight the moment the weakness is patched, and worse, it can mask genuine improvements because your evaluation harness still routes around a problem that no longer exists.

What does transfer

Task decomposition is durable because it reflects something true about problem structure, not about any model. Breaking a vague request such as 'summarise this contract and flag risks' into explicit sub-steps, extract clauses, classify clause type, check against a risk checklist, produce a summary, is a skill that improves output quality on essentially any sufficiently capable model, because it reduces the chance that the model silently skips a step under time or attention pressure. This is closer to systems analysis than to conjuring, and it does not go stale.

Precise constraint specification is the second durable skill. Saying 'answer in under 100 words, cite only facts present in the source, and say insufficient information if the source does not cover the question' does real work regardless of which model receives it, because it narrows the output space and gives you something concrete to check. The exact wording that maximises compliance may shift slightly between models, but the discipline of writing explicit, checkable constraints instead of vague aspirations transfers completely.

The third and most undervalued transferable skill is building a small, honest evaluation set before you start tweaking prompts at all. Say you are building a support-ticket triage prompt: pulling together forty real tickets with agreed correct categories, holding out ten as a check set you do not look at while iterating, and scoring candidate prompts against the other thirty, is a habit that works identically whether you are prompting an open model on your own hardware or a closed API. It also protects you from the classic failure of tuning a prompt until it looks good on the five examples you happened to glance at, then watching it fail in production. That habit of building a leakage-aware, representative test set before optimising is arguably the single most transferable skill in the whole discipline, because it is really just applied measurement discipline borrowed from good experimental practice, and it works the same way whether you are testing a prompt, a model or a whole pipeline.

A practical takeaway

When you notice yourself getting good at prompting a particular model, pause and sort what you have learned into two piles. One pile is genuinely about specifying tasks and constraints more precisely, and about measuring output quality honestly against a held-out set you did not overfit to by eye. The other pile is folklore tied to a checkpoint that will be retired. Write the first pile down as reusable process documentation. Do not bother memorising the second; it will not survive the next model update anyway.

If you manage a team, the same split should shape how you evaluate people. Someone who can decompose a messy task, write checkable constraints and build a small held-out test set before tuning anything is bringing a skill that will still be valuable in two years, regardless of which vendor you use by then. Someone who only knows the current magic words is bringing something closer to a temporary cheat code, useful today, worthless soon. Invest your training budget accordingly.