A Plain Guide to Fine-Tuning LLMs

Why the distinction matters

People talk about fine-tuning as if it teaches a model something new, the way a textbook teaches a student. In practice it usually does something narrower: it nudges a model that already has broad competence toward a particular style, format, or task distribution. The model is not learning language again. It already learned language, facts, and reasoning patterns during pretraining on a huge, general corpus. Fine-tuning adjusts a comparatively small number of weight updates on top of that foundation, and understanding which weights move, and which stay frozen, explains both the power and the limits of the approach.

This matters practically because teams routinely over-promise what fine-tuning can fix. If a base model has never seen a domain's vocabulary or reasoning pattern at all, a modest fine-tuning run will not implant deep new knowledge; it will mostly teach the model to produce outputs in a certain shape using what it already knows, sometimes badly if the underlying knowledge is not there. Conversely, if the base model already has the relevant knowledge but expresses it inconsistently, fine-tuning can be remarkably effective because it is steering something that already exists rather than building it from nothing.

Think of pretraining as building a vast library with a research assistant who has read almost everything and can speak in almost any register. Fine-tuning is more like giving that assistant a short briefing on house style, preferred tone, and the exact format you want reports in. The assistant does not relearn how to read; they adjust which parts of their knowledge they foreground and how they phrase the answer. That is a useful mental model to keep in your head for the rest of this post.

What actually changes during fine-tuning

In full fine-tuning, every parameter in the network is technically eligible to update, but gradient magnitudes are not uniform across layers. Later layers, closer to the output, tend to shift more because they are closer to the loss signal and because they encode more task-specific, less general representations. Early layers, which tend to capture broad syntactic and low-level statistical patterns, often shift comparatively little unless the fine-tuning dataset is large and very different from the pretraining distribution. This is not a hard rule, but it is a reasonable intuition backed by how backpropagation distributes gradient signal through depth.

Consider a concrete, illustrative scenario. Suppose you have a base model pretrained on a broad web-scale corpus, and you fine-tune it on twenty thousand examples of customer support transcripts formatted in a specific question-and-answer style. After fine-tuning, the model reliably produces answers in that format, uses the company's preferred tone, and references the kinds of issues present in the transcripts. What it has not necessarily gained is new factual knowledge about the company's products beyond what appeared in those twenty thousand examples. If a customer asks about a product released after the fine-tuning data was collected, the model will still answer fluently, because fluency was never the limiting factor, but it may confidently invent details, because the underlying facts were never there to begin with.

This is where parameter-efficient fine-tuning methods, such as low-rank adaptation, make the picture even clearer. Instead of updating all the weights, these methods freeze the entire pretrained network and insert small trainable matrices alongside existing weight matrices, often with a tiny fraction of the total parameter count, sometimes well under one percent. The frozen base still contains essentially all of the model's world knowledge and general capability. The small trainable addition learns a targeted transformation: a kind of steering vector applied at each relevant layer. This makes explicit what was always true in spirit, even in full fine-tuning: most of what the model knows is untouched, and what you are really training is a relatively compact adjustment layered on top.

What stays frozen, and why that is usually a feature

The vast bulk of the network, its embeddings, its general attention patterns, its learned associations between concepts, typically remains close to its pretrained state even after fine-tuning, especially with parameter-efficient methods or with full fine-tuning on modest data volumes. This is desirable more often than people assume. If fine-tuning rewired the whole network every time, you would risk catastrophic forgetting, where the model loses general capabilities it had before, in exchange for narrow gains on the new task. A support bot fine-tuned aggressively on a narrow dataset can become excellent at that dataset's patterns while becoming noticeably worse at general reasoning or at handling questions slightly outside its training distribution.

This also explains why small, well-curated fine-tuning sets often outperform large, noisy ones. If the goal is to adjust behaviour rather than rebuild knowledge, you do not need millions of examples; you need examples that clearly demonstrate the target format, tone, and edge-case handling, without contradicting what the base model already knows reliably. A few thousand carefully written examples that consistently demonstrate a desired behaviour can outperform a much larger dataset scraped together with inconsistent labelling, because the inconsistent dataset sends a noisier gradient signal and risks degrading capabilities that were previously sound.

The practical implication for evaluation is important too. If you fine-tune a model and test it only on examples drawn from the same distribution as the fine-tuning data, you will likely see strong improvement, but that improvement tells you little about whether the model's general capability improved or degraded elsewhere. A rigorous evaluation should include held-out data from the fine-tuning distribution, held-out data from the original general distribution, and ideally a few adversarial or out-of-domain probes that were never part of training. Without that split, it is easy to mistake narrow overfitting for genuine improvement, and to miss regressions in capabilities the model used to have.

A practical takeaway

If you remember one thing, let it be this: fine-tuning is far better understood as steering than as teaching. The base model's pretrained weights encode the overwhelming majority of what it knows and can do, and fine-tuning, whether full or parameter-efficient, typically adjusts a comparatively small set of behaviours layered on top of that foundation. This is why fine-tuning works brilliantly for format, tone, and task specialisation, and why it is a poor tool for injecting large amounts of new factual knowledge that was never present at pretraining time.

Before investing in a fine-tuning run, it is worth asking plainly what you are actually trying to change: behaviour, or knowledge. If it is knowledge, retrieval-based approaches or additional pretraining on relevant data are usually more honest tools. If it is behaviour, format, or consistency, fine-tuning on a carefully curated, leakage-free dataset, evaluated against both in-distribution and out-of-distribution held-out sets, is the right instinct. Knowing which parameters move and which stay frozen is not just a mechanistic curiosity; it is the difference between designing a fine-tuning project that works as intended and one that quietly breaks things you never tested for.