What a Fine-Tuning Run Actually Costs

The bill people quote is only one line item

When someone asks what a fine-tuning run costs, they usually mean the compute. How many GPU hours, what instance type, what's the total in pounds or dollars. That number is real and it matters for budgeting, but it is also the easiest part of the whole exercise to measure and the easiest to be misled by. A run that costs forty pounds in compute can still be a bad decision if the model it produces is worse than a simpler baseline, or if nobody can tell whether it's better at all.

I think the honest cost of a fine-tuning run has three parts: compute, data, and evaluation. Compute is visible and gets all the attention because it arrives as an invoice. Data preparation is semi-visible, it shows up as days of someone's time cleaning and labelling. Evaluation is the part that routinely gets skipped, not because it's unimportant but because it's unglamorous and because a sloppy evaluation will still produce a number, just not a trustworthy one.

The danger is that all three costs trade off against each other in ways that are easy to miss. Spend less time on data and you'll likely spend more time later debugging why the model behaves oddly. Spend less time on evaluation and you save nothing, you just defer the cost to production, where it shows up as silent failures instead of a line in a report.

A worked example of where the money actually goes

Suppose you're fine-tuning a mid-sized model on a classification task with twenty thousand labelled examples. The compute for a few epochs on a single reasonable GPU might take an afternoon and cost a modest amount, let's say comparable to a few hours of cloud rental. That's the number that goes in the budget spreadsheet.

But getting to twenty thousand clean, correctly labelled examples is rarely free. If even five percent of your labels are wrong, which is not an unusual rate for data labelled under time pressure, that's a thousand mislabelled examples quietly teaching the model the wrong thing. Auditing a sample of the data, checking label agreement, and fixing systematic errors can easily take longer than the training run itself. This work doesn't appear as a GPU bill, so it's tempting to shortcut it, but a model trained on noisy labels will plateau at a worse accuracy no matter how much compute you throw at it.

Then there's evaluation, which is where I think most of the real cost is hidden. Suppose your twenty thousand examples were split into train and test by simply shuffling and taking the last ten percent. If the data has any structure, duplicate near-identical entries, examples from the same source document, or time-ordered records, that shuffle can leak information from train into test. The test accuracy you report might be a few points higher than the model's true generalisation performance, and you won't know by how much unless you deliberately check for that leakage: grouping by source, holding out by time, or deduplicating near-matches before splitting. That checking work takes real hours, and skipping it means your headline number is simply wrong, not approximately right.

Add it up and the compute might be the smallest of the three costs in actual hours spent, even though it's the only one that shows up on an invoice.

Why the evaluation debt compounds

The reason I call it a debt you still owe is that an inadequate evaluation doesn't just give you an uncertain number, it actively misleads future decisions. If you report ninety-two percent accuracy on a leaky test set, and the true figure is eighty-six, every subsequent comparison against that baseline is distorted. A genuinely better model might look worse than the leaky baseline, and you'd reject it. A genuinely worse model might inherit the inflated number and get shipped.

A strong baseline suffers from the same neglect. It's common to compare a fine-tuned model only against a zero-shot or off-the-shelf version of the same base model, skipping a simple baseline like a logistic regression on bag-of-words features or a nearest-neighbour lookup. These baselines are nearly free to compute and often surprisingly competitive. If your fine-tuned model beats a weak comparison by a wide margin but only edges out the simple baseline by a point or two, that's the real signal about how much value the fine-tuning added, and it changes whether the compute spend was justified at all.

There's also the question of variance. Training a model once and reporting one test score tells you almost nothing about how reliable that score is. Running the same fine-tuning procedure three times with different seeds and reporting the spread is cheap relative to the original training cost, often adding only a fraction more compute, but it tells you whether your headline number is a stable property of the method or a lucky roll. Teams that skip this step are, in effect, borrowing confidence they haven't earned.

The practical takeaway

If you're budgeting a fine-tuning project, I'd suggest costing it in three columns rather than one: compute hours, data curation hours, and evaluation hours, including time for leakage checks, baseline comparisons, and repeated runs for variance. In my experience the data and evaluation columns often exceed the compute column, even though compute is the one everyone asks about first.

The payoff for doing this properly isn't just a more honest number. It's that when the fine-tuned model does get deployed, you actually know what it's good at, what it struggles with, and how much to trust the headline metric. That knowledge is the actual deliverable of a fine-tuning project. The model weights are just the artefact that carries it.

So the next time someone asks what a run costs, it's worth answering with all three numbers, not just the one on the invoice. The compute bill ends when the job finishes. The evaluation debt keeps accruing interest until someone pays it down properly.