What is Fine-Tuning?

Why It Matters

Fine-tuning matters because it lets a company adapt a powerful general-purpose model to its own specific terminology, tone, or task without the cost and complexity of training a model from scratch. That distinction is what makes fine-tuning practical for most companies, since training from scratch requires far more data and compute than most organizations have access to. Knowing when fine-tuning is worth the investment versus when prompt engineering will do matters directly for project timelines and budget, since fine-tuning is typically slower and more expensive to iterate on. It's also the mechanism that closes accuracy gaps prompting alone can't fix, particularly for narrow, specialized domains like a company's own product jargon or internal support conventions. Skipping this evaluation and jumping straight to fine-tuning when prompting would have worked is a common source of wasted effort.

Example

A general-purpose LLM correctly answers broad customer questions but struggles with a software company's specific product terminology and internal jargon. After fine-tuning the model on 5,000 historical support tickets specific to that product, its accuracy on product-specific questions rises measurably, while its general-purpose capabilities stay largely intact. Fine-tuning is typically more expensive and slower to iterate on than prompt engineering, which is why teams usually try prompt engineering first and only fine-tune when prompting alone can't close the accuracy gap.

Frequently Asked Questions

• How is Fine-Tuning different from prompt engineering?

  Prompt engineering changes only the instructions given to a model at the moment of use, with no change to the model itself. Fine-tuning actually retrains the model's underlying parameters on new examples, which is a deeper and more permanent change, but also slower and more expensive to iterate on.

• How much data does Fine-Tuning require?

  It's typically much less than training a model from scratch, since fine-tuning builds on a model that's already broadly capable. The exact amount varies by task, but it can range from hundreds to several thousand well-chosen, task-specific examples.

• When should a team choose Fine-Tuning over prompt engineering?

  Fine-tuning is usually the right call once prompt engineering has been tried and still can't close the accuracy gap on a specific task, particularly one involving narrow domain terminology, a specific tone, or a pattern that's hard to fully specify in a prompt.

• How is Fine-Tuning success measured?

  It's measured by comparing task-specific accuracy or evaluation scores on a held-out test set before and after fine-tuning, checking that the improvement is real and that general-purpose capabilities haven't meaningfully degraded in the process.

• Does Fine-Tuning risk making a model worse at general tasks?

  It can, a phenomenon sometimes called catastrophic forgetting, where a model over-specializes on the fine-tuning data and loses some of its broader capability. That's why teams typically test general performance alongside task-specific performance after fine-tuning, not just the narrow metric being improved.