Diagnose what is missing
If the model does not have the operative agreement, improve the source route. If it has the agreement but the task is vague, improve the brief. If a stable, repeated behavior remains poor despite good instructions and examples, training may be worth investigating. These interventions address different problems.
Fine-tuning is additional training: taking an already trained model and giving it many example input and output pairs, often thousands or more. [source] LoRA is a method that trains relatively small update matrices while keeping original model weights frozen. It can reduce the number of parameters that need training. [source] Quantization concerns numerical representation and memory; it is not a mechanism for adding your matter documents to an application's search. [source]
Use a decision table
| Observed problem | First thing to investigate |
|---|---|
| Wrong or missing source | Ingestion, versions, retrieval, or access |
| Inconsistent output format | Clearer schema, examples, structured outputs, and validation |
| Repeated style issue | Specific instructions and accepted examples |
| Narrow classification task at scale | Measured baseline; consider training if justified |
| Slow local model | Placement, model size, context, runtime, and workload |
The table proposes an order of investigation. It does not establish that prompting can solve every behavior problem or that training is always needed for classification.
Why prompting comes first
Language models learn new tasks from the prompt itself: from an instruction alone (zero-shot), one example (one-shot), or a few examples (few-shot). [source] Teaching through the prompt takes minutes to change and test. Fine-tuning has these downsides: [source]
- Overhead. You must create and clean a large set of examples.
- Velocity. Every new capability means new data, a new training run, and a new model to test, instead of a few new lines in a prompt.
- It does not replace prompts. You still need to supply the matter's current documents and facts at run time. Fine-tuning can make prompts shorter and behavior more consistent, but it is not a filing system.
- Availability and cost change. In 2023 the leading chat models could not be fine-tuned through their vendors' APIs at all, and fine-tuned models cost several times more per use than stock models. [source] As of October 2026, OpenAI's documentation lists supervised, preference, and reinforcement fine-tuning methods, but also states that it is winding down its fine-tuning platform and no longer accepts new users. [source] Check current terms before planning a project around any vendor's training service.
Keep learning datasets separate from client records
A fine-tuned model can memorize parts of its training data and reproduce them to other users who should never see them. Use synthetic data for training sets. [source] If accepted and rejected drafts contain confidential facts, do not assume they are suitable training material simply because the original task was authorized. Determine the actual permission and architecture. Avoiding fine-tuning prevents this particular route for adding your supplied client data to weights; it does not prevent disclosure through prompts, logs, retrieval, or existing training data.
Work a style example
Suppose the model routinely turns qualified statements into categorical ones. Start with a rule and three contrasting examples. Test on fresh sentences. If the failure persists across a large, stable workload, consider whether a curated training set could improve behavior.
Before training, write a baseline evaluation. Separate examples used for development from held-out evaluation.
Understand the continuing maintenance
A trained adaptation can reproduce an unwanted pattern as readily as a useful one. Label quality, dataset balance, and version tracking matter. If the task changes, old training may encode assumptions you no longer want. Keep the baseline and compare against both old and new cases.
For changing matter facts, a source system with identifiable versions and corrections is usually easier to audit than trying to make model weights serve as the authoritative record. The practical question is how an error can be found, corrected, and traced through later outputs.
A useful stopping rule
Do not fine-tune merely because the technique is available. State the target behavior, the measured baseline, the lawful and suitable dataset, the evaluation method, and the cost of maintaining it. If those are undefined, you do not yet have a training project; you have a problem that still needs diagnosis.