Fine-Tuning vs Prompting vs RAG: A Decision Framework, Not a Trend Chart
When to use prompting, RAG or fine-tuning, what each changes in a model, what they cost, and why most teams should try them in that order.

Every few months the conversation shifts. First prompting was everything. Then RAG was everything. Then fine-tuning came back. Trend charts are not a strategy. The three approaches solve different problems, and choosing between them is a question about your task, not about what is fashionable.
What each one actually changes
Prompting changes the instructions and examples you send with each request. The model stays the same. It is fast to try, cheap to change and easy to roll back.
Retrieval-augmented generation (RAG) adds relevant information from your own documents to each request. The model stays the same; it gets better context. It is the right tool when the model lacks knowledge.
Fine-tuning changes the model itself by training it further on your examples. It shapes behaviour, format and style, and can make a smaller model perform a narrow task as well as a larger one.
Put simply: prompting changes the instructions, RAG changes the knowledge available, fine-tuning changes the behaviour.
The decision framework
Start from the problem, not the technique.
The model does not know something it needs to know. Your policies, products, documentation, customer history. → RAG. Knowledge changes, and RAG lets you update a document rather than retrain a model. Answers can cite sources, which builds trust and makes errors visible.
The model knows enough but does not follow instructions consistently. → Better prompting first: clearer instructions, good examples, structured outputs, splitting the task into steps. See prompt engineering as a system.
Prompting has been measured and still falls short on format, tone or behaviour. → Consider fine-tuning.
Cost or latency at high volume is the problem. A fine-tuned smaller model can sometimes replace a larger general model for a narrow task. → Consider fine-tuning, and measure the saving against the ongoing maintenance.
Specialised classification or extraction with lots of labelled data. → Fine-tuning often shines here.
Why fine-tuning is not for facts
A common mistake is fine-tuning a model on company documents so it "knows" them. Fine-tuning is good at teaching patterns of behaviour; it is much less reliable at adding facts that must be recalled accurately. And when those facts change, the model needs retraining.
For knowledge, RAG is almost always the better tool. It is cheaper to update, and it can show where each answer came from.
Try them in order
- Prompting, with an evaluation set of real inputs and accepted outputs.
- Add RAG if failures are caused by missing or outdated knowledge.
- Consider fine-tuning only if a measured gap remains that the first two cannot close.
Each step is more expensive to build and maintain than the last. Most teams find that steps one and two meet their quality bar. For the running cost of step two, see what a RAG system costs per month.
What fine-tuning really requires
If the evidence points to fine-tuning, plan for:
- Data: at least hundreds of high-quality, consistent examples, often thousands. Inconsistent examples teach inconsistent behaviour.
- A training pipeline with versioned datasets and reproducible runs.
- Evaluation of every fine-tuned version against the same set, compared with the prompting baseline.
- Retraining when the base model is updated or retired by the provider.
- Deployment and monitoring of the fine-tuned model, like any other release.
Fine-tuning is an ongoing capability, not a one-off task. For how to run that capability reliably, see MLOps for getting models to production.
Combining approaches
The approaches are not exclusive. A fine-tuned model can be better at using retrieved context in your preferred format, while RAG supplies current facts. That combination is powerful and the most expensive to run. Use it when simpler setups have been measured and fall short.
Decide with an evaluation set
The only reliable way to choose is to test. Build an evaluation set, then compare approaches on quality by category, cost per request and latency. The numbers usually make the decision obvious, and they give you a baseline to measure future changes against.
Three examples
A support assistant that must answer from policy documents. The model lacks the knowledge, and policies change. → Prompting plus RAG. Fine-tuning would bake in facts that go stale.
Classifying incoming emails into forty categories at high volume. Prompting works but is expensive at scale, and there are thousands of labelled historical emails. → Fine-tune a smaller model, compare against the prompting baseline on cost and accuracy, and keep the prompting version as a fallback.
Drafting product descriptions in a strict house style. Prompting with good examples gets close; a measured gap remains in tone consistency. → Consider fine-tuning on a few hundred approved descriptions, then re-evaluate.
In each case the decision came from the task and the evaluation numbers, not from which approach was fashionable.
Mistakes to avoid
- Fine-tuning before building an evaluation set, so there is no baseline to beat.
- Fine-tuning on documents to add knowledge, instead of using retrieval.
- Using inconsistent or low-quality examples as training data.
- Forgetting that fine-tuned models need retraining when the base model is retired.
Start with the cheapest thing that could work
Our first fixed-scope project is two to four weeks: we build the evaluation set, test prompting and, if needed, RAG, and report the numbers. Fine-tuning comes later, if the evidence supports it.
We deliver this for teams across India, the United States, the United Kingdom, the UAE and Canada; see AI and ML services.
Choosing an approach, in numbers
Blurbee's assessment, scored out of 10. Effort covers build and maintenance; opinion based on our engagements.
Why teams choose their AI approach with Blurbee
We start with the cheapest approach that could work, measure it against your own examples, and move to more complex options only when the numbers say so. No fine-tuning because it sounds advanced.
- Fixed-scope first projectTwo to four weeks, priced as a number rather than an estimate, so you have evidence before a longer commitment.
- One delivery lead, namedYou always know who to ask. No account manager relaying questions to a team you never meet.
- Code and IP are yoursAssigned in writing, handed over with repositories, infrastructure definitions and documentation whether or not you continue with us.
- Performance budgets in CICore Web Vitals and bundle size gate the build. Slow does not ship.
- Accessibility to WCAG 2.2 AAReviewed with automated and manual passes, not claimed on a slide.
- Written overlap disciplineStanding calls inside your working window, everything else asynchronous with a decision log.