RAG Without the Hype: What It Actually Costs to Run a Retrieval System Per Month
The monthly bill for a RAG system, line by line: model tokens, embeddings, vector storage, hosting, evaluation and the people who keep answers right.

Most RAG proposals price the build and say little about the monthly bill. Then the system launches, usage grows, and someone in finance asks why the AI line keeps moving. The cost of running a retrieval-augmented generation system is predictable if you model it properly, and it is rarely where people expect.
This is the line-by-line view. For how to build RAG that answers correctly in the first place, see our RAG and knowledge base guide.
The six cost lines
Every production RAG system has the same six costs:
- Generation tokens. The language model reads the question plus retrieved passages and writes an answer.
- Embeddings. Documents are converted to vectors once; each new query is embedded at request time.
- Vector storage and search. Where the vectors live and how fast they can be searched.
- Hosting and compute. The application, ingestion pipelines, OCR and any self-hosted models.
- Evaluation. Running the evaluation set on every change.
- People. Reviewing answers, fixing retrieval, updating content.
The first five are infrastructure. The sixth is often the largest, and the one most proposals leave out.
Generation: input tokens dominate
Each query sends the model a system prompt, the retrieved passages and the question. A typical request might carry several thousand input tokens of context and produce a few hundred output tokens. Even though output tokens usually cost more per token, the volume of input makes it the bigger share.
That makes retrieval quality a cost lever as well as a quality lever. Sending five precise passages instead of fifteen loose ones cuts generation cost and usually improves the answer.
A simple formula:
Monthly generation cost = queries × (input tokens × input price + output tokens × output price)
Use your own measured token counts and your provider's current prices. Model prices change often, and usually downward, so re-run the numbers every quarter.
Embeddings: mostly a one-off
Embedding the initial corpus is a one-time cost, usually small. Afterwards you pay only for new or changed documents and for each incoming query, which are short.
The exception is switching embedding models. Vectors from different models are not compatible, so a switch means re-embedding everything. Choose deliberately and budget for the occasional migration.
Storage and search
For many business knowledge bases, PostgreSQL with the pgvector extension is enough. It runs on a database you probably already pay for and operate. Dedicated vector databases earn their cost at larger scale, very high query rates, or when you need specific features such as advanced filtering at scale.
Storage cost scales with the number of chunks times vector dimensions. Smaller chunks mean more vectors; very large embedding dimensions mean more storage per vector. Neither usually dominates the bill.
Hosting and compute
The application server, the ingestion pipeline that syncs and re-indexes documents, OCR for scanned files, and reranking all need compute. Ingestion is spiky: heavy when a large batch arrives, light the rest of the time. Serverless or scheduled jobs avoid paying for idle servers.
Self-hosted versus hosted models
Self-hosting an open-weights model on a GPU server can be cheaper at high, steady volume. At low or spiky volume it is usually more expensive, because the server costs the same whether or not anyone is asking questions, and someone has to run it.
Compare using your real traffic pattern and include engineering time. Many teams start hosted, measure, and move specific high-volume steps to a self-hosted model later.
Evaluation: cheap in tokens, important in practice
Running a few hundred evaluation questions on each change costs little in tokens. The real cost is people time: maintaining the set and reviewing failures. It is also the cheapest protection you have against a silent quality drop after a prompt, model or content change.
People: the line nobody quotes
A RAG system is never finished. Documents change, new question types appear, and retrieval misses show up in the logs. Someone needs to review a sample of answers each week, trace failures, fix chunking or metadata, update content, and add new cases to the evaluation set.
For a stable internal knowledge base, a few hours a week is typical. Customer-facing systems need more, plus a clear escalation path when the assistant cannot answer. In our cost models this is frequently the largest single line, and it is what keeps the other lines worth paying.
Cost controls that work
- Better retrieval, with reranking and metadata filters, so fewer passages are sent.
- Model routing: a small model for simple questions, a larger one only when needed.
- Caching answers to repeated questions, with invalidation when source documents change.
- Shorter system prompts, since they are paid for on every request.
- Budgets and alerts per day, per user or per team, set before launch.
Together these routinely cut infrastructure spend substantially without reducing answer quality, as measured by the evaluation set.
A worked example with round numbers
To show the method, take an internal assistant answering 20,000 questions a month. Suppose each request carries 3,000 input tokens (system prompt, five retrieved passages and the question) and produces 300 output tokens.
That is 60 million input tokens and 6 million output tokens a month. Multiply each by your provider's current price per million tokens and you have the generation line. With many current mid-tier models, that lands in the tens to low hundreds of US dollars for this volume; with the largest frontier models it can be several times higher.
Now change one variable at a time:
- Cut retrieved passages from five to three with better reranking, and input tokens fall by roughly a quarter.
- Route half the questions to a smaller model, and the generation line can drop by more than a third.
- Cache the top 10% of repeated questions, and those requests cost almost nothing.
The point of the exercise is not the exact figure, which changes whenever prices do. It is knowing which levers move your bill, so you can decide on evidence rather than surprise.
Budgeting beyond infrastructure
When finance asks for an annual figure, include:
- infrastructure at expected volume, plus headroom for growth;
- people time for review and tuning;
- a periodic re-embedding or model migration;
- security and compliance reviews where personal data is involved.
A budget that covers only tokens will be wrong in the first quarter.
Estimate before you build
During a two-to-four-week proof of value we measure real context sizes, answer lengths and retrieval settings on your documents, then multiply by expected volume and current prices. You get a cost per query, a monthly range at low, expected and high usage, and a list of the levers that matter for your case, before committing to production.
We deliver this for teams across India, the United States, the United Kingdom, the UAE and Canada; see our locations for market-specific rates and compliance notes.
RAG running costs, in numbers
Blurbee cost model for an illustrative system: 20,000 queries a month, 50,000 documents, hosted model API. Shares vary widely with model choice and volume. Not a quotation.
Why teams plan RAG budgets with Blurbee
We model cost per query before we build, set budget alerts before launch, and treat evaluation and maintenance as line items rather than surprises. The system you get is one you can afford to keep running.
- Fixed-scope first projectTwo to four weeks, priced as a number rather than an estimate, so you have evidence before a longer commitment.
- One delivery lead, namedYou always know who to ask. No account manager relaying questions to a team you never meet.
- Code and IP are yoursAssigned in writing, handed over with repositories, infrastructure definitions and documentation whether or not you continue with us.
- Performance budgets in CICore Web Vitals and bundle size gate the build. Slow does not ship.
- Accessibility to WCAG 2.2 AAReviewed with automated and manual passes, not claimed on a slide.
- Written overlap disciplineStanding calls inside your working window, everything else asynchronous with a decision log.