What does a RAG system cost?

RAG development cost sits between roughly $14,000 and $50,000 for most enterprise builds, with running costs that are modest next to the build. The number moves on three things: what kinds of documents you have and how clean they are, whether answers must respect who is allowed to see what, and how often the content changes. The model and the vector database are almost never the expensive part. This article explains where the money goes, gives a cost table by scope, and shows how to write a RAG project budget you can defend.

What a RAG system is, and what you are paying for

Retrieval-augmented generation answers questions by finding the relevant passages in your documents and giving them to a language model to write from. That sounds like a weekend project, and a demo is. Production is different: documents must be parsed without destroying tables, chunked by structure, indexed with both keyword and vector search, filtered by permission, refreshed when the source changes, and measured so you know the answers are grounded. Each of those is an engineering stage with its own cost. We covered why the demo version fails in why basic RAG fails in production.

RAG cost by scope

Our retrieval and knowledge engineering service starts at $14,000 / ₹8.8L for the first row and is quoted fixed price once the sources are known.

Document types drive the first third of the cost

Clean markdown from a help centre costs little to parse. Contracts with numbered clauses, spreadsheets with merged headers, slide decks, scanned invoices and photographed forms each need their own parsing and chunking strategy, and each needs testing. A source inventory is the first thing we build in a discovery sprint: for each source, the format, the volume, the owner, the refresh rate and the sensitivity. That inventory is a better predictor of RAG cost than anything else in the brief. Chunking choices per document type are covered in chunking strategies for RAG.

Permissions drive the second third

If everyone may see everything, retrieval is simple. If a sales rep must not see HR documents and a contractor must not see pricing, the index needs permission metadata mirrored from the source system, the query needs a filter applied before ranking, and the whole thing needs tests that prove a user cannot retrieve a passage they cannot open. Permission-aware search is a large part of enterprise search AI cost, and it is the part that most often gets discovered late. Decide early whether you need it.

Refresh needs drive the rest

A policy library that changes quarterly can be re-indexed by hand. A ticket system that changes every minute needs incremental indexing with change detection, deletions propagated, and a way to expire chunks that are no longer true. The refresh design also decides running cost: re-embedding everything nightly is wasteful; embedding only what changed is cheap but takes engineering. Ask the vendor how deletions and updates reach the index. If the answer is "we re-index", ask how long that takes at your volume.

The parts that cost less than people expect

The vector database is rarely a budget item. For most enterprise volumes, pgvector inside the Postgres you already run is sufficient, and a managed vector service is a few hundred dollars a month at most. Embedding a corpus of tens of thousands of pages is a one-off cost measured in tens of dollars. Inference per query is small because a well-built retriever sends a few relevant passages, not whole documents, to the model. Where teams overspend is on a large model for every query when a cheaper one answers most of them; model-agnostic routing fixes that, and so does caching the answers to questions that are asked hundreds of times a week.

Evaluation: the cost that pays for itself

Every RAG budget should carry a line for evaluation, and it is the line vendors most often cut to win on price. The work is unglamorous: collect fifty to two hundred real questions from the people who will use the system, record the correct answer and the passage that supports it, and run the retriever and the generator against that set on every change. The output is three numbers you can track: how often the right passage is retrieved, how often the answer is grounded in it, and how often the system correctly says it does not know. Without those numbers a chunking change or a model upgrade is a coin toss. With them, the team can swap a model or tune retrieval in an afternoon and know whether it helped. The evaluation set also becomes the acceptance test for the fixed-price build, which protects you as much as it protects us.

Why the eval set must be yours

Public benchmarks measure public documents. Your questions are about your contracts, your product and your policies, phrased the way your staff phrase them. A vendor who offers to test on a generic dataset is testing something else. Insist that the graded question set is built from your logs, owned by you and handed over with the code.

What a knowledge base AI price should include

• A source inventory with parsing strategy per document type
• Hybrid retrieval (keyword plus vector) with a re-ranker
• Citations back to the source passage in every answer
• Permission filtering if any source is restricted
• Incremental refresh with deletions handled
• An evaluation set of real questions with graded answers, run on every change
• A dashboard showing unanswered questions and low-confidence answers
• Handover of code, prompts, index configuration and documentation to your team

A worked example

A non-banking financial company needed to pull fields from customer onboarding documents: identity proofs, address proofs, bank statements, many of them photographed on phones. The initial ask was "a chatbot over our documents". Discovery showed the real job was a document intelligence pipeline: layout-aware parsing, OCR for images, extraction into named fields, validation against business rules and a review queue for low-confidence cases. That scope sits at the top of the cost table, and it was worth it because the manual review time it replaced was the operations team's largest line. The KYC document intelligence case study describes the outcome qualitatively. Had the sources been clean PDFs with no field extraction, the same client would have been in the first row of the table.

How to write a RAG project budget

Start with the source inventory and mark each source clean or messy, open or restricted, static or changing. Count how many are messy, restricted or changing; each adds a stage. Add an evaluation budget, because a retrieval system without graded questions is a demo. Add running cost for embeddings, inference, hosting and refresh jobs, then a care plan for monitoring drift. Keep a contingency for one source that turns out worse than it looked, because one always does. The AI development cost guide has the wider framing.

Team and timeline

A single-source assistant takes three to four weeks with one retrieval engineer and a content owner on your side. A multi-source, permission-aware system takes six to ten weeks: a retrieval engineer, a backend engineer for connectors and access control, and an evaluation owner who grades answers. A document intelligence pipeline adds a data engineer and runs eight to twelve weeks. Where the sources are uncertain, a Sprint Zero discovery at $3,250 / ₹2,00,000 produces the inventory and a fixed quote and is credited to the build. Ongoing refresh monitoring and retrieval tuning sit under a Care Plan from $1,000 a month. Price bands are on the pricing page.

Before you start: a checklist

• Inventory every source: format, volume, owner, refresh rate, sensitivity
• Decide whether answers must respect per-user permissions
• Collect fifty real questions with the answer and the source passage
• Agree what the system does when it cannot find an answer
• Confirm who owns content quality after launch
• Estimate query volume per day for the running-cost forecast
• Check where data may be stored and which models may see it
• Ask each vendor how updates and deletions reach the index

Glossary

• RAG: retrieval-augmented generation; the model answers from retrieved passages rather than memory
• Hybrid search: keyword and vector retrieval combined, then re-ranked
• Re-ranker: a model that scores candidate passages against the question
• Permission-aware retrieval: filtering results by what the asking user may see
• Incremental refresh: indexing only what changed since the last run
• Groundedness: whether the answer is supported by the retrieved passages
• Golden question set: real questions with graded answers used to test every change

Related reading

Read why basic RAG fails in production, RAG vs fine-tuning and how to measure RAG quality. The pgvector project documents what Postgres can do before you buy a separate vector database.

Budget for your documents, your permissions and your refresh rate; the model and the database will be the cheap part.