RAG Development Services: a practical implementation guide

You implement RAG in five phases: write the question set the system must answer, build the ingestion and chunking pipeline, build hybrid retrieval with reranking and permission filtering, ground generation in cited passages with an honest refusal path, then measure against the question set and roll out behind a flag. Evaluation is phase one, not the last phase.

This guide walks each phase in the order we build them on real RAG development services engagements, names the architecture decisions that are expensive to reverse once you have indexed a corpus, and is deliberately specific about the parts that look trivial in a tutorial and consume most of a project in practice.

Phase 0: define what the system must answer

Before a single document is parsed, write down between one hundred and three hundred real questions with the sources that correctly answer each. That artefact is the golden question set, and it is the contract for the whole engagement. It comes from support tickets, sales calls, internal search logs and an afternoon with the people who currently answer these questions by hand.

Two rules make it useful. Questions must be in the user's words, including the misspellings and internal jargon, not in the tidy language of the documentation. And each entry names the passage that answers it, so retrieval can be scored independently of the language model.

Teams that skip this step do not discover their retrieval is weak until a launch demo, at which point the only available response is prompt tinkering. Teams that write it first can tell you, in week three, that recall at ten is 0.71 and which document type is dragging it down.

The architecture decisions that are expensive to reverse

Most RAG choices can be changed cheaply. A handful cannot, because changing them means re-embedding and re-indexing everything and re-running the entire evaluation. Decide these deliberately, in writing, in the first fortnight.

The identity row is the one that bites hardest. Access rights have to be captured at ingest, alongside the text, because reconstructing them afterwards means walking every source system again. We wrote about the consequences in permission-aware retrieval.

Phase 1: ingestion and chunking

Connectors before cleverness

Each source needs an authenticated, incremental pull: Confluence, SharePoint, Drive, a ticketing system, a document store, sometimes a database view. Incremental matters, because a nightly full re-crawl of a large corpus is both slow and expensive. Build change detection on the source's own modified timestamps or webhooks from day one.

Parsing the awkward formats

Clean HTML and Markdown are easy. Scanned PDFs need OCR, spreadsheets used as databases need row-level extraction with the header row carried into each chunk, and slide decks need the speaker notes as much as the slides. Budget explicitly for whichever of these you own; they are the difference between a four-week ingestion phase and a nine-week one.

Chunking that preserves meaning

A chunk must be answerable on its own. That usually means splitting on the document's real structure rather than on a fixed token count, and prefixing each chunk with its document title and section path so a retrieved passage still tells the model what it is looking at. Our guide to chunking strategies by document type covers the windows we use for contracts, manuals, tickets and transcripts.

Metadata is not optional

Capture source system, document owner, effective date, version, business unit and access groups. Every one of those becomes a retrieval filter later, and every one is painful to backfill.

Phase 2: retrieval

Retrieval is where accuracy is won or lost, and it is three components rather than one.

Vector search finds passages that mean the same thing as the query. Keyword search finds exact matches, which is what saves you when the user types a part number, an error code or an internal acronym that embeddings flatten into nonsense. Running both and fusing the results is the baseline, not an optimisation; the argument is set out in why vectors alone miss the answer.

A reranker then takes the top forty or fifty candidates and reorders them with a model that reads the query and passage together. This is the cheapest single accuracy gain available in most systems. Finally, the permission filter is applied as part of the query, never as a post-filter on the results, otherwise a user learns from an empty answer that a document exists. Where the index lives is a separate decision, covered in pgvector versus Pinecone versus Qdrant.

Phase 3: generation and grounding

The generation prompt does four jobs: answer only from the supplied passages, cite each claim with the passage it came from, say plainly that it does not know when the passages do not contain the answer, and match the format the interface expects. The refusal path is the one teams under-invest in, and it is what earns user trust in the first fortnight.

Citations belong in the interface, not in a footnote. A user who can click through to the source paragraph will forgive an imperfect answer; a user who cannot verify anything will stop using the system by the third wrong result.

Phase 4: measure it properly

Run the golden question set on every change and track a small number of metrics. These are the ones we report weekly.

• Recall at k. Share of questions where a correct source appears in the top k retrieved passages. If this is low, no prompt will save you.
• Precision at k. How much of what you retrieved was relevant. Low precision wastes context and raises cost per query.
• Groundedness. Share of answer sentences traceable to a retrieved passage. This is your hallucination measure.
• Answer correctness. Judged against the reference answer by a human reviewer or a scored model, sampled rather than exhaustive.
• Refusal accuracy. How often the system declines when it should, and how often it declines when it should not.
• Latency and cost per query. Measured at the ninety-fifth percentile, with the reranker in the path.

Open tooling exists for the retrieval-side metrics: the Ragas documentation defines faithfulness and context recall in a way you can implement directly and, more usefully, compare across builds. Our own note on measuring RAG quality explains how to read the numbers together, because each is misleading alone.

Phase 5: rollout and operations

Launch to one team behind a feature flag, with every answer logged alongside its retrieved passages and a one-click way to flag a bad one. Review the flagged set weekly. Most fixes in the first month are content fixes, not model fixes: a policy page that was never written, a superseded document that should have been archived, a term nobody outside one team uses.

Then set the operating cadence: re-index on a schedule that matches how fast the content changes, re-run evaluation on every prompt and model change, and keep a dashboard of cost per query. A Care Plan with the AI add-on at $750 or ₹40,000 a month exists for exactly this work, because these chores get dropped when the delivery team leaves.

Where this approach is the wrong choice

RAG is the wrong pattern when the answer requires joining and aggregating structured records, when the corpus is small and stable enough to fit in a context window, or when the knowledge is genuinely relational and the questions are multi-hop. That last case is the interesting one: if users ask how three entities connect, vector similarity keeps returning passages about each entity separately, and a graph-backed approach is the better fit, as we set out in GraphRAG explained.

What an engagement looks like

An NBFC came to us with KYC and loan documents in three formats and a compliance rule that nothing could leave their environment. The build ran as a private deployment with extraction, validation and a reviewer queue rather than an open chat box, and the accuracy target was agreed on a labelled document set before the first sprint. The shape of that work is described in the KYC document intelligence case study, and the lesson generalises: the interface matters less than the question set you agreed to hit.

Commercially, our retrieval and knowledge engineering programme runs from $14,000 to $49,000, or ₹8,80,000 to ₹32,00,000, fixed price against locked scope, typically eight to sixteen weeks. Where the corpus is unproven, a three-week AI POC Sprint at $6,250 to $10,500, or ₹4,00,000 to ₹6,80,000, indexes a slice and reports measured recall before you commit.

Related reading

Five ways RAG projects fail covers the patterns this sequence is designed to avoid, and Eazy Knowledge AI is the productised version for teams that want the same pipeline without a bespoke build.

Build the question set first and the rest of the sequence becomes an engineering problem with a number attached to it.