Five ways RAG development services projects fail, and how to avoid each

RAG projects fail for five reasons: the corpus cannot answer the questions people actually ask, retrieval quality was never measured, permissions were treated as a later phase, nobody owns the content after launch, and the task was never a retrieval problem in the first place. Four of the five are visible before the first sprint, if somebody checks.

This article is about project failure rather than technical debugging. If you are already live and answers are wrong, our engineering note on why basic RAG fails in production covers the fixes at the pipeline level. What follows is the set of decisions that determine whether you ever reach that point, ordered by how often we are called in to repair them.

Why RAG development services fails: the five patterns

Each pattern has a characteristic symptom in month three, a root cause that sits earlier than people expect, and one decision that prevents it. The table is the short version; the sections after it explain each.

Pattern one: the corpus cannot answer the questions

This is the most common and the most expensive, because it is only discovered after the pipeline is built. Retrieval can only return what exists. If the answer to "what is our escalation path for a delayed shipment in the Gulf" lives in one operations manager's memory, no chunking strategy will find it, and the system will return the nearest plausible paragraph with a citation that looks authoritative.

The check takes an afternoon. Take fifty real questions from support tickets or internal chat, and have a person try to answer each using only the documents in scope. If they can answer fewer than forty, you have a content problem rather than an AI problem, and the correct next spend is writing the missing pages. Do that first and the same build works; skip it and you have paid five figures to industrialise a gap.

Pattern two: nobody measured retrieval

The visible symptom is a team six weeks into prompt engineering with no shared view of whether anything improved. The underlying cause is that retrieval and generation are being judged together, by vibe, in a demo. When a wrong answer appears, nobody can say whether the right passage was retrieved and the model ignored it, or the passage never arrived.

Separating the two is the whole discipline. Score retrieval on recall and precision at k against a golden question set with known-correct sources, and score generation on groundedness and refusal accuracy against the same set. Then a change is either an improvement or it is not. Our stance on this is not subtle: we build the evaluation harness before the system, which is the argument in evals over demos, and the metrics themselves are explained in measuring RAG quality.

One practical warning. A system that never refuses is not confident, it is ungrounded. If groundedness is high but the refusal rate is zero across a question set that deliberately includes unanswerable questions, the model is inventing coverage, and the wider handling of that problem sits in how to handle hallucinations in production.

Pattern three: permissions as a later phase

Two versions of this failure exist and both are bad. In the first, security blocks the launch in week fourteen because the assistant can quote any document to any employee, and retrofitting access control means re-ingesting every source to capture rights that were never recorded. In the second, the system ships and a salary band, a board paper or another customer's contract surfaces to somebody who should not see it.

The prevention is architectural and cheap if taken early: capture access groups on the chunk at ingest time, and apply the filter inside the retrieval query rather than on the results afterwards, so a restricted document cannot even be inferred from an empty answer. The reasoning is set out in permission-aware retrieval.

Treat the surrounding risks properly too. Sensitive information disclosure and prompt injection are both named in the OWASP Top 10 for LLM applications, and both apply directly to a system that reads your internal documents and returns their contents in natural language. A retrieved passage is untrusted input, and a system that follows instructions found inside a document is exploitable by anyone who can add a document.

Pattern four: nobody owns the content

This failure is quiet, which is what makes it dangerous. The system launches well, scores well, and then decays over two quarters as policies change, documents are superseded without being archived, and new products arrive with no documentation. Accuracy falls a few points a month. By the time somebody complains, trust has gone and the usage curve has already flattened.

The fix is organisational, not technical: a named content owner, a weekly review of flagged answers and unanswered questions, a re-indexing schedule matched to how fast the content changes, and evaluation re-run on every model version change. That is exactly the work our Care Plans cover, from $1,000 or ₹68,000 a month with the AI add-on at $750 or ₹40,000 for evals, cost monitoring, prompt regression and re-indexing, published on the pricing page. Whether you buy it from us or staff it internally matters far less than somebody owning it by name.

Pattern five: it was never a retrieval problem

RAG answers questions whose answers exist as text. It is the wrong tool for three other shapes of question, and a surprising number of projects are approved without anyone classifying which shape they are dealing with.

Aggregation and calculation over structured records, such as which region grew fastest last quarter, is a querying problem. Vector search over a document corpus cannot count rows, and the right service is natural language data querying against a governed semantic layer. Teaching a fixed house style, tone or rigid output format is a fine-tuning problem, and the boundary is set out in our RAG versus fine-tuning comparison. Multi-hop questions about how entities relate to each other are a graph problem, because similarity search keeps returning passages about each entity separately.

Classify your top twenty questions by shape during scoping. If fewer than twelve are genuinely document lookups, the RAG budget is being asked to solve a problem it cannot reach.

Early warning signs in the first month

These are the signals we watch for on our own engagements, and they are worth watching for on anyone else's.

• No golden question set by the end of week two. Everything downstream becomes unfalsifiable.
• Subject-matter experts cannot free two hours a week. The question set and answer review will not happen, and the plan is fiction.
• Source credentials still pending in week three. The ingestion block has already started slipping.
• Demos to stakeholders instead of scores. A demo is a sample of one chosen by the person presenting it.
• Scope growing by sources rather than by questions. Each added source costs connectors plus re-chunking, re-indexing and a fresh evaluation run.
• No agreed accuracy threshold. Without one, the project has no definition of done and the last block expands indefinitely.
• No named content owner for after launch. Pattern four is already scheduled.

When a RAG build is the wrong purchase entirely

Beyond the five patterns, there are situations where the right advice is not to buy. If your whole body of knowledge is fifty stable pages, it fits in a modern context window and a retrieval pipeline is scaffolding around a problem you no longer have. If query volume is genuinely low, the arithmetic will not repay a five-figure build no matter how well it works. And if the organisation is mid-reorganisation and nobody can name the owner of the documentation, the system will be orphaned before it is measured. We say so in these cases, because an honest no costs a conversation and a bad yes costs a quarter.

How to de-risk before you commit

The cheapest insurance is to buy evidence before you buy the system. A three-week AI POC Sprint at $6,250 to $10,500, or ₹4,00,000 to ₹6,80,000, indexes a real slice of your content, runs your own questions against it and reports measured recall. That number tells you which of the five patterns you are exposed to while the decision is still reversible. The full build then runs as our retrieval and knowledge engineering programme, from $14,000 to $49,000, or ₹8,80,000 to ₹32,00,000, fixed price against a locked scope.

Related reading

A practical RAG implementation guide sets out the build order these failures come from ignoring, and how long a RAG build takes shows where each risk sits in the calendar.

Every one of these failures is cheap to prevent in week one and expensive to repair in month six, which is the whole argument for spending that first fortnight on questions rather than code.