RAG is not the whole story. When classic search works better

Vector databases have gone mainstream, and RAG has become shorthand for “AI in the business”. In practice, in half of the projects we deliver, classic full-text search gives better results at a fraction of the cost. Here is how to recognise when RAG is not worth forcing.

The real question

Before you reach for embeddings and a vector database, ask the basic question - is the user looking for a specific piece of information, or for an answer synthesised from several sources? RAG handles the second case well, but for the first a good full-text index with facets is usually better.

Three signs against RAG

1. Queries are short and specific - “holiday policy”, “invoice number 2024/03”. A classic inverted index in Postgres FTS or Elastic will find that better and faster.
2. The data is structured - tables, forms, lists. Vectors do not help here; SQL with GIN or GIN-trgm indexes is enough.
3. Users are experts - they know the vocabulary of the field. Keyword search with synonyms gives them more control than semantic search.

When RAG pays off

RAG makes sense when you have to: (a) answer in full sentences rather than return a list of documents; (b) combine passages from different sources into a single answer; (c) handle imprecise queries that call for “reading the intent” - typical of customer service and internal assistants.

The best implementations we have seen combine both approaches: hybrid retrieval (BM25 plus embeddings) and a reranker that decides what actually reaches the model’s context.

A practical heuristic

Start with the simplest thing that could work. Postgres FTS and a well-designed search interface are often 80% of the value of RAG at 20% of the cost. If that turns out not to be enough, you can add vector search on top.