Vector databases went mainstream and RAG became a synonym for “AI in the enterprise”. In practice, in half of our deployments classic full-text search delivers better results at a fraction of the cost. Here is how to tell when not to force RAG.
What are you actually trying to do?
Before reaching for embeddings, ask yourself - is the user looking for a specific piece of information, or for an answer synthesized from many sources? RAG shines at the latter, but for the former a good full-text index with facets usually wins.
Three signals you don’t need RAG
- Queries are short and concrete - “vacation policy”, “invoice 2024/03”. A classic inverted index in Postgres FTS or Elastic will find it faster and better.
- Data is structured - tables, forms, lists. Vectors don’t help here; SQL with GIN/trigram indexes is enough.
- Users are experts - they know the domain vocabulary. Keyword search with synonyms gives them more control than semantic search.
When RAG actually pays off
RAG makes sense when you must: (a) answer in full sentences, not return a list of documents; (b) combine fragments from many sources into one answer; (c) handle imprecise queries that require “intent understanding” - typical for customer support and internal assistants.
The best deployments we’ve seen combine both: hybrid retrieval (BM25 + embeddings) and a reranker that decides what actually reaches the model context.
Practical heuristic
Start with the simplest thing that might work. Postgres FTS plus solid search UX is often 80% of a RAG’s value at 20% of the cost. If that’s not enough, add vector search.