RAG vs vector database: what vector search leaves for you to build
A vector database indexes and searches embeddings. RAG also needs extraction, chunking, updates, ordering and evaluation. Here is where each job lands.
A team loads its employee handbook into pgvector, asks how many vacation days contractors get, and receives a confident answer about full-time staff. The nearest-neighbour query did its job. The chunk it returned had lost the table header saying which column applied to contractors, and the exception sat in the next paragraph, which landed in a different chunk.
That is the gap behind most "RAG vs vector database" comparisons. A vector database stores vectors with their payloads, builds indexes, and answers similarity queries. RAG is the whole path from a source file to an answer the model can support with evidence. The database is one stage of that path, and our view is that most teams should spend their first week on the stages before it.
A vector database indexes and searches, and stops there
pgvector is a good reference point because it is small and clear about its scope. It adds a vector type to Postgres, exact nearest-neighbour search, and approximate indexes such as HNSW and IVFFlat. Its README also shows hybrid search: run Postgres full-text search beside the vector query and merge the two lists with reciprocal rank fusion or a cross-encoder. That is a retrieval design you assemble yourself, and a capable one.
What the index does not know is that a row came from page 37 of the March revision of the handbook. It has no opinion on whether the running header is content, whether a table should stay in one piece, or whether deleting the source PDF should remove the twelve chunks derived from it. Source systems hold files, pages, tickets, permissions, and revision histories. An index holds searchable units with an ID, a vector, some text, and metadata. Something has to translate one into the other without losing provenance or access rules, and keep doing it every time a source changes.
If your application already produces clean chunks, stable IDs, and reliable change events, a vector database may be the only retrieval infrastructure you need to add. You keep control of parsing, embedding, and data movement, and the database does the part it is good at. The other jobs are still there; they just have no owner until someone names one.
Integrated databases move the boundary a little
Some products cover more than storage. Pinecone accepts precomputed vectors, but an index with integrated embedding also takes text records and embeds them with a hosted model. The same docs describe hybrid search over keyword and semantic signals and integrated reranking. OpenAI File Search goes further: you upload files of a documented set of types into a vector store, and the tool runs semantic and keyword search over them with metadata filters.
Those features remove real integration work. They still start from records or files that someone chose, and they do not decide how a scanned contract, a wiki tree, or a support ticket becomes a good record. Deletion propagation from the system of record, document-level permissions, source-revision tracking, and evaluation criteria stay with you unless a specific documented feature takes them over. "Managed RAG" is not a standard category either, and each service draws its line in a different place.
The stages between a source file and an answer
The diagram lays out the path. The dashed box is the part a vector database owns.
After the context stage, a complete application sends the evidence to a model and checks whether the answer is supported by it. Reranking is optional. Synchronization, access policy, observability, and reindexing cut across every box, and for a corpus that changes they run again whenever a file, a permission, or the embedding model changes.
| Stage | Vector database | Managed RAG: what to verify |
|---|---|---|
| Extract and chunk | Usually receives prepared vectors or text records. | Supported file types, extraction fidelity, chunk controls, and whether you can review the output. |
| Embed and retrieve | Stores and searches vectors; may embed text or support hybrid search. | Embedding choices, retrieval methods, filters, and access boundaries. |
| Update and delete | Changes indexed records when instructed. | Source synchronization, stable IDs, deletion propagation, and retry behaviour. |
| Rerank and generate | May rerank; context assembly and generation usually stay outside. | Reranking, context assembly, source attribution, and model integration. |
| Evaluate and rebuild | Database health says nothing about answer quality. | Ingestion status, retrieval evidence, model-change handling, and rebuild cost. |
Most wrong answers are decided before the embedding
Go back to the handbook. Swapping the embedding model would not have helped, because the words "contractor" and "vacation" were in the returned chunk; the column header that tied the number to contractors was not. Retrieval cannot reconstruct what extraction threw away, so we start upstream: read the extracted text of the page that produced the bad answer, check that the table header and the exception survived, then read the chunk on its own and ask whether a person could answer the question from it. No chunk size settles that in general. A small sample with facts you already know, as in how to check PDF extraction quality before embedding, tells you more than a comparison of embedding models.
Synchronization is the next quiet failure. Each derived chunk needs a stable ID tied to its source revision; otherwise an update leaves the old version searchable next to the new one, and the model receives two pieces of evidence that contradict each other. A metadata change should not trigger a re-embed and a text change must. When the embedding model changes, old and new vectors do not share a space, so someone has to plan a staged rebuild and pay for it.
Ordering comes last. A reranker only sees the candidates the first stage selected, so it can lift a relevant document from fifth place and can do nothing for one that was never retrieved. Our post on semantic search vs. reranking shows how to tell those two cases apart, and metadata pre-filtering is often the cheaper fix when the right document keeps losing to another department's version of it.
A storage price per million vectors also understates the bill. Extraction, embedding, repeated synchronization, query volume, reranking, and the occasional full rebuild all cost money, and only the first demo query is cheap.
Do you need a vector database for RAG?
No. RAG needs retrieval, not a particular engine. Full-text search can feed passages to a model, and on corpora full of product codes and policy numbers it sometimes beats embeddings outright. Vector search pays for itself when users describe things in words the documents do not use. Postgres with pgvector plus full-text search covers both cases for many teams; you would still write the extraction, chunking, synchronization, context assembly, and generation around it.
Where AIVAX draws its own boundary
We build AIVAX collections, so here is where our line sits, as the documentation states it today. You import prepared text, one document at a time or as JSONL, and each document has a stable name (docid in JSONL) that later imports match against. When the contents change, the document is queued again and its vectors are regenerated; a metadata-only change does not re-embed. Direct imports are not chunked for you. Documents below about 32 tokens may not be retrieved reliably, documents above about 4,096 tokens are indexed with a warning in the account logs, and text far beyond that is truncated for embedding.
Preparation is a separate, explicit choice. Text Segmentation splits long text and returns the segments for review before import, and the PDF to Markdown and JSONL walkthrough runs that path end to end. For PDFs, images, audio, and video, the Console's Media Injector writes focused documents from the file; they are generated knowledge rather than a transcription, so we import reviewed text when exact wording matters. On the retrieval side, search reranks the candidate pool with Reflex unless you sort by a field such as createdAt, and attaching a collection to an AI Gateway puts the results into the model's context.
The handbook question would show up in the last stage, RAG Observability in the Console. Its Transactions tab records every search made by gateways, MCP clients, and your own code, and the Low quality view lists the lowest-scoring ones with the documents each returned and their scores before and after reranking, which is where you see that the right chunk was there and still useless. The same pages show each collection as Indexed, Queued, or Outdated (indexed with a different embedding model than the collection's current one) and a query-coverage figure for documents no recorded search ever returned. Update outdated documents in the collection settings regenerates the stale vectors and bills the embedding. Deciding who may retrieve which document, syncing deletions from your source system, and judging whether the answers are good enough for your users stay with you. To try the path on a small corpus, the Free plan allows 500 document insertions and 2 Media Injector files per day (limits); the daily allowances behind those requests are small, and what exceeds them is billed to your balance.
Choose by the work you cannot operate
Our recommendation is to pick the smallest managed boundary that removes work your team cannot reliably run. If you already own clean chunks, IDs, embeddings, access rules, and change events, a vector database or pgvector is enough, and you keep full control along with the ingestion failures. If preparing and maintaining files is the hard part, evaluate a managed service on its documented import path, whether you can inspect what it produced, how it handles updates and deletions, and how it hands context to a model. If the right evidence is already retrieved but ranked badly, add a reranker to what you have and measure the latency it adds.
Before signing anything, get written answers to four questions. Who coordinates the reindex when the embedding model changes, and who pays for it? Can every retrieved chunk be traced to a source revision and to the access policy applied on that request? Can your telemetry follow one answer back through reranking, retrieval, and ingestion? During a partial failure or a full rebuild, what keeps serving? If a product cannot show which source revision produced a given chunk, we would not put it in front of users.
Checked against the AIVAX documentation and the pgvector, Pinecone, and OpenAI pages linked above on 9 November 2026. The handbook example is illustrative, not a measured incident.