How to check PDF extraction quality before embedding for RAG

To check ingestion quality before embedding, take a small sample of real pages, write down by hand the facts a user will ask about, run the import into a scratch collection, and grade every generated document against those facts. Treat one wrong number as a blocking defect. A missing fact is a coverage gap you can fix by changing the source or the processing context. Search tuning comes after this check, because no retriever or reranker can recover a value that was misread on the way in.

The check below is written for two AIVAX ingestion routes: Media Injector, which turns a file into generated knowledge documents, and Text Segmentation, which splits text you already control. The acceptance thresholds are yours to set.

Inspect the evidence before you embed the document

Once a misread total sits in a collection, it is retrieved with the same confidence as a correct one, and the answer built on it cites a real document. A vector database is only one part of a RAG system; the text that goes into it decides what the system can honestly say.

Developer reports point to the same failures. One r/LangChain write-up describes a table split across two pages whose second chunk has no header information, and a later thread on PDF parsing for RAG shows people still comparing setups. They are anecdotes, but they show why the check has to run on your pages.

Use OmniDocBench to choose what to measure

OmniDocBench is a public reference for what to measure. Its README describes 1,651 PDF pages across 10 document types, with annotations for text, tables, formulas, and reading order, and it scores parsers on four things: text (normalized edit distance), tables (TEDS), formulas (CDM), and reading order. The README dates its current version, v1.7, to April 2026.

Use it to choose what to measure, not to rank a method for your files. It ranks parsers that turn a page into Markdown; it does not measure retrieval, and it says nothing about Media Injector or any AIVAX component. Its scores rely on a page-level ground truth that your invoices and policies do not have, and its own documented limits — inconsistent symbol recognition, text evaluation restricted to Chinese and English — are not your limits. Your documents, languages, and scan quality differ. Take the four axes as a vocabulary, then build a ground truth for a few of your own pages by hand.

Media Injector answers questions it writes itself

Media Injector accepts PDFs, images, audio, and video from the dashboard and produces RAG documents. It is not a page-by-page transcription. The current implementation runs an interview: a model reads the file and your optional processing context, proposes factual questions, and each question is answered from the source into one self-contained document. It stops when it judges that no materially new knowledge remains, and it is instructed to ignore pagination, decorative text, boilerplate, and repeated summaries.

Three consequences shape the check. Each document is a concise factual answer in the source's language, so the wording is the model's even when the values come from the page: verify facts against the source, and do not diff text. The interview may decide a table row is not worth a question, and an absent fact is invisible unless you wrote down what you expected — when a fact never appears, add it to the processing context as something to prioritize, rerun the sample, and check whether it shows up; if the source does not state it, no context can supply it. And the processing context steers priority without being evidence: the docs state that it cannot add facts absent from the file, so a document containing something only the context said is a defect.

Generated documents are tagged autogenerated and interview, plus a tag derived from the source file name. Their metadata records the source file name and the question that produced them, so each document can be traced to what it was meant to answer. Only the document text is embedded: tags and metadata come back with results for filtering and audits, but they do not influence semantic matching, so the words a user would search for belong in the text (Documents).

Build the sample and write the expected facts first

Pick pages that break things, not the cleanest ones: a scanned page, a table that crosses a page break, a two-column page, a chart or photo with values, and a page dense with repeated headers. Fifteen to twenty pages spread across those kinds is a workable start; adjust it to your document mix.

For each page, write the expected facts before you import anything. A fact is one sentence a user could plausibly ask about, with its exact values:

invoice-0412.pdf, p.2: the invoice total is 4,180.00 BRL, due 2026-10-15.
invoice-0412.pdf, p.2: line 3 (freight) is 320.00 BRL and is not taxed.

Write them before importing: once you have read the generated documents, it is easy to accept them as the answer key.

Grade each document against four axes

Import the sample into a scratch collection, wait for indexing to finish, then open the generated documents. The first three rows are the OmniDocBench axes, restated for retrieval; the last is specific to indexing.

For the retrievable-unit axis, the Documents guide gives concrete signals. Keep most documents between 20 and 700 words, which is a target and not an API rule, and prefer one answerable topic per document, with its subject named in the text. Below about 32 tokens a document may not be retrieved reliably; above about 4,096 tokens the indexer records a warning in the account logs, and text far beyond that size is truncated for embedding. The size section lists the current thresholds. The Get Document endpoint returns each document's character and word counts, so you can read the smallest and largest first.

Then score coverage. For every expected fact, mark it as found and correct, found and wrong, or missing, and record any generated fact that is neither on your list nor on the page. To list the generated documents, browse the collection with a filter; -t filters by tag and -c by content:

curl -G "https://inference.aivax.net/api/v1/collections/COLLECTION_ID/documents" \
  -H "Authorization: Bearer YOUR_AIVAX_API_KEY" \
  --data-urlencode "filter=-t autogenerated -c 4,180.00"

If a fact is found but wrong, decide whether the fault is in the source or in the extraction. A faint scan, a mislabeled extension, or a stamp over a total is a source problem: fix or replace the file. A correct source with a wrong result is a signal to retry with a sharper processing context, or to use the text route in the next section.

Segment text yourself when exact wording matters

Use Media Injector when you want a managed pipeline that turns a file into question-shaped knowledge units. Choose a text route when exact wording matters: the docs advise direct import when you need exact source wording, and reviewing generated documents when legal fidelity or sensitive-data handling is important. A common shape is Fetch and OCR to extract text, your own review or cleanup, Text Segmentation to propose boundaries, and then an import of the segments you accept. If the source is in another language, translate the extracted text before segmenting so the review and the segments share one language.

For the end-to-end version of this text route, with a runnable script that ends in a JSONL import, see How to convert a PDF to Markdown for RAG and split it into chunks. The same path exists as a command: the AIVAX CLI runs aivax ocr document <file> --format jsonl locally and hands the result straight to aivax rag import.

Segmentation chooses boundaries; it does not fix extraction errors. A misread character in the OCR text stays in the output, so review the text before segmenting. With sanitize left at false, no language model is involved: the endpoint covers the whole document in source order without overlap and keeps its line breaks, although a boundary may fall at a sentence end inside a line (changelog, 6 October 2026). So the check below compares text, not line counts. Test for it:

const source = ocrText;

const response = await fetch(
  "https://inference.aivax.net/api/v1/generations/segment",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.AIVAX_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ documents: [source], sanitize: false }),
  },
);

const { segments } = (await response.json()).data.result[0];

const compact = (text) => text.replace(/\s+/g, "");

if (compact(segments.join("")) !== compact(source)) {
  console.warn("The segments do not reproduce the source text");
}

This check catches dropped and duplicated text, not a segment boundary that cuts a table between its header and its rows. A table or fenced code block of more than about 400 tokens is split with its header row or fence lines repeated in each part, so it fails the comparison although nothing is missing. Both cases need a manual read; the docs list lost table headers first among the failure modes of messy sources. Segmentation returns text to your application; it neither embeds nor stores it, so nothing is indexed until you send the accepted segments to the collection.

Fix the cause and rerun the same pages

Delete the affected documents with the Delete Document endpoint, or the whole scratch collection, fix the cause, and rerun the same pages so results stay comparable. Failed or cancelled Media Injector jobs can be retried while their uploaded data is still available. Keep the expected-facts file with your ingestion setup: when you change the processing context, the file type mix, or the segmenter settings, rerun it as a regression check before the production import.

Once the sample passes, semantic search and reranking can be evaluated on documents you have already verified.

Thresholds and endpoint behavior here were read from the AIVAX code and maintained documentation on November 4, 2026; the OmniDocBench description is as read on its README. No accuracy figures were measured for AIVAX.