Designing your own custom RAG pipeline.

Your question set does two jobs at once. It's a product spec — the concrete thing you're building toward — and it's an evaluation set — the thing you'll measure every later decision against. Most teams skip straight to picking a vector database because that feels like progress. It isn't; it's motion without direction.

Scope is the real risk. A RAG system aimed at 'anything about the company' has no way to know what a good answer looks like, so it can't tell a grounded answer from a confident-sounding fabrication. Narrow the target first: which team, which documents, which questions. You can always widen later — widening a system that never worked narrowly just multiplies the failure.

Define the failure mode you want, not just the success mode. A good answer is grounded in a real source, cited, and current. A missing answer should say so plainly — 'I couldn't find this in the documentation' is a better outcome than a fluent guess. Then inventory your sources honestly: where the documents actually live, how often they change, and who is allowed to see what. That inventory becomes the access and freshness constraints every later step has to respect.

Parsing is unglamorous because it produces no demo-able output — nobody screenshots a clean text file. But it's the highest-leverage decision in the pipeline: garbage in at this step survives every downstream step unchanged. A model can't reason about a table it never saw, because the parser flattened it into word soup.

Read outputs looking for specific failure modes, not a vague 'does it look okay.' Tables collapsed into unstructured text with rows and columns scrambled together. Reading order broken on multi-column layouts, so a sentence jumps from column one to column two mid-thought. Headers and footers bleeding into every page's content, polluting the actual text. Scanned pages silently skipped entirely — the most dangerous failure, because nothing in the output tells you it happened.

Then weigh the operational trade-off: hosted parsers send your documents to someone else's infrastructure in exchange for less engineering work; local, open-source parsers keep data in-house but you own the compute and the upkeep. At corpus scale, price per page adds up fast — model it before you commit, not after the invoice arrives.

There's no single product called 'document processing' — it's a set of decisions and the glue code that implements them, made per document type. A scanned invoice, a wiki page, and a 300-page manual all need different treatment, and treating them identically is how quality quietly degrades for whichever type didn't fit the default path.

The metadata you extract here is what makes retrieval filterable later — you can't filter search results by date, document type, product line, or access level if that information was never pulled out during processing. It's much easier to decide this now than to re-derive it from raw chunks after the fact.

Access rules deserve special attention: they must be carried through the pipeline from the very first step, tagged onto documents and chunks alike. Bolting security on after the fact means re-processing your entire corpus, because the permission information was never captured when it was cheapest to capture.

Chunking failures are easy to spot once you know what to look for: a table split from its header, so the numbers lose their meaning; a procedure step separated from the rest of the procedure; a contract clause cut off from the definitions it depends on. Each of these turns a perfectly good document into a chunk that can't answer the question it was supposed to.

In plain terms, the strategies break down like this. Fixed or page-based chunking is dumb but predictable — it always produces the same shape, which makes it easy to reason about even though it ignores meaning. Semantic chunking places boundaries where the meaning actually shifts. Hierarchical chunking keeps small chunks for precise search but links them to larger parent chunks for context. Hybrid — different rules for different document types — is usually the right answer, because a manual and a spreadsheet don't share a natural chunk shape.

Chunk size trades recall against precision: smaller chunks retrieve more precisely but each one carries less context; larger chunks carry more context but dilute relevance. You don't have to solve this purely at chunking time — retrieval-time tricks like fetching neighboring chunks or pulling in the parent document can recover context that a hard boundary cut off.