Retrieval / RAG
Grounded question answering over dense technical documents. Every claim is cited to the exact source page, and the system refuses rather than guesses.
Anyone who has had to answer a question from a 200-page contract, manual, or filing knows the real work isn’t reading. It’s finding, and then trusting, what you found. General-purpose chat assistants will happily answer questions about a document they were never given, inventing plausible page numbers along the way. For any high-stakes domain that’s worse than useless.
InferLens is a retrieval-augmented question-answering system built around one non-negotiable rule: every claim in an answer is grounded in a retrieved passage, and every passage points back to the exact page it came from. If the document doesn’t support an answer, the system says so rather than guessing.
A PDF is extracted page by page with PyMuPDF, then split into ~500-word overlapping chunks. The detail that matters: a chunk never crosses a page boundary. Each chunk carries the page number it came from, so a citation can resolve to “page 12” rather than an opaque passage index. That costs a few extra chunks on documents with short pages, but it’s the right trade for a product where “where did this come from?” is the most common follow-up question.
Chunks are embedded with all-MiniLM-L6-v2 and indexed in FAISS. At query time the
question is embedded the same way, the top five nearest passages are retrieved, and
they’re handed to Claude with a system prompt that allows it to use only those
passages, emitting inline [1], [2] markers after each claim. When the context
is insufficient, the model returns a fixed refusal string instead of improvising.
Exact search, not approximate. The index is a faiss.IndexFlatL2: exact,
brute-force nearest-neighbour. For single-document corpora (hundreds to low
thousands of chunks) an approximate index buys nothing but a new class of silent
recall bugs. I wrote up the full trade-off in
Exact over approximate.
ONNX embeddings to fit the box. Running the embedding model through ONNX Runtime
(via fastembed) instead of PyTorch cut per-process memory from ~400 MB to ~100 MB.
That’s the difference between fitting on a 512 MB hosting tier and OOM-ing on cold
start. Same model, same vectors, a third of the footprint.
Citations are a contract, not a nicety. The generation prompt requires a citation marker after every factual claim and forbids inventing page numbers or passages. Grounding is enforced at the prompt boundary, and the UI renders the markers as clickable references to the source passages, so a reader can verify any sentence in one click.
InferLens began as my graduate capstone (originally DocuMind) and became the reference design I reach for whenever a RAG system has to be auditable rather than merely conversational: exact retrieval, page-level provenance, and a generation step that refuses to exceed its evidence.