Browser RAG: Retrieval-Augmented Generation Without a Server
REPORT_ABSTRACT
Browser RAG implements a full Retrieval-Augmented Generation pipeline that runs entirely on-device. Drop in a PDF or text file and it gets parsed with pdfjs-dist, chunked, embedded with MiniLM, and indexed in-memory. Questions are embedded in the same latent space, cosine-similarity retrieves the top-k chunks, and a local GPT-2 model generates a grounded answer conditioned on that retrieved context. No document text, query, or embedding ever leaves the browser.
01. THE PIPELINE SHAPE
The system follows the classic embed → retrieve → generate pattern, but every stage is implemented in JavaScript. pdfjs-dist extracts text, a sentence-aware chunker splits it into retrieval units, transformers.js embeds both corpus and queries, an in-memory cosine-similarity index handles retrieval, and a GPT-2 model generates the final answer. There is no server between the user and their document.
02. INGESTION AND CHUNKING
PDF text extraction runs through pdfjs-dist, the same library that powers Firefox's built-in viewer. Rather than naive fixed-size splits, the chunker is sentence-aware: it respects paragraph boundaries so each chunk is a self-contained unit of meaning. This matters because retrieval quality is bounded by chunk quality — badly split text produces badly grounded answers.
03. EMBEDDING AND RETRIEVAL
Corpus chunks and user queries are both embedded with MiniLM (all-MiniLM-L6-v2), producing 384-dimensional, L2-normalized vectors. Retrieval is a brute-force cosine-similarity scan over the in-memory index — fast at this corpus scale and trivially correct. The top-3 chunks are selected as the grounding context for generation.
04. GROUNDED GENERATION
The retrieved chunks are injected into a prompt template and fed to a local GPT-2 model running through transformers.js on WebGPU or WASM. Conditioning the decoder on retrieved evidence — instead of answering from model weights alone — is what turns the raw model into a RAG system: answers point at the document rather than hallucinating from pretraining.
05. CITATION-AWARE UX
The chat interface makes retrieval visible. Every assistant response carries collapsible source citations listing which chunks informed the answer, with expandable snippet previews, so the user can audit grounding at a glance. Combined with zero data egress, this makes the tool safe for confidential documents that cannot be uploaded anywhere.