The final project addresses the failure mode this platform keeps returning to: models inventing detail when asked to recall. The fix is structural — supply the text.
Goal. Answer questions about your own documents, citing the passages used, and say so when the answer is not there.
The pipeline
1. Ingest. Load documents, extract clean text, keep the source filename and position for every piece.
2. Chunk. Split into passages of roughly 500–1000 tokens with about 10–15% overlap.
Chunking decides retrieval quality more than any other choice. Too small and a passage lacks the context to be understood. Too large and the relevant sentence is buried among irrelevant text that also fills the context window. Split on natural boundaries — headings, paragraphs — rather than a fixed character count.
3. Embed. Convert each chunk to a vector and store it with its text and source.
4. Retrieve. Embed the question, find the nearest chunks, take the top 3–5.
5. Generate. Give the model the question and only those chunks:
Answer using only the passages below. Cite the source for each claim. If the passages do not contain the answer, say so — do not use outside knowledge.
Diagnose the two failures separately
This is the most valuable habit in the project. When an answer is wrong, ask which half broke:
Retrieval failed — the right passage was never fetched. No prompt change will fix this. Look at chunking, at whether the question's wording matches the document's, at how many chunks you retrieve.
Generation failed — the right passage was present and the model answered badly. Now the prompt matters.
Citations that can be checked
Return the source and enough of the passage that a person can verify it without opening the file. A citation nobody can check is decoration.
Test the absence case
Ask something your documents do not cover. The correct answer is "these documents do not address that". If it answers from general knowledge, the grounding instruction is not holding — and an assistant that quietly falls back to memory is worse than none, because users trust it precisely because it is supposed to be grounded.
Evaluate
Twenty questions: fifteen answerable from the documents, five not. Score retrieval (was the right chunk fetched?) and generation (was the answer correct and cited?) separately. Track them separately over time.
Extend it
- Combine keyword and vector search, which fixes many exact-term misses
- Filter by metadata — date, author, document type
- Return "I found three passages that disagree" rather than silently picking one