Build a Document QA System Using RAG: 2026 Resume Guide

A document QA system answers natural language questions about any PDF by searching the document itself, not the model’s general training data.

That distinction matters for placement season. Interviewers can test it live by asking your system a question whose answer is in your uploaded PDF. If the system returns the right passage, you have a working retrieval pipeline. If it hallucinates, you have a debugging story. Either way, the interviewer learns something real about your build, which is more than most projects offer.

What a Document QA System Actually Does

The core pattern is called Retrieval-Augmented Generation, or RAG. It works in two phases.

In the indexing phase, run once offline: load a document, split it into chunks of text, convert each chunk into a numerical vector using an embedding model, and store those vectors in a searchable index.

In the query phase, run each time a question arrives: convert the question into a vector using the same embedding model, find the chunks whose vectors are most similar to the question vector, and pass those chunks to a language model as context. The model writes an answer grounded in the retrieved text rather than in its training data alone.

Three properties make this architecture worth knowing for a technical interview:

• The system only answers from what is in the document. A properly built system says it does not know rather than fabricating an answer when the relevant content is absent.
• The retrieval step is transparent. You can log exactly which chunks were retrieved for any query, which makes debugging tractable.
• The embedding and retrieval components are independent of the LLM. You can swap the language model without rebuilding the indexing layer.

The Four-Part Stack

Four components make up the pipeline. The choices below run entirely on a laptop, with no API key and no per-query cost. For a student at a Tier-2 or Tier-3 engineering college without a dedicated GPU, this matters: every component runs on CPU.

Component	Tool	Why
Document loader	PyPDFLoader (LangChain)	Parses multi-page PDFs into page objects; handles exports from Word and Google Docs
Text splitter	RecursiveCharacterTextSplitter	Splits at paragraph and sentence boundaries before cutting mid-sentence, preserving semantic coherence per chunk
Embedding model	all-MiniLM-L6-v2 (HuggingFace)	22 MB download; runs on CPU only; produces 384-dimensional vectors; fast enough for documents up to a few hundred pages
Vector store	FAISS	In-memory; no server required; one function call to build; save and reload in two lines
LLM backend	Ollama	Runs Llama 3.2 or Gemma 2 locally on 8 GB RAM; no API key, no cost per query

LangChain provides the connective tissue: loaders, splitters, chains, and retriever interfaces that wire these components together without writing the plumbing yourself.

The HuggingFace sentence-transformers library hosts hundreds of pre-trained embedding models. all-MiniLM-L6-v2 is the standard starting point for English text retrieval at this scale; it requires no GPU and no API account.

Building It, Step by Step

Install the required packages:

pip install langchain langchain-community langchain-core faiss-cpu pypdf sentence-transformers

For the local LLM, install Ollama from ollama.com and run ollama pull llama3.2 to download the model (approximately 2 GB on disk).

The complete pipeline in Python:

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama

# Step 1: Load the PDF
loader = PyPDFLoader("company_report.pdf")
documents = loader.load()

# Step 2: Split into chunks
splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50
)
chunks = splitter.split_documents(documents)

# Step 3: Embed and store in FAISS
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
vectorstore = FAISS.from_documents(chunks, embeddings)

# Step 4: Build the retrieval QA chain
llm = Ollama(model="llama3.2")
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
)

# Step 5: Ask a question
response = qa_chain.invoke({"query": "What are the key financial highlights?"})
print(response["result"])

Three parameter decisions worth explaining in an interview:

• chunk_size=500: each chunk is at most 500 characters. Smaller chunks give more precise retrieval but risk splitting a sentence mid-thought. Larger chunks give more context but dilute the relevance signal. Start at 500 and test 300 and 800 on a handful of queries to find what works for your document.
• chunk_overlap=50: adjacent chunks share 50 characters at their boundary, preventing a fact that straddles two chunks from being missed. Roughly one sentence of overlap.
• k=3 in the retriever: the system returns the 3 most similar chunks to the query. Too few risks missing relevant context; too many adds noise. Three is the standard starting value.

To avoid re-embedding the document on every run, save the index with vectorstore.save_local("faiss_index") and reload it with FAISS.load_local("faiss_index", embeddings, allow_dangerous_deserialization=True).

Deploying and Demonstrating

A script that prints answers to a terminal is a project. A URL that a recruiter can open on any device is a project that gets remembered.

The fastest path to a shareable demo is Streamlit. Add a text-input widget, call qa_chain.invoke, display the result, and push to Streamlit Community Cloud in under 20 minutes. For the detailed tradeoff between Streamlit and a GitHub repository, the article on deploying your AI project for fresher interviews covers what each format signals to different interviewers.

One practical tip: index a publicly available document (a company’s annual report, an open-access research paper, or a textbook chapter in the public domain) so the recruiter can verify answers independently. Using a confidential document during a live demo introduces unnecessary risk.

Before adding the Streamlit layer, run 5 to 10 test queries whose answers you can verify from the source document. Note which queries retrieve the wrong chunks and why: the most common cause is a chunk boundary that splits a multi-sentence answer. That debugging experience gives you specific, credible things to say when an interviewer asks what went wrong during development.

How to Write It on Your Resume and in Interviews

The weak version of this project on a resume: “Built a PDF chatbot using Python and OpenAI.”

The stronger version names the architecture:

• “Built a RAG document QA system using LangChain, FAISS, and HuggingFace sentence-transformer embeddings; retrieves the top-3 relevant chunks from a PDF before generating an answer, grounding responses in the document rather than model training data.”

That bullet covers the stack, the architectural pattern, and a design decision. For the full template library on writing AI projects as resume bullets, the guide on writing AI projects on your resume has formats that hold up in both technical panels and HR screens.

Prepare clear answers to these two interview questions on this project:

• Question: “How does the system find the right part of the document?” Answer: Both the query and each document chunk are converted to vectors by the same embedding model. FAISS finds the chunks whose vectors are closest to the query vector by cosine distance. Those chunks go to the LLM as context.
• Question: “What happens if the answer is not in the document?” Answer: A properly built system notes that no relevant context was found rather than generating a confident wrong answer. If your implementation handles this, say so; it shows you thought about the failure mode rather than shipping only the happy path.

This project also stands apart from a simpler API wrapper build, which matters when a technical panel compares portfolios. The guide to framing ChatGPT wrapper projects on your resume explains the distinction in detail. In a RAG system, the retrieval and chunking layers are yours, not the model provider’s. That ownership is defensible under questioning in a way that a thin API wrapper is not.

If you are mapping this project to a broader skill sequence, the 2026 AI roadmap for Indian engineering students shows where RAG fits in the full learning path from Python fundamentals through to deployed systems. Once the build is complete and working, the guide on writing AI projects on your resume is the next stop for turning it into bullets that survive a technical panel.