IMEN891K / IDSC501 / AXGS-501 · Data Engineering · Fall 2026

16. RAG

Sangwoo Mo · POSTECH
Industrial & Management Engineering

Where We Are

Part III connects stored data to models, retrieval, and actions

Outline: RAG

01 · How RAG Answers Questions

§ Embeddings and Storage

Recap: LLM WorkflowA retrieval step supplies external evidence to the model

Question → Retrieval → Prompt → LLM

Retrieval-Augmented GenerationRAG conditions generation on evidence retrieved at query time

Indexing prepares documents; retrieval supplies answer context

Why RetrievalPrivate and changing facts need evidence outside model parameters

Retrieval reads the approved manual at question time

Document LoaderA loader supplies text and source metadata to the indexing pipeline

The source reference travels with the loaded content

Text EmbeddingsAn embedding maps text into a vector used for similarity search

Similar wording or meaning can produce nearby vectors

Embedding SimilaritySemantic similarity can differ from exact word overlap

The two animal sentences are closer than the car sentence

Cosine SimilarityCosine similarity compares vector direction

Same direction, same score, whatever the length

Embedding InterfaceUse one compatible embedding space for documents and queries

import { OllamaEmbeddings } from '@langchain/ollama';

const embeddings = new OllamaEmbeddings({
  model: '<installed-embedding-model>',
  baseUrl: 'http://127.0.0.1:11434'
});
const documents = await embeddings.embedDocuments([
  'P-17 pressure alarm threshold is 5 bar.'
]);
const query = await embeddings.embedQuery('P-17 pressure threshold');
// Use one embedding model and compatible preprocessing for both.

Vector StorageStore vectors together with the original chunks and metadata

A vector is a search key; the text remains answer evidence

Vector SearchA query vector retrieves nearby chunk vectors

The query selects candidate documents through the vector index

Vector Index ChoicesSelect the index using collection size, latency, and recall

Index recall and embedding relevance are separate questions

Caveat: Embedding VersionA model change requires a compatible query and document index

Mixing models compares points from two different spaces

§ Answer Construction

Answer GenerationThe model receives the question and selected evidence

The answer prompt receives retrieved chunks, not the whole corpus

Context RecordKeep the selected text and its source reference together

context = [
  {'source_id': 'manual_v4:p12',
   'text': 'Steel flange: tighten to 12 N·m.'},
  {'source_id': 'manual_v4:p13',
   'text': 'Polymer flange: tighten to 8 N·m.'}
]

Grounded Answer PromptAsk for evidence references and an explicit unsupported outcome

# Prompt template
Use only the evidence below for factual claims.
If the evidence is insufficient, say what is missing.
Attach source IDs to the claims they support.

Question: {question}
Evidence: {context}

Context Composition ChoicesChoose how retrieved evidence enters generation

Four ways evidence enters generation

02 · Preparing Documents for Search

§ Loading

Document LoadingA loader reads a source and returns content with metadata

Loaders normalize heterogeneous sources into documents

Document RecordPreserve identity and provenance alongside the text

document = {
  'doc_id': 'pump_manual_v4',
  'text': extracted_text,
  'metadata': {
    'source': 'manuals/pump_v4.pdf',
    'page': 12, 'version': 4,
    'access_group': 'maintenance'
  }
}

Loader ValidationLoading success does not imply faithful content extraction

Inspect extracted content before it enters the index

Caveat: PDF Reading OrderFlattening a table can detach a number from its meaning

Flattening detaches 12 from Pump A

§ Splitting

Document SplittingSplitting creates the retrieval units from loaded documents

Chunk boundaries determine which evidence travels together

Splitting StrategiesChoose boundaries that match the document structure

The chunk must keep the evidence an answer needs

Chunk Size and OverlapOverlap keeps some boundary context at the cost of duplication

Neighboring chunks share a controlled overlap

Splitting the ManualRun a real text splitter and inspect the retained metadata

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 85, chunkOverlap: 15
});
const chunks = await splitter.splitDocuments(manuals);
return chunks.map(d => ({id:d.metadata.id,
  text:d.pageContent, length:d.pageContent.length}));

Demo: Manual ChunksChange chunk size and overlap; inspect text and source IDs

Set chunkSize to 35 and inspect split conditions.
Compare chunkOverlap 0 and 15. Keep overlap smaller than chunk size.

Run code to inspect its output.

Ready

LangChain.js · JavaScript · Ctrl/Cmd+Enter · Real library execution with local data

Answer: Manual ChunksThe baseline output comes from executing the shown code

Overlap repeats a few words; split conditions still separate

Chunk IdentityAssign each chunk a stable source reference

chunk = {
  'chunk_id': 'pump_manual_v4:p12:c2',
  'doc_id': 'pump_manual_v4',
  'text': 'Tighten the flange bolts to 12 N·m.',
  'metadata': {'page': 12, 'section': 'Flange assembly'}
}

Caveat: Broken EvidenceSplitting a condition from an instruction can reverse the answer

A chunk boundary can drop the condition

03 · Finding Evidence and Checking Answers

§ Retrieval

Retriever InterfaceA retriever returns evidence records for a query

Question → Candidate search → Selected chunks

Search MethodsLexical and dense search fail on different queries

Exact IDs need lexical matching; rewording needs dense search

Caveat: Similar but WrongA high similarity score can retrieve the wrong entity

The closest vector belongs to the wrong pump

Retrieval RequestCarry constraints separately from the natural-language question

query = {
  'text': 'What is the flange torque?',
  'filters': {'equipment_id': 'P-17', 'approved': True},
  'top_k': 5
}
# Conceptual interface; actual filter syntax depends on the store.

Metadata FiltersRestrict candidate records using source properties

Each mismatched tag removes a candidate

Searching the ManualRetrieve documents with a real vector store and an exact filter

const chunks = await new RecursiveCharacterTextSplitter({
  chunkSize: 100, chunkOverlap: 15
}).splitDocuments(manuals);
const store = await MemoryVectorStore.fromDocuments(chunks, makeEmbeddings());
const k = 2;
const filter = d => d.metadata.equipment === "P-17" && d.metadata.approved;
const hits = await store.similaritySearch("P-17 pressure alarm threshold", k, filter);
return {embedding:EMBEDDING_MODE, hits:hits.map(d => ({
  source:d.metadata.id, text:d.pageContent}))};

Demo: Filtered RetrievalChange k and the filter; inspect which version becomes evidence

Remove approved from the filter. Does the obsolete 7 bar rule appear?
Search P-18 and update the equipment filter together.

Execution

Run code to inspect its output.

Ready

LangChain.js · JavaScript · Ctrl/Cmd+Enter · Offline replay uses fixed responses; local mode calls your model

Answer: Filtered RetrievalThe baseline output comes from executing the shown code

{"embedding": "Local token-presence vectors (not learned)", "hits": [{"source": "p17-v4", "text": "P-17 pressure alarm threshold is 5 bar. Stop if pressure exceeds 5 bar. Use Celsius for temperature."}, {"source": "p17-vibration", "text": "P-17 vibration inspection: record vibration and temperature before changing the operating speed."}]}

Query ReformulationA rewritten query can recover evidence missed by the original wording

Original → Rewrite → Search → Merge

Caveat: Query DriftA rewrite can silently change the question

Retrieval succeeded on the wrong question

Evidence Selection MethodsChoose the operation by what the candidate set still lacks

Choose by what the candidate set still lacks

RerankingA reranker scores the query together with each candidate

Reranking reorders only what retrieval found

Maximum Marginal RelevanceMMR balances query relevance with novelty among selected chunks

Diverse candidates reduce repeated evidence

MMR ObjectivePenalize similarity to evidence that has already been selected

MMR trades relevance against repetition

Context CompressionCompression removes irrelevant text before generation

Only selected evidence enters the answer prompt

Caveat: Compression LossCompressing away a condition can change a safety instruction

Compression removed the condition

Complete RAG ApplicationSearch approved records, build context, and invoke the answer chain

const chunks = await new RecursiveCharacterTextSplitter({
  chunkSize: 100, chunkOverlap: 15
}).splitDocuments(manuals);
const store = await MemoryVectorStore.fromDocuments(chunks, makeEmbeddings());
const retriever = RunnableLambda.from(q => store.similaritySearch(q, 2,
  d => d.metadata.equipment === "P-17" && d.metadata.approved));
const prompt = ChatPromptTemplate.fromMessages([
  ["system", "Use only evidence. Cite source IDs. If missing, return unknown."],
  ["human", "Question: {question}\nEvidence: {context}"]

]);
const model = makeModel(["P-17: 5 bar [p17-v4]"]);
const chain = prompt.pipe(model).pipe(new StringOutputParser());
const question = "What is the P-17 pressure alarm threshold?";
const hits = await retriever.invoke(question);
const context = hits.map(d => `[${d.metadata.id}] ${d.pageContent}`).join("\n");
return {mode:MODEL_MODE, sources:hits.map(d=>d.metadata.id),
  answer:await chain.invoke({question,context})};

§ Evaluation and Updates

Question Answering PipelineA QA pipeline combines a retriever with an answer generator

A question produces evidence before an answer

Conversational RetrievalResolve the current question before retrieving new evidence

Conversation history helps form the retrieval question

Caveat: Follow-Up AmbiguityHistory can resolve a pronoun to the wrong equipment

"Its" points to two pumps

RAG Evaluation LayersDiagnose the first stage that loses the required evidence

A wrong answer may start with parsing or retrieval

Retrieval Recall at kRecall measures how many relevant chunks are retrieved

Relevance labels come before the score

Answer EvaluationCorrectness and groundedness are different questions

A cited answer still needs claim-level checks

Case study: Versioned ManualsA correct fact from an obsolete manual can be the wrong answer

Entity, material and version decide the answer

Index RefreshSource changes must reach chunks, vectors, and cached answers

Detect → Transform → Index → Validate

Caveat: Stale ChunksDeleting a source file may leave its indexed text searchable

Deleting the file left its chunks searchable

RAG Release BundleVersion the corpus and index together with the answer system

Corpus → Index → Retrieval → Generation

Takeaway: Retrieve the evidence before trusting the answer

1Index content together with identity, versions, and access metadata

2Evaluate retrieval before changing the answer prompt

3Return supported answers and keep the index current