IMEN891K / IDSC501 / AXGS-501 · Data Engineering · Fall 2026
15. LLM Systems
Sangwoo Mo · POSTECH
Industrial & Management Engineering
Where We Are
Part III connects stored data to predictions, answers, and actions
Outline: LLM Systems
The P-17 QuestionWhat is the pressure alarm threshold for P-17?
The answer needs the current P-17 manual
01 · Inputs and Evidence
Large Language ModelsA large language model (LLM) generates text from its input
The input is visible; the generated answer still needs checking
Language Models: A Turning PointIn 2020, GPT-3 showed how task instructions and examples could guide one pretrained model
The input can describe a task without retraining the model for each request
Question Without EvidenceA question alone does not supply the current manual
7 bar is an unsupported example answer
Question With EvidenceThe manual gives the model a source it can use
The supplied P-17 rule states 5 bar
PromptThe prompt is the input sent to the model
Instructions and the question share one input
ContextContext includes the material supplied for this request
The manual text becomes part of the model input
Answer With a SourceA source reference lets a reader inspect the claim
5 bar points back to p17-v4
A Collection of ManualsThe collection contains similar rules for different cases
P-17 has a current rule, an obsolete rule, and a neighbouring pump
02 · Documents and Search
Why Document PreparationThe answer needs the rule and the equipment name together
A value without its row can describe the wrong pump
Loading the ManualLoading extracts readable text from the source
The equipment name, value, and unit survive extraction
Document PiecesA chunk is a small piece that can be retrieved
Each piece keeps a route back to its source
Caveat: Broken ConditionsA cut can separate an instruction from its condition
The stop instruction loses its threshold after a bad cut
Source LabelsA piece keeps its equipment and version alongside the text
The text and its labels travel together
RetrievalRetrieval finds pieces related to the question
A search returns evidence candidates rather than a finished answer
Embeddings: Text Becomes a VectorAn embedding represents text as numbers that can be compared
Related meanings can have similar vectors even when their words differ
Store the Text Beside Its VectorThe stored vector helps find a piece while the text remains the evidence
Keep the original text so the answer can quote or cite its source
Search with the Question VectorEncode the question in the same way, then compare it with stored pieces
The selected vectors lead back to candidate source text
Caveat: Similar but WrongSimilar wording can belong to an obsolete rule
Both P-17 versions mention the same pressure question
Eligible EvidenceOnly approved P-17 sources can support this request
The current rule remains after the equipment and approval checks
Demo: Find the ManualChange the request and approval filter, then inspect the returned source text
Predict which sources will appear, then select Search.
| Source | Returned text | Similarity |
|---|
| ? | Search has not run | ? |
Read the text: does it give the current pressure threshold for this equipment?
Local word-presence search, not a learned embedding or a model answer
Answer: Find the ManualThe baseline returns one pressure rule and one unrelated inspection note
Only p17-v4 answers the pressure threshold question
03 · Answers and Checks
Why Answer ChecksFinding the right source does not force the right answer
The source says 5 bar while the candidate says 7 bar
Evidence in the InputSelected text enters the prompt with its source label
The 5 bar rule reaches the model as p17-v4
Retrieval-Augmented Generation (RAG)A 2020 RAG paper connected document retrieval with text generation
Retrieval-augmented generation connects the source to the answer
Checking the EvidenceDoes the selected text answer this question for this equipment?
Current P-17 pressure text supports the requested value
Checking the AnswerDoes the cited text support the value and unit in the answer?
The citation must support the claim beside it
Answer or AbstainMissing evidence calls for an explicit unknown
No supplied document gives the bearing replacement interval
Case study: A Withdrawn ManualA document can disappear while its searchable copy remains
A withdrawn source can still leave searchable pieces
A Small Answer CheckTest the same path with a current, wrong-equipment, and missing-evidence question
One collection exposes the main ways this answer can fail
Takeaway: Answers need evidence that readers can check
1Put the material in the model input
2Retrieve complete pieces from the right current source
3Check the evidence and the answer; say unknown when evidence is missing
