IMEN891K / IDSC501Fall 2025
Data Engineering
This course covers the data side of AI systems: how data is collected and prepared, how it is stored and processed at scale, and how models and LLM applications run on it. In a team project, students go through an end-to-end data engineering pipeline.
- Instructor
-
Prof. Sangwoo MoPOSTECH IME
- Location & Time
Science Building 4, Room 302
Tuesdays & Thursdays 9:30 – 10:45 KST
- Office Hours
- Thursdays 11:00 – 12:00, Room 319
- Other Courses
- Programming for AX
Course Outline
- Part ILectures 2 – 6 Collection and Preparation Collect data, explore it, and clean, integrate, transform and reduce it.
- Part IILectures 7 – 15 Storage Store and query it with SQL and NoSQL, then process it with MapReduce, Spark, BigQuery, Airflow and streams.
- Part IIILectures 16 – 19 Monitoring and Updates Run models on the data: ML systems, LLM systems, RAG and agents.
Grading
- Attendance10%
- Midterm Exam25%
- Final Exam25%
- Team Project40%
Resources
Reference Courses
- [DMCT] Data Mining: Concepts and Techniques UIUC CS412
- [DMS] Database Management Systems Wisconsin CS564
- [IDS] Introduction to Database Systems Berkeley CS186
- [PDIS] Principles of Data-Intensive Systems Stanford CS245
- [MLSD] Machine Learning Systems Design Stanford CS329S