Data Pipeline Fundamentals

Describe the role of a data engineering pipeline and its typical stages from source to consumption.

Contrast ETL and ELT architectures and identify when each is appropriate.

Compare batch and streaming processing in terms of latency, throughput, and use cases.

Explain idempotency in data pipelines and why it is critical for reliability.

Define the medallion (bronze, silver, gold) data layering pattern and its purpose.

Outline the target architecture for the project using the selected technology stack.