Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Data Engineering Pipeline Building
·
Module 4
Orchestrating Workflows with Airflow
1
Airflow Fundamentals: DAGs, Tasks, Operators, and Scheduling
Explain Airflow's core concepts: DAGs, tasks, operators, and scheduling.
2
Building a Standalone Ingestion Script in Python
Author a Python script for a self-contained ingestion task.
3
Orchestrating Data Ingestion with Airflow's PythonOperator
Create an Airflow DAG that orchestrates the API and CSV ingestion scripts using the PythonOperator.
4
Scheduling DAGs with Cron and Airflow's Data Interval
Define a schedule for a DAG using a cron expression and explain Airflow's data-interval semantics.
5
Triggering DAGs with File Sensors
Use a FileSensor to trigger a DAG run when a new file arrives.
6
Incremental Data Ingestion with Postgres High-Watermark
Implement an incremental ingestion strategy using a high-watermark timestamp stored in Postgres.
7
Testing Data Ingestion with Pytest and Test Databases
Write unit tests for an ingestion function using pytest and a test database.
Next module isn't ready yet,
Back to course
Previous module
Implementing Batch Ingestion Patterns
Next module