Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Databricks data engineer certification
·
Module 7
Processing Streaming Data with Structured Streaming
Incremental vs. Full Batch Processing
Explain the concept of incremental data processing versus full batch reprocessing.
2
Unbounded Tables in Structured Streaming
Describe how Structured Streaming treats incoming data as an unbounded table.
3
Reading Data Streams with `spark.readStream`
Read a data stream from a source (e.g., Delta table, cloud files) using `spark.readStream`.
4
Streaming to Delta Lake with Checkpointing
Write streaming query results to a Delta table sink using `spark.writeStream` with checkpointing.
5
Choosing Output Modes for Streaming Data
Select the appropriate output mode (append, complete, update) for a given streaming scenario.
6
Controlling Streaming Execution with Trigger Options
Use trigger options (e.g., availableNow, processingTime) to control streaming execution frequency.
Previous module
Incremental Data Ingestion
Next module
Declarative Pipelines with Delta Live Tables (DLT)