Real-Time Data Processing with Azure Databricks
Spark Structured Streaming with Event Hubs — windowing, stateful processing and real-time anomaly detection.
Platform: Azure Databricks + Azure Event Hub
Sign in to read this course
A free account unlocks all 514 courses. 20 are readable without one.
What's inside
23 sections- 1 Table of Contents
- 2 Module 1. Batch vs Real-Time — Fundamentals
- 3 Module 2. Azure Event Hub — Architecture and Components
- 4 Module 3. Event Hub vs Apache Kafka
- 5 Module 4. Real-Time Streaming Use Cases
- 6 Module 5. Environment Setup
- 7 Module 6. Spark Structured Streaming — Introduction
- 8 Module 7. Connecting Databricks to Azure Event Hub
- 9 Module 8. Simulated IoT Data Producer
- 10 Module 9. Spark Structured Streaming Consumer
- 11 Module 10. Schema Evolution and Late-Arriving Data
- 12 Module 11. Checkpointing and Fault Tolerance
- 13 Module 12. Temporal Aggregations — Windowed Operations
- 14 Module 13. Stateful Processing — Complex Alerts
- 15 Module 14. Real-Time Anomaly Detection (ML)
- 16 Module 15. Data Stream Enrichment
- 17 Module 16. Autoscaling for Streaming Workloads
- 18 Module 17. Monitoring Streaming Jobs
- 19 Module 18. Troubleshooting Latency Issues
- 20 Module 19. Streaming Cost Optimization
- 21 Module 20. Real-Time Reference Architecture
- 22 Module 21. Summary and Best Practices
- 23 Module 22. Glossary
More Azure Databricks & Spark courses
View all 14Administering Clusters and Configuring Policies with Databricks
Databricks architecture, cluster types and runtimes, autoscaling, cluster policies, pools and init scripts.
ETL Pipelines with Azure Databricks and Data Factory
Build ETL with Spark and PySpark, Unity Catalog governance, Delta Lake and Databricks vs Data Factory.
Manage Data with Azure Databricks and Azure Data Lake
Connect Databricks to ADLS Gen2 securely, ingest with Auto Loader and govern with Unity Catalog.
Optimize Storage and Performance with Delta Lake
Delta Lake internals, ACID, OPTIMIZE, Z-Order, liquid clustering, caching and Photon acceleration.
Machine Learning with Azure Databricks
The Databricks ML lifecycle: MLflow tracking, tuning with Ray, the model registry, serving and AutoML.
Building Deep Learning Models on Databricks
Build, train, tune and serve deep-learning models on Databricks with TensorBoard integration.
Interested in this course?
Contact us to book it or get a custom training plan for your team.