DPH

Discover, Share & Showcase
the Best Data Engineering Projects

Explore curated Data Engineering projects from the community. Be recognized for your projects, vote for your favorites and share your own creations.

Last Week

Apr 5 - 11

0votes

1.Minimalist Data Stack - Nasa data viz

Production-ready data visualization for NASA dataset

Analytics & BIETL/ELT PipelineData Platform
Apache AirflowSnowflakedbtGrafana+1
by thoutmose

Week of Mar 29

Mar 29 - Apr 4

1votes

2.airflow-ai-ops

A self-healing data pipeline platform built on Airflow

Data PlatformExperiments
Apache AirflowPostgreSQLSQLPython
by Rahul Rajasekharan

3.Basalt

A synthetic data generator that guarantees deterministic output

Data Platform
Python
by phanindra562

Week of Mar 22

Mar 22 - 28

0votes

4.Asphalt

Real-Time Vehicle Data Processing Pipeline

Real-time StreamingETL/ELT Pipeline
Apache FlinkApache KafkaJavaPython
by markov-ngz

Week of Mar 15

Mar 15 - 21

2votes

5.Instacart-Analytics-PIpeline

Production-grade analytics pipeline

ETL/ELT PipelineAnalytics & BIData Platform
Apache AirflowdbtPostgreSQLSnowflake+1
by otienoduncan99

6.Airflow Schedule Insights

Predict, simulate, and debug Airflow schedules before they fail.

ETL/ELT PipelineBatch ProcessingData Quality & Testing
Apache AirflowPostgreSQLSQLPython
by egor

Week of Mar 8

Mar 8 - 14

4votes

7.airflow-watcher

An Airflow UI plugin for monitoring DAG failures and SLA misses/delays.

Data Quality & TestingData PlatformExperiments
Apache AirflowPython
by ramsolaimalai

8.WikiStream Event Data Analytics Pipeline in AWS

Content monitoring analytics service using latest AWS S3 Tables along with MSK, EMR (SLA=20 mins)

ETL/ELT PipelineData Quality & TestingReal-time Streaming
Apache SparkAWSdeequSQL+1
by mrshihabullah

Week of Mar 1

Mar 1 - 7

0votes

9.airflow-schedule-heatmap

Visualize how your Airflow DAG schedules are distributed across the day with an interactive heatmap

ExperimentsBatch ProcessingData Platform
Apache AirflowPython
by Rahul Rajasekharan

10.PySemantic

A lightweight, graph-based Semantic Layer. Define metrics once, Generate SQL everywhere.

Analytics & BIData PlatformData Quality & Testing
Python
by nitesh8j

Week of Feb 22

Feb 22 - 28

4votes

11.Flowrs - a TUI for Airflow

Monitor, inspect and trigger Airflow DAGs from the comforts of your terminal

ExperimentsData Platform
Apache Airflow
by jan.vanbuel

Week of Dec 28, 2025

Dec 28 - Jan 3

4votes

12.UFC Data Warehouse

UFC Data Warehouse and dashboards based on up-to-date data

Analytics & BIETL/ELT Pipeline
dbt
by reshefsharvit21

13.Drift Detective

Drift Detective is a Python library for tracking schema evolution using versioned JSON snapshots

Data PlatformData Quality & Testing
Python
by varga.dani6

Week of Dec 21, 2025

Dec 21 - 27

4votes

14.Silism Commerce 360 - Local AI Native Lakehouse

AI-native e-commerce data platform you can run locally (Airflow + dbt + MCP)

Data PlatformETL/ELT PipelineAnalytics & BI
Apache KafkaApache AirflowdbtDuckDB+1
by Silism

15.Real-Time-Sales-Streaming-Pipeline

Modern Lakehouse Architecture with Kafka + Spark Structured Streaming + Delta Lake

Batch ProcessingReal-time Streaming
Apache KafkaApache Spark
by imen.bnamar

16.AIRFLow Medical Data Pipeline

Enterprise-grade ETL pipeline transforming medical XML data into actionable business intelligence

Analytics & BIETL/ELT Pipeline
Apache AirflowPostgreSQLPandasPython+1
by imen.bnamar

17.Bluesky NBA Real-Time Sentiment Analysis

A real-time data streaming pipeline that captures live posts from Bluesky regarding the NBA, perform

Batch ProcessingETL/ELT PipelineReal-time Streaming
Apache KafkaApache SparkElasticsearchGrafana
by imen.bnamar

Week of Dec 14, 2025

Dec 14 - 20

7votes

18.Airflow Python React Widgets

Python React Experiment

ExperimentsData Platform
Apache AirflowPython
by Rahul Rajasekharan

19.Yelp Batch ETL Pipeline

A batch ETL pipeline that processes Yelp business raw data to generate analytics and insights

ETL/ELT PipelineBatch ProcessingAnalytics & BI
Apache AirflowApache SparkScalaMongoDB+1
by darracq.aurelien

20.Airflow and DBT Analytics Accelerator

An Open-source accelerator for a ready-to-run, end-to-end analytics platform

Data PlatformETL/ELT PipelineAnalytics & BI
Apache AirflowdbtClickHouseSQL+1
by gsaipurushoth7

21.Cricket Analytics Data Pipeline

CAP is an end-to-end cricket analytics platform built on Cricsheet ball-by-ball data

Analytics & BIML/AI PipelineExperiments
SnowflakeApache AirflowPythonSQL+1
by Rahul Rajasekharan

Week of Dec 7, 2025

Dec 7 - 13

2votes

22.Yelp Medallion Batch Pipeline

Building medallion architecture for crowd-sourced reviews using Snowflake native features

ETL/ELT PipelineBatch Processing
SnowflakeSQL
by vikneshwararb

Week of Nov 23, 2025

Nov 23 - 29

37votes

23.Smart Wardrobe Suggestion

LLM Based Smart Clothing Suggestion

Batch ProcessingETL/ELT PipelineML/AI Pipeline
Apache Airflow
by Rahul Rajasekharan

24.Reddit ETL Pipeline in Docker

Reddit Data Engineering ETL Pipeline: Spark, Airflow, MinIO in Docker Medallion Architecture

ETL/ELT PipelineData PlatformBatch Processing
Apache SparkApache AirflowMinIOPython+1
by Abdullah

25.Flink Sales Pipeline

Real-Time E-Commerce Sales Analytics Pipeline

Analytics & BIReal-time StreamingETL/ELT Pipeline
Apache FlinkApache KafkaElasticsearchPostgreSQL+1
by nuti.krish4

26.Baskpipe

Fully AWS-native data pipelines for processing basketball (NBA) data.

Analytics & BIETL/ELT PipelineBatch Processing
PostgreSQLAWS S3AWSSQL+1
by dominik.zsajovic

27.Data Warehousing for Realtime Pipelines

Building a real-time data warehouse with the use of state-of-the-art tools like Apache Kafka..etc

Real-time StreamingData Platform
Apache AirflowPostgreSQLApache Kafka
by wahomewilberforce

28.Github Stars Monitor

Never miss a new top starred repository

Analytics & BIETL/ELT PipelineBatch Processing
dbtPythonDuckDBSQL
by maxime.lemaitre

29.Macro Agents Economic Data Platform

From FRED to Forecasts: A Modern Data Stack for Economic Intelligence

ML/AI PipelineETL/ELT PipelineData Quality & Testing
DagsterdbtDuckDBPolars
by a.noonan

30.E2E Real-Time Data Pipeline

Real-time data pipeline with Kafka, Flink, Iceberg, Trino, and Superset.

Analytics & BIData PlatformReal-time Streaming
Apache FlinkApache IcebergApache KafkaAWS S3+1
by abelst9