A live transit-data lakehouse that ingests MBTA real-time feeds, computes on-time performance, and runs, heals, and improves itself — with an agentic layer that writes its own insights and opens pull requests.
A live transit-data lakehouse that ingests MBTA real-time feeds, computes on-time performance, and runs, heals, and improves itself — with an agentic layer that writes its own insights and opens pull requests.
A production-grade ELT pipeline that automates daily identification of high-value customers using Apache Airflow, dbt-spark, and Apache Iceberg.
A comprehensive guide to implementing modern analytics engineering practices using dbt, BigQuery, and Looker Studio. Learn to build scalable data transformation pipelines with dimensional modeling, testing, and deployment strategies.
A comprehensive guide to implementing a scalable, cost-effective data pipeline and warehouse using Google BigQuery, featuring external tables, partitioning, clustering, and performance optimization with NYC taxi data.
A practical guide to building and orchestrating robust data pipelines using Apache Airflow, covering local, cloud (GCP), and Kubernetes deployments.
Workflow Orchestration using Kestra tool
A step by step guide of a simple data pipeline
How artificial intelligence is transforming data systems and workflows
Best practices for creating efficient and scalable ML pipelines
An introduction to the fundamentals of data engineering
Introduction and overview of what you can find on this site