README.md

Hi, I'm João Blasques 👋

I'm an AI-Enabled Data Engineer with around 8 years across data and software engineering, 3 of them full-time in data engineering. I build data platforms the way software should be built: tested transforms, CI/CD, infrastructure as code, and data-quality gates that fail the build instead of writing a report nobody reads.

Core expertise

  • Data Engineering — ELT pipelines, lakehouse design (medallion), dimensional modelling
  • Databricks & Spark — Unity Catalog, Delta, Asset Bundles, Structured Streaming
  • Transformation & quality — dbt, Great Expectations and dbt tests wired into CI
  • CI/CD & IaC — GitHub Actions, Terraform, keyless WIF/OIDC, Docker
  • AI-enabled — agent-operated workflow, feature stores, embeddings, LLM pipelines

Curious about my work? Browse my posts and projects, or read more about me.

1,181 contributions in the last year

Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Mon
Wed
Fri
Less More

Pinned

View all projects →
vitals Site

From raw clinical signals to trusted, AI-ready data — a governed health-data medallion lakehouse with OMOP standardization, a Great Expectations DQ gate, real Kafka streaming, and cross-engine parity (DuckDB/Databricks/PySpark) serving analytics marts, an ML feature store and a vector index.

Python
corpus Site

A knowledge base that tends itself. Sources in, cited pages out — an LLM agent reads everything, writes it into a cross-linked web of citable pages, and keeps it consistent as new sources arrive.

Python
nora Site

An email-native AI executive assistant. Nora drafts, the human sends — that asymmetry is the product's trust model.

Python

A self-managing MBTA on-time-performance lakehouse on Databricks + GCP, with an agentic layer that writes its own insights and opens pull requests.

Python

Daily ELT pipeline that identifies high-value customers for sales outreach — Airflow, dbt, Spark, Iceberg, Docker.

Python

Real-time e-commerce analytics platform using Apache Spark, PySpark and Kafka.

Python

Popular posts

An email-native AI executive assistant built on one hard guarantee: she drafts, you send, and every action she takes is cryptographically gated behind DKIM sender verification on the way in and explicit human confirmation on the way out.

AIllmagenticpython

A health-data lakehouse that turns messy FHIR, claims, wearable, and notes data into three trusted outputs — analytics marts, an ML feature store, and a vector index — with de-identification, OMOP standardization, and cross-engine parity testing that caught a real de-id gap.

data engineeringdatabrickslakehousedbt

A live transit-data lakehouse that ingests MBTA real-time feeds, computes on-time performance, and runs, heals, and improves itself — with an agentic layer that writes its own insights and opens pull requests.

data engineeringdatabricksgcplakehouse

A production-grade ELT pipeline that automates daily identification of high-value customers using Apache Airflow, dbt-spark, and Apache Iceberg.

data engineeringairflowdbtspark