Core Expertise: Data Engineer with 8+ years of experience in data management, specializing in designing DWH architectures from scratch, building fault-tolerant ETL/ELT pipelines, and integrating heterogeneous data sources. Focused on batch and stream processing for high-load systems (2+ TB / 10M+ daily events) using Python, Apache Airflow, PostgreSQL, and ClickHouse.
Combining:- DWH & Data Architecture: ER diagram modeling, Staging/ODS/Core layer design, and analytical data mart engineering for BI and ML.
- Pipeline Engineering & Orchestration: Airflow DAG development, CDC (Change Data Capture) implementation, pipeline idempotency, and load consistency.
- Data Governance & Reliability: Automated Data Quality and Freshness monitoring, anomaly alerting, and query performance optimization.
Operating Across Three Levels:- Ingestion & Infrastructure: Raw log ingestion (APIs, CDC, flat files), stream validation, deduplication, and reliable loading into DWH.
- Modeling & Transformation: Optimizing data marts (dbt, SQL, Materialized Views), configuring incremental refreshes (pg_cron, dblink), and reducing processing time by 3–4x.
- Business Value: Delivering dependable data infrastructure powering executive BI dashboards (Metabase/Tableau), feature stores, and end-to-end analytics systems.