Scope of Responsibility:Building and maintaining end-to-end data infrastructure (Data Engineering), testing product hypotheses (A/B testing), and delivering advanced analytics (Data Science / ML & Data Analytics): spanning source integrations, ETL/ELT pipelines, DWH architecture, ML modeling, and experimental design.
System Scale & Data Volume:- DWH Capacity: Managing a Data Warehouse containing 2+ TB of raw and aggregated data (ClickHouse, PostgreSQL).
- Data Flow: Ingesting and validating high-throughput streams of up to 5–10 million daily events (raw web/mobile analytics logs, transactional data, CRM event triggers).
- Pipelines (ETL/ELT): Orchestrating and maintaining 30+ production Airflow DAGs/pipelines with automated SLA, latency, and Data Quality/Freshness checks.
Key Responsibilities:Data Engineering, ETL/ELT & DWH Architecture:- Architected and deployed a 2+ TB Data Warehouse ecosystem: Structured Staging, ODS, and Core (Marts) layers to consolidate data from marketing platforms, CRMs, and web analytics tools.
- Automated batch and incremental ETL/ELT data processing pipelines (Python, SQL) handling up to 10M daily events with built-in deduplication and ingestion validation.
- Engineered and optimized complex SQL scripts, stored procedures, and Materialized Views (dbt, pg_cron) to accelerate data mart aggregation by 3–4x and prepare feature stores for ML tasks.
- Configured automated data freshness and accuracy test suites (Data Quality) to validate data prior to downstream reporting and ML consumption.
Data Science, Machine Learning & A/B Testing:- A/B Testing: Designed and executed end-to-end A/B experiments (sample size determination, experiment design, MDE, p-value statistical significance testing), building event logging infrastructure to ensure proper traffic splitting.
- ML & Data Science: Developed and deployed machine learning models (Python, scikit-learn, statsmodels) addressing core business cases (churn forecasting, LTV prediction, customer segmentation/classification, sales forecasting).
- Built automated feature stores, scoring pipelines, and feedback loops back into DWH and CRM systems.
Data Analytics & End-to-End BI Reporting:- Engineered end-to-end attribution models evaluating marketing channel performance, ROMI, CAC, and LTV.
- Designed and delivered executive BI dashboards tracking sales funnels, user retention, and core operational KPIs for leadership and product teams.
- Bridged communication between business stakeholders and technical teams, authoring technical specs for event logging and tracking enhancements.
Key Achievements:- Scalable Infrastructure & ML: Built an automated DWH ingestion and consolidation pipeline from scratch handling 10M daily events and establishing reliable ML data prep and A/B testing pipelines.
- Performance Optimization: Migrated core transformations to incremental dbt models in ClickHouse, reducing daily data mart processing time from 15–20 minutes down to just a few minutes.
- Analytics & Business Impact: Established a single source of truth for analytics and BI dashboards, providing transparent tracking of critical metrics (CAC, LTV, funnels) and enabling data-driven decision-making through statistically validated hypotheses.