Hi There! I'm Mani Kumar Singh. A Data Engineer Building Scalable Pipelines & Lakehouse Systems
Years of Experience
Senior Data Engineer specializing in scalable lakehouse architectures, production Snowflake & dbt pipelines, and high-reliability data systems for Fortune 500 enterprises.
Lakehouse & Warehousing
Architecting scalable cloud data warehouses and lakehouses on Snowflake and GCP BigQuery with optimized partitioning, clustering, and dimensional models.
Automated ETL/ELT & dbt
Building robust modular data transformation pipelines using dbt, SQL, and PySpark following the Medallion Architecture (Bronze → Silver → Gold).
Governance & Automation
Implementing automated CI/CD deployment workflows, schema validation, and stringent data quality checks that eliminate 20+ hours of weekly manual toil.
Production Data Systems
Customer Reward Analytics & BI Rationalization
Reverse-engineered 300+ complex SQL queries into 105 centralized dbt mart models on Snowflake, significantly reducing cloud compute costs while ensuring data integrity.
Enterprise E-Commerce & Retail Delivery Platform
Architected end-to-end retail data pipelines with automated dbt tests and native Snowflake Alerts, establishing an INFORMATION_SCHEMA DQ system.
GCP Cloud-Based Medallion Retail Lakehouse
Cleaned and transformed multi-layer retail transactional datasets across Medallion architecture using Dataproc, Dataprep, and BigQuery on Google Cloud Platform.
End-to-End Medallion Pipeline Flow
Multi-Source Ingestion
Batch & streaming ingestion from retail POS logs, marketing APIs, transactional databases, and event streams.
Raw Data Lake
Immutable append-only raw storage. Preserves source fidelity, lineage audit history, timestamps, and schema-on-read flexibility.
Cleansed & Conformed
Deduplication, schema enforcement, data type casting, business logic transformations, and data quality testing via dbt models.
Curated Business Marts
Star & snowflake dimensional schemas optimized for low-latency BI queries, executive reporting, and downstream ML features.
Professional Experience
Tredence Inc.
Promoted to ConsultantConsultant
- End-to-End E-Commerce & Retail Delivery: Architected production data pipelines for omnichannel retail domains (online/offline store transactions, e-commerce orders, customer demographics, and store analytics). Managed full data lifecycle from source extraction and profiling to dimensional modeling and delivering governed business tables.
- BI Rationalization & Query Consolidation: Reverse-engineered 300+ redundant, complex custom SQL queries powering Sigma BI reward reports, consolidating them into 105 centralized, highly optimized dbt mart models, significantly cutting cloud compute expenditure while guaranteeing data integrity.
- Snowflake Alert & Monitoring Framework: Developed an automated real-time monitoring system combining dbt tests with native Snowflake Alerts to intercept source discrepancies before loading into target tables, instantly notifying stakeholders.
-
Automated Data Quality (DQ) System: Leveraged Snowflake
INFORMATION_SCHEMAand scheduled audit tasks to enforce schema validation, referential integrity, and business rule conformance. - Production Support & Incident RCA: Owned pipeline uptime SLAs, conducted root cause analysis (RCA) on production incidents, implemented automated archive models for missing transactions, and mentored junior engineers on dbt standards.
Analyst
-
Enterprise E-Commerce & Retail Pipeline Engineering (Fortune 500 Client):
Tech Stack: Snowflake, dbt Core, SQL, Python, GitLab CI/CDBuilt and maintained core production data pipelines for enterprise retail transactions, inventory updates, and store-level feeds. Developed dbt transformation models, executed regular data reconciliation routines, and provided production support and incident resolution.
-
Marketing Campaign Analytics Platform (Fortune 500 Client):
Tech Stack: SQL, Power BI, SQL Server, PythonEngineered automated ingestion and transformation pipelines for omnichannel marketing campaign data (push notifications and email events). Built executive Power BI dashboards tracking sales, YoY revenue growth, and campaign performance, discovering high-value customer patterns that drove a 20%+ marketing-to-sales conversion uplift.
-
T-Discover — Enterprise AI Product Development:
Tech Stack: Azure Databricks, Azure Data Factory (ADF), PySpark, Python, ML Similarity ModelsCore engineer building Tredence's flagship AI product to revolutionize BI report rationalization and eliminate redundant data assets using ML similarity scoring. Optimized Databricks Spark clusters, tuned PySpark functions, and automated production workflows with ADF.
-
GCP Cloud-Based Medallion Retail Lakehouse:
Tech Stack: Google Cloud (GCP), BigQuery, Dataproc, Dataprep, Cloud Storage, SQL, PythonCleaned and transformed multi-layer retail transactional datasets across Medallion architecture (Bronze → Silver → Gold) using Dataproc, Dataprep, and BigQuery, establishing automated execution logging and technical documentation.
Bijli Solutions
Data Engineering & Analytics Intern
Built automated data extraction and transformation pipelines using Python and SQL. Streamlined reporting and audit reconciliation routines, reducing analysis turnaround time by 60% and eliminating 20+ weekly manual operational hours.
NIT Jalandhar
Bachelor of Technology (B.Tech)
Dr. B. R. Ambedkar National Institute of Technology Jalandhar
Premier technical institute established under the NIT Act by the Government of India, recognized for engineering and research excellence.
Core Engineering Foundation
- Computational Logic: Algorithms, data structures, and distributed computation.
- Data Foundations: Relational Database Management Systems (RDBMS) & SQL.
- Quantitative Analysis: Engineering mathematics, statistics, and probability.
- Analytical Problem Solving: Structured systems design & architecture principles applied directly to enterprise data engineering.
Delivering for Fortune 500 Retailers
Architecting and operating high-throughput production data pipelines on Google Cloud and Snowflake, ensuring critical sales, customer, and marketing datasets are delivered with high availability.
Core Tech Stack & Tools
Cloud & Warehousing
ETL & Modeling
Code & Automation
Quality & Analytics
Certifications & Honors
SnowPro Core Certified
Certified in cloud data warehousing, performance optimization, data sharing, security governance, and scalable storage architecture.
Verify Credentialdbt Certified Developer
Demonstrated expertise in building modular, tested, and documented analytics engineering transformation workflows with dbt Core.
Verify CredentialProfessional Data Engineer
Certified in architecting data processing systems, operationalizing machine learning models, and building scalable BigQuery & GCP pipelines.
Verified ProfessionalPAT ON THE BACK Award
Honored with enterprise recognition for outstanding engineering performance, dedication, and exceptional contribution to retail client projects.
Excellence Honor