Prateek Dubey

Director — AI & Data Engineering

• Principal AI & Data Engineer in the Technology Consulting division, leading a team of 10+ AI, Data & Software Engineers.
• Working with clients across ASEAN on GenAI, Agentic AI and real-time data solutions and architecture.
• Building EY’s Agentic AI Accelerator — the Decisions Operation Layer platform — bringing Agents to enterprise customers with end-to-end Agent observability, governance and guardrails.
• Delivering GenAI projects for FairPrice Group (POS AI Knowledge Assistant) and Temasek (AI Fluency Assessment Platform).

Head of Data Engineering & Architecture

• Partnered with global senior leadership to define and execute Chubb Life’s enterprise-wide Data & AI strategy, shaping technology roadmaps to drive measurable business impact.
• Built, coached and scaled a high-performing regional team of 50+ Engineers, establishing a COE for Data Engineering and applied AI.
• Directed the implementation of a cloud-native data platform with scalable frameworks for ingestion, transformation and governance.
• Led development of applied ML solutions, including RAG-based AI assistants for self-service analytics, and pioneered AI Agents to automate and scale DE and ML.

Data Engineering Manager / Staff Data Engineer

• Managed, mentored and coached a team of 20+ Data Engineers covering Data Engineering, MLOps, AIOps, and Data & AI Architecture.
• Data architecture lead for Public and Private sector clients — standardized architectural frameworks and methodologies for scalable, reusable solutions.
• Delivered the Industry Partnerships Platform for Ngee Ann Polytechnic using LLMs for conversational AI on Azure GCC.
• Lead Engineer & Architect for FWD Insurance — rolled out LLM-powered voice-to-voice conversational AI avatars for sales agent training, scaled on EKS to serve thousands of users across South East Asia.
• Worked with Temasek Trust & Laconic on Lakehouse design on Azure & AWS using Synapse, EKS, ADF, Spark, dbt and Open Metadata.
• Led pre-sales, business development, proposals and tech consulting; closed deals worth 500k SGD.
• Developed internal solutions to automate ETL development using LLMs and data governance using Great Expectations & Open Metadata.

Data Engineering Manager / Staff Data Engineer

• Worked with Dafiti (LATAM) on data strategy, restructuring, prioritisation, roadmap planning and adopting data mesh architecture by treating data as a product.
• Managed a team of 8 Data & MLOps Engineers across Vietnam and India covering Data Engineering, DevOps, MLOps, Solution Architecture, Data Governance, Observability and Cloud.
• Staff Engineer / Architect for the Global Data Platform (GDP) built on OSS CNCF, LF AI & ASF stack on Kubernetes (AWS EKS) — the central data platform for GFG, Zalora, The Iconic and Dafiti, including Model Serving.
• Technical leader for GDP (Infrastructure as a Service) and DaaS (Data as a Service); delivered data and ML solutions for Pricing Engine, Recommendation and Personalisation with Zalora (SEA) and The Iconic (ANZ).
• Led the shutdown of Snowflake and Redshift cloud DWHs to cut cost, migrating to a Trino-backed Iceberg Lakehouse.
• Helped Zalora, The Iconic and Dafiti migrate Seller Dashboards to GDP for a subscription-based data product.
• Ingested GA3 (The Iconic, Dafiti) and Segment (Zalora) events into GDP for behavioural data recommendation.
• Improved observability and availability using Loki, Thanos, Grafana and Prometheus with Slack and email alerts.
• Incorporated GitOps methodology; set up CI/CD with GitHub Actions, Argo CD, Ansible and Terraform.
• Set up Data Governance & Quality for the Iceberg Lakehouse using Apache Ranger, DataHub and Great Expectations.
• Set up the ML-Ops stack: MLflow (experimentation), Feast (feature store), KServe (model serving), Kong (API gateway).

Data Engineering Manager / Lead Data Engineer

• Built the Telco Cyber Data Lake & Platform using Hybrid Cloud for Cyber Analytics and Threat Intelligence for Singtel Cybersecurity.
• Lead Architect and Engineer for the Telco CDL Kubernetes Platform — on-premise with Rancher and on AWS with EKS.
• Processed TBs of data using Spark on K8s stored in Kafka, Ceph and S3 across AWS and on-premise.
• Built streaming and batch pipelines with Apache Kafka and Spark Streaming, orchestrated with Airflow.
• Set up logging and monitoring using Elasticsearch, FluentD, Grafana and Prometheus.
• Set up DevOps and CI/CD processes using GitLab CI, Argo CD, Ansible and Terraform.
• Set up Data Governance for Kafka, Ceph, Hive and S3 using Apache Atlas, Apache Ranger and Amundsen.
• Set up ML-Ops with MLflow to track experiments, metrics and parameters.

Data Science Lead

• Technical lead for the ML Infrastructure team — cloud architecture across AWS and Azure for Data Science, spanning ML-Ops, DevOps, Cloud, Data Engineering and Data Science.
• Led a team of ML Engineers building and supporting ML Platforms on AWS and Azure — data engineering, feature engineering, model training and deployment.
• Data Science PIC for vendor engagement with AWS, Microsoft and Databricks; owner of all DS cloud accounts.
• Implemented Data and Infrastructure Governance across ML Platforms on AWS and Azure.
• Developed the GrabFood restaurant behavioural recommendation system with the Azure CAT team, increasing click-through rate by 170% over existing recommenders.
• Developed a multi-user JupyterHub and Kubeflow Data Science ML Platform on AWS EKS for ML and analytics.
• Automated cloud infrastructure with Terraform and Azure ARM; used Helm on EKS and AKS.
• Architected platform tooling: Spark UI Proxy, JupyterHub resource tracking, AWS and Datadog monitoring for EKS, and Agartha — Docker build automation via Git pull requests.

Senior Data Engineer

• Part of Cloud Engineering, responsible for migrating the Athena SAS DWH onto AWS for Risk & Finance Solutions.
• Designed the Data Lake on AWS used by Data Engineering & Analytics for ETL, analytics and machine learning.
• Automated AWS cloud infrastructure using Terraform.
• Built a Python parser to auto-convert HiveQL to PySpark (Spark SQL) jobs on AWS EMR, reducing conversion time by over 90%.
• Reduced job workflow creation time by 80% with an automated Oozie workflow creation framework.
• Built monitoring for active EMR clusters and SageMaker instances across regions using Boto3 on Lambda, with SES notifications scheduled via CloudWatch.
• Enabled Amazon SageMaker for Risk Modelling with Spark, locally and via remote EMR clusters; persisted notebooks in S3.
• Installed and set up Apache Airflow to schedule core Spark ETL workflows.

Senior Data Engineer

• Migrated the SAS Datawarehouse onto Cloudera for the Risk Solutions vertical of Barclays.
• Designed an on-premise data lake using Cloudera CDH for Data Engineering & Analytics ETL workloads.
• Developed real-time and batch data pipelines using Spark Streaming, PySpark and HBase.

Senior Data Engineer

• Delivered end-to-end Big Data projects: data ingestion, ETL, data lake design, data migration, unit test plans and POCs.
• Designed and developed the HRBPO Attendance Tracking System using Hive and Hadoop.
• Ran POCs across the Hadoop ecosystem: HDFS Snapshots, HA for HDFS/Hive/YARN/HUE, HBase, Active Directory integration, Hive on Spark, Scala on Spark.
• Owned production ETL monitoring via Control-M, plus fixes and enhancements in Pig Latin, HiveQL, Unix and SQL.

University of Pune, Maharashtra, India

Army Public School, Dhaula Kuan, New Delhi, India

Army Public School, Delhi Cantt, New Delhi, India

AWS Certified Developer Associate