Sr. Data Engineer

Title: Sr. Data Engineer
Location: Gurugram
Exp: 5+ yrs

Job Description:
Required Credentials:
5 - 7 Years of Experience
Required Qualifications:
GCP Data Stack: Expertise in BigQuery, Cloud Composer, Cloud SQL, and Dataplex.
Transformation: Proficiency in Dataform, Advanced SQL, and Python.
Legacy Migration: Ability to read and interpret Oozie, Pig, Hive, and proprietary ETL configurations.
DevOps: Experience with GitHub Actions, Docker, and basic Terraform knowledge.
Useful Qualifications:
Experience with large-scale historical data migrations from on-prem Oracle/MySQL sources.
Familiarity with "Silver and Gold" data modeling layers.
Contractor Scope of Work Delivery Expectations: Below covers the scope of work we anticipate the contractor supporting throughout the project timeline.
Scope and Solution Expectations:
Pipeline Development: Design and implement GCP-based ETL/ELT pipelines using Cloud Composer, BigQuery, and Dataform to replace legacy Hadoop-stack jobs.
Data Modeling: Build core data models (Silver and Gold Layers) in BigQuery for downstream analytics and reporting.
Governance Implementation: Ensure data quality and governance by implementing Dataplex, Data Validation Tools (DVT), and automated quality checks.
Migration Execution: Perform large-scale historical data migrations from on-prem Oracle/MySQL to BigQuery and Cloud SQL.
CI/CD: Develop and implement CI/CD pipelines for all data transformation jobs using GitHub Actions.