Senior Data Engineer

Role Overview :

We are seeking a Senior Data Engineer with strong developer DNA — someone who codes daily in PySpark, SQL, and Python, builds end-to-end data solutions, and leads design of robust, production-grade data platforms on Microsoft Fabric, Azure Databricks, and ADF. This is a hands-on leadership role for engineers who can think architecturally and code efficiently.

Key Responsibilities:

• Design, develop, and optimize complex data pipelines and lakehouse architectures.

• Lead development of batch and streaming ingestion frameworks using ADF, Databricks, and Event Hubs.

• Write optimized PySpark/SQL transformations for performance, scalability, and cost efficiency.

• Build reusable data frameworks and automation scripts for continuous integration and deployment.

• Establish unit testing, code review, and versioning standards across engineering teams.

• Integrate with DevOps CI/CD pipelines for automated releases and environment management.

• Mentor junior engineers on coding practices and architecture design.

• Collaborate with data architects on Fabric-based Lakehouse implementations.

Skills & Qualifications:

• 6–12 years of strong hands-on experience in data engineering and development.

• Expertise in:

o PySpark, SQL, Python, and performance tuning of large-scale transformations.

o Databricks, ADF, ADLS, Fabric Data Pipelines, and Delta Lake.

• Deep understanding of data architecture, schema design, job orchestration, and data governance. • Strong knowledge of CI/CD (Azure DevOps) and infrastructure-as-code for data pipelines.

• Optional: Exposure to GCP BigQuery, Dataproc, or Dataflow as secondary platforms.

Preferred Certifications: • Databricks Certified Data Engineer Professional • Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) • Microsoft Certified: Azure Solutions Architect Expert (AZ-305) • Optional: Google Cloud Certified – Professional Data Engineer