Sr Data Engineers

Experience: 7+Years

Employment Type: Full-Time

Location: Pan India (Hybrid)

Job Overview

We are looking to onboard a Senior Data Engineer to form and support a dedicated OGA Scrum Team . The ideal candidate will bring strong hands-on expertise in Databricks and Azure-based data platforms , contribute as a senior individual contributor, and collaborate closely with onshore and offshore team members in an Agile delivery model.

Key Responsibilities

• Design, develop, and maintain scalable batch and streaming data pipelines using Azure Databricks and Azure Data Factory (ADF) .
• Build and manage Databricks workflows for orchestration, monitoring, and dependency management.
• Develop robust data transformations using PySpark, Python, and Databricks SQL .
• Implement config-driven data pipelines to enable reusable, scalable, and low-maintenance data processing.
• Leverage Databricks Genie for conversational, AI-driven data insights using no-code/low-code approaches .
• Utilize GitHub Copilot for code completion and productivity acceleration within Databricks environments.
• Design and optimize data models , including Star Schema and Snowflake Schema , for analytics and reporting use cases.
• Work with open table formats such as Delta Lake and Apache Iceberg to support modern lakehouse architectures.
• Implement and optimize data virtualization schemes to enable seamless data access across heterogeneous sources.
• Ensure data quality, reliability, performance, and cost optimization across pipelines.
• Collaborate with US onshore stakeholders and offshore team members within Agile Scrum ceremonies.
• Contribute to sprint planning, estimation, code reviews, and production support.

Required Skills & Qualifications

• Strong hands-on experience with Azure Databricks .
• Proven expertise in Azure Data Factory (ADF) and Databricks Workflows .
• Advanced programming skills in PySpark, Python, and Databricks SQL .
• Experience building config-driven and metadata-driven data pipelines .
• Solid understanding of data modeling techniques (Star Schema, Snowflake Schema).
• Hands-on experience with Delta Lake and Apache Iceberg .
• Strong experience working on Azure cloud platforms (preferred).
• Knowledge of data virtualization concepts and architectures .
• Experience working in Agile/Scrum teams , preferably in an OGA delivery model .
• Strong problem-solving, communication, and collaboration skills.

Nice to Have

• Exposure to Databricks Genie or similar AI-assisted analytics tools.
• Experience using GitHub Copilot or AI-based developer productivity tools.
• Experience with streaming technologies (Kafka, Event Hubs).
• Understanding of security, governance, and access control in Databricks (Unity Catalog).
• Prior experience working with US-based clients or stakeholders.

What We’re Looking For

• Senior-level engineers who can operate independently while mentoring junior team members.
• Engineers comfortable working across time zones in an OGA model.
• Strong ownership mindset with a focus on quality and delivery excellence.