Data Engineer

Role - Azure Databricks

Location - Bangalore, Chennai, Delhi, Mumbai, Pune, Hyderabad

Exp range - 5+ years

• Architecture of Azure Databricks, differentiating between the control plane and data plane.
• SDLC of Azure Databricks. Describe the role of Databricks Workspaces, Notebooks, and Jobs How to create workspaces, providing access to users.
• Setup and Provisioning How to configure and manage different types of Databricks clusters (e.g., all-purpose, job, high concurrency).
• How to secure data in Databricks. How to manage cluster resources & optimize cluster performance.
• Integration and Other Azure Services, Azure Databricks integrate with Azure Data Lake Storage (ADLS)? & Azure Synapse Analytics? How to implement CI/CD pipelines for Databricks notebooks and jobs using Azure DevOps Monitor and log Databricks workloads for performance and error analysis.
• Monitoring and Cost Management. Required level of experience on the Databricks platform. To support our business unit–facing data platforms,
• Hands-on experience with Databricks, Proficiency in Azure services, particularly ADF and ADLS, A good understanding of CI/CD and Terraform, even if the implementation experience is limited Strong communication skills, as this role involves close collaboration with business stakeholders.