Senior Engineer Azure Databricks

Position: Senior Engineer Azure databricks

Overall IT experience: 6 to 9 years

Primary Skills: Azure Databricks, PySpark, SQL

Secondary Skills: ADF/Azure Data factory

Work Location: Bengaluru/Hyderabad

Mode of work: Hybrid

Notice Period: 0 to 30 Days

Experience & Expertise:

• Total Experience: 6 to 9 years in Cloud Data Engineering.
• Azure Databricks & PySpark: Minimum 3 to 5 years of hands-on experience.

Technical Skills & Responsibilities:

• SQL Expertise:
• Proficient in SQL with 5 to 6 years of experience in complex joins, subqueries, functions, and procedures.
• Strong ability to perform SQL tuning and query optimization independently.
• ETL & Data Processing:
• Design, develop, and maintain ETL pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
• Extract, transform, and load (ETL) data from various sources.
• Optimize and fine-tune existing ETL workflows for performance and scalability.
• Azure Cloud Expertise:
• Hands-on experience in implementing large-scale hosted solutions using Azure ADF, Azure Data Lake Storage, and Databricks.
• Big Data & Streaming:
• Experience working with large datasets.
• Exposure to batch and streaming data processing (good to have).
• Security & Compliance:
• Ability to develop reliable, secure, and compliant data processing systems.
• Collaboration & Independence:
• Work closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.
• Self-driven and capable of working independently without support.