Job Description
About the Role
We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.
Key Responsibilities
Design, develop, and optimize scalable data pipelines using
Databricks (PySpark/Spark SQL) .
Implement and tune data workloads using
Azure Data Factory ,
Azure Synapse Analytics , and
Microsoft Fabric .
Apply best practices in
Delta Lake , including performance tuning, partitioning, indexing, and caching.
Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
Implement data governance, lineage, and security using
Unity Catalog
and Azure capabilities.
Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
Collaborate w...
We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.
Key Responsibilities
Design, develop, and optimize scalable data pipelines using
Databricks (PySpark/Spark SQL) .
Implement and tune data workloads using
Azure Data Factory ,
Azure Synapse Analytics , and
Microsoft Fabric .
Apply best practices in
Delta Lake , including performance tuning, partitioning, indexing, and caching.
Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
Implement data governance, lineage, and security using
Unity Catalog
and Azure capabilities.
Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
Collaborate w...
Apply for this Position
Ready to join Senpiper PTY LTD? Click the button below to submit your application.
Submit Application