Job Description
About the Role We are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.
Key Responsibilities Design, develop, and optimize scalable data pipelines using Databricks (PySpark/Spark SQL) .
Implement and tune data workloads using Azure Data Factory , Azure Synapse Analytics , and Microsoft Fabric .
Apply best practices in Delta Lake , including performance tuning, partitioning, indexing, and caching.
Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
Implement data governance, lineage, and security using Unity Catalog and Azure capabilities.
Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
Collaborate with data scientists, analysts, and business teams to deliver clean and ...
Key Responsibilities Design, develop, and optimize scalable data pipelines using Databricks (PySpark/Spark SQL) .
Implement and tune data workloads using Azure Data Factory , Azure Synapse Analytics , and Microsoft Fabric .
Apply best practices in Delta Lake , including performance tuning, partitioning, indexing, and caching.
Troubleshoot complex ingestion, transformation, orchestration, and performance issues.
Implement data governance, lineage, and security using Unity Catalog and Azure capabilities.
Automate workflows using CI/CD, Databricks Jobs, and REST APIs.
Collaborate with data scientists, analysts, and business teams to deliver clean and ...
Apply for this Position
Ready to join Senpiper PTY LTD? Click the button below to submit your application.
Submit Application