Job Description
Job Summary
We are seeking a highly skilled Data Engineer with deep expertise in the Azure data ecosystem and Databricks, combined with strong DevOps, CI/CD, and Infrastructure as Code (Terraform) capabilities.
The ideal candidate will be responsible for designing, developing, deploying, and maintaining scalable data pipelines and platforms with full ownership of the data lifecycle in production environments.
Key ResponsibilitiesData Engineering & Architecture
- Design and develop scalable, high-performance data pipelines using Azure Databricks (PySpark/SQL).
- Build batch and streaming ingestion pipelines using Azure Data Factory / Synapse / Event Hub / Kafka.
- Implement data modeling and transformation using Delta Lake, Medallion Architecture (Bronze/Silver/Gold).
- Ensure data quality, reliability, and performance tuning (partitioning, indexing, caching, cluster optimization).
DevOps, CI/CD & Automation
- Design and man...
We are seeking a highly skilled Data Engineer with deep expertise in the Azure data ecosystem and Databricks, combined with strong DevOps, CI/CD, and Infrastructure as Code (Terraform) capabilities.
The ideal candidate will be responsible for designing, developing, deploying, and maintaining scalable data pipelines and platforms with full ownership of the data lifecycle in production environments.
Key ResponsibilitiesData Engineering & Architecture
- Design and develop scalable, high-performance data pipelines using Azure Databricks (PySpark/SQL).
- Build batch and streaming ingestion pipelines using Azure Data Factory / Synapse / Event Hub / Kafka.
- Implement data modeling and transformation using Delta Lake, Medallion Architecture (Bronze/Silver/Gold).
- Ensure data quality, reliability, and performance tuning (partitioning, indexing, caching, cluster optimization).
DevOps, CI/CD & Automation
- Design and man...
Apply for this Position
Ready to join Senpiper PTY LTD? Click the button below to submit your application.
Submit Application