Job Description
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Apache Spark.
- Build data ingestion, transformation, and processing frameworks for structured and unstructured data.
- Work with Azure services including ADLS Gen2, Azure Data Factory (ADF), Azure Synapse, Azure SQL, Delta Lake, and Event Hubs.
- Optimize and tune Spark jobs for performance, cost efficiency, and scalability.
- Implement advanced analytics and machine learning workflows using Databricks notebooks.
- Ensure data quality, data governance, and compliance throughout the data lifecycle.
- Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver data solutions.
- Manage CI/CD pipelines for data engineering workloads using Azure DevOps or GitHub Actions.
- Troubleshoot performance, security, and operational issues in Databricks environments.
- Document architecture, data flows, and...
Apply for this Position
Ready to join Hucon Solutions? Click the button below to submit your application.
Submit Application