Senior Data Engineer

Responsibilities:

· Design and develop data objects and data processing jobs in Azure Data Platform

· Design and implement E2E analytics solutions using Palantir Foundry Platform
,
including:

· Various types of data sources and data connections

· Scalable data transformation pipelines in PySpark

· Optimize and automate data processing jobs and transformation pipelines through reviewing of logs and query plan, partitioning and code simplification

· Ensure data quality, security (access concept) and governance across the whole flow of data & analytics solution

· Collaborate with cross-functional teams and stakeholders to understand and translate business requirements into technical specifications and actionable plans, ensuring successful delivery

· Troubleshoot and resolve functional and technical data issues for stakeholders, actively contributing to shared goals and supporting peers in solving technical and analytical cha...