Lead I - Data Engineerings

Overview

Join to apply for the Lead I - Data Engineerings role at UST .

Role description and responsibilities below reflect the data engineering focus across Azure-based data platforms.

Key Responsibilities

• Design, build, and maintain robust data pipelines using Azure Data Factory (ADF) to ingest data from diverse sources including APIs, Excel files, Azure Data Lake Storage Gen2 (ADLS Gen2), and SharePoint, ensuring secure access and efficient data flow.
• Configure and manage SharePoint integration within ADF pipelines, handling authentication and access criteria effectively.
• Develop complex data transformations using ADF Mapping Data Flows, applying best practices to create scalable, maintainable ETL/ELT workflows.
• Implement and champion metadata-driven architecture for enhanced scalability, modularity, and reusability of data pipelines.
• Perform rigorous data validation and testing to maintain...