AWS Data Engineer

Overview

Descrição: Project Description

We are looking for an experienced AWS data engineer, who will join the team responsible for the second phase of data warehouse cloud migration project. The initial lift-and-shift of an on-premise MS SQL data warehouse to AWS RDS has been completed. This next phase focuses on refactoring the legacy system into a modern, serverless data architecture. The primary goal is to migrate all data transformation logic from MS SQL to AWS Glue and prepare the data for consumption by business intelligence tools like AWS QuickSight.

Responsibilities

• Analyze existing MS SQL stored procedures and data structures within the PMM data warehouse to understand the current business and transformation logic.
• Design, develop, and deploy scalable and efficient ETL/ELT pipelines using AWS Glue and PySpark to replace the legacy SQL processes.
• Ingest and process data from various sources currently landing ...