Job Description
Lead PySpark Engineer (Cloud Migration)
Role Type: 5-Month Contract
Location: Remote (UK-Based)
Experience Level: Lead / Senior (5+ years PySpark)
Role Overview
We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance AWS cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape.
Key Responsibilities
Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated tools (SAS2PY) and manual refactoring.
Pipeline Engineering: Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS.
Performance Tuning: Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness.
Quality Assurance: Implement clean coding princip...
Role Type: 5-Month Contract
Location: Remote (UK-Based)
Experience Level: Lead / Senior (5+ years PySpark)
Role Overview
We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance AWS cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape.
Key Responsibilities
Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated tools (SAS2PY) and manual refactoring.
Pipeline Engineering: Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS.
Performance Tuning: Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness.
Quality Assurance: Implement clean coding princip...
Apply for this Position
Ready to join Randstad Technologies Recruitment? Click the button below to submit your application.
Submit Application