PySpark- Senior Developer

Job Description:

Key responsibilities include the following:

• Develop and maintain scalable data pipelines using Pyspark and proven experience as developer with expertise in PySpark.
• Good to have knowledge on Ab Initio.
• Experience with distributed computing and parallel processing.
• Proficiency in SQL and experience with database systems.
• Collaborate with data engineers and data scientists to understand and fulfil data processing needs.
• Optimize and troubleshoot existing PySpark applications for performance improvements.
• Write clean, efficient, and well-documented code following best practices.
• Participate in design and code reviews.
• Develop and implement ETL processes to extract, transform, and load data.
• Ensure data integrity and quality throughout the data lifecycle.
• Stay current with the latest industr...