Data Engineer

We're hiring a highly motivated Data Engineer with expertise in Python, AWS Glue/pyspark, AWS Lambda, Kafka, and relational databases (specifically Postgres). You'll be responsible for designing, developing, and maintaining data processing and management solutions, ensuring data integrity and availability through collaboration with multidisciplinary teams.

Responsibilities:

• Design and develop efficient data pipelines using Python, AWS Glue/pyspark, AWS Lambda, Kafka, and related technologies.
• Implement and optimize ETL processes for data extraction, transformation, and loading into relational databases, especially Postgres.
• Collaborate on data warehouse architectures, ensuring proper data modeling, storage, and access.
• Utilize tools like StitchData and Apache Hudi for data integration and incremental management, improving efficiency and enabling complex operations.
• Identify and resolve data quality and consistency issues...