Data Engineer (UK Shift)

Job Title: Consultant/ Sr. Consultant - Data Engineer

Job Summary:

Responsible for implementation of data ETL to on-board and ingest data sets to cloud data management platforms.

Work closely with analytics and data science teams to support their end-to-end data acquisition needs. Interact with internal stakeholders to ensure data capabilities are aligned with overall data strategy.

Key Responsibilities:

• Develop and maintain ETL pipelines to support data processing, analytics, and visualization needs.
• Collaborate with cross-functional teams to understand business requirements and provide data-driven solutions.
• Manage multiple disparate data sources and integrate them into a cohesive data platform and dashboards.
• Develop and implement validation processes to ensure data accuracy and consistency.
• Utilize PySpark, Spark, SQL to optimize data processing and querying performance.
• Design and implement scalable, fault-tolerant data processing systems in Amazon Web Services
• Work independently and be comfortable walking through code with the team lead.
• Document all processes, code, and work, as well as be willing to ask questions and raise issues when roadblocks are encountered
• Extensive experience in developing API services for data consumption
• Working knowledge in AWS infrastructure with hands-on knowledge in Glue, Redshift, Amazon Dynamo DB or any other NoSQL or Columnar databases
• Demonstrate extensive knowledge in data quality management and reference data management.
• Working experience on data stewardship process and data governance is an advantage.
• Ability to multi-task and work in an agile environment is a must.

Requirements:

• Bachelor's or Master's degree in Computer Science,
• 3 to 7 years of experience in data science
• Strong proficiency in python, ETL, PySpark, spark, and SQL.
• Experience with Amazon Web Services, specifically with data processing services such as S3, EC2, Lambda, and Glue.
• Experience in data visualization tools such as Tableau is an added plus.
• Strong problem-solving skills and attention to detail
• Ability to work independently and in a team environment.
• Excellent written and verbal communication skills
• Strong documentation processes and ability to maintain code.

Location: Hyderabad