M03 - Data Engineer

Roles and Responsibilities

Work closely with Product Owner for the development, implementation and maintenance of data engineering products for Business Intelligence (BI) and Data warehouse projects. Scope of work includes:

• Design, develop and deploy data engineering solutions (including data tables, views, marts and etc) in data pipeline, data warehouse, operational data store, data lake and virtualisation.
• Perform data extraction, cleaning, transformation, and flow. Web scraping may be also a part of the work scope in data extraction.
• Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks.
• Integrate and collate data silos in a manner which is both scalable and compliant.
• Ensure adherence to quality processes throughout the data product lifecycle, including development, deployment, validation, change management and documentation, with particular ...