Job Description
Job Description
Key Responsibilities
Design, develop, and optimize ETL pipelines using PySpark on Google Cloud Platform (GCP).
Work with BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage for data transformation and orchestration.
Develop and optimize Spark-based ETL processes for large-scale data processing.
Implement best practices for data governance, security, and monitoring in a cloud environment.
Collaborate with data engineers, analysts, and business stakeholders to understand data requirements.
Troubleshoot performance bottlenecks and optimize Spark jobs for efficient execution.
Automate data workflows using Apache Airflow or Cloud Composer.
Ensure data quality, validation, and consistency across pipelines.
5+ years of experience in ETL development with a ...
Key Responsibilities
Design, develop, and optimize ETL pipelines using PySpark on Google Cloud Platform (GCP).
Work with BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage for data transformation and orchestration.
Develop and optimize Spark-based ETL processes for large-scale data processing.
Implement best practices for data governance, security, and monitoring in a cloud environment.
Collaborate with data engineers, analysts, and business stakeholders to understand data requirements.
Troubleshoot performance bottlenecks and optimize Spark jobs for efficient execution.
Automate data workflows using Apache Airflow or Cloud Composer.
Ensure data quality, validation, and consistency across pipelines.
5+ years of experience in ETL development with a ...
Apply for this Position
Ready to join Orison Solutions? Click the button below to submit your application.
Submit Application