Data Engineer

Job Title

Full Stack Data Engineer

Job Description

● Programming: Python, SQL, Shell scripting.

● GCP Data Stack:

○ Data Storage: BigQuery, Cloud Storage

○ Data Processing: Dataflow (Apache Beam), Dataproc (Spark), Cloud Composer (Airflow) ○ Streaming: Pub/Sub

○ Orchestration: Cloud Composer / Airflow

○ APIs & Services: Cloud Run, Cloud Functions, API Gateway

○ Monitoring: Cloud Logging, Cloud Monitoring, Stackdriver, or Prometheus/Grafana ● DevOps: Terraform, Git, Docker, Kubernetes, Cloud Build / Jenkins.

● Data Visualization: Looker Studio, Power BI, or Tableau.

● Version Control & CI/CD: GitHub, GitLab, or Bitbucket pipelines

Key Responsibilities

● Build, and maintain end-to-end GCP data pipelines (batch and streaming).

● Ensure data platform uptime and performance in alignment with defined SLAs/SLOs. ● Develop ETL/ELT workflows using Cloud Composer (Airflow), Data...