Job Description

Databricks Engineer

Positions: 2 | Duration: 3 Months | Location: Dubai

Bank is building a scalable data ingestion and streaming platform that ingests change data capture (CDC) events from diverse source systems (databases and applications), processes them in real time and lands curated data into our analytics lake. The platform uses Confluent connectors (Debezium/Oracle CDC) to emit Parquet files into cloud storage and leverages Databricks Auto Loader to incrementally ingest, deduplicate and write this data into Delta Lake Bronze table.

Key Skills

  • Kafka, Confluent
  • Real‑Time, Streaming Data Ingestion
  • Producers, Consumers, Topics
  • Data Lake Integration, Lakehouse Integration
  • Azure Databricks, Python, Scala, Java

Responsibilities

  • Design and develop streaming ingestion pipelines using Apache Spark (Structured Streaming) and Databricks Auto Loader to ingest data into Delt...

Apply for this Position

Ready to join Virtusa? Click the button below to submit your application.

Submit Application