Data Engineer

Role: Data Engineer 

Location - Remote

Experience: Min. 5 Years

 

Key Responsibilities:


Design and implement manual test strategies for real-time streaming use cases using Azure Service Bus, Event Hubs, Kafka, and Azure Functions.
Validate Spark Streaming applications, including unbounded data flows, streaming DataFrames, checkpoints, and streaming joins.
Develop test plans for containerized microservices deployed on Kubernetes, ensuring scalability and fault tolerance.
Test data ingestion and transformation workflows across open table formats like Delta Lake, Apache Iceberg, and Hudi.


Good to Have:


Monitoring and troubleshooting system performance using observability stacks such as Prometheus, Grafana, and ELK.
functional and performance testing on analytical databases and query engines such as Trino, StarRocks, and ClickHouse.
Testing and validation of data products designed under data mesh architecture, ensuring domain-oriented data quality and governance.
. Skillset Required: Azure service , Event hub, custom azure func, Data Engineer, Data lakes, Azure data