Sr Data Engineer -- pipeline

KAYGEN is an emerging leader in providing top talent for technology-based staffing services. We specialize in providing high-volume contingent staffing, direct hire staffing and project-based solutions to companies worldwide ranging from startups to Fortune and Managed Service Providers (MSP) across a wide variety of industries. 

Design, develop, and optimize data pipelines using Databricks and Medallion architecture for structured and unstructured data.

Implement data ingestion strategies from diverse sources (SAP, TrackWise, Salesforce, and others) ensuring high-quality, reliable data flow.

Ingest and process large volumes of data from Azure Blob Storage into Databricks for transformation and analytics.

Build and maintain silver and gold layers for curated, analytics-ready datasets.

Apply advanced statistical methods , including: Anomaly Detection f...