Job Description
Azure Data Sources : Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics. External Sources: APIs, on-prem databases, flat files (CSV, Parquet, JSON).
Tools: Azure Data Factory (ADF) for orchestration, Databricks connectors. •
Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing.
Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution.
Performance Optimization: Partitioning, caching, broadcast joins.
Delta Lake: Implementing ACID transactions, time travel, and schema enforcement.
Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks.
Azure Key Vault: Secure credential management.
Azure Monitor & Logging: For ETL job monitoring and alerting.
Networking & Security: VNET integration, private endpoints
Apply for this Position
Ready to join ? Click the button below to submit your application.
Submit Application