JD15 - Associate Consultant - Azure

Job purpose: The Azure Data Engineer is responsible for designing, implementing, and optimizing cloud-based data solutions using Microsoft Azure. This role involves ETL/ELT development, data modeling, and data pipeline orchestration using services like Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake. The ideal candidate has strong experience in data engineering, cloud technologies, data modeling concepts, and performance optimization to support analytical and reporting needs. Duties and responsibilities of position: General areas of responsibility relate to: Collaborate with clients and internal teams to understand business requirements and translate them into scalable Azure data solutions. Ensure security, performance, and cost optimization of Azure-based data platforms. Assist in preparing documentation, architecture diagrams, and client presentations for data engineering solutions. Maintain compliance with industry standards such as PHIPPA, CIS, and NIST as required Provide after-hours support for Infrastructure related emergencies, regular and occasional weekend maintenance Requirements Specific areas of responsibility include but are not limited to: Design, develop, and optimize data engineering solutions on Azure using services like Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage (ADLS). Develop and optimize data ingestion and transformation processes from structured and unstructured sources. Automate data workflows using Azure Functions, Logic Apps. Implement incremental load strategies and CDC (Change Data Capture) mechanisms for real-time data updates. Implement parameterized and dynamic ETL pipelines using ADF for scalability and reusability. Design data models for Azure SQL Database, Synapse Analytics, and Data Lakes. Implement dimensional modeling (star/snowflake schema) for reporting and analytics use cases. Optimize query performance using partitioning, indexing, and caching techniques. Implement data governance best practices, including data cataloging, lineage tracking, and metadata management. Implement Slowly Changing Dimensions (SCD) Type 1 and Type 2 strategies for maintaining historical data in dimensional models. Monitor data pipelines using Azure Monitor, Log Analytics, and Application Insights. Enforce data security best practices using RBAC, Managed Identities, and Data Masking. Ensure compliance with GDPR, HIPAA, and SOC 2 where applicable. Implement data encryption and access control policies for sensitive data. Set up monitoring and alerting for real-time data pipelines. Monitor and optimize Azure resource utilization to minimize cloud costs. Implement reserved instances and serverless architectures where applicable. Qualifications: Bachelors degree in Computer Science, Information Technology, or a related field; or equivalent professional experience. Minimum of 2+ years of experience administering Azure Cloud environments with a focus on Azure data warehouses and data lakes. Strong hands-on experience with Azure Data Factory (ADF), Synapse Analytics, and Data Lake Storage (ADLS Gen2), Key Vault. Proficiency in o SQL (queries, joins, window functions, basic performance tuning) o Python (data processing, scripting, utilities; familiarity with PySpark is a plus) Experience with Azure SQL, Cosmos DB, or other NoSQL databases. Hands-on experience in data modeling, indexing, and query performance tuning. Experience working with big data processing frameworks such as Databricks or Spark. PowerShell scripting for task automation. Knowledge of virtual and physical infrastructure, network administration, and vulnerability management. Virtual and Physical Infrastructure and Network administration. Certifications (Preferred): o Microsoft Certified: Azure Administrator Associate (AZ-104) o Microsoft Certified: Azure Data Engineer Associate (DP-203) Experience with Microsoft 365 administration (e.g., Exchange, SharePoint, OneDrive, Teams). Experience working in a team, and execute tasks in a high-pressure environment and provide or make sound decisions in emergencies.
Bachelor’s/Master’s in Computer Science, IT, or related field (or equivalent experience).
1-3 years
The Azure Data Engineer is responsible for designing, implementing, and optimizing cloud-based data solutions using Microsoft Azure. This role involves ETL/ELT development, data modeling, and data pipeline orchestration using services like Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake. The ideal candidate has strong experience in data engineering, cloud technologies, data modeling concepts, and performance optimization to support analytical and reporting needs.