Senior Software Engineer - Azure Databricks Developer

Position Description:

We are seeking a highly skilled Databricks Developer with strong hands-on experience in Azure Integration services. The candidate will be responsible for designing, developing, and operationalizing data pipelines to migrate Oracle API data from the staging database into Databricks. The role requires strong data engineering expertise, cloud knowledge, and the ability to work closely with cross-functional teams to deliver scalable and reliable data solutions.

Responsibilities:
•Design, develop, and maintain scalable data pipelines in Databricks.
• Migrate Oracle API data from staging databases to Databricks-based data platforms.
• Implement data ingestion, transformation, and orchestration workflows using Azure Integration Services (Azure Data Factory, Azure Functions, Logic Apps, etc.).
• Optimize Databricks notebooks, Delta Lake tables, and ETL processes for performance and reliability.
• Collaborate with Oracle development teams, integration teams, and business stakeholders to understand data flows and requirements.
• Monitor and troubleshoot data pipelines, ensuring data consistency and availability.
• Ensure adherence to data governance, security, and compliance standards.
Prepare technical documentation and support knowledge transfer activities

Must-Have Skills:
• 7–12 years of overall experience in data engineering, ETL development, or integration roles.
• Strong hands-on experience with Azure Databricks, Spark (PySpark/Scala), and Delta Lake.
• Good understanding of Azure Integration tools such as Azure Data Factory (ADF), Logic Apps, or related components.
• Experience working with Oracle databases, including staging tables, APIs, and data extraction techniques.
• Strong SQL, performance tuning, and data transformation skills.
• Experience working in cloud-based data architectures and CI/CD pipelines.
• Ability to analyze, troubleshoot, and optimize complex data workflows.
Excellent communication and stakeholder management skills

CGI is an equal opportunity employer. In addition, CGI is committed to providing accommodation for people with disabilities in accordance with provincial legislation. Please let us know if you require reasonable accommodation due to a disability during any aspect of the recruitment process and we will work with you to address your needs.

Skills:

English

Python

Analytical Thinking