Job Description

Responsibilities

:

Data Pipeline Development:

  • Build and maintain scalable ETL/ELT pipelines using Databricks.
  • Leverage PySpark/Spark and SQL to transform and process large datasets.
  • Integrate data from multiple sources including Azure Blob Storage, ADLS and other relational/non-relational systems.
  • Collaboration & Analysis:
  • Work Closely with multiple teams to prepare data for dashboard and BI Tools.
  • Collaborate with cross-functional teams to understand business requirements and deliver tailored data solutions.

    • Performance & Optimization:

  • Optimize Databricks workloads for cost efficiency and performance.
  • Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
  • Governance & Security
  • Implement and manage data security, access controls and governance standards using Unity Catalog.
  • Ensure compliance with organizational and regulat...

    • Apply for this Position

    Ready to join IntellectFaces? Click the button below to submit your application.

    Submit Application