Job Description
Job Summary
We are seeking a Senior Data Architect with deep Big Data Engineering expertise to design and modernize large-scale, cloud-native data platforms. This role emphasizes distributed data processing, real-time pipelines, data platform automation , and Gen AI enablement on top of strong Big Data foundations.
Key Responsibilities
Architect and govern enterprise Big Data platforms (data lake, lakehouse, warehouse, real-time).
Design high-volume, high-velocity data pipelines using batch and streaming frameworks.
Lead implementation of distributed processing architectures (Spark, Py Spark, EMR).
Build event-driven and real-time streaming solutions (Kafka, Kinesis, Flink).
Define ETL/ELT patterns , metadata-driven pipelines, and reusable ingestion frameworks.
Drive data platform automation (Airflow/Step Functions, CI/CD, data quality, observability).
Optimize performance, scalability, fault tolerance, and cost across Big Data workloads.
Integrate Gen AI architectures (LLMs, embeddings, vector databases, RAG) with enterprise data lakes.
Ensure security, governance, lineage, and compliance across data platforms.
Provide hands-on leadership and technical mentoring to data engineering teams.
Required Technical Skills & Experience
12+ years in Big Data Engineering / Data Architecture roles.
Expert-level experience with Spark, Py Spark, SQL , and distributed compute engines.
Strong knowledge of AWS Big Data stack : S3, EMR, Glue, Athena, Redshift, Lambda, Step Functions.
Hands-on experience with Snowflake (performance tuning, data sharing, optimization).
Expertise in streaming platforms : Kafka, Kinesis, Flink, or Spark Streaming.
Strong experience with data modeling (dimensional, Data Vault 2.0).
Proficiency in Python , schema evolution, partitioning, and data versioning.
Experience with orchestration and automation tools (Airflow, Dagster, CI/CD).
Working knowledge of Gen AI data integration (feature stores, vector DBs, RAG pipelines).
Experience with Agile delivery and leading globally distributed engineering teams.
We are seeking a Senior Data Architect with deep Big Data Engineering expertise to design and modernize large-scale, cloud-native data platforms. This role emphasizes distributed data processing, real-time pipelines, data platform automation , and Gen AI enablement on top of strong Big Data foundations.
Key Responsibilities
Architect and govern enterprise Big Data platforms (data lake, lakehouse, warehouse, real-time).
Design high-volume, high-velocity data pipelines using batch and streaming frameworks.
Lead implementation of distributed processing architectures (Spark, Py Spark, EMR).
Build event-driven and real-time streaming solutions (Kafka, Kinesis, Flink).
Define ETL/ELT patterns , metadata-driven pipelines, and reusable ingestion frameworks.
Drive data platform automation (Airflow/Step Functions, CI/CD, data quality, observability).
Optimize performance, scalability, fault tolerance, and cost across Big Data workloads.
Integrate Gen AI architectures (LLMs, embeddings, vector databases, RAG) with enterprise data lakes.
Ensure security, governance, lineage, and compliance across data platforms.
Provide hands-on leadership and technical mentoring to data engineering teams.
Required Technical Skills & Experience
12+ years in Big Data Engineering / Data Architecture roles.
Expert-level experience with Spark, Py Spark, SQL , and distributed compute engines.
Strong knowledge of AWS Big Data stack : S3, EMR, Glue, Athena, Redshift, Lambda, Step Functions.
Hands-on experience with Snowflake (performance tuning, data sharing, optimization).
Expertise in streaming platforms : Kafka, Kinesis, Flink, or Spark Streaming.
Strong experience with data modeling (dimensional, Data Vault 2.0).
Proficiency in Python , schema evolution, partitioning, and data versioning.
Experience with orchestration and automation tools (Airflow, Dagster, CI/CD).
Working knowledge of Gen AI data integration (feature stores, vector DBs, RAG pipelines).
Experience with Agile delivery and leading globally distributed engineering teams.
Apply for this Position
Ready to join ? Click the button below to submit your application.
Submit Application