AI Data Pipeline Engineer

Experience Level: 2- 4 years

Key Responsibilities

• Data and AI Engineer responsible for Implementing AI data pipelines that bring together structured, semi-structured and unstructured data to support AI and Agentic solutions. This Includes pre-processing with extraction, chunking, embedding and grounding strategies to get the data ready.
• Real-Time Data Streaming: Build and maintain scalable and robust real-time data streaming pipelines using technologies such as Apache Kafka, AWS Kinesis, Spark streaming, or similar.
• Develop AI-driven systems to improve data capabilities
• Implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate with enterprise data infrastructure.
• Develop data domains and data products for various consumption archetypes including Reporting, Data Science, AI/ML, Analytics etc.
• Ensure the reliability, availability, and scalability of data pipelines and systems thr...