Data Engineer

What You'll Build 

Core Responsibilities 

Data Architecture & Infrastructure (40%) 

● Design and implement a multi-database architecture (MongoDB, Redis, Milvus, Neo4j, BigQuery) 

● Build scalable data pipelines for real-time conversation processing and personalization

● Architect ETL/ELT workflows for data migration from legacy systems

● Implement data partitioning, sharding, and optimization strategies for high-throughput systems 

● Create data governance frameworks ensuring quality, security, and compliance Vector & Graph Database Systems (25%)

● Design and optimize Milvus vector collections for semantic search (1024-dim embeddings) 

● Build graph schemas in Neo4j for customer journey mapping and persona relationships

● Implement HNSW indexing strategies and similarity search optimization

● Create hybrid search systems combining vector, full-text, and graph queries

...