Apache Flume Developer (Data Engineer)

We at Coforge are hiring for a detail-oriented Data Engineer (Apache Flume) to join our team. The ideal candidate will have 4+ years of hands-on experience in development, with strong expertise in Apache Flume, data modeling, and migration. This role involves leading the migration of existing logs from SQL Server databases to an Apache Flume application and establishing a robust pipeline for daily log ingestion.

Key Responsibilities

• Log Migration:
• Design and implement solutions to extract and transfer historical logs from SQL Server to Apache Flume.
• Ensure data integrity, completeness, and consistency during migration.
• Daily Log Ingestion:
• Develop and automate pipelines to move newly generated logs from SQL Server to Flume daily.
• Monitor and maintain pipeline performance and reliability.
• System Integration & Optimization:
• Collabora...