Apache Flume Developer (Data Engineer)

We at Coforge are hiring for a detail-oriented Data Engineer (Apache Flume) to join our team. The ideal candidate will have 4+ years of hands-on experience in development, with strong expertise in Apache Flume , data modeling , and migration . This role involves leading the migration of existing logs from SQL Server databases to an Apache Flume application and establishing a robust pipeline for daily log ingestion.

Key Responsibilities

• Log Migration:
• Design and implement solutions to extract and transfer historical logs from SQL Server to Apache Flume.
• Ensure data integrity, completeness, and consistency during migration.
• Daily Log Ingestion:
• Develop and automate pipelines to move newly generated logs from SQL Server to Flume daily.
• Monitor and maintain pipeline performance and reliability.
• System Integration & Optimization: ...