Data Engineer Big Data (Batch & Streaming)

• Job Title- Data Engineer – Big Data (Batch & Streaming)
• Location: Bangalore, India
• Hybrid (3 days onsite)

Company Overview:
Flexton is a growing IT services and staffing company serving some of the world's most renowned and Fortune-listed companies across industries such as eCommerce, HighTech, Banking & Financial Services, Insurance, and Healthcare. With a global presence spanning the US, Canada, Europe, Colombia, Mexico, and India, Flexton offers its clients cost-effective solutions through onsite, offshore, and nearshore models. Flexton provides its employees with opportunities for growth, meaningful work, and continuous learning!

---

About the Role
Project focuses on linking and correlating user account data with transactional data to enable fraud detection and risk analysis.
The platform processes large-scale datasets stored in Hadoop using MapReduce, Spark SQL, and Flink SQL, ensuring reliable daily data generation, pipeline health, and production stability.
The role supports enhancement, maintenance, and operational reliability of data pipelines critical to fraud detection systems.
Role Summary - The Data Engineer will be responsible for developing, enhancing, and supporting batch and streaming data pipelines used for account linking and fraud detection.


The role includes Spark SQL and Flink SQL development , light Java work for MapReduce integration , and production support to ensure timely, accurate, and complete data availability .
The engineer will follow established SOPs for data readiness , proactively manage pipeline dependencies, and resolve delays or data issues impacting downstream consumers.

Key Responsibilities (Concise)

• Develop and enhance batch and streaming data pipelines using Spark SQL and Flink SQL
• Maintain and support production data pipelines ensuring daily data availability
• Troubleshoot and resolve data delays, dependency failures, and data quality issues
• Perform pipeline monitoring, re-runs, and backfills as needed
• Follow SOPs for data readiness and production operations
• Provide L2/L3 production support for fraud detection datasets

Must-Have Technical Skills

• Spark SQL – strong hands-on experience
• Flink SQL – streaming data processing
• Hadoop ecosystem – HDFS & MapReduce
• SQL – advanced querying and optimization
• Java (basic to intermediate) – reading/debugging MapReduce jobs

Other Required Skills

• Experience supporting large-scale data pipelines in production
• Strong troubleshooting and root-cause analysis skills
• Understanding of data dependencies and pipeline orchestration
• Ability to handle time-sensitive data issues in critical systems
• Clear communication and documentation skills

---

Work Model

• Hybrid role requiring 3 days onsite in Bangalore, India .

Why Flexton?
Be the driver of your way forward, no matter what your level. At Flexton you can be both an idea and action person. Flexton is the place to be!
Flexton offers competitive benefits, as well as an industry-leading practice of performance-based bonuses for all employees. We believe that global innovation demands diverse employees and attractive work/life initiatives that sustain, and retain, them. Flexton gives you the power to design your workday, and your life, according to your unique styles and needs.
Flexton is an Equal Opportunity Employer.