Cloud Data Architect

Purpose of the Position:

You will be a critical member of the InfoCepts Cloud Data Architect Team. This position requires a sound knowledge on Big Data using Cloud technologies such as such as Databricks, EMR, Athena, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required.

/ Bangalore/Chennai

Type of Employment: Full time

Key Result Areas and Activities:

Technology Assessment and Design

• Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
• Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect.
• Ensure optimal balance between cost and performance.

Documentation and Stakeholder Communication

• Project documentation, adheres to quality guidelines & schedules.
• Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
• Articulate design decisions very clearly with stakeholders.
• Perform Proof-of-Concepts and document all observations before proposing a new solution.
• Conduct design review sessions with other teams and identify scope of improvement.

Process Improvement and Automation

• Suggests automation to improve existing processes.
• Assist junior Data Engineers by providing expert advice or troubleshooting steps whenever required.
• Create technology-focused training plans whenever required.
• Deliver technology-focused training sessions with team members whenever required.
• Conduct Expert Knowledge Sharing sessions with Client Stakeholders whenever required.
• Assist in designing case study documents whenever required.

Roles and Responsibility

Work and Technical Experience:

Must-Have:

• In-depth knowledge of the following AWS services: S3, EC2, EMR, Athena, AWS Glue, Lambda
• Experience with at least one MPP database: AWS Redshift, Snowflake, SingleStore
• Proficiency in Big Data technologies: Apache Spark, Databricks
• Must have strong programming skills in Python
• Responsible for building data pipelines in AWS And Databricks
• Experience with Big Data table formats, such as Delta Lake (open source)
• Must have very strong SQL skills
• Experience with orchestration tools like Apache Airflow
• Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.

Good to Have:

• Cloud Databases - Snowflake, AWS Aurora
• Big Data - Hadoop, Hive
• Cloud Databases- AWS Aurora
• Associate Level or Professional Level AWS Certification or Databricks Certification

Qualifications:

• Overall 7+ years of IT Exp
• 5+ years of relevant experience in AWS related project
• Bachelor's degree in computer science, engineering, or related field (master's degree is a plus)

Qualities:

• Strong hold technical knowledge and experience.
• Should have the capability to deep dive and research in various technical related fields
• Self-motivated and focused on delivering outcomes for a fast-growing team and firm.
• Prior experience of working in a large media company would be added advantage