Pyspark, Azure_AMR

Pyspark, Azure

Greetings from TCS!! !

TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project.

Make sure you have a valid EP number before interview.

To create an EP Number, please visit

Kindly complete the registration if you have not done it yet.

Position: Pyspark, Azure

Experience: 6 to 8 years

Location: Kolkata, Chennai, Bengaluru, Pune, Gurugram

Job Description

Job Requirements

• 4- 6yrs of experience with strong proficiency in Python and PySpark.
• Experience working on Spark SQL, RDD, and DataFrame APIs.
• Good understanding of Hadoop ecosystem (Hive, HDFS, YARN).
• Knowledge of data formats like Parquet, Avro, JSON, etc.
• Experience with SQL and writing efficient queries.
• Familiarity with job orchestration tools (Airflow, Oozie, or similar).
• Version control systems like Git.
• Exposure to cloud platforms (AWS/GCP/Azure) is a plus.

Key responsibilities:

1. Design, develop, and maintain robust ETL/ELT pipelines using PySpark and other big data technologies. Optimize Spark jobs for performance and scalability.

2. Implement data transformations, aggregations, and joins over large datasets. Perform batch and real-time data processing tasks.

3. Collaborate with data scientists, analysts, and other engineers to understand requirements and deliver quality solutions. Integrate PySpark solutions with data warehouses (like Hive, Redshift, Snowflake) and other data stores.

4. Write clean, maintainable, and well-documented code. Follow version control and CI/CD practices using tools like Git, Jenkins, or Azure DevOps.

5. Troubleshoot data quality and performance issues. Monitor pipeline health and ensure SLAs are met.

6. Work with tools and platforms like Hadoop, Hive, HDFS, AWS EMR, Databricks, or Azure Synapse as required.

TCS Eligibility Criteria:*BE/B.tech/MCA/M.Sc./MS with minimum 3 years of relevant IT-experience post

Qualification.*Only Full-Time courses would be considered.

*Candidates who have attended TCS interview in the last 1 months need not apply.

Referrals are always welcome!!!

Kindly don‘t apply if already attended interview in last 1 months.

Thanks & Regards

• Amrutha