Data Engineering Lead-Aws

Technical Skills Required:
- At least Intermediate level in AWS ETL (Glue, lambda, Batch, EMR) & API Development(Fast API + Core python), Redshift
- AWS Redshift:
- Worked on creating provisioned clusters
- Expert knowledge of Workload management
- Materialized View, DDL Optimizations, Json handling
- AWS Glue to Redshift Loads
- Row & Column Level Security
- AWS Glue (ETL, Jobs, Crawlers, Catalogue including Iceberg, Workflows, Dynamic Data Frames)
- Worked on Batch & Stream workloads
- Performance Tuning & Cost Optimization
- IAM, KMS, Secret Manager and Fine-Grained Access Control (Encryption good to have)
- PySpark, Python
- Lambda, Amazon S3, Athena, RDS
- Apache Parquet, JSON, CSV
- Data Lake Design & Implementation
Good to have:
- CI/CD (Terraform or AWS CDK or CloudFormation)
- Data Lineage & Governance
- Kinesis/Kafka/Glue Streaming/AWS Batch
Key Responsibilities:
- Collaborate with business stakeholders to analyze...