Job Description
We are seeking an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments. The role requires strong hands-on experience in big data processing, pipeline optimization, and deployment using modern data engineering tools and frameworks.
Key Responsibilities
Design, develop, and maintain robust, scalable data pipelines using Python and PySpark
Perform data ingestion, transformation, cleansing, and validation across structured and unstructured datasets
Conduct Exploratory Data Analysis (EDA) to identify data patterns, anomalies, and quality issues
Apply data imputation techniques, data linking, and cleansing to ensure high data quality
Implement feature engine...
Apply for this Position
Ready to join Black Pearl Consult? Click the button below to submit your application.
Submit Application