Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)

Position: Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)

Education Required: Bachelor’s / Masters / PhD:

Bachelor’s or master’s in computer science, Statistics, Mathematics, Data Science, Engineering

AWS certification (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Developer)

Must have skills:

Proficiency in AWS cloud services: AWS Glue, QuickSight, S3, Lambda, Athena, Redshift, EMR, and related technologies

Strong experience with PySpark

Expertise in SQL and data modeling for relational and non-relational databases

Familiarity with business intelligence and visualization tools, especially Amazon QuickSight

Good to have:

Proficiency in Python and ML libraries (scikit-learn, TensorFlow, PyTorch).

Understanding of MLOps and model deployment best practices.

Hands-on experience with AWS services for ML.

Experience or familiarity with HVAC domain is a plus

Key Responsibilities:

Design, develop, and maintain data pipelines using AWS Glue, PySpark, and related AWS services to extract, transform, and load (ETL) data from diverse sources

Build and optimize data warehouse/data lake infrastructure on AWS, ensuring efficient data storage, processing, and retrieval

Develop and manage ETL processes to source data from various systems, including databases, APIs, and file storage, and create unified data models for analytics and reporting

Implement and maintain business intelligence dashboards using Amazon QuickSight, enabling stakeholders to derive actionable insights

Collaborate with cross-functional teams (business analysts, data scientists, product managers) to understand requirements and deliver scalable data solutions

Ensure data quality, integrity, and security throughout the data lifecycle, implementing best practices for governance and compliance5.

Support self-service analytics by empowering internal users to access and analyze data through QuickSight and other reporting tools1.

Troubleshoot and resolve data pipeline issues, optimizing performance and reliability as needed

Required Skills:

Proficiency in AWS cloud services: AWS Glue, QuickSight, S3, Lambda, Athena, Redshift, EMR, and related technologies

Strong experience with PySpark for large-scale data processing and transformation

Expertise in SQL and data modeling for relational and non-relational databases

Experience building and optimizing ETL pipelines and data integration workflows

Familiarity with business intelligence and visualization tools, especially Amazon QuickSight

Knowledge of data governance, security, and compliance best practices.

Strong programming skills in Python; experience with automation and scripting

Ability to work collaboratively in agile environments and manage multiple priorities effectively

Excellent problem-solving and communication skills.

*******************************************************************************************************************************************