Data Engineer

Role Overview

Data Engineer

Key Responsibilities

• Design, build, and maintain robust, scalable, and efficient data pipelines and ETL/ELT processes on AWS.
• Work closely with data scientists, analysts, and business teams to understand data requirements and deliver solutions.
• Integrate data from multiple internal and third-party sources into unified data platforms.
• Optimize data lake and data warehouse performance (e.g., S3, Redshift, Glue, Athena).
• Ensure data quality, governance, and lineage using appropriate tools and frameworks.
• Implement CI/CD practices for data pipelines and workflows.
• Monitor and troubleshoot production data pipelines to ensure reliability and accuracy.
• Ensure compliance with data privacy and information security policies.

Must-Have Qualifications

• 5 years of experience in data engineering or a related role.
• Strong programming skills in Python, PySpark, or Scala.
• Proficient in SQL and working with structured, semi-structured, and unstructured data.
• Solid experience with AWS services such as:

• S3, Glue, Lambda, Redshift, Athena, Kinesis, Step Functions, CloudFormation

• Hands-on experience with workflow orchestration tools like

  Apache Airflow

  or AWS-native alternatives.
• Must have an active

  AWS certification

  (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect).
• Experience with infrastructure as code (IaC) and DevOps practices is a plus.

Preferred Skills

• Experience with Delta Lake, Apache Hudi, or similar formats.
• Exposure to data cataloging and metadata management tools.
• Familiarity with data security frameworks and GDPR/data privacy considerations.
• Experience in client-facing roles and agile delivery environments.