Lead Data Engineer

Job Title:

Location:

About the Role:

Lead Data Engineer

Key Responsibilities:

• Design, develop, and maintain large-scale distributed data processing systems using

  Spark (on EMR)

  and

  Scala

• Build and manage real-time data pipelines with

  Apache Kafka

• Leverage

  SQL

  ,

  Athena

  , and other AWS data tools for efficient data querying and transformation
• Orchestrate workflows using

  Apache Airflow

• Deploy and manage infrastructure using

  AWS components

  (e.g.,

  EKS

  , EMR, S3, etc.)
• Collaborate with stakeholders to build interactive dashboards and reports using

  Superset

  or other

  data visualization tools

• Ensure high-quality data availability, integrity, and governance across systems
  
• Provide technical leadership, code reviews, and mentoring to junior engineers
  

Required Skills:

• Strong experience with

  Apache Spark

  and

  Scala

• Hands-on expertise in

  Apache Kafka

  for streaming data solutions
• Strong command of

  SQL

  ; experience with

  Athena

  is a plus
• Experience working with

  AWS services

  (especially

  EMR

  ,

  EKS

  ,

  S3

  , etc.)
• Experience with

  Airflow

  for job scheduling and orchestration
• Familiarity with

  Superset

  or similar

  data visualization

  tools (e.g., Tableau, Power BI)
• Understanding of data warehouse technologies like

  Hive

  and

  Presto

  (deep expertise not required if strong in SQL)

Preferred Qualifications:

• Prior experience in a

  leadership or mentoring role

• Exposure to best practices in data engineering, data governance, and security
  
• Strong problem-solving skills and ability to work in a fast-paced environment
  

What We Offer:

• Opportunity to work on cutting-edge data technologies
  
• Collaborative and inclusive work culture