Data Engineer

Job Title: Senior Data Engineer/Module Lead

Location: Pune, Maharashtra, India

Experience Level: 5-8 Years

Work Model: Full-time

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing team in Pune. The ideal candidate will have a strong background in data engineering, with a particular focus on Google Cloud Platform (GCP) data services and Apache Airflow. You will be responsible for designing, developing, and maintaining robust and scalable data pipelines, ensuring data quality, and optimizing data solutions for performance and cost. This role requires a hands-on approach and the ability to work independently and collaboratively within an agile environment.

Responsibilities:

• Design, develop, and deploy scalable and efficient data pipelines using Google Cloud Platform (GCP) data services, specifically BigQuery, Cloud Composer (Apache Airflow), and Cloud Storage.
• Develop, deploy, and manage complex DAGs in Apache Airflow for orchestrating data workflows.
• Write complex SQL and PL/SQL queries, stored procedures, and functions for data manipulation, transformation, and analysis.
• Optimize BigQuery queries for performance, cost efficiency, and scalability.
• Ensure data quality, integrity, and reliability across all data solutions.
• Collaborate with cross-functional teams including data scientists, analysts, and other engineers to understand data requirements and deliver effective solutions.
• Participate in code reviews and contribute to best practices for data engineering.
• Troubleshoot and resolve data-related issues in a timely manner.
• Manage and maintain version control for data engineering projects using Git.
• Stay up-to-date with the latest industry trends and technologies in data engineering and GCP.

Required Skills and Qualifications:

• 5-8 years of hands-on experience in Data Engineering roles.

• Strong hands-on experience with Google Cloud Platform (GCP) data services, specifically BigQuery, Cloud Composer (Apache Airflow), and Cloud Storage.

• Mandatory expertise in Apache Airflow, including designing, developing, and deploying complex DAGs.

• Mandatory strong proficiency in SQL and PL/SQL for data manipulation, stored procedures, functions, and complex query writing.

• Experience with Informatica.
• Proven ability to optimize BigQuery queries for performance and cost.
• Familiarity with version control systems (e.g., Git).
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and collaboratively in an agile environment.
• Bachelor's degree in Computer Science, Engineering, or a related field.