Lead Data Engineer

ITQube LTD

3 - 5 years

0 Lacs

pune maharashtra india

Posted:1 day ago| Platform: Foundit logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Looking for a Data Engineer to join our engineering will contribute directly to the design, automation, and optimization of our data processes, primarily developing solutions in Python within the AWS cloud ecosystem, including Lambda, Glue, Redshift, and other key services.
The role also involves working with infrastructure as code (Terraform), Git-based version control, and designing scalable data architectures. The ideal candidate has solid data engineering knowledge, a passion for automation, experience working with large volumes of data, and the ability to proactively propose technical and architectural improvements. Responsibilities:
Design, develop, and maintain efficient ETL processes using Python, SQL, AWS Glue, and Redshift.
Automate data flows and integrations using AWS Lambda and other serverless services. Propose improvements and optimizations to existing pipelines, prioritizing performance, scalability, and maintainability.
Collaborate on the design of scalable and resilient data architectures in AWS.
Develop and manage infrastructure as code using Terraform.
Actively participate in code reviews and collaborative version control workflows using Git.
Document technical solutions and promote best practices in data engineering.
Ensure that data processing pipelines can handle large-scale datasets (Big Data). Who we're looking for: Technical Requirements:
Programming Languages: Python 3+ years of experience (Intermediate to advanced level) o SQL 3+ years of experience, capable of working with complex data models
Large-scale data processing:
Experience with PySpark or Big Data environments 12+ years (preferred)
Familiarity with distributed processing and performance optimization for high-volume data pipelines
AWS Technologies: Lambda, Glue, Redshift, S3 23 years of hands-on experience
Experience designing ETL workflows using native AWS services
Proactive mindset with a drive to propose and implement improvements
Analytical thinker with the ability to identify bottlenecks and performance issues.
Effective collaborator with both technical teams and non-technical stakeholders
Strong documentation and technical communication skills Nice to have (not mandatory):
Experience with monitoring and observability tools for data pipelines (CloudWatch, logging, alerting), Knowledge of event-driven architecture design
Familiarity with orchestration tools like Apache
Experience with automated testing for data pipelines
Background in Fintech or experience handling financial datasets

More Jobs at ITQube LTD

MS Dynamics 365 F&O Developer

Gurugram, Haryana, India

5 - 8 yrs

Salary: Not disclosed

Business Analyst

Kochi, Kerala, India

Experience: Not specified

Salary: Not disclosed

Full Stack Engineer

Coimbatore, Tamil Nadu, India

Experience: Not specified

Salary: Not disclosed

Lead Data Engineer

pune, maharashtra, india

3.0 - 5.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.