Lead Data Engineer

Themathcompany

6 - 10 years

8 - 12 Lacs

Bengaluru

Posted:1 day ago| Platform:

Apply

Skills Required

Cloud computing NoSQL Data modeling SCALA Software development life cycle Data processing Data quality Unit testing SQL Python

Work Mode

Work from Office

Job Type

Full Time

Job Description

As a Lead Data Engineer you will be responsible for:

Leading a team of talented data engineers responsible for designing, building, and maintaining scalable data pipelines and infrastructure
Work closely with cross-functional teams to ensure client data systems meet the highest standards of quality and performance
Lead, mentor, and develop a team of data engineers, fostering a collaborative and inclusive team environment
Conduct regular performance reviews, provide feedback, and set goals for team members
Identify and address skill gaps, and provide opportunities for professional development
Plan, execute, and deliver data engineering projects on time and within scope
Coordinate with stakeholders to gather requirements, set priorities, and define project timelines.
Ensure projects align with overall business objectives and data strategy
Oversee the design, development, and maintenance of data pipelines, ETL processes, and data warehouse
Ensure data quality, integrity, and security across all data engineering projects.
Identify opportunities for process improvements and drive initiatives to enhance the efficiency and effectiveness of data operations.
Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing
Ability to build/drive reusable frameworks that can drive efficiency of the overall data system
Has executed and lead multiple projects including on - streaming, batch, large data pipelines, etc.
Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.

Required Tech Stack:

Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
Working knowledge of an ETL and/or orchestration tools like IICS, Metallion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Hive, Big Query, etc.
Proficient in Spark and can optimize Spark jobs with ease.
Proficient in at least one programming language used in data engineering, such as Python (Non-negotiable) and Scala/Rust/Java
Understanding of Medallion architecture pattern
Has strong SQL knowledge along with optimization skills

Required Non-Tech Stack

Strong problem-solving skills with an ability to assess the financial impact of decisions, both in running the delivery team and delivering solutions to clients.
Proficient in written and verbal communication and able to hold conversations with mid-management-level clients.
Ability to recognize pragmatic alternatives vis- -vis a perfect solution and get the delivery teams on board to pursue them, balancing time priorities with potential business impact.
Strong people skills, including conflict resolution, empathy, communication, listening, and negotiation.
Shows proficiency in providing technical guidance and provide leadership and mentorship to the delivery team.
Self-driven with a strong sense of ownership.

Preferred Educational Qualifications

B.E/B.Tech, MCA, M.Sc. (Mathematics, Statistics)

More Jobs at Themathcompany

Senior Power BI Consultant

Bengaluru

4 - 6 yrs

INR 10 - 15 Lacs

Sr. Research Specialist

Bengaluru

6.0 - 9.0 yrs

INR 10 - 14 Lacs

AI Delivery Lead

Bengaluru

11.0 - 14.0 yrs

INR 50 - 55 Lacs

Data Science Manager

Bengaluru

5.0 - 8.0 yrs

INR 25 - 35 Lacs

Product Engineer - II (QA)

Bengaluru

2.0 - 4.0 yrs

INR 11 - 15 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.