D-TechWorks - Data Engineering Lead - ETL

0 years

0 Lacs

Posted:4 days ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Data Engineer Lead - Pune

Job Title :

Data Engineer Lead

Experience :

5 to 9 Yrs

Location :

Pune

Responsibilities

  • Lead and mentor a team of data engineers, providing technical guidance, setting best practices, and overseeing task execution for the migration project.
  • Design, develop, and architect scalable ETL processes to extract, transform, and load petabytes of data from on-premises SQL
Server to GCP Cloud SQL PostgreSQL.
  • Oversee the comprehensive analysis of existing SQL Server schemas, data types, stored procedures, and complex data models, defining strategies for their optimal conversion and refactoring for PostgreSQL.
  • Establish and enforce rigorous data validation, quality, and integrity frameworks throughout the migration lifecycle, ensuring accuracy and consistency.
  • Collaborate strategically with Database Administrators, application architects, business stakeholders, and security teams to define migration scope, requirements, and cutover plans.
  • Lead the development and maintenance of advanced scripts (primarily Python) for automating large-scale migration tasks, complex data transformations, and reconciliation processes.
  • Proactively identify, troubleshoot, and lead the resolution of complex data discrepancies, performance bottlenecks, and technical
challenges during migration.
  • Define and maintain comprehensive documentation standards for migration strategies, data mapping, transformation rules, and
post-migration validation procedures.
  • Ensure data governance, security, and compliance standards are meticulously applied throughout the migration process, including data encryption and access controls within GCP.
  • Implement Schema conversion or custom schema mapping strategy for SQL Server to PostgreSQL shift
  • Refactor and translate complex stored procedures and T-SQL logic to PostgreSQL-compatible constructs while preserving functional equivalence.
  • Develop and execute comprehensive data reconciliation strategies to ensure consistency and parity between legacy and migrated datasets post-cutover.
  • Design fallback procedures and lead post-migration verification and support to ensure business continuity.
  • Ensuring metadata cataloging and data lineage tracking using GCP-native or integrated tools.
(ref:hirist.tech)

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You