Data Engineer Lead

Profile - Data Engineer Lead

Exp- 8+ Years

Location - Noida

Work Mode - Hybrid

Skills - Data Engineer , AWS/Azure/GCP , Snowflake , ETL , Data Warehousing

Seeking a Lead Data Engineer to drive the development and optimization of data infrastructure supporting public, private, and purchased healthcare datasets. This role will focus on designing and implementing scalable data pipelines, ensuring data quality, and collaborating with internal teams and external vendors to refine data acquisition and integration strategies.

The ideal candidate will have strong expertise in healthcare data engineering, cloud-based platforms, and ETL development, with experience working across multiple data sources, including government databases, commercial claims, and research-funded datasets.

Key Responsibilities

Data Engineering & Pipeline Development:

• Develop and maintain robust, scalable ETL pipelines to ingest, clean, and transform public (e.g., CMS, Medicaid), private (e.g., commercial claims, EHRs), and purchased (e.g., Clarivate, Doceree) datasets.
• Optimize data workflows and storage solutions to enhance data accessibility and usability for analytics and business intelligence teams.
• Implement data quality frameworks, ensuring the integrity and accuracy of healthcare datasets used for decision-making.
• Collaborate with data architects to design and enhance data models, ensuring efficient query performance and integration across multiple sources.

Collaboration with External Vendors & Partners:

• Work closely with external data vendors, research organizations, and commercial agencies to define projects, ensure data consistency, and improve ingestion processes.
• Provide technical oversight on vendor-sourced data projects, ensuring compliance with Avalere’s data quality and governance standards.
• Support the evaluation and acquisition of new datasets, integrating them into Avalere’s data ecosystem for analytics and modeling.

Internal Team Leadership & Support:

• Partner with data scientists, analysts, and market access teams to develop data solutions that drive business insights.
• Provide technical mentorship to junior data engineers, ensuring best practices in data pipeline development, automation, and cloud-based infrastructure.
• Work cross-functionally with data architects and analytics teams to align data engineering efforts with business and research objectives.

Required Skills and Experience

Education:

• Bachelor’s degree in computer science, information systems, data engineering, or a related field. Master’s degree preferred.

Technical Expertise:

• Extensive experience in building and managing ETL pipelines for large-scale healthcare data processing.
• Strong knowledge of public healthcare datasets, including Medicare/Medicaid claims, state health registries, and government-funded research data.
• Experience with private-sector datasets, such as commercial claims, EHRs, and provider-generated data.
• Proficiency in SQL, Python, and Spark for large-scale data processing and transformation.
• Hands-on experience with cloud platforms (AWS, Azure, GCP) and big data technologies (Databricks, Snowflake, Hadoop).
• Deep understanding of data modeling, data warehousing, and governance best practices.
• Experience implementing data security and compliance frameworks, including HIPAA, HITRUST, and other healthcare regulations.

Professional Experience:

• 8+ years of experience in data engineering, pipeline development, or a related field, preferably within healthcare, consulting, or analytics.
• Demonstrated ability to integrate, process, and optimize multi-source datasets for analytics and research applications.
• Experience managing data acquisition projects and working with external vendors to refine data ingestion approaches.
• Proven track record in developing scalable cloud-based data solutions for enterprise-wide use.

Collaboration & Leadership:

• Experience leading technical teams and mentoring junior engineers.
• Strong communication skills to engage with business stakeholders, data scientists, and external partners.

Ability to translate business needs into scalable, technical data solutions

Regards

Yogendra Joshi

Mob - 6397457648