Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Home
>
Jobs in pune
>
Mastercard
>
Senior Software Engineer (Pyspark, Hadoop, Python, SQL, Airflow)

Senior Software Engineer (Pyspark, Hadoop, Python, SQL, Airflow)

Mastercard

5 - 10 years

11 - 15 Lacs

pune

Posted:4 days ago| Platform:

Apply

Skills Required

software engineer python oracle airflow workflow orchestration pyspark power bi elt data engineering sql data quality apache nifi spark gcp apache spark jenkins splunk hadoop big data aws etl apache airflow architecture azure

Work Mode

Work from Office

Job Type

Full Time

Job Description

Title and Summary

Senior Software Engineer (Pyspark, Hadoop, Python, SQL, Airflow)Job Summary:As a Senior Software Engineer focused on Data Quality, you will lead the design, development, and deployment of scalable data quality frameworks and pipelines. You will work closely with data engineers, analysts, and business stakeholders to build robust solutions that validate, monitor, and improve data quality across large-scale distributed systems.

Key Responsibilities:

Lead the design and implementation of data quality frameworks and automated validation pipelines using Python, Apache Spark, and Hadoop ecosystem tools.Develop, deploy, and maintain scalable ETL/ELT workflows using Apache Airflow and Apache NiFi to ensure seamless data ingestion, transformation, and quality checks.
Collaborate with cross-functional teams to understand data quality requirements and translate them into technical solutions.Define and enforce data quality standards, rules, and monitoring processes.Perform root cause analysis on data quality issues and implement effective fixes and enhancements.
Mentor and guide junior engineers, conducting code reviews and fostering best practices.
Continuously evaluate and integrate new tools and technologies to enhance data quality capabilities.Ensure high code quality, performance, and reliability in all data processing pipelines.
Create comprehensive documentation and reports on data quality metrics and system architecture.

Required Skills & Experience:

* Bachelors or Masters degree in Computer Science, Engineering, or a related field with Data Engineering Experience.

* 5+ years of professional experience in software development, with at least 2 years in a lead or senior engineering role.

* Strong proficiency in Python programming and experience building data processing applications.

* Hands-on expertise with Apache Spark and Hadoop for big data processing.

* Solid experience with workflow orchestration tools like Apache Airflow.

* Experience designing and managing data ingestion and integration pipelines with Apache NiFi.

* Understanding on Data Quality automation, CI/CD, Jenkins, Oracle, Power BI, Splunk

* Deep understanding of data quality concepts, data validation techniques, and distributed data systems.

* Strong problem-solving skills and ability to lead technical discussions.

* Experience with cloud platforms (AWS, GCP, or Azure) is a plus.

* Excellent communication and collaboration skills

More Jobs at Mastercard

Senior Analyst - Project Management

Pune

5 - 9 yrs

INR 13 - 18 Lacs

Sr. Principal Software Engineer

Pune

9 - 13 yrs

INR 20 - 25 Lacs

Lead Software Engineer-R-245726

Vadodara, Gujarat, India

2 - 4 yrs

Salary: Not disclosed

Consultant-2

Gurugram, Haryana

Experience: Not specified

Salary: Not disclosed

Consultant-2

Gurugram, Haryana

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Software Engineer Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

Mastercard

IT Services and IT Consulting

Purchase NY

Login to

Please Verify Your Phone or Email

Confirm Action

Senior Software Engineer (Pyspark, Hadoop, Python, SQL, Airflow)