Python Hadoop Data Engineer

Tech Mahindra

7 - 10 years

8 - 9 Lacs

bengaluru

Posted:1 hour ago| Platform:

Apply

Skills Required

data processing pyspark data warehousing microsoft azure problem solving data architecture transformation relational databases machine learning data engineering sql docker data architecture principles git data modeling gcp kafka hadoop aws etl

Work Mode

Work from Office

Job Type

Full Time

Job Description

Design, develop, and maintain robust data pipelines using PySpark to process large datasets efficiently
Collaborate with data scientists and analysts to understand data requirements and deliver high quality data solutions
Implement data integration processes from various sources, ensuring data quality and consistency
Optimize existing data workflows and troubleshoot performance issues to enhance data processing efficiency
Work with cloud based data platforms and tools to manage and store data securely
Document data engineering processes and maintain clear communication with stakeholders regarding project status and challenges
Stay updated with the latest industry trends and technologies in data engineering and big data
Mandatory Skills:Strong expertise in PySpark for data processing and transformation
Proficient in SQL and experience with relational databases
Experience with data warehousing solutions and ETL processes
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud
Solid understanding of data modeling and data architecture principles
Ability to work in a fast paced environment and manage multiple priorities effectively
Preferred Skills:Experience with Apache Kafka or similar streaming technologies
Knowledge of machine learning concepts and frameworks
Familiarity with containerization technologies like Docker
Experience with version control systems such as Git
Strong analytical and problem solving skills

More Jobs at Tech Mahindra

Customer Support Associate || Banking Process || Gandhinagar, Gujrat

Gandhinagar, Dehgam, Ahmedabad

Experience: Not specified

INR 0 - 2 Lacs

Urgent Hiring For Voice & Accent Trainer For Hyderabad.

Hyderabad

2 - 6 yrs

INR 0 - 0 Lacs

Fresher Requirement- Immediate Joiner- Hyderabad

Hyderabad

1 - 4 yrs

INR 1 - 3 Lacs

Java Script Developer

Bengaluru

5 - 8 yrs

INR 10 - 20 Lacs

Api Engineer

Bengaluru

5 - 9 yrs

INR 12 - 22 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.