Data Engr/Sr Data Pipeline Strictly SCALA , Spark Streaming German MNC

Full time with top German MNC for location Bangalore - Experience on SCALA is a must

Job Overview:

• To work on development, monitoring and maintenance of Data pipelines across clusters.

Primary responsibilities:

• Develop, Monitor and Maintain data pipeline for various plants.

• Create and maintain optimal data pipeline architecture.

• Assemble large, complex data sets that meet functional / non-functional business requirements.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.

• Work with stakeholders including the Data officers and stewards to assist with data-related technical issues and support their data infrastructure needs.

• Work on incidents highlighted by the data officers.

• Incident diagnosis, routing, evaluation & resolution.

• Analyze the root cause of incidents.

• Create incident closure report.

Qualifications

Qualifications

• Bachelors degree in Computer Science, Electronics & Communication Engineering, a related technical field, or equivalent practical experience.

• 6-8 years of experience in Spark, Scala software development.

• Experience in large-scale software development.

• Excellent software engineering skills (i.e., data structures, algorithms, software design).

• Excellent problem-solving, investigative, and troubleshooting skills.

• Experience in Kafka is mandatory

Additional Information

Skills

• Self-starter and empowered professional with strong execution and project management capabilities

• Ability to collaborate effectively, well developed inter personal relationships with all levels in the organization and outside contacts.

• Outstanding written and verbal communication skills.

• High Collaboration & a perseverance to drive performance & change

Additional information

Key Competencies-

• Distributed computing systems

• Experience with CI/CD tools such as Jenkins or Github Actions

• Experience with Python programming

• Working knowledge of Docker & Kubernetes

• Experience in developing data pipelines using spark & scala.

• Experience in debugging pipeline issues.

• Experience in writing python and shell scripts.

• In-Depth Knowledge of SQL and Other Database Solutions

• Having a strong understanding of Apache Hadoop-based analytics

• Hands on experience on InteliJ, Github /Bitbucket, HUE.