Software Engineer

We are seeking an experienced Software Engineer with deep expertise in Scala programming and Big Data technologies to design, develop, and maintain large-scale distributed data processing systems. The ideal candidate will be a hands-on developer with a strong understanding of data pipelines, Spark ecosystem, and related technologies, capable of delivering clean, efficient, and scalable code in an Agile environment.

Key Responsibilities

• Develop and maintain scalable, efficient, and robust data processing pipelines using Scala and Apache Spark (Spark Core, Spark SQL, Spark Streaming).
  
• Write clean, maintainable, and well-documented Scala code following industry best practices and coding standards.
  
• Design and implement batch and real-time data processing workflows handling large volumes of data.
  
• Work closely with cross-functional teams to understand business requirements and translate them into technical solutions that meet quality standards.
  
• Utilize Hadoop ecosystem components such as HDFS, Hive, Sqoop, Impala, and related tools to support data storage and retrieval needs.
  
• Develop and optimize ETL processes and data warehousing solutions leveraging Big Data technologies.
  
• Apply deep knowledge of Data Structures and algorithms to ensure efficient data processing and system performance.
  
• Conduct unit testing, code reviews, and performance tuning of data processing jobs.
  
• Automate application job scheduling and execution using UNIX shell scripting (advantageous).
  
• Participate actively in Agile development processes including daily standups, sprint planning, reviews, and retrospectives.
  
• Collaborate effectively with upstream and downstream teams to identify, troubleshoot, and resolve data pipeline issues.
  
• Stay current with emerging technologies, frameworks, and industry trends to continuously improve the architecture and implementation of data solutions.
  
• Support production environments by handling incidents, root cause analysis, and continuous improvements.
  

Required Skills & Experience

• Minimum 8 years

  of professional software development experience with strong emphasis on Scala programming.
• Extensive experience designing and building distributed data processing pipelines using

  Apache Spark

  (Spark Core, Spark SQL, Spark Streaming).
• Strong understanding of Hadoop ecosystem technologies including

  HDFS, Hive, Sqoop, Impala

  , and related tools.
• Proficient in SQL and NoSQL databases with sound knowledge of database concepts and operations.
  
• Familiarity with Data Warehousing concepts and ETL methodologies.
  
• Solid foundation in Data Structures, Algorithms, and Object-Oriented Programming.
  
• Experience in UNIX/Linux shell scripting to manage and schedule data jobs (preferred).
  
• Proven track record of working in Agile software development environments.
  
• Excellent problem-solving skills, with the ability to analyze complex issues and provide efficient solutions.
  
• Strong verbal and written communication skills, with experience working in diverse, global delivery teams.
  
• Ability to manage multiple tasks, collaborate across teams, and adapt to changing priorities.
  

Desired Qualifications

• Bachelors or Master’s degree in Computer Science, Engineering, or a related technical field.
  
• Previous experience working in a global delivery or distributed team environment.
  
• Certification or formal training in Big Data technologies or Scala programming is a plus.