Senior Big Data Engineer

The Applications Development Programmer Analyst position is an intermediate level role where you will participate in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. Your main objective will be to contribute to applications systems analysis and programming activities. Your responsibilities will include utilizing your knowledge of applications development procedures and concepts to identify and define necessary system enhancements. You will be expected to identify and analyze issues, make recommendations, and implement solutions based on your understanding of business processes, system processes, and industry standards to solve complex issues. Your role will also involve analyzing information to recommend solutions and improvements, conducting testing and debugging, writing basic code for design specifications, and assessing risk when making business decisions. To excel in this role, you should have hands-on experience in designing, developing, and optimizing scalable distributed data processing pipelines using Apache Spark and Scala. You should possess 3 to 6+ years of experience in big data development, focusing on Apache Spark, Scala/Python, and distributed systems. Advanced knowledge of Scala, a good understanding of Python for data engineering tasks, solid understanding of data modeling principles and ETL processes in big data environments, strong analytical and problem-solving skills in analyzing and solving performance issues in Spark jobs and distributed systems, familiarity with Git, Jenkins, and other CI/CD tools for automating the deployment of big data applications, as well as experience with streaming platforms such as Apache Kafka or Spark Streaming. You should hold a Bachelor's degree/University degree or equivalent experience to be considered for this position. This job description provides an overview of the work performed, and other job-related duties may be assigned as required.,