Bigdata Engineer

About the Role:

Seeking a highly skilled Bigdata Engineer with an experience of 4 years to join our dynamic team.

Requirements:

• At least 4 to 8 years of experience in application development using Spark and Scala.


• Hands-on exposure to working on the Hadoop ecosystem including HDFS, Hive and Spark.


• Good understanding of Hadoop file formats.


• Expertise in Hive, HDFS, PySpark, Spark, Jupyter Notebook, ELT Talend, Control-M, Unix scripting, Python, CI/CD, Git, Jira, Hadoop, TOM, Oozie and Snowflake.


• Proven expertise in implementing data quality controls.


• Ability to interpret the Spark UI, identify bottlenecks in the Spark process and provide optimal solutions.


• Exposure to Agile methodology and processes.


• Good to have exposure to CI/CD processes.


• Ability to understand complex business rules and translate them into technical specifications and designs.


• Proficiency in writing highly efficient and optimized code that is scalable.


• Adherence to coding quality and security standards.


• Proficiency in Spark and Scala for coding and transforming raw requirements into technical solutions.


• Experience with Hadoop HDFS infrastructure and platform, including proficiency in HDFS commands.


• Strong skills in SQL for data analysis, querying and retrieval.


• Expertise in GitHub for code management throughout the development lifecycle.

#LI-MS2 #LI-Hybrid