Data Engineer_8+years_Hyderabad

As a Data Engineer, you will be responsible for developing and enhancing data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation. Collaborating with product and technology teams to design and validate the capabilities of the data platform will be a key part of your role. You will also identify, design, and implement process improvements, automate manual processes, optimize for usability, and redesign for greater scalability. Providing technical support and usage guidance to the users of our platform services is also a crucial aspect of this position. Additionally, you will drive the creation and refinement of metrics, monitoring, and alerting mechanisms to provide visibility into our production services. The ideal candidate will have experience building and optimizing data pipelines in a distributed environment, supporting and working with cross-functional teams, and proficiency working in a Linux environment. A minimum of 4 years of advanced working knowledge of SQL, Python, and PySpark is required, with a strong emphasis on PySpark queries. Knowledge of Palantir and experience using tools such as Git/Bitbucket, Jenkins/CodeBuild, and Code Pipeline will be advantageous. Experience with platform monitoring and alert tools is also desirable. This is a Data Engineer role based in Hyderabad with a contract duration of 12+ months, likely to be extended. Strong skills in Python programming, PySpark queries, and AWS are essential for this position, with Palantir being a secondary skill. If you are passionate about data engineering, possess the necessary technical skills, and thrive in a collaborative environment, we encourage you to apply for this exciting opportunity.,