Data Engineer - Python,Pyspark,AWS - 5 DAYS ONSITE

You will be required to have strong Python programming skills and expertise in Pyspark queries and AWS. As part of your responsibilities, you will be developing and enhancing data-processing, orchestration, monitoring, and more by utilizing popular open-source software, AWS, and GitLab automation. Collaboration with product and technology teams to design and validate the capabilities of the data platform will be a key aspect of your role. You will also be expected to identify, design, and implement process improvements, automate manual processes, optimize for usability, and redesign for greater scalability. Providing technical support and usage guidance to the users of the platform services will also be part of your responsibilities. You will drive the creation and refinement of metrics, monitoring, and alerting mechanisms to provide visibility into production services. To be successful in this role, you should have experience building and optimizing data pipelines in a distributed environment, supporting and working with cross-functional teams, and proficiency working in a Linux environment. A minimum of 4 years of advanced working knowledge of SQL, Python, and PySpark is required, with expertise in PySpark queries being a must. Knowledge of Palantir and experience using tools such as Git/Bitbucket, Jenkins/CodeBuild, and Code Pipeline will be beneficial. Experience with platform monitoring and alert tools will also be an advantage.,