Data Engineer Python

You will be part of our Data Engineering Team in Gurgaon, Noida, or Pune, contributing to the development and management of large enterprise data and analytics platforms. Your role will involve collaborating with the data engineering and data science teams to implement scalable data lakes, data ingestion platforms, machine learning analytics platforms, and more. With at least 3 years of industry experience, you will be responsible for creating end-to-end data solutions and optimal data processing pipelines for handling large volumes of diverse data types. Proficiency in Python, including knowledge of design patterns and strong design skills, is essential. You should have expertise in working with PySpark Dataframes, Pandas Dataframes, and developing efficient data manipulation tasks. Experience in building Restful web services, API platforms, SQL, NoSQL databases, and stream-processing systems like Spark-Streaming and Kafka will be crucial. You will collaborate with the data science and infrastructure teams to deploy machine learning solutions in production environments. It would be advantageous if you have experience with testing libraries like pytest, knowledge of Dockers, Kubernetes, model versioning with mlflow, and microservices libraries. Familiarity with machine learning algorithms and libraries is a plus. Our ideal candidate is proactive, independent, and enjoys problem-solving. Continuous learning is encouraged as we are a rapidly growing company. Being a team player and an effective communicator is essential for success in this role.,