Data Engineer

At 3Pillar, the focus is on leveraging cutting-edge technologies to revolutionize industries through enabling data-driven decision-making. As a Data Engineer, you will play a crucial role within the dynamic team, actively contributing to exciting projects that redefine data analytics for clients, providing them with a competitive edge in their respective industries. If you are passionate about data analytics solutions that have a real-world impact, this position opens the door to the captivating world of Data Science and Engineering! You will be based at the office location in Vikroli/Churchgate, Mumbai. Responsibilities - Design, develop, and maintain data pipelines and workflows using PySpark to efficiently process large volumes of data. - Implement data ingestion, transformation, and storage solutions to meet business requirements. - Develop and optimize Python scripts and algorithms for data manipulation, analysis, and modeling tasks. - Design and manage database systems, including schema design, indexing, and performance optimization. - Collaborate with cross-functional teams to understand data requirements and deliver solutions that align with business needs. - Perform data quality assessment, validation, and cleansing to ensure data accuracy and integrity. - Monitor and troubleshoot data pipeline issues, performance bottlenecks, and system failures. - Stay updated with emerging technologies and best practices in data engineering and analytics. Qualifications - Bachelor's degree in Computer Science, Engineering, or a related field. - Minimum of 3 years of experience in Database or Data Analytics. - At least 3 years of experience in data engineering, with expertise in Spark, Python, and database technologies. - Experience with PySpark for data processing and analysis tasks. - Proficiency in Python programming for scripting and automation. - Strong understanding of database concepts and hands-on experience with SQL and NoSQL databases. - Experience with data modeling, schema design, and ETL processes. - Strong problem-solving skills and ability to work independently and collaboratively in a fast-paced environment. - Excellent communication and interpersonal skills, with the ability to effectively convey technical concepts to non-technical stakeholders. - Experience with cloud platforms would be an advantage. If you meet the qualifications and are excited about the opportunity to work on impactful data analytics projects with a focus on innovation, apply for this job.,