Pyspark Data Engineer

As a Senior Data Engineer, you will be responsible for: - Designing and implementing solutions for problems arising out of large-scale data processing - Attending/driving various architectural, design, and status calls with multiple stakeholders - Ensuring end-to-end ownership of all tasks aligned, including development, testing, deployment, and support - Designing, building, and maintaining efficient, reusable, and reliable code - Testing implementation, troubleshooting, and correcting problems - Working both as an individual contributor and within a team - Ensuring high-quality software development with complete documentation and traceability - Fulfilling organizational responsibilities by sharing knowledge and experience with other teams/groups Qualifications: - Strong experience working with the Apache Spark framework, including a solid grasp of core concepts, performance optimizations, and industry best practices - Proficiency in PySpark with hands-on coding experience; familiarity with unit testing, object-oriented programming (OOP) principles, and software design patterns - Experience with code deployment and associated processes - Proven ability to write complex SQL queries to extract business-critical insights - Hands-on experience in streaming data processing - Familiarity with machine learning concepts is an added advantage - Experience with NoSQL databases - Good understanding of Test-Driven Development (TDD) methodologies - Demonstrated flexibility and eagerness to learn new technologies Please note that the experience required for this position is 5 to 8 years.,