Senior Pyspark Data engineer

Job Description

Job Title: Senior PySpark Data Engineer Share only the quality profiles Location: Pune/Hybrid Experience: 5-8 years Budget: 8LPA-11 LPA Notice Period: Immediate to 15 days Mandatory Skills: · Python · SQL · ETL · Informatica PowerCenter · AWS/Azure Good to Have: · IDMC Tech Stack Table Skills Experience Rating out of 10 Python SQL ETL Informatica PowerCenter Aws/Azure Job Summary: We are seeking a Senior PySpark Data Engineer with extensive experience in developing, optimizing, and maintaining data processing jobs using PySpark. The ideal candidate will possess a robust background in SQL and ETL processes, along with proficiency in cloud platforms such as AWS or Azure. This role will require excellent analytical skills and the ability to communicate effectively with both technical and non-technical stakeholders. Key Responsibilities: · Design, develop, and optimize PySpark jobs for enhanced performance and scalability. · Collaborate with data architects and business analysts to understand data requirements and translate them into technical specifications. · Redesign and maintain complex SQL queries and stored procedures to support data extraction and transformation processes. · Utilize ETL tools, specifically Informatica PowerCenter, to build effective data pipelines. · Troubleshoot and resolve data quality issues and performance bottlenecks. · Mentor and provide technical guidance to a team of developers to enhance productivity and code quality. · Stay updated with new technologies and practices to continually improve data processing capabilities. Qualifications: · Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. · Experience: 5-8 years of experience in data engineering, with a strong focus on PySpark and ETL processes. Technical Skills: Must-Have: 1. Extensive experience with PySpark, focusing on job optimization techniques. 2. Proficiency in SQL, with experience in SQL Server, MySQL, or other relational databases. 3. Strong knowledge of ETL concepts and tools, particularly Informatica PowerCenter and IDMC. 4. Excellent analytical and troubleshooting skills. 5. Strong communication skills for effective collaboration. Good to Have: 1. Basic knowledge of Unix commands and Shell scripting. 2. Experience in leading and mentoring development teams. 3. Familiarity with Azure/Fabric. Kindly share a profile only in this tracker format ,attach the tracker to the body of the mail. Without this tracker format and Tech Stack Table profile will not be considered. S.no Date Position Names of the Candidate Mobile Number Email id Total Experience Relevant Experience CUrrent CTC Expected CTC Notice Period / On Paper Current Organisation Current Location Address with Pin code Reason of leaving DOB Offer in hand VENDOR NAME - Regards Damodar 91-8976334593 info@d-techworks.com D-TechWorks Pvt Ltd USA | INDIA www.d-techworks.com Information Technology Services Technology | Consulting | Development | Staff Augmentation Show more Show less