Associate II - Data Engineering

Experience Range : 3 to 7 yearsHiring Locations : Chennai, Trivandrum, KochiRole DescriptionThe role demands proficiency in designing and developing robust data pipelines using ETL tools and programming languages such as Python, PySpark, and SQL. The candidate will be responsible for coding, testing, and implementing data ingestion and transformation pipelines. Additionally, the role includes L1 Data Operations responsibilities, such as monitoring dashboards, identifying anomalies, and escalating issues as per standard operating procedures.Key ResponsibilitiesData Pipeline Development :Independently develop, test, and implement data processing pipelines.Use tools likeInformatica, Glue, Databricks, and DataProc .Write clean, scalable, and optimized code inPython, PySpark, and SQL .Conduct thorough unit testing to ensure data accuracy and pipeline stability.Create clear documentation and maintain project artifacts.Data Operations (L1 Monitoring) :Monitor data pipelines, dashboards, and databases on a shift basis (24x7 support including night shifts).Identify, log, and escalate anomalies or failures using SOPs and runbooks.Execute basic SQL queries for data validation and issue resolution.Collaborate with L2\/L3 teams for escalation and root cause analysis.Maintain logs of incidents and escalations.Additional Responsibilities :Adhere to project timelines, SLAs, and compliance standards.Participate in estimation of effort and timelines for assigned work.Obtain foundational certifications in cloud platforms (Azure, AWS, or GCP).Contribute to knowledge management, documentation repositories, and release management processes.Mandatory SkillsProficiency inPython, PySpark, and SQLExperience withETL tools like Informatica, AWS Glue, Databricks, or DataProcStrong understanding ofdata pipeline design and data wranglingHands-on experience incloud platforms \u2013 AWS, Azure, or GCP (especially with data services)Knowledge ofdata schemas, transformations, and modelsStrong ability to debug and test data processes and troubleshoot issuesGood to Have SkillsFamiliarity withApache Airflow, Talend, Azure ADF, or GCP DataFlowCertification inAzure\/AWS\/GCP data servicesExperience inproduction monitoring , L1 data ops support, and incident escalationExposure towindowing functions in SQL andadvanced Excel analysisKnowledge of Agile\/Scrum development processesSoft SkillsStrongwritten and verbal communication skillsExcellentanalytical and problem-solving abilityAbility towork independently with minimal supervisionKeenattention to detail and precision in monitoring tasksCollaboration and coordination skills for working with cross-functional support teamsAbility tomultitask and remain calm in high-pressure, fast-paced environmentsWillingness to work in24x7 shift schedules , includingnight shifts as required ","