Lead AWS Data Engineer (Python & Scala)

Job Description

Key Responsibilities: Develop & Optimize Data Pipelines Architect, build, and enhance scalable data pipelines for high-performance processing. Troubleshoot & Sustain Identify, diagnose, and resolve data pipeline issues to ensure operational efficiency. Data Architecture & Storage Design efficient data storage and retrieval strategies using Postgres, Redshift, and other databases. CI/CD Pipeline Management Implement and maintain continuous integration and deployment strategies for smooth workflow automation. Scalability & Performance Tuning Ensure the robustness of data solutions while optimizing performance at scale. Collaboration & Leadership Work closely with cross-functional teams to ensure seamless data flow and lead engineering best practices. Security & Reliability – Establish governance protocols and ensure data integrity across all pipelines. Technical Skills Required: Programming: Expert in Python and Scala Big Data Technologies: Proficient in Spark, Kafka DevOps & Cloud Infrastructure: Strong understanding of Kubernetes SQL & Database Management: Skilled in SQL administration, Postgres, Redshift CI/CD Implementation: Experience in automating deployment processes for efficient workflow Job Location Bangalore Notice Period: Immediate to 15 days. Interested candidates can share your profiles t o marygracy.antony@ilink-systems.com