Senior Data Engineer

Job Description

Job Title: Senior Data Engineer Location: India preferably Bengaluru Experience Level: 10+ years Employment Type: Full-Time Job Summary: Our organization is seeking a highly experienced and technically proficient Senior Data Engineer with over 10 years of experience in designing, building, and optimizing data pipelines and applications in big data environments. The ideal candidate must have strong hands-on experience in workflow orchestration, data processing, and streaming platforms, and possess full-stack development capabilities. Key Responsibilities: 1. Design, build, and maintain scalable and reliable data pipelines using Apache Airflow. 2. Develop and optimize big data workflows using Apache Spark, Hive , and Apache Flink . 3. Lead the implementation and integration of Apache Kafka for real-time and batch data processing. 4. Apply strong Java full-stack development skills to build and support data-driven applications. 5. Utilize Python to develop scripts, utilities, and support data workflows and integrations. 6. Work closely with data scientists, analysts, and platform engineers to support a high- volume,high-velocity data environment. 7. Drive performance tuning, monitoring, and troubleshooting across the data stack. 8. Ensure data integrity, security, and governance across all processing layers. 9. Mentor junior engineers and contribute to technical decision-making processes. Required Skills and Experience: Minimum 10 years of experience in data engineering or related fields. Proven experience with Apache Airflow for orchestration. Deep expertise in Apache Spark, Hive, and Apache Flink . Mandatory experience as a Full Stack Java Developer. Proficiency in Python programming for data engineering tasks. Demonstrated experience in Apache Kafka development and implementation. Prior hands-on experience in a Big Data ecosystem involving distributed systems and large-scale data processing. Strong understanding of data modeling, ETL/ELT design , and streaming architectures. Excellent problem-solving, communication, and collaboration skills. Preferred Qualifications: Experience working in cloud-based environments (e.g., AWS, Azure, GCP ). Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes). Exposure to CI/CD pipelines and DevOps practices in data projects.