Senior Data Engineer

About Delhivery:

Delhivery is India’s leading fulfillment platform for digital commerce. With a vast logistics network spanning 18,000+ pin codes and over 2,500 cities, Delhivery provides a comprehensive suite of services including express parcel transportation, freight solutions, reverse logistics, cross-border commerce, warehousing, and cutting-edge technology services. Since 2011, we’ve fulfilled over 550 million transactions and empowered 10,000+ businesses, from startups to large enterprises.

Vision

:To become the operating system for commerce in India by combining world-class infrastructure, robust logistics operations, and technology excellence

.About the Role: Senior Data Enginee

a Senior Data Engine

n.This is a hands-on role where you'll work with a wide range of big data technologies (Spark, Kafka, Hive, Hudi/Iceberg, Databricks, EMR), data modeling best practices, and real-time systems to power analytics, data products, and machine learnin

g.As a senior engineer, you'll review complex pipelines, manage SLAs, and mentor junior team members — while leveraging GenAI tools to scale your impac

t.What You’ll

• Do:Build and optimize scalable batch and streaming data pipelines using Apache Spark, Kafka, Flink, Hive, and Airfl
• ow.Design and implement efficient data lake architectures with Hudi, Iceberg, or Delta for versioning, compaction, schema evolution, and time trav
• el.Architect and maintain cloud-native data systems (AWS EMR, S3, Glue, Lambda, Athena), focusing on cost, performance, and availabili
• ty.Model complex analytical and operational data workflows for warehouse and data lake environmen
• ts.Own pipeline observability — define and monitor SLAs, alerts, and lineage across batch and real-time syste
• ms.Debug performance bottlenecks across Spark, Hive, Kafka, and S3 — optimizing jobs with broadcast joins, file formats, resource configs, and partitioning strategi
• es.Leverage AI tools (e.g., Cursor AI, Copilot, Gemini, Windsurf) f
• or:Code generation and refactoring of DAGs or Spark j
• obsDebugging logs, stack traces, and SQL err
• orsGenerating tests for data pipeli
• nesDocumenting complex pipeline dependencies and architect
• ureCollaborate with product, analytics, data science, and platform teams to deliver end-to-end data produc
• ts.Mentor junior engineers and establish AI-native development workflows, including prompt libraries and automation best practic

es.What We’re Looking

• For:Experience in building and maintaining large-scale data syst
• ems.Strong hands-on experience with Apache Spark, Kafka, Hive, and Airflow in product
• ion.Deep knowledge of Hadoop ecosystem (HDFS, YARN, MapReduce tuning, NameNode
• HA).Expert in SQL (windowing, recursive queries, tuning) and experience with NoSQL stores (e.g., DynamoDB, HBa
• se).Experience with trino/pr
• estoExperience with cloud-native data platforms — especially AWS Glue, S3 lifecycle policies, EMR, and Ath
• ena.Working knowledge of file formats and internals like Parquet, Avro, and best practices for efficient stor
• age.Familiarity with modern Lakehouse formats (Hudi, Iceberg, Delta Lake) and their compaction, versioning, and schema evolut
• ion.Hands-on experience managing Databricks or
• EMR.Solid grounding in data modeling, DWH design, and slowly changing dimensions (S
• CD).Strong programming in Python/Scala/Java, and ability to write clean, modular, testable c
• ode.Proficiency with CI/CD practices, Git, Jenkins/GitHub Actions for data engineering workfl
• ows.Bonus: Experience with distributed systems, consensus protocols, and real-time data guarant
• ees.Passion for AI-native engineering — using and evolving prompt-based workflows for greater efficiency and qual