Lead Data Engineer

Experience:

What you’ll do

• Own and evolve our end-to-end data platform, ensuring robust pipelines, data lakes, and warehouses with 100% uptime.
• Build and maintain real-time and batch pipelines using Debezium, Kafka, Spark, Apache Iceberg, Trino, and Clickhouse.
• Manage and optimize our databases (PostgreSQL, DocumentDB, MySQL RDS) for performance and reliability.
• Drive data quality management — understand, enrich, and maintain context for trustworthy insights.
• Develop and maintain reporting services for data exports, file deliveries, and embedded dashboards via Apache Superset.
• Use orchestration tools like Maestro (or similar DAGs) for reliable, observable workflows.
• Leverage LLMs and other AI models generate insights and automate agentic tasks that enhance analytics and reporting.
• Build domain expertise to solve complex data problems and deliver actionable business value.
• Collaborate with analysts, data scientists, and engineers to maximize the impact of our data assets.
• Write robust, production-grade Python code for pipelines, automation, and tooling.

What you’ll bring

• Experience with our open-source data pipeline and datalike, warehouse stack
• Strong Python skills for data workflows and automation.
• Hands-on orchestration experience with Maestro, Airflow, or similar.
• Practical experience using LLMs or other AI models for data tasks.
• Solid grasp of data quality, enrichment, and business context.
• Experience with dashboards and BI using Apache Superset (or similar tools).
• Strong communication and problem-solving skills.