Lead Data Engineer (Spark, SQL)

You will be joining Lifesight as a Data Engineer in our Bengaluru office, playing a pivotal role in the Data and Business Intelligence organization. Your primary focus will be on leading deep data engineering projects and contributing to the growth of our data platform team. This is an exciting opportunity to shape our technical strategy and foster a strong data engineering team culture in India. As a Data Engineer at Lifesight, you will be responsible for designing and constructing data platforms and services, managing data infrastructure in cloud environments, and enabling strategic business decisions across Lifesight products. Your role will involve building highly scalable, fault-tolerant distributed data processing systems, optimizing data quality in pipelines, and owning data mapping, transformations, and business logic. You will also engage in low-level system debugging, performance optimization, and actively participate in architecture discussions to drive new projects forward. The ideal candidate for this position will possess proficiency in Python and PySpark, along with a deep understanding of Apache Spark, Spark tuning, and building data frames. Experience with big data technologies such as HDFS, YARN, Map-Reduce, Hive, Kafka, and Airflow, as well as NoSQL databases and cloud platforms like AWS and GCP, are essential. You should have at least 5 years of professional experience in data or software engineering, demonstrating expertise in data quality, data engineering, and various big data frameworks and tools. In summary, as a Data Engineer at Lifesight, you will have the opportunity to work on cutting-edge data projects, collaborate with a talented team of engineers, and contribute to the ongoing success and innovation of Lifesight's data platform.,