Data Engg - Sr Technical Lead

About Birlasoft

Key Responsibilities

• Experience building maintainable, testable, and scalable data pipelines for big data using PySpark according to best practices
• Deep understanding of Spark's execution engine for processing billion of records
• Strong skills in performance optimization as well as experience configuring Spark clusters for efficient resource usage at scale.
• Advanced Python skills
• Basic knowledge in SQL
• Basic understanding of Git and Git workflow
• Collaborate with business teams to gather requirements and translate them into technical solutions.
• Build and optimize data models and workflows for performance and scalability.
• Ensure data quality, integrity, and governance across all solutions.
• Create clear and concise documentation and deliver presentations to stakeholders.
• Work independently and take ownership of projects from inception to deployment.
  

Required Skills

• Strong experience with PySpark and Python.
• Solid understanding of data engineering principles and ETL processes.
• Experience in working with large-scale data systems.
• Excellent communication, documentation, and presentation skills.
• Ability to work independently and manage multiple priorities.
  

Nice To Have

• Familiarity with Palantir Foundry or similar platforms.
• Experience with cloud platforms (AWS, Azure, GCP).
• Knowledge of data warehousing and SQL.