Job
Description
Project Role :Data Platform Engineer
Project Role Description :Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
Must have skills :Databricks Unified Data Analytics Platform
Good to have skills :NAMinimum
15 year(s) of experience is required
Educational Qualification :15 years full time education
Summary:Highly experienced Senior Databricks Expert to provide critical technical guidance and support for the productionization of key data products. This role requires a deep understanding of the Databricks Lakehouse Platform and a pragmatic approach to implementing robust, scalable, and performant solutions.
Roles & Responsibilities:- Expected to be a Subject Matter Expert with deep knowledge and experience.- Should have influencing and advisory skills.- Responsible for team decisions.- Engage with multiple teams and contribute on key decisions.- Expected to provide solutions to problems that apply across multiple teams.- Facilitate workshops and discussions to gather requirements and feedback from stakeholders.- Mentor junior professionals in best practices and emerging technologies. Professional & Technical
Skills:
This role requires a deep understanding of the Databricks Lakehouse Platform and a pragmatic approach to implementing robust, scalable, and performant solutions.Key Technical Skills & Experience:Must Have:Databricks Platform Mastery:Deep, hands-on expertise across the Databricks Lakehouse Platform, including Delta Lake, Spark SQL, Databricks compute, and an understanding of Unity Catalog principles. Advanced Spark Development & Optimization:Proven ability to write, review, and significantly optimize complex Apache Spark (Python/Scala) applications for performance, stability, and efficiency in production. Production Data Engineering & Architecture:Strong experience in designing, validating, and troubleshooting production-grade data pipelines and architectures on Databricks, adhering to data modeling and software engineering best practices. CI/CD & DevOps for Databricks:Practical experience implementing and advising on CI/CD practices for Databricks projects (e.g., using Databricks CLI, Repos, dbx, Azure DevOps, GitHub Actions) for automated testing and deployment. Databricks Security & Governance:Solid understanding of Databricks security features, including access control models, secrets management, and network configurations, with the ability to advise on their practical application. Operational Excellence on Databricks:Experience with monitoring, logging, alerting, and performance tuning strategies for Databricks jobs and clusters to ensure operational reliability and efficiency. Machine Learning on Databricks:Experience with ML on Databricks:MLflow, Model Serving, and best practices for securely exposing model endpoints. Problem-Solving & Mentorship:Excellent analytical and troubleshooting skills, with a proven ability to diagnose complex issues and effectively communicate solutions and best practices to technical teams in a supportive, advisory capacity.Good to have:Familiarity with Google Cloud Platform
Additional Information:- The candidate should have minimum 15 years of experience in Databricks Unified Data Analytics Platform.- This position is based at our Hyderabad office.- A 15 years full time education is required.Qualification
15 years full time education