Senior Databricks Engineer - Python/SQL/Scala

Key Responsibilities:

• Design and develop robust data processing pipelines and analytics solutions using Databricks.
• Architect scalable and efficient data models and storage solutions tailored for the Databricks environment.
• Collaborate with architects and cross-functional teams to migrate existing data solutions onto Databricks.
• Optimize performance and reliability of Databricks clusters and jobs to consistently meet SLAs and business requirements.
• Implement and uphold best practices for data governance, security, and compliance within the Databricks platform.
• Mentor junior engineers and provide technical guidance to foster team growth.
• Stay up-to-date with emerging technologies and industry trends in data engineering and analytics to continuously improve our solutions.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.
• 3 to 6 years of overall experience in data engineering or related roles.
• Minimum 2 years of hands-on experience designing and implementing data solutions on the Databricks platform.
• Proficiency in Python, Scala, and/or SQL for data processing and analytics.
• Strong understanding of distributed computing principles and extensive experience with Apache Spark.
• Experience with major cloud platforms such as AWS, Azure, or GCP, including their data service offerings.
• Proven ability to deliver scalable and reliable data solutions in fast-paced, dynamic environments.
• Excellent analytical and problem-solving skills with strong attention to detail.
• Effective communication and collaboration skills to work efficiently in cross-functional teams.

Preferred Skills:

• Experience with containerization technologies such as Docker and Kubernetes.
• Familiarity with DevOps practices for automated deployment, monitoring, and maintenance of data pipelines.