Senior Databricks Architect

Title: Senior Databricks Architect

Location : Hybrid ( Chennai, TN)

Key Responsibilities:

The role requires a strong focus on engaging with clients and showcasing thought leadership. The ideal candidate will be skilled in developing insightful content and presenting it effectively to diverse audiences

Cloud Architecture & Design:

• Understand customers’ overall data platform, business and IT priorities and success measures to design data solutions that drive business value.

• Design scalable, high-performance, end-to-end data architectures, solutions and pipelines using Databricks Lakehouse architecture and related technologies that includes data ingestion, processing, storage and analytical capabilities.

• Work with major cloud providers to integrate Databricks solutions seamlessly into customer’s enterprise environments.

• Assess and validate non-functional attributes and build solutions that exhibit high levels of performance, security, scalability, maintainability, and reliability.

• Optimize Databricks clusters and queries for efficiency, reliability, and cost-effectiveness

Technical Leadership:

• Guide technical teams in best practices on cloud adoption, migration and application

modernization and provide thought leadership and insights into existing and emerging

Databricks capabilities.

• Ensure long term technical viability and optimization of cloud deployments by identifying and resolving bottlenecks proactively thereby ensuring high availability and efficient resource utilization.

Stakeholder Collaboration:

• Work closely with business leaders, developers, and operations teams to ensure alignment of technological solutions with business goals.

• Work with prospective and existing customers to implement POCs/MVPs and guide through to deployment, operationalization, and troubleshooting.

• Identify, communicate, and mitigate the assumptions, issues, and risks that occur throughout the project lifecycle.

• Ability to judge and strike a balance between what is strategically logical and what can be

accomplished realistically.

Innovation and Continuous Improvement:

• Stay updated with the latest advancements in Databricks and big data technologies and drive their adoption.

• Implement innovative solutions to improve data processing, storage, and analytics efficiency.

• Identify opportunities to enhance existing data engineering processes using AI and machine learning tools.

Documentation:

• Create comprehensive blueprints, architectural diagrams, technical collaterals, assets and implementation plans for Databricks solutions.

Required Qualifications:

Education:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.

Experience:

• Minimum of 8 years in architecture roles, including at least 3-5 years working with Databricks.

Technical Expertise:

• Strong proficiency in Spark, Python, Scala, SQL, and cloud platforms (Azure, AWS, GCP).

• Strong proficiency in Databricks workflows, Lakehouse architecture, Delta Tables, and MLflow.

• Proficiency in architectural best practices in cloud around user management, data privacy, data security, performance and other non-functional requirements.

• Familiarity in building AI/ML models on cloud solutions using Databricks.

Soft Skills:

• Strong analytical, problem-solving and troubleshooting skills.

• Excellent communication and ability to mentor and inspire teams.

• Strong leadership abilities with experience managing cross-functional teams.

Preferred Skills:

• Databricks certification as a professional or architect.

• Experience with BFSI or Healthcare or Retail domain.

• Experience with hybrid cloud environments and multi-cloud strategies.

• Experience with data governance principles, data privacy and security.

• Experience with data visualization tools like Power BI or Tableau.