Applications Leveraging Azure Databricks

We are seeking a skilled Application & Platform Support Engineer to provide production support for applications built on Azure Databricks and associated technologies. The role involves ensuring the stability, performance, and resilience of critical data platforms used in Capital Markets. The ideal candidate will have strong system administration, CI/CD expertise, and experience with monitoring, troubleshooting, and automation.

Key Responsibilities:

• Provide 24/7 production support for applications leveraging:
  

• Azure Databricks

• Collibra

• Kafka

• Cassandra

• Azure Kubernetes Service (AKS)

• Monitor platform health, performance, and security to proactively identify and resolve issues.
  
• Troubleshoot and resolve incidents related to system performance, application failures, and infrastructure issues.
  
• Collaborate with development, infrastructure, and QA teams to deploy and automate CI/CD pipelines.
  
• Manage system administration tasks including configuration, patching, upgrades, and capacity planning.
  
• Automate routine tasks and processes to improve efficiency and reduce manual intervention.
  
• Utilize

  Dynatrace

  for application and infrastructure monitoring, diagnostics, and root cause analysis.
• Maintain and improve documentation related to support processes, runbooks, and system configurations.
  
• Work closely with vendors and third-party support teams when required.
  
• Participate in on-call rotation for production support.
  
• Drive continuous improvements in system reliability and operational excellence.
  

Mandatory Skills:

• Strong

  system administration

  skills on cloud platforms, especially Azure.
• Extensive experience in

  CI/CD

  pipeline creation, maintenance, and troubleshooting.
• Proven expertise in

  production support

  for distributed applications.
• Solid troubleshooting and monitoring skills using tools such as

  Dynatrace

  .
• Experience with automation tools and scripting (PowerShell, Bash, Python, or similar).
  
• Knowledge of

  Azure Databricks

  ,

  Kafka

  ,

  Cassandra

  , and

  AKS (Kubernetes)

  .
• Familiarity with monitoring, alerting, and incident management processes.
  

Desired Skills:

• Exposure to other big data technologies and data governance tools like

  Collibra

  .
• Experience in

  Capital Markets

  domain or financial services.
• Knowledge of cloud security best practices.
  
• Experience with container orchestration and microservices architecture.