Azure Data Engineer

Job Title: Azure Data Engineer

Location: Bengaluru - Onsite

We are looking for a skilled Operations Engineer to support our Azure-based Data Engineering platform powered by Databricks. This role blends DevOps practices, system reliability, and security auditing to ensure our data infrastructure is robust, secure, and scalable. You&aposll be instrumental in automating workflows, monitoring system health, and enforcing security best practices across our data ecosystem.

Key Responsibilities:

• Maintain and monitor Databricks-based data pipelines and Azure data services (e.g., Data Lake, Synapse, Key Vault).
• Implement CI/CD pipelines using Azure DevOps for data engineering workflows.
• Conduct regular system health checks and performance tuning of clusters and jobs.
• Automate infrastructure provisioning using tools like Terraform or ARM templates.
• Use monitoring tools (e.g., Azure Monitor, Log Analytics, Prometheus) to track system metrics and alert on anomalies.
• Perform security audits using tools like Databricks Security Analysis Tool (SAT) to ensure compliance with best practices.
• Manage secrets and credentials securely using Azure Key Vault and Databricks secret scopes.
• Collaborate with security teams to enforce network security, access control, and data encryption policies.
• Maintain operational runbooks and incident response

Qualifications:

• Bachelor&aposs degree in computer science, Engineering, or related field.
• 5+ years of experience in DevOps, cloud operations, or data platform support.
• Proficiency in Azure services, Databricks, and scripting languages (Python, Bash).
• Experience with CI/CD tools (Azure DevOps, GitHub Actions) and infrastructure-as-code.
• Familiarity with security frameworks and auditing tools in cloud environments.
• Strong troubleshooting skills and a proactive approach to system reliability.