Azure Data Engineer

Role - Azure Data Engineer

Location - Bangalore

Experience - 6-12 Years

Senior Azure Data Engineer

Key Responsibilities

• Data Engineering & Architecture

• Design and implement scalable data pipelines using

  Databricks Delta Live Tables (DLT)

  ,

  Structured Streaming

  , and

  PySpark

  .
• Develop and optimize

  real-time and batch data processing

  solutions for high-volume datasets.
• Manage and enhance

  Unity Catalog

  for secure and efficient data governance.
• Build and maintain data integration workflows leveraging

  Azure Data Factory (ADF)

  ,

  Azure Data Lake Storage (ADLS)

  , and

  Azure Synapse Analytics

  .
• Implement

  Stream Analytics

  for real-time event processing and analytics.

• Performance Optimization

• Fine-tune

  Databricks clusters

  , caching strategies, and partitioning for optimal performance.
• Ensure cost-effective and high-performing data solutions aligned with business SLAs.

• Data Quality & Security

• Enforce data quality standards, lineage, and governance using

  Unity Catalog

  and Azure-native tools.
• Implement robust security measures, including RBAC, encryption, and compliance frameworks.

• Leadership & Collaboration

• Lead cross-functional teams to deliver strategic data initiatives.
• Act as a trusted advisor to stakeholders, providing technical guidance and actionable insights.
• Communicate effectively with executives and business leaders to align data solutions with organizational goals.

Required Skills & Qualifications

• Technical Expertise

• Deep experience with

  Databricks DLT

  ,

  Structured Streaming

  ,

  PySpark

  ,

  Python

  , and

  SQL

  .
• Strong knowledge of

  Azure Data Services

  : ADF, ADLS, Synapse, Stream Analytics.
• Proven ability to optimize

  Databricks

  and

  Unity Catalog

  for high-volume, real-time data processing.

• Leadership & Communication

• Exceptional stakeholder management and executive communication skills.
• Demonstrated ownership, accountability, and passion for delivering impactful solutions.
• Ability to lead and mentor teams, driving innovation and best practices.

• Preferred

• Experience with

  CI/CD for data pipelines

  using Azure DevOps.
• Familiarity with

  data governance tools

  like Azure Purview.
• Relevant certifications:

  DP-203 (Azure Data Engineer Associate)

  ,

  AZ-104

  ,

  AZ-305

  .