Azure Databricks Developer

Core Responsibilities

8 + years

• Developing ETL pipelines using Python, SQL, and Delta Live Tables
• Managing job execution through Databricks Jobs, including scheduling and parameterisation
• Implementing data transformation using built-in Databricks features Collaborating with data architects and analysts to ensure data models align with business needs
• Implementing security and governance using features like encryption, access control and data lineage tracking Data performance management using built-in Databricks features (i.e. concurrency for better query execution) Must be an expert in writing complex SQL queries.

Technical Skills

The following technical proficiencies are commonly required:

• Languages:

  Python, SQL Databricks

• Features:

  Delta Lake, Delta Live Tables, MLflow, Databricks Jobs, Secrets Management, encryption access control etc.

• Azure Integration:

  Azure Key Vault, Azure Functions, Azure Data Lake, Azure DevOps
• Deployment Pipelines: Experience with CI/CD for Databricks and integration with Snowflake schemas (DEV, PRE-PROD, PROD)

• Monitoring & Optimisation:

  Cost analysis of clusters, performance tuning, and job concurrency management Desirable Experience Working knowledge of Control-M for orchestrating Databricks jobs
• Familiarity with Power BI integration and semantic modelling Exposure to geospatial or time-series data processing using Databricks Basic knowledge of data modelling

Aarushi.Shukla@coforge.com

• Good to have exposure to Agile methodology of software development
• Soft Skills & Qualifications Strong problem-solving and communication and interpersonal skills.
• Ability to work collaboratively across teams (e.g. HR, Architecture, Data Engineering).
• A degree in Computer Science, Data Engineering, or a related field is typically preferred