Azure Data Engineer

Role - Azure Data Engineer

Required Technical Skill Set

Azure Data Factory + Azure Databricks.

Desired Experience Range - 8 TO 12 years

Location of Requirement

Mumbai, Bangalore, Hyderabad, Chennai, Gurgaon, Pune.

Desired Competencies (Technical/Behavioral Competency)

Must-Have

Azure Data Factory, Azure Databricks

Good-to-Have

Python/Pyspark

Responsibility of / Expectations from the Role

1 Dev Implementing highly performant, scalable and re-usable data ingestion and transformation pipelines across Azure components and services

2 Dev Strong experience using Logic Apps, Functions & Data Factory

3 Dev Strong experience of development using Microsoft Azure as well as experience of developing integrations for large, complex pieces of software.

4 Dev Strong understanding of the technical side of CI / CD

5 Dev Strong understanding of Agile

6 Des Designing and implementing a data warehouse on Azure using Azure HDInsight, Azure Data Factory, ADLS, Databricks, SQL Server, SQL DWH, Analytics Service, Event Hubs, KeyVault and other Azure services

7 Des Designing, orchestrating and implementing highly performant, scalable and re-usable data ingestion and transformation pipelines across Azure components and services

8 Des Designing and implementing Event based and streaming data ingestion and processing using Azure PaaS services

9 Des Data ingestion, data engineering and/or data curation using native Azure services or tools available in Azure Marketplace

10 Des Designing and implementing data governance, data cataloguing and data lineage solutions using tools such as Azure Data Catalog/Informatica Data Catalog

11 Des Developing physical data models in MongoDB, SQL DWH , ...etc.

12 Des Developing APIs

13 Des Have worked in agile delivery teams with DevOps ways of working to continuously deliver iterative deployments with experience in using Jira, Git Repositories, Confluence, or similar

14 Des Have had experience in data migration activities in the past including migration strategy and approach, source and target system discovery, analysis, mapping, development and reconciliation