Lead Azure Data Engineer

6 years

0 Lacs

Posted:1 week ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Job Title: Lead Azure Data Engineer

Experience Level: Mid - Senior Level

Location: Delhi

Duration: Fulltime

Experience Required: 6-8+ Years


Description: We are seeking a highly skilled and experienced Lead Azure Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with a focus on working with Databricks, PySpark, Scala-Spark, and advanced SQL. This role requires hands-on experience in implementing or migrating projects to Unity Catalog, optimizing performance on Databricks Spark, and orchestrating workflows using various tools.


Must Have Skills:

  • MS Fabric
  • ADF (Azure Data Factory)
  • Azure Synapse


Key Responsibilities:

  • Data engineering and analytics project delivery experience Minimum 6 years
  • Min. 2 project done in past of Databricks Migration (Ex. Hadoop to Databricks, Teradata to Databricks, Oracle to Databricks, Talend to Databricks etc)
  • Hands on with Advanced SQL and Pyspark and/or Scala Spark
  • Min 3 project done in past on Databricks where performance optimization activity was done
  • Design, develop, and optimize data pipelines and ETL processes using Databricks and Apache Spark.
  • Implement and optimize performance on Databricks Spark, ensuring efficient data processing and management.
  • Develop and validate data formulation and data delivery for Big Data projects.
  • Collaborate with cross-functional teams to define, design, and implement data solutions that meet business requirements.
  • Conduct performance tuning and optimization of complex queries and data models.
  • Manage and orchestrate data workflows using tools such as Databricks Workflow, Azure Data Factory (ADF), Apache Airflow, and/or AWS Glue.
  • Maintain and ensure data security, quality, and governance throughout the data lifecycle.


Technical Skills:

• Extensive experience with PySpark and Scala-Spark.

• Advanced SQL skills for complex data manipulation and querying.

• Proven experience in performance optimization on Databricks Spark across at least three projects.

• Hands-on experience with data formulation and data delivery validation in Big Data projects.

• Experience in data orchestration using at least two of the following: Databricks Workflow, Azure Data Factory (ADF), Apache Airflow, AWS Glue.

• Experience in Azure Synapse

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Celebal Technologies logo
Celebal Technologies

Technology Consulting and Services

Ahmedabad

RecommendedJobs for You

Hyderabad / Secunderabad, Telangana, Telangana, India