Lead Data Engineer-Databricks

Experience-9+ years

Location-Hyderabad

Job type-Permanent

Role & responsibilities

Azure Data Services (Azure Data Factory, Synapse, Databricks, SQL, SSIS, and Data Lake)

Key Responsibilities:

1. Data Pipeline Development & ETL/ELT

• Design and build scalable data pipelines using Azure Data Factory, Synapse Pipelines ,Databricks, SSIS and ADF Connectors like Salesforce.
• Implement ETL/ELT workflows for structured and unstructured data processing.
• Optimize data ingestion, transformation, and storage strategies.

2. Cloud Data Architecture & Integration

• Develop data integration solutions for ingesting data from multiple sources (APIs, databases, streaming data).
• Work with Azure Data Lake, Azure Blob Storage, and Delta Lake for data storage and processing.

3. Database Management & Optimization

• Design and maintain cloud data bases (Azure Synapse, BigQuery, Cosmos DB).
• Optimize SQL queries and indexing strategies for performance.
• Implement data partitioning, compression, and caching for efficiency.

4. Collaboration & Documentation

• Document data models, pipeline architectures, and data workflows.

Immediate joiners are preferred.