3167 Azure Databricks Jobs

We looking for a candidate who has hands on experience on Azure Cloud and Microsoft Fabrics. Also candidates should have hands on experience with Python, SQL and Pyspark. Knowledge on Azure Data Factory and Azure Data Bricks is a plus. Required Candidate profile Azure Data Engineer

We are seeking a skilled Application & Platform Support Engineer to provide production support for applications built on Azure Databricks and associated technologies. The role involves ensuring the stability, performance, and resilience of critical data platforms used in Capital Markets. The ideal candidate will have strong system administration, CI/CD expertise, and experience with monitoring, troubleshooting, and automation. Key Responsibilities: Provide 24/7 production support for applications leveraging: Azure Databricks Collibra Kafka Cassandra Azure Kubernetes Service (AKS) Monitor platform health, performance, and security to proactively identify and resolve issues. Troubleshoot and r...

Please find below for reference. Key Qualifications: 10+ years of ETL/data engineering experience, with at least 3+ years on Azure Strong experience with Azure Data Factory, Synapse, ADLS Gen2, and Databricks (PySpark) Solid SQL skills and deep understanding of data modelling (dimensional, star/snowflake) Experience with CI/CD tools (e.g., Azure DevOps), source control, and infrastructure as code for data workflows Familiarity with Azure security features (Key Vault, Managed Identity, RBAC) Proven ability to troubleshoot and optimize large-scale data pipelines Strong documentation and communication skills Preferred (Nice to Have): Experience with real-time data ingestion (Kafka/Event Hubs) A...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for: - Experience in Azure Fabric, Azure Data factory, Azure Databricks, Azure Synapse, Azure SQL, ETL - Creating Pipelines, datasets, dataflows, Integration runtimes, and monitoring Pipelines, and trigger runs - Extracting Transformation and Load Data from the source system and processing the data in Azure Databricks - Creating SQL scripts to perform complex queries - Creating Synapse pipelines to migrate data from Gen2 to Azure SQL - Implementing Data Migration pipeline to Azure cloud (Azure SQL) - Handling database migration from on-prem SQL server to Azure Dev Environment using Azure DMS and Data Migration A...

Role Overview: As an IT professional at Capgemini Invent, you will be responsible for creating data warehouses, data lakes, ETL/ELT, and data pipelines on cloud platforms. Your focus will be on delivering cutting-edge solutions tailored to the challenges faced by clients in the Life Sciences Domain. You will leverage your expertise in cloud storage, cloud database, and data warehousing solutions such as Snowflake, BigQuery, AWS Redshift, ADLS, and S3. Additionally, you will use cloud data integration services like Azure Databricks, Azure Data Factory, Azure Synapse Analytics, AWS Glue, AWS EMR, Dataflow, and Dataproc for handling structured, semi-structured, and unstructured data. Your role ...

About Position: We are seeking a highly analytical and detail-oriented Cloud Data Engineer. Role: Cloud Data Engineer Location: All Persistent Locations Experience: 5+ Years Job Type: Full Time Employment What You'll Do: Minimum of 5+ years of experience as a Cloud Data Engineer. Hands-on experience with Azure Cloud data tools (ADF, SHIR, Logic Apps, ADLS Gen2, Blob Storage) and Snowflake. Strong experience in ETL development using on-premises databases ETL technologiesExperience with Python or other scripting languages for data processing.Proficiency in DevOps and CI/CD practices using GitActions. Experience with Agile methodologies. Excellent problem-solving skills and ability to work inde...

Organization Background: At Providence, we are grounded in our goal to serve all as we engineer the future of healthcare. Providence Global Innovation Center, launched in Feb 2020, is the first development and innovation center of Providence outside the United States. Providence is a $26B non-profit organization and is one of the largest health systems in the United States. The ESI Enablement & Platform Technology - Power BI Platform Team provides analytical services to business units and IS Stakeholders. In this role you will engineer, develop, and support the complete lifecycle of the Enterprise PowerBI/ Fabric Services Platform, PowerBI Reports for ESI EPT Service Lines & Strategic Projec...

About Position: We are seeking a highly skilled and experienced Data Engineer with expert-level proficiency in Azure Databricks, Apache Spark, and Azure Data Services. The ideal candidate will have a strong background in building production-grade Delta Live Tables (DLT) pipelines, implementing Medallion Architecture, and managing infrastructure using Terraform. This role demands hands-on experience in real-time data ingestion, structured streaming, and CI/CD practices. Familiarity with the Healthcare domain and Snowflake is a plus. Role: Senior Data Engineer Location: All Persistent Locations Experience: 6+ Years Job Type: Full Time Employment What You'll Do: Design, develop, and maintain sc...

About Position: As a Data Engineer, you will design and manage robust data pipelines using Azure Databricks, Spark, and Kafka. You'll process large-scale datasets and implement CDC pipelines to power analytics platforms. Your role includes modeling data using the Medallion Architecture and ensuring data quality and governance. Collaboration with analytics teams for visualization and insights is key to success. Role: Data Engineer (Azure Databricks) Location: Navi Mumbai Experience: 5 to 11 Years Job Type: Full Time Employment What You'll Do: Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory and Azure Databricks. Implement Change Data Capture (CDC) pip...

Role & responsibilities Understand Requirement, Analyze, Design, Develop, Test and Deploy End to End Azure based Data and Analytics solutions Develop, Test, Debug, Monitor data solutions, like data pipelines, data warehousing, and big data solutions, with Azure Data Fabric and Azure Synapse Analytics Follow Best Practices, Guideline, Review, Tune and optimize Microsoft Azure Synapse Analytics experience is essential. (Dedicated SQL Pool, Azure Data Factory, Azure Storage) End-to-end Data Warehouse experience: ingestion, ETL, big data pipelines, data architecture, message queuing, stream processing, BI/Reporting, and Data Security. Ability to build processes supporting data transformation, da...

What you'll do Develop, maintain, and enhance new data sources and tables, contributing to data engineering efforts to ensure comprehensive and efficient data architecture. Serves as the liaison between Data Engineer team and the Airport operation teams, developing new data sources and overseeing enhancements to existing database; being one of the main contact points for data requests, metadata, and statistical analysis Migrates all existing Hive Metastore tables to Unity Catalog, addressing access issues and ensuring smooth transition of jobs and tables. Collaborate with IT teams to validate package (gold level data) table outputs during the production deployment of developed notebooks Deve...

Educational Requirements Bachelor of Engineering Service Line Application Development and Maintenance Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and facilitate deployment resulting in client delight. You will develop a proposal by owning parts of the proposal document and by giving inputs in solution design based on areas of expertise. You will plan the activities of configuration, configure the product as per the design, conduct conference room pilots and will assist in resolving any queries related to requirements and solutio...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to lead the engagement effort of providing high-quality and value-adding consulting solutions to customers at different stages- from problem definition to diagnosis to solution design, development and deployment. You will review the proposals prepared by consultants, provide guidance, and analyze the solutions defined for the client business problems to identify any potential risks and issues. You will identify change Management requirements and propose a structured approach to client for mana...

Role & responsibilities 8 to 10 years of experience is Data Modeling and Design Design and maintain conceptual, logical, and physical data models using tools Exp in Azure Cloud is mandatory Develop dimensional models for analytics and 3NF models for transactional systems. Create and manage source-to-target mappings for ETL/ELT processes. Write and optimize SQL scripts for transformation, and analysis. Collaborate with cross-functional teams to gather requirements and deliver data solutions. Support data integration from various sources including APIs, files, and databases .Contribute to data governance efforts including metadata management and data quality Ensure documentation of data archit...

Metyis is a Netherlands based fast-growing international company with a diverse team of over 700+ professionals with a mix of Analytics, Data Science and Technology Team. We are looking for an experienced Data Engineer to join our team with below requirements: Responsibilities: Independently lead & manage execution of data engineering projects Engineer complete technical solutions to solve concrete business challenges in the areas of digital marketing, eCommerce, Business Intelligence and self-service analytics Collect functional and non-functional requirements, consider technical environments, business constraints and enterprise organizations Support our clients in executing their Big Data ...

Metyis is a Netherlands based fast-growing international company with a diverse team of over 700+ professionals with a mix of Analytics, Data Science and Technology Team. We are looking for an experienced Data Engineer to join our team with below requirements: Responsibilities: Independently execute data engineering projects Diagnose problems, disaggregate issues, develop hypotheses and execute relevant solutions Undertake processing of structured and unstructured data Undertake work related to ELT activities Development of data processing codes for automation & building scalable solutions Develop and maintain data solutions for clients / projects Communicate and interact with clients at the...

Role & responsibilities • Experience in ADF • Experience in SQL Server • Ready to work in Shift (We have 24 X 7 Shift) • Maintaining and monitoring of Data Pipelines • Ability to understand the issues with pipelines and perform root cause analysis • Monitor data quality pillars like uniqueness, completeness, correctness, timeliness, consistency, availability • Good documenting skills for capturing issue logs, sequence of events, root cause analysis and short term, long term fix details • Updating wiki page for each pipeline enhancement or change. • Assisting partner teams on information related to data issues so as to arrive at issue resolution • Send regular updates on status of open issues...

Role & responsibilities Mandatory Skills: Microsoft Fabric, Azure datafactory, Azure Data Lake Storage, Azure data bricks with Py-Spark Good to have: Power BI, Azure IAAS services. Very strong understanding on ETL and ELT Very strong understanding on Lakehouse architecture. Very strong knowledge in Pyspark and Spark architecture. Good knowledge in Azure data lake architecture and access controls Good knowledge in Microsoft Fabric architecture Good knowledge in T-SQL Good knowledge in CI /CD process using Azure devops Power BI Preferred candidate profile

Job description Job Summary: We are seeking a talented Data Engineer with strong expertise inAzure , Databricks, specifically in Unity Catalog, PySpark, and SQL, to join our data team. Youll play a key role in building secure, scalable data pipelines and implementing robust data governance strategies using Unity Catalog. Key Responsibilities: Design and implement ETL/ELT pipelines using Databricks and PySpark. Work with Unity Catalog to manage data governance, access controls, lineage, and auditing across data assets. Develop high-performance SQL queries and optimize Spark jobs. Collaborate with data scientists, analysts, and business stakeholders to understand data needs. Ensure data qualit...

As a Data Engineer, you will be responsible for the following key responsibilities: - Proficient in Python, including popular Python packages such as Pandas and NumPy, as well as SQL. - Strong background in distributed data processing and storage, such as Apache, Spark, and Hadoop. - Ability to work with large-scale data (TBs of data) and skills in data engineering to model data and create production-ready ETL pipelines. - Development experience with at least one cloud platform, with a high preference for Azure, and familiarity with AWS and GCP. - Knowledge of data lake and data lake house patterns, ETL performance tuning, and cost optimization. - Understanding of data structures, algorithms...

About Position: We are seeking Data Engineer with hands on experience in Python, Pyspark, Azure Databricks, ADF, etc., Role: Data Engineer Location: All Persistent Location Experience: 4 to 6 years Job Type: Full Time Employment What You'll Do: Technical Design and implement client requirements. Build data pipeline. Perform transformations using pyspark in data bricks. Create automated workflows with the help of triggers, Scheduled Jobs in Airflow. Design and development of Airflow dags to orchestrate data processing jobs. Design and development for ensuring timely notifications through email alerts in the event of job failures or critical system issues. Developing and implementing code to w...

We are looking for an experienced Data Engineer with strong hands-on expertise in PySpark , Azure Databricks , and Fabric development. The ideal candidate should have solid development experience (not support) in building and optimizing scalable data solutions within a fast-paced financial domain environment. Roles and Responsibilities Design, develop, and implement scalable data pipelines using PySpark , Azure Databricks , and Fabric . Work extensively on SQL for data transformations, optimization, and performance tuning. Collaborate with cross-functional teams to translate business requirements into efficient data engineering solutions. Ensure data quality, integrity, and compliance with o...

Technology Stack Data engineer with experience in working with Big data technologies e.g. spark/hadoop Design and implement scalable cloud infrastructure solutions using Google Cloud Platform services such Cloud Storage, BigQuery , Cloud Data flow, Cloud run etc Strong scripting and programming skills on python

Job Title Databricks Engineer- CHRTN Job Description Design, build, and maintain scalable and efficient ETL/ELT pipelines in Azure Databricks to process structured, semi-structured, and unstructured insurance data from multiple internal and external sources. • Collaborate with data architects, modellers, analysts, and business stakeholders to gather data requirements and deliver fit-for-purpose data assets that support analytics, regulatory, and operational needs. • Develop, test, and optimize data transformation routines, batch and streaming solutions (leveraging tools such as Azure Data Factory, Data Lake Storage Gen2, Azure Event Hubs, and Kafka) to ensure timely and accurate data deliver...

Role & responsibilities - Design and implement robust data pipelines using Azure Data Factory, Azure Databricks, and Azure SQL Database. - Work with large datasets to ingest, process, and analyze data from various sources. - Utilize Azure services for data storage and processing, ensuring optimal performance and scalability. - Collaborate with data scientists and business analysts to understand data needs and deliver solutions that meet requirements. - Ensure data quality, security, and compliance by implementing best practices in data governance. - Troubleshoot and resolve data-related issues, providing support for end-users and stakeholders. - Stay up-to-date with the latest Azure technolo...