Senior Azure Data Engineer

Critical Skills to Possess:

• Expertise in data ingestion, data processing and analytical pipelines for big data, relational databases, and data warehouse solutions
• Hands-on experience with Agile software development
• Experience in designing and hands-on development in cloud-based analytics solutions.
• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required.
• Designing and building of data pipelines using API ingestion and Streaming ingestion methods.
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.
• Thorough understanding of Azure and AWS Cloud Infrastructure offerings.
• Expertise in Azure Databricks, Azure Stream Analytics, Power BI is desirable
• Knowledge of SAP and BW/ BPC is desirable.
• Expert in Python, Scala, SQL is desirable
• Experience developing security models.

Preferred Qualifications:

• BS degree in Computer Science or Engineering or equivalent experience

• Design, develop, and deploy data pipelines and ETL processes using Azure Data Factory.
• Implement data integration solutions, ensuring data flows efficiently and reliably between various data sources and destinations.
• Collaborate with data architects and analysts to understand data requirements and translate them into technical specifications.
• Build and maintain scalable and optimized data storage solutions using Azure Data Lake Storage, Azure SQL Data Warehouse, and other relevant Azure services.
• Develop and manage data transformation and cleansing processes to ensure data quality and accuracy.
• Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
• Optimize data pipelines for performance, cost, and scalability