Senior Data Engineer

You are a Senior Data Engineer with 7 to 11 years of experience who is proficient in building and managing scalable, secure, and high-performance data pipelines and storage solutions. Your primary focus will be on Microsoft Azure data services. In Requirement 1 (Azure Focus), your key responsibilities will include designing and implementing robust data pipelines using Azure Data Factory, Azure Data Lake, and other Azure services. You will also develop and optimize large-scale data processing using Databricks and PySpark. Additionally, you will work with both relational (SQL) and NoSQL databases, write clean code in Python and/or Scala, and collaborate with cross-functional teams. To excel in this role, you must have 7-11 years of professional experience as a Data Engineer, deep expertise with Azure Data Services, solid experience with Databricks and PySpark, proficiency in SQL and NoSQL databases, and a strong background in Python, Scala, and object-oriented programming. In Requirement 2 (GCP & Databricks Specialist), you will be expected to build, maintain, and optimize scalable data pipelines using Databricks and Google Cloud (BigQuery, Cloud Storage, etc.). You will collaborate with analytics and data science teams, perform exploratory data analysis (EDA), and ensure best practices in data modeling, governance, and lifecycle management. For this role, you should have 7-11 years of professional experience in data engineering, expert-level proficiency in Databricks, BigQuery, and Python, strong SQL skills with large datasets, exposure to data science concepts and EDA methodologies, and familiarity with CI/CD processes and version control systems.,