Azure Data Engineer

job description

Technical Experience

experience in Data Platform Engineering

• Extensive coding experience (3+ years) including Python/Pyspark (1 years +)
• Experience developing in Azure with key data technologies (e.g. ADLS, ADF, Azure Databricks etc.)
• Software development methodologies
• Experience working with at least one DevOps tool (GIT, Azure DevOps, Maven, Jenkins)
• Prior roles that demonstrate utilisation of high-quality Agile development best practices.
• In depth knowledge of at least one scheduling tool (e.g. Control-M, Oozie, ADF, etc.)

Data lake, JIRA experience. Experience working on Agile projects

Responsibilities

• Design, build, implement and maintain data pipeline components for the ingestion, validation, and structuring of disparate data sources into Azure data environments
• Produce operational ready code that is easily maintainable
• Produce standardised coding designs using industry best practices
• Document generic solutions to the point they can be re-used by other members of the team to perform builds
• Interact with the business, architecture or other members of the team to define and refine requirements

Peer review and test components