Data Engineer

We are seeking a highly experienced and passionate Data Engineer to join our growing data team
As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our robust data infrastructure, ensuring data quality, availability, and scalability You will play a critical role in empowering our data-driven decision-making by building and optimizing data pipelines, ETL processes, and data warehousing solutions This is a senior-level position requiring extensive experience with database technologies, data modelling, and cloud platforms

Responsibilities:

Data Pipeline Development & Maintenance: Design, build, and maintain end-to-end data pipelines for ingesting, processing, and transforming large datasets from various sources (e
g, relational databases, APIs, flat files, streaming data)

ETL/ELT Process Optimization: Develop and optimize ETL/ELT processes using industry-standard tools and techniques, ensuring data accuracy, efficiency, and scalability

Database Design & Management: Design, implement, and manage relational databases, including schema design, indexing, performance tuning, and data governance

Data Warehousing: Design and implement data warehousing solutions to support business intelligence and reporting needs
Experience with star schema, snowflake schema, and other data modelling techniques is essential

Database Administration: Perform database administration tasks, including performance monitoring, capacity planning, backup and recovery, and security management

Data Quality & Governance: Implement data quality checks, validation rules, and data governance policies to ensure data accuracy and consistency

Cloud Platform Expertise: Leverage cloud platforms (e
g, AWS, Azure, GCP) for data storage, processing, and management

Collaboration & Communication: Collaborate with cross-functional teams (e
g, data scientists, business analysts, software engineers) to understand data requirements and deliver effective data solutions Clearly communicate technical concepts to both technical and non-technical audiences

Mentoring & Knowledge Sharing: Mentor junior engineers and share knowledge and best practices within the team

Automation & Scripting: Automate data engineering tasks using scripting languages (e
g, Python, Bash)

Stay Up to Date: Continuously research and evaluate new data technologies and techniques to improve our data infrastructure

Production Support: Monitoring the batch/jobs daily, no matter itweekdays or weekend

Production Release: Actively participation in release process

Qualifications:

Experience: Minimum of 6 years of experience as a Data Engineer or in a similar role

Database Expertise:

o Expert proficiency in SQL Server: Extensive experience with SQL Server, including database design, performance tuning, query optimization, and database administration

o Expert proficiency in PostgreSQL: Deep understanding of PostgreSQL, including database design, performance tuning, query optimization, and database administration

o Solid experience with other relational databases like MySQL, Oracle, etc is a plus

ETL/ELT Tools: Proven experience with ETL/ELT tools (e
g, Apache Airflow, Informatica, Talend, SSIS, ADF etc)

Data Modelling: Strong understanding of data modelling principles and techniques (e
g, dimensional modelling, star schema, snowflake schema)

Cloud Computing: Experience with cloud platforms (AWS, Azure, or GCP) and related data services (e
g, S3, Redshift, Snowflake, Azure Data Lake Storage)

Programming & Scripting: Proficiency in scripting languages such as Python, Bash, or similar

Data Governance & Quality: Experience implementing data quality checks, data governance policies, and data validation rules

Problem-Solving & Analytical Skills: Excellent problem-solving and analytical skills with the ability to identify and resolve complex data-related issues

Communication & Collaboration: Excellent communication, collaboration, and interpersonal skills
Ability to work effectively in a team environment

Education: Bacheloror masterdegree in computer science, Information Technology, or a related field

Bonus Points:

Experience with data streaming technologies (e
g, Kafka, Spark Streaming)

Experience with data visualization tools (e
g, Tableau, Power BI, Looker)

Relevant certifications (e
g, AWS Certified Data Engineer, Microsoft Certified: Azure Data Engineer Associate)