Data Engineer

Project Highlights:

• Prestigious Client / Project: Delivered a high-impact data engineering solution for a renowned hospitality tech company, focused on unifying and streamlining their operational data landscape.
• Revolutionizing Technologies: Built a cloud-native Data Lakehouse using AWS (S3, Glue, Athena, Kinesis), Apache Airflow, and modern storage formats like Parquet, Delta Lake, and Hudi enabling real-time analytics and boosting reporting efficiency.

Roles and Responsibilities:

• Design, develop and own robust data pipelines, ensuring optimal performance, scalability, and maintainability.
• Design and implement Data Lake, Dare Warehouse and Lakehouse solutions with different architecture patterns.
• Ensure data quality, integrity and governance across all stages of data lifecycle.
• Monitor and optimize performance of data engineering pipelines.
• Contribute to design principles, best practices, and documentation.
• Collaborate closely with cross-functional teams to deeply understand business requirements, translating them into effective technical design, implementations that support the organizations data-driven initiatives.
• Provide mentorship and guidance to other team members of the data engineering team, promoting
• knowledge transfer, a culture of continuous learning and skills development.

Requirements:

• Bachelors degree in computer science, Information Systems, or a related field Masters degree is a plus.
• A seasoned Data Engineer with a minimum of 7-8 years of experience.
• Deep experience in designing and building robust, scalable data pipelines both batch and real-time using modern data engineering tools and frameworks.
• Proficiency in AWS Data Services (S3, Glue, Athena, EMR, Kinesis etc.).
• Strong grip on SQL queries, various file formats like Apache Parquet, Delta Lake, Apache Iceberg or Hudi and CDC patterns.
• Experience in stream processing frameworks like Apache Flink or Kafka Streams or any other distributed data processing frameworks like pySpark.
• Expertise in workflow orchestration using Apache Airflow.
• Strong analytical and problem-solving skills, with the ability to work independently in a fast-paced environment.
• In-depth knowledge of database systems (both relational and NoSQL) and experience with data warehousing concepts.
• Hands-on experience with data integration tools and a strong familiarity with cloud-based data warehousing and processing is highly desirable.
• Excellent communication and interpersonal skills, facilitating effective collaboration with both technical and non-technical stakeholders.
• A strong desire to stay current with emerging technologies and industry best practices in data landscape.