Big Data Engineer

Description

We are seeking a skilled Big Data Engineer to join our dynamic team in India. The ideal candidate will be responsible for developing and maintaining our big data infrastructure, ensuring efficient data processing and analytics. The role requires a strong understanding of big data technologies, programming skills, and the ability to work collaboratively in a fast-paced environment.

Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures to handle large volumes of data.
• Collaborate with data scientists and analysts to understand data requirements and provide data solutions.
• Implement ETL processes to extract, transform, and load data from various sources into data warehouses and data lakes.
• Optimize and tune existing data systems for performance and efficiency.
• Ensure data quality and integrity by implementing appropriate validation and monitoring processes.
• Stay updated with emerging technologies in big data and recommend best practices to enhance data processes.

Skills and Qualifications

• 4-8 years of experience in big data technologies and data engineering roles.
• Proficiency in Hadoop ecosystem (HDFS, MapReduce, Hive, Pig) and distributed computing frameworks.
• Strong programming skills in languages such as Java, Scala, or Python.
• Experience with data warehousing solutions such as Amazon Redshift, Google BigQuery, or Snowflake.
• Familiarity with NoSQL databases like MongoDB, Cassandra, or HBase.
• Knowledge of data modeling techniques and data architecture principles.
• Experience with data pipeline orchestration tools such as Apache Airflow or Apache NiFi.
• Strong analytical and problem-solving skills, with a focus on data-driven decision-making.