Big Data Developer

About the Role

As a Big Data Engineer, you will design, develop, and optimize data pipelines, ETL workflows, and large-scale data systems. You will partner with business stakeholders, data scientists, and software engineers to deliver high-quality, scalable, and timely data solutions.

hr@hiremaze.in

Key Responsibilities

Design, develop, and maintain scalable and fault-tolerant data pipelines using AWS big data services.

Build and optimize ETL workflows to transform and process large datasets from diverse sources.

Work with structured and unstructured data (batch and real-time streaming).

Ensure data integrity, security, and compliance with internal and external standards.

Collaborate with BI teams, data scientists, and product managers to deliver insights.

Monitor, troubleshoot, and tune big data systems for cost-efficiency and performance.

Stay updated with emerging big data technologies, tools, and best practices.

Basic Qualifications

Bachelor’s/Master’s degree in computer science, information systems, engineering, or a related field.

5–7 years of experience in data engineering, Big Data, or related roles.

Strong experience with SQL and at least one programming language (Python, Java, or Scala).

Hands-on expertise in big data frameworks: Hadoop, Spark, Hive, and Presto.

Experience with data modeling, schema design, and distributed data storage systems.

Exposure to AWS services such as S3, Redshift, EMR, Glue, Kinesis, Athena, and DynamoDB.

Strong analytical, problem-solving, and debugging skills.

Preferred Qualifications

Experience with streaming data pipelines (Kafka, Flink, and Spark Streaming).

Knowledge of machine learning pipelines and integration with big data systems.

Experience with DevOps, CI/CD, and infrastructure automation tools.

Excellent written and verbal communication skills.

Ability to thrive in a fast-paced, ambiguous environment with minimal supervision.