Big Data Developer

Role & responsibilities

We are seeking a talented and motivated Big Data Developer to design, develop, and maintain large-scale data processing applications. You will work with modern Big Data technologies, leveraging PySpark and Java/Scala, to deliver scalable, high-performance data solutions on AWS. The ideal candidate is skilled in big data frameworks, cloud services, and modern CI/CD practices.

Preferred candidate profile

• Design and develop scalable data processing pipelines using PySpark and Java/Scala.
  
• Build and optimize data workflows for batch and real-time data processing.
  
• Integrate and manage data solutions on AWS services such as EMR, S3, Glue, Airflow, RDS, and DynamoDB.
  
• Implement containerized applications using Docker, Kubernetes, or similar technologies.
  
• Develop and maintain APIs and microservices/domain services as part of the data ecosystem.
  
• Participate in continuous integration and continuous deployment (CI/CD) processes using Jenkins or similar tools.
  
• Optimize and tune performance of Big Data applications and databases (both relational and NoSQL).
  
• Collaborate with data architects, data engineers, and business stakeholders to deliver end-to-end data solutions.
  
• Ensure best practices in data security, quality, and governance are followed.
  

Must-Have Skills

• Proficiency with Big Data frameworks and programming using

  PySpark and Java/Scala

• Experience designing and building data pipelines for large-scale data processing
  
• Solid knowledge of distributed data systems and best practices in performance optimization
  

Preferred Skills

• Experience with AWS services (EMR, S3, Glue, Airflow, RDS, DynamoDB, or similar)
  
• Familiarity with container orchestration tools (Docker, Kubernetes, or similar)
  
• Knowledge of CI/CD pipelines (e.g., Jenkins or similar tools)
  
• Hands-on experience with relational databases and SQL
  
• Experience with NoSQL databases (e.g., MongoDB, Cassandra, DynamoDB)
  
• Exposure to microservices or API gateway frameworks
  

Qualifications

• Bachelors or Master’s degree in Computer Science, Engineering, or a related field
  
• 5+ years of experience in Big Data development
  
• Strong analytical, problem-solving, and communication skills
  
• Experience working in an Agile environment is a plus