Java Spark Developer

Key Skills

• Strong Core Java (8+) & OOP fundamentals
• Apache Spark (RDD/DataFrame API, Spark SQL, performance tuning)
• Advanced SQL for large‑scale data manipulation and analytics
• Version control (Git), CI/CD pipelines, Unix/Linux scripting
• Good understanding of data‑modelling and ETL best practices

Nice to have:experience with Hadoop/Hive, Kafka, cloud data services (AWS EMR, Databricks, GCP Dataproc, or Azure HDInsight), containerization (Docker/K8s).

Responsibilities

• Design & Development: Build scalable data‑processing jobs in Java and Spark; refactor legacy MapReduce/SQL workloads into efficient Spark applications.
• Data Engineering: Develop robust ETL/ELT pipelines, implement data quality checks, and maintain metadata/catalogue integration.
• Performance Optimisation: Profile jobs, tune Spark configurations, and optimise SQL queries to meet SLAs on throughput and latency.
• Code Quality: Write clean, modular, and test‑covered code; enforce coding standards through peer reviews and automation in CI/CD.
• Collaboration: Liaise with architects, analysts, and cloud engineers to integrate data solutions with microservices, dashboards, and ML workflows.
• Monitoring & Support: Implement logging/alerting, troubleshoot production issues, and participate in on‑call rotations (as needed).
• Continuous Improvement: Evaluate new frameworks, contribute to PoCs, and share best practices to uplift team capability.