Kafka Developer

Location: Hyderabad [Hybrid]

Job Summary:

We are looking for a skilled Kafka Engineer

Key Responsibilities:

• Design and maintain Kafka-based streaming pipelines

  for high-throughput, low-latency real-time data processing.
• Ensure

  scalability

  ,

  fault tolerance

  , and

  schema governance

  using tools like

  Schema Registry (Avro/Protobuf)

  .
• Develop robust

  Kafka producers and consumers

  using

  Java

  , adhering to performance and reliability standards.
• Collaborate with DevOps teams to automate

  CI/CD pipelines

  , deployment, and infrastructure provisioning for Kafka components.
• Implement

  security best practices

  including encryption, authentication, and access control for Kafka clusters.
• Monitor and troubleshoot Kafka cluster health, latency, throughput, and message delivery guarantees.
• Document architecture, deployment strategies, and operational runbooks for Kafka-based systems.
• Participate in code reviews, knowledge-sharing sessions, and architectural discussions.

Must-Have Skills:

• 5–7 years of experience in backend or data engineering, with strong focus on

  Apache Kafka

  .
• Proficiency in

  Java

  for building Kafka producers/consumers and integration with backend services.
• Deep understanding of

  Kafka internals

  (brokers, partitions, consumer groups, offsets, replication, etc.).
• Experience with

  schema management

  using

  Confluent Schema Registry

  , Avro, or Protobuf.
• Experience with

  CI/CD pipelines

  for Kafka component deployment and integration.
• Strong collaboration skills with DevOps teams to manage

  Kafka infrastructure

  , security, and automation.
• Solid understanding of

  real-time data processing concepts

  , fault tolerance, and data consistency.

Good-to-Have Skills:

• Experience with

  Kafka Streams

  ,

  ksqlDB

  , or

  Apache Flink

  for stream processing.
• Familiarity with

  Kafka Connect

  and connectors for data ingestion and export.
• Exposure to

  Confluent Kafka Platform

  and

  Kafka REST Proxy

  .
• Experience with

  monitoring and observability tools

  (e.g., Grafana, ELK Stack).
• Understanding of

  container orchestration platforms

  like Kubernetes or OpenShift.
• Knowledge of

  cloud-native Kafka deployments

  (AWS MSK).
• Familiarity with

  Spring Boot

  for integrating Kafka in microservice environments.