Applore Technologies - Kafka Administrator

Job Title :

Location :

Employment Type :

Role Summary

Key Responsibilities

Kafka & Zookeeper Administration :

• Manage multiple Kafka clusters with high-availability Zookeeper setups.
• Perform end-to-end operational support including deployment, configuration, and health monitoring of Kafka brokers and Zookeeper nodes.
• Conduct capacity planning, partition strategy optimization, and topic lifecycle management.
• Implement and manage backup and disaster recovery processes with defined RPO/RTO targets.
• Enforce security configurations, including TLS encryption, authentication (SASL, mTLS), and ACL management.
• Optimize Kafka producer and consumer performance to meet low-latency, high-throughput requirements.
• Plan and execute Kafka and Zookeeper upgrades and patching with minimal/zero downtime.
• Integrate Kafka with monitoring platforms like Prometheus, Grafana, or similar tools.
• Define and enforce log retention and archival policies in line with compliance requirements.
  

Monitoring & Logging

• Integrate Kafka metrics and logs with centralized observability and logging tools.
• Create dashboards and alerts to monitor Kafka consumer lag, partition health, and broker performance.
• Collaborate with DevOps/SRE teams to ensure visibility into Kafka services.
  

Security & Compliance

• Apply CIS benchmarks and perform automated security scans across Kafka nodes.
• Manage secret and certificate rotation using tools like Vault or AWS ACM, as applicable.
• Support regular vulnerability assessments and ensure timely remediation.
  

Additional (Nice-to-Have) Responsibilities

• Collaborate on Kong API Gateway support, especially in areas such as mTLS, monitoring, and certificate management.
• Participate in cross-functional discussions around observability, alerting, and compliance strategies.
  

Infrastructure Snapshot

• Kafka Deployments :
• o1 Cluster : 5 Kafka + 5 Zookeeper nodes.
• o5 Clusters : 3 Kafka + 3 Zookeeper nodes each.
• API Gateway (Kong) : AWS EKS with mTLS support (if experience applicable).
  

Required Skills & Experience

• 3+ years of hands-on Kafka administration experience in production environments.
• Strong understanding of Kafka internals (broker behavior, ISR, partitions, replication, etc.)
• Proficient in Zookeeper management and configuration.
• Experience with Kafka performance tuning and troubleshooting.
• Familiar with TLS/mTLS, ACLs, SASL, and other Kafka security mechanisms.
• Proficient with monitoring and logging tools (e.g., Prometheus, Grafana, ELK).
• Scripting skills (e.g., Bash, Python) for operational automation.
  

Preferred Qualifications

• Experience with API gateways (Kong or equivalent).
• Exposure to Kubernetes-based environments (EKS preferred).
• Familiarity with compliance standards and security hardening practices.
• Experience with IaC tools (e.g., Terraform, Ansible).
  

What We Offer

• A mission-critical role in managing large-scale real-time data infrastructure.
• Flexible work environment and opportunities for growth.
• Supportive team and access to modern observability and automation tools.