SRE-2 (Big Data)

Roles and Responsibilities:

• Ensure the ongoing stability, scalability, and performance of PhonePes Hadoop ecosystem and associated services.
• Exhibit a high level of ownership and accountability that ensures reliability of the Distributed clusters.
• Manage and administer Hadoop infrastructure including Apache Hadoop,HDFS, HBase, Hive, Pig, Airflow, YARN, Ranger, Kafka, Pinot,Ozone and Druid.
• Automate BAU operations through scripting and tool development.
• Perform capacity planning, system tuning, and performance optimization.
• Set-up, configure, and manage Nginx in high-traffic environments.
• Administration and troubleshooting of Linux + Bigdata systems, including networking (IP, Iptables, IPsec).
• Handle on-call responsibilities, investigate incidents, perform root cause analysis, and implement mitigation strategies.
• Collaborate with infrastructure, network, database, and BI teams to ensure data availability and quality.
• Apply system updates, patches, and manage version upgrades in coordination with security teams.
• Build tools and services to improve observability, debuggability, and supportability.
• Enabling cluster security using Kerberos and LDAP.
• Experience in capacity planning and performance tuning of Hadoop clusters.
• Work with configuration management and deployment tools like Puppet, Chef, Salt, or Ansible.

Preferred candidate profile:

• Minimum 1 year of Linux/Unix system administration experience.
• Over 4 years of hands-on experience in Apache Hadoop administration.
• Minimum 1 years of experience managing infrastructure on public cloud platforms like AWS, Azure, or GCP (optional ) .
• Strong understanding of networking, open-source tools, and IT operations.
• Proficient in scripting and programming (Perl, Golang, or Python).
• Hands-on experience with maintaining and managing the Hadoop ecosystem components like HDFS, Yarn, Hbase, Kafka .
• Strong operational knowledge in systems (CPU, memory, storage, OS-level troubleshooting).
• Experience in administering and tuning relational and NoSQL databases.
• Experience in configuring and managing Nginx in production environments.
• Excellent communication and collaboration skills.

Good to Have

• Experience designing and maintaining Airflow DAGs to automate scalable and efficient workflows.
• Experience in ELK stack administration.
• Familiarity with monitoring tools like Grafana, Loki, Prometheus, and OpenTSDB.
• Exposure to security protocols and tools (Kerberos, LDAP).
• Familiarity with distributed systems like elasticsearch or similar high-scale environments