Hadoop Administrator

Hi All ,

We have immediate openings for Below Requirement

Role

Skill

Experience

Work location

Interview Mode

Notice Period

Interested candidates can share your cv to

Mail

Contact

JD FOR Hadoop Admin:

• Administer, manage, and maintain Hadoop EMR clusters, ensuring optimal performance, high availability, and resource utilization.
• Handle the provisioning, configuration, and scaling of Hadoop clusters, with a focus on EMR, ensuring seamless integration with other ecosystem tools (e.g., Spark, Kafka, HBase).
• Oversee HBase configurations, performance tuning, and integration within the Hadoop ecosystem.
• Manage OpenSearch(formerly known as Elasticsearch) for log analytics and large-scale search applications.

Data Integration & Processing:

• Oversee the performance and optimization of Apache Spark workloads across distributed data environments.
• Design and manage efficient data pipelines between Snowflake, Kafka, and the Hadoop ecosystem, ensuring seamless data movement and transformation.
• Implement data storage solutions in Snowflake and manage seamless data transfers to/from Hadoop(EMR) and other environments.

Cloud & AWS Services:

• Work closely with AWS services such as EC2, S3,ECS, Lambda, IAM, RDS, and CloudWatch to build scalable, cost-efficient solutions for data management and processing.
• manage AWS EMR clusters, ensuring they are optimized for big data workloads and integrated with other AWS services.

Security & Compliance:

• Manage and configure Kerberos authentication and access control mechanisms within the Hadoop ecosystem (HDFS, YARN, Spark) to ensure data security.
• Implement encryption and secure data transfer policies within Hadoop clusters, Kafka, HBase, and OpenSearch to meet compliance and regulatory requirements.
• Manage user roles and permissions for access to Snowflake and ensure seamless integration of security policies across platforms.

Monitoring & Troubleshooting:

• Set up and manage monitoring solutions to ensure the health of the Hadoop ecosystem and related components.
• Actively monitor and troubleshoot issues with Spark, Kafka, HBase, OpenSearch, and other distributed systems.
• Provide proactive support to address performance issues, bottlenecks, and failures.

Automation & Optimization:

• Automate the deployment, scaling, and management of Hadoop and other big data systems using scripting languages (Bash, Python) .
• Optimize the configurations and performance of EMR, Spark, Kafka, HBase, OpenSearch.
• Develop scripts and utilities for backup, job monitoring, and performance tuning.