Posted:5 days ago|
Platform:
On-site
Full Time
L3 Big data
developer:
1. Design, develop, and implement highly scalable and distributed big data
solutions using Hadoop ecosystem technologies such as HBase, Hive, Kudu, and
Spark.
2. Architect HBase schemas and data models to accommodate evolving business
requirements and ensure optimal performance for data storage and retrieval
operations.
3. Develop complex Hive queries and data processing pipelines to transform raw
data into structured formats suitable for analysis and reporting.
4. Implement data ingestion pipelines using Spark Streaming and Spark SQL for
real-time processing of streaming data sources, ensuring high throughput and
low latency.
5. Optimize Spark applications for performance and resource utilization,
including tuning RDD transformations, optimizing data partitioning strategies,
and leveraging in-memory caching.
6. Utilize advanced features of Spark MLlib for machine learning tasks such as
classification, regression, clustering, and collaborative filtering.
7. Design and deploy Kudu tables for fast analytical queries and real-time
analytics, leveraging Kudus unique combination of fast analytics and fast data
ingestion.
8. Collaborate with data scientists to integrate machine learning models into
Spark workflows and productionize them for real-time predictions and analytics.
9. Troubleshoot performance bottlenecks, data quality issues, and system
failures in big data applications and infrastructure, and implement solutions
to address them.
10. Stay abreast of emerging technologies and best practices in big data processing
and analytics, and evaluate their potential impact on our architecture and
solutions.
Pradeepit Consulting Services
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Hyderabad
5.0 - 7.0 Lacs P.A.
Pune, Maharashtra, India
Salary: Not disclosed
bangalore, chennai, hyderabad, mumbai city, delhi
0.00021 - 0.00024 Lacs P.A.
Andhra Pradesh
Experience: Not specified
Salary: Not disclosed
Andhra Pradesh, India
Experience: Not specified
Salary: Not disclosed
17.0 - 19.0 Lacs P.A.
Bengaluru
18.0 - 30.0 Lacs P.A.
Hyderabad, Telangana, India
Salary: Not disclosed
Gurugram, Haryana, India
Experience: Not specified
Salary: Not disclosed
Pune, Maharashtra, India
Salary: Not disclosed