Tech Lead

Virtusa

6 - 9 years

0 Lacs

Andhra Pradesh India

Posted:5 days ago| Platform:

Apply

Skills Required

data design java scala apache spark kafka processing debugging storage nosql aws engineering workflow orchestration docker code management github deployment analytics governance automate architecture

Work Mode

On-site

Job Type

Full Time

Job Description

Data Lead Engineer
Exp : 6 - 9 Years

Design, develop, and maintain efficient and reliable data pipelines using Java, Scala, Apache Spark and Confluent Cloud (Kafka, KStreams, kSQLDB, Schema Registry)
Leverage Apache Spark (Java/Scala) for large-scale data processing and transformation.
Experience with building, maintaining and debugging applications and data pipelines using Confluent Cloud (Kafka, KStreams, kSQLDB, Schema Registry).
Build and optimize data storage solutions using NoSQL databases such as ScyllaDB and/or Cassandra.
Experienced with AWS services required for Data Engineering such as EMR, ServerlessEMR, AWS Glue, CodeCommit, EC2, S3 etc.
Familiarity with workflow orchestration tools such as Airflow
Experience with building and deploying applications using Docker or AWS ECS or AWS EKS
Well versed with code management using tools like GitHub and CI/CD pipelines and deployment of data pipelines on AWS cloud.
Implement and manage search and analytics capabilities using AWS OpenSearch and/or Elasticsearch.
Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver effective solutions.
Monitor and troubleshoot data pipelines to ensure data quality and performance.
Implement data governance and data quality best practices.
Automate data ingestion, processing, and deployment processes.
Stay up-to-date with the latest data engineering trends and technologies.
Contribute to the design and architecture of our data platform on AWS.

More Jobs at Virtusa

Servicenow QA

Bengaluru, Karnataka

3 - 3 yrs

Salary: Not disclosed

React T3

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Java-Pankaj

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

C ++

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Java Architect (ATC)

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Java Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.