Data Engineer (Pyspark)

Virtusa

4 years

0 Lacs

Chennai Tamil Nadu India

Posted:1 week ago| Platform:

Apply

Skills Required

data pyspark development design testing python hadoop spark pipeline software engineering linting integration git flow algorithms troubleshooting debugging scripting assessment profiling architecture relational etl

Work Mode

On-site

Job Type

Full Time

Job Description

Minimum 4+ years of development and design experience in experience as Data Engineer
Hands-on experience in building PySpark
Should be able to perform data cleansing and transformations in PySpark.
Unit testing using Python/PySpark code
Experience on Big Data platforms and distributed computing (e.g. Hadoop, Map/Reduce, Spark, HBase, Hive)
Experience in data pipeline software engineering and best practice in python (linting, unit tests, integration tests, git flow/pull request process, object-oriented development, data validation, algorithms and data structures, technical troubleshooting and debugging, bash scripting )
Experience in Data Quality Assessment (profiling, anomaly detection) and data documentation(schema,dictionaries)

Experience in data architecture, data warehousing and modelling techniques (Relational, ETL, OLTP) and consider performance alternatives

More Jobs at Virtusa

Servicenow QA

Bengaluru, Karnataka

3 - 3 yrs

Salary: Not disclosed

React T3

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Java-Pankaj

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

C ++

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Java Architect (ATC)

Bengaluru, Karnataka

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.