"Spark & Delta Lake
Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL and Spark Streaming. Experience with Spark optimization techniques. Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database) Understanding of data quality best practices and data validation techniques. Other
Skills:

Understanding of data warehouse concepts, data modelling techniques. Expertise in Git for code management. Familiarity with CI/CD pipelines and containerization technologies. Nice to have experience using data integration tools like DataStage/Prophecy/Informatica/Ab Initio"

More Jobs at IDESLABS PRIVATE LIMITED

Bank Fast Platform Engineer

Bengaluru

3.0 - 7.0 yrs

INR 5 - 9 Lacs

Risk Business Analyst

Bengaluru

5.0 - 10.0 yrs

INR 8 - 13 Lacs

Mainframe TDM Professional

Hyderabad

3.0 - 5.0 yrs

INR 4 - 6 Lacs

SharePoint Developer

Pune

4.0 - 9.0 yrs

INR 6 - 10 Lacs

Informatica PowerCenter ETL Professional

Bengaluru

5.0 - 10.0 yrs

INR 7 - 12 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview Now

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

IDESLABS PRIVATE LIMITED

2317 Jobs

Login to

Please Verify Your Phone or Email

Confirm Action

Search

Profile

Upskill and Grow with AI

Spark & Delta Lake Professional