Informatics IDMC to PySpark

Cloud 9 Infosystems, Inc.

0 years

0 Lacs

Mumbai Maharashtra India

Posted:2 months ago| Platform:

Apply

Skills Required

pyspark data informatica migration etl spark aws azure orchestration analyze design migrate integration code efficiency databricks deployment automate monitoring governance compliance development apache processing gcp sql tuning git airflow

Work Mode

On-site

Job Type

Contractual

Job Description

Dear Professionals, Find below JD for Informatics IDMC to PySpark NOTE: DON'T APPLY IF YOU DON'T HAVE EXPERIENCE WITH PYSPARK. Job Summary : We are looking for a skilled Data Engineer with strong experience in Informatica IDMC and PySpark to lead the migration of ETL pipelines from Informatica IDMC to scalable, Spark-based solutions. The ideal candidate will have hands-on experience in cloud environments like AWS or Azure and a strong understanding of data warehousing, transformation, and orchestration. Key Responsibilities : Analyze existing ETL workflows built in Informatica IDMC and design equivalent pipelines in PySpark . Migrate and re-engineer data integration pipelines from Informatica Cloud to PySpark on distributed platforms. Optimize PySpark code for performance and cost efficiency in cloud data platforms (Databricks/EMR/HDInsight). Collaborate with data architects and business stakeholders to validate transformed data and maintain data quality. Ensure ETL processes are reliable, maintainable, and well-documented. Use CI/CD pipelines for deployment and version control of PySpark workflows. Work with large-scale structured and unstructured datasets from multiple sources. Automate data quality and monitoring mechanisms. Follow data governance and compliance policies. Required Skills : 5+ years of hands-on experience in Informatica Cloud (IDMC) and ETL development . Strong hands-on expertise in PySpark and Apache Spark for building scalable data processing pipelines. Good understanding of data lakes , data warehouses , and big data ecosystems . Experience working in cloud environments (AWS/Azure/GCP); experience with Databricks is a plus. Proficiency in SQL and performance tuning of queries. Knowledge of version control (Git) and CI/CD practices. Experience with job orchestration tools like Airflow , Control-M , or Azure Data Factory . Show more Show less

More Jobs at Cloud 9 Infosystems, Inc.

Informatics IDMC to PySpark

Mumbai, Maharashtra, India

Experience: Not specified

Salary: Not disclosed

Informatica IDMC and PySpark

Mumbai, Maharashtra, India

5.0 - 5.0 yrs

Salary: Not disclosed

Azure Fabric

India

Experience: Not specified

Salary: Not disclosed

Project Manager Technical Lead

Mumbai, Maharashtra, India

15.0 - 15.0 yrs

Salary: Not disclosed

.net architect

Mumbai, Maharashtra, India

12.0 - 12.0 yrs

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.