8 Spark Optimization Jobs

As an experienced Data Engineer, your role will involve designing, implementing, and optimizing a global data handling and synchronization solution across multiple regions. You will work with cloud-based databases, data lakes, and distributed systems, ensuring compliance with data residency and privacy requirements such as GDPR. Key Responsibilities: - Design and implement data pipelines for global and regional data synchronization using Azure SQL, Data Lake, Data Factory, PySpark, etc. - Develop solutions for secure handling of PII and non-PII data, ensuring compliance with GDPR and other regulations. - Build and optimize ETL processes for anonymization, transformation, global synchronizati...

As an employee at Impetus Technologies, a digital engineering company, you will be part of a team dedicated to providing expert services and products to help enterprises achieve their transformation goals. Specializing in solving challenges related to analytics, AI, and cloud technologies, we empower businesses to drive innovation and growth. Your role will involve: - Working with technologies such as Bigdata, Pyspark, Hive, and Spark Optimization to develop cutting-edge solutions for our clients. Qualifications required for this position include: - Proficiency in Bigdata, Pyspark, Hive, and Spark Optimization. - Experience in working with GCP would be beneficial. About Impetus Technologies:...

As a candidate for the position, you should possess the following qualifications and skills: - Experience: 4 to 7 years - Proficiency in Bigdata, Pyspark, Hive, and Spark Optimization - Good to have: GCP Please note the Job Reference Number for this position is 13159.,

To transform data into a format that can be easily analyzed, you will be responsible for developing, maintaining, and testing infrastructures for data generation. Working closely with data scientists, you will play a key role in architecting solutions that enable data scientists to perform their tasks efficiently. Your main responsibilities will include: - Developing and maintaining scalable data pipelines and building new API integrations to support increasing data volume and complexity - Connecting offline and online data to enhance customer behavior understanding for personalization - Cleansing data, improving data quality, and preparing data for analysis - Processing data through cleansi...

We are seeking a highly skilled PySpark Developer with expertise in Distributed data processing to optimize Spark Jobs and ensure efficient data processing in a Big Data platform. This role demands a deep understanding of Spark performance tuning, distributed computing, and Big data architecture. Responsibilities include creating data tools for analytics and data scientist team members, collaborating with data and analytics experts to enhance functionality in the data system, and having the ability to adjust priorities quickly based on circumstances. The ideal candidate should have 8+ years of relevant experience in Apps Development or systems analysis. Key Responsibilities: - Analyze and co...

You will be responsible for developing, optimizing, and maintaining business intelligence and data warehouse systems. Your main accountabilities include developing scalable data pipelines, integrating offline and online data for improved customer understanding, managing data quality and governance strategies, transforming data for analysis, and ensuring data accessibility for the enterprise. To qualify for this role, you should have a Masters/Bachelors degree in Engineering/Computer Science/Math/Statistics or equivalent, strong programming skills in Python/Pyspark/SAS, experience with large datasets and technologies like Hadoop, Hive, and Spark optimization. Additionally, experience with clo...

Hiring, Data Engineer 4+ years of hands-on experience Experience in building and optimizing Big Data data pipelines, architectures, and data sets. Experience with big data tools: Hadoop, Spark Optimization, Spark-Streaming, Hive, Kafka, Hbase, Airflow, etc. Hands-on in SQL

About Impetus: Impetus Technologies is a digital engineering company dedicated to offering expert services and products to support enterprises in accomplishing their transformation objectives. Specializing in solving the analytics, AI, and cloud challenges, we empower businesses to foster unparalleled innovation and expansion. Established in 1991, we stand out as leaders in cloud and data engineering, catering cutting-edge solutions to Fortune 100 corporations. Our headquarters are located in Los Gatos, California, while our development centers span across NOIDA, Indore, Gurugram, Bengaluru, Pune, and Hyderabad, boasting a global team of over 3000 professionals. Additionally, we have operati...