32 Bigdata Frameworks Jobs - Page 2

ABOUT AMGEN Amgen harnesses the best of biology and technology to fight the world’s toughest diseases, and make people’s lives easier, fuller and longer. We discover, develop, manufacture and deliver innovative medicines to help millions of patients. Amgen helped establish the biotechnology industry more than 40 years ago and remains on the cutting-edge of innovation, using technology and human genetic data to push beyond what’s known today. ABOUT THE ROLE Role Description: We are looking for an Associate Data Engineer with deep expertise in writing data pipelines to build scalable, high-performance data solutions. The ideal candidate will be responsible for developing, optimizing and maintaining complex data pipelines, integration frameworks, and metadata-driven architectures that enable seamless access and analytics. This role prefers deep understanding of the big data processing, distributed computing, data modeling, and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management. Roles & Responsibilities: Data Engineer who owns development of complex ETL/ELT data pipelines to process large-scale datasets Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions Ensuring data integrity, accuracy, and consistency through rigorous quality checks and monitoring Exploring and implementing new tools and technologies to enhance ETL platform and performance of the pipelines Proactively identify and implement opportunities to automate tasks and develop reusable frameworks Eager to understand the biotech/pharma domains & build highly efficient data pipelines to migrate and deploy complex data across systems Work in an Agile and Scaled Agile (SAFe) environment, collaborating with cross-functional teams, product owners, and Scrum Masters to deliver incremental value Use JIRA, Confluence, and Agile DevOps tools to manage sprints, backlogs, and user stories. Support continuous improvement, test automation, and DevOps practices in the data engineering lifecycle Collaborate and communicate effectively with the product teams, with cross-functional teams to understand business requirements and translate them into technical solutions Must-Have Skills: Experience in Data Engineering with a focus on Databricks, AWS, Python, SQL, and Scaled Agile methodologies Proficiency & Strong understanding of data processing and transformation of big data frameworks (Databricks, Apache Spark, Delta Lake, and distributed computing concepts) Strong understanding of AWS services and can demonstrate the same Ability to quickly learn, adapt and apply new technologies Strong problem-solving and analytical skills Excellent communication and teamwork skills Experience with Scaled Agile Framework (SAFe), Agile delivery, and DevOps practices Good-to-Have Skills: Data Engineering experience in Biotechnology or pharma industry Exposure to APIs, full stack development Experienced with SQL/NOSQL database, vector database for large language models Experienced with data modeling and performance tuning for both OLAP and OLTP databases Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops Education and Professional Certifications Any degree and 2-5 years of experience AWS Certified Data Engineer preferred Databricks Certificate preferred Scaled Agile SAFe certification preferred Soft Skills: Excellent analytical and troubleshooting skills. Strong verbal and written communication skills Ability to work effectively with global, virtual teams High degree of initiative and self-motivation. Ability to manage multiple priorities successfully. Team-oriented, with a focus on achieving team goals. Ability to learn quickly, be organized and detail oriented. Strong presentation and public speaking skills. EQUAL OPPORTUNITY STATEMENT Amgen is an Equal Opportunity employer and will consider you without regard to your race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or disability status. We will ensure that individuals with disabilities are provided with reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request an accommodation.

Job Title / Primary Skill: Sr. Big Data Developer Years of Experience: 4 to 8 years Job Location: Bangalore/Kochi (Hybrid) Must Have Skills: BDF with Hadoop, Spark, Scala, and SQL Educational Qualification: BE/BTech/ MTech/ MCA Experience: • Minimum 2+ years of experience in Big Data development. • Good understanding of SDLC. • Experience with Agile or iterative development methodologies is a plus. • Prior experience in Healthcare Analytics domain is a plus. Required Skills: Strong experience with big data technologies and associated tools such as Hadoop, Unix, HDFS, Hive, Impala, etc. Proficient in using Spark/Scala Experience with data Import/Export using Sqoop or similar tools Experience using Airflow, Jenkins or similar other automation tools Excellent knowledge of SQL Server and database structures Demonstrate ability to write and optimize T-SQL queries and stored procedures Experience working with Jira/Confluence/GitLab Excellent organizational skills and ability to handle multiple activities with changing priorities simultaneously Professional Attributes: Should have good communication skill. Team player willing to collaborate throughout all phases of development, testing and deployment. Ability to solve problems and meet the deadlines within minimal supervision.

About Oracle FSGIU - Finergy: The Finergy division within Oracle FSGIU is dedicated to the Banking, Financial Services, and Insurance (BFSI) sector. We offer deep industry knowledge and expertise to address the complex financial needs of our clients. With proven methodologies that accelerate deployment and personalization tools that create loyal customers, Finergy has established itself as a leading provider of end-to-end banking solutions. Our single platform for a wide range of banking services enhances operational efficiency, and our expert consulting services ensure technology aligns with our clients' business goals. Job Summary: We are seeking a skilled Senior Data Migration Engineer with expertise in AWS, Databricks, Python, PySpark, and SQL to lead and execute complex data migration projects. The ideal candidate will design, develop, and implement data migration solutions to move large volumes of data from legacy systems to modern cloud-based platforms, ensuring data integrity, accuracy, and minimal downtime. Job Responsibilities Software Development: Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications. Implement efficient and maintainable code using best practices and coding standards. AWS & Databricks Implementation: Work with Databricks platform for big data processing and analytics. Develop and maintain ETL processes using Databricks notebooks. Implement and optimize data pipelines for data transformation and integration. Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines. Leverage PySpark for large-scale data processing and transformation tasks. Continuous Learning: Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks. Share knowledge with the team and contribute to a culture of continuous improvement. SQL Database Management: Utilize expertise in SQL to design, optimize, and maintain relational databases. Write complex SQL queries for data retrieval, manipulation, and analysis. Qualifications & Skills: Education: Bachelors degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees are a plus. 4 to 8 Years of experience inDatabricksand big data frameworks Proficient in AWS services and data migration Experience inUnity Catalogue Familiarity withBatch and real time processing Data engineering with strong skills in Python, PySpark, SQL Certifications: AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus. Soft Skills: Strong problem-solving and analytical skills. Excellent communication and collaboration abilities. Ability to work in a fast-paced, agile environment.

Overview: As a Sr Machine Learning Engineer of the Data Science & AI team, you will develop analytical products and solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms built on data sets measured in the billions of transactions that allow our users to derive insights from big data that in turn drive their businesses with a keen eye for data privacy and governance. Role: - Leads talent acquisition efforts and initiatives, facilitates training programs and conducts performance management for team of direct reports - Lead teams in the creation of portfolio robust ML solutions through effective use of Mastercards global data assets and software platform - Build, productionize and maintain data driver AI/ML application and data processing workflows or pipelines - Consult with clients/ stakeholders to understand and translate their needs into a data analyses and/or solution, ensuring that their requirements are accurately captured and technically feasible - Guide others in comprehensive technical analyses and allocates work across teams to ensure the delivery of high quality and effective solutions - Liaise with internal stakeholders (e.g., MA TECH, Data Strategy Management, AI governance) to identify and elaborate on opportunities as they relate to analytical solution development, feasibility, and other technical offerings - Lead development of presentations and technical documentation - Identify and recommend opportunities to standardize and automate efforts to ensure quality and enable scaling of ML products - Meet project deadlines for accountable deliverables and anticipates delays or foreseeable barriers to progress and escalates issues when necessary - Conduct due diligence quality assurance testing for prototypes and tools in stage and resolves reoccurring complex issues and bugs - Ensure that all machine learning processes, from data preparation to model deployment, are well-documented for internal use and compliance. - Mentor and guide junior developers All about you: - Expertise in Big Data Technologies: Proficiency in big data frameworks and tools such as Hadoop, Spark, Hive - Technical Proficiency: Strong programming skills in languages such as Python and SQL. Experience with data visualization tools (e.g., Tableau, Power BI) and understanding of cloud computing services (AWS, Azure, GCP) related to data processing and storage is a plus. Experience with testing frameworks and test-driven development (TDD) practices - Advanced Analytical Skills: Strong applied knowledge and hands on experience in machine learning algorithms and deep learning frameworks. Familiarity with AI and machine learning platforms such as TensorFlow, PyTorch, or similar. Familiar with training and deploying models with large datasets including strategies for parallelizing and optimizing the training/deployment workflows. Experience in productionizing of GenAI products a plus. - Leadership and Strategic Planning: Proven experience in leading engineering teams, defining vision and strategy for data-driven initiatives, and driving projects from conception to implementation. Ability to mentor and develop talent within the team. - Problem-Solving Skills: Strong analytical and critical thinking abilities to solve complex problems, along with the creativity to find innovative solutions. - Communication and Collaboration: Excellent verbal and written communication skills, with the ability to explain complex analytical concepts to non-technical stakeholders. Experience in working cross-functionally with departments and flexibility to work as a member of a matrix based diverse and geographically distributed project teams. - Project Management Skills: Proficiency in managing multiple projects simultaneously, with a focus on delivering results within tight deadlines. - Responsible AI knowledge: Awareness of the principles and practices surrounding responsible AI, including fairness, transparency, accountability, and ethics in AI deployments. - Innovation and Continuous Learning: A mindset geared towards innovation, staying abreast of industry trends, emerging technologies in big data and analytics, and continuously seeking opportunities for personal and professional growth. Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercards security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercards guidelines.

Job Description Summary As a member of the GE Aerospace Digital Technology organization aligned to the Technology & Operations High Performance Computing as a Service (HPCaaS) team, this role will focus on providing excellent service to Engineering stakeholders while operating a quality production simulation engineering infrastructure environment to deliver critical outcomes. You will work with a global team of like-minded HPC professionals to ensure the stable, reliable performance of our products and services while inspiring evolution and continuous improvement in how we build, operate, monitor, maintain and secure our infrastructure for a globally consistent user experience. You will drive excellence while identifying opportunities for continuous improvement, simplification, automation and global consistency in HPCaaS software, applications and tools. Role Overview: In this role, you will: Work collaboratively with a globally diverse team of employees and contractors through FLIGHT DECK and with our SQDC mindset to deliver on established business priorities and commitments. Provide excellent global customer/stakeholder support in a collaborative, consultative style with a constant focus on operational efficiency. Install, configure, manage, secure, run and support large scale Linux-based clusters and related Lustre/NAS storage, visualization and data management services. Support and maintain a broad spectrum of engineering software packages aligned to technical disciplines such as finite element analysis, computational fluid dynamics and lifing analytics or equivalent. Build, manage and execute complex infrastructure management and operational projects (i.e. installation, upgrades, andmigration/decommission.) Embrace change and promote the identification of service improvement opportunities across the globalteam while applying robust change management processes. Contribute to evolving current legacy service and solution offerings into next generation capabilities aligned to new and emerging business needs. Identify opportunities to drive the automation of repeatable processes, efficiency and to remove non-value added tasks and overhead from team activities. Drive effective written and spoken communication within the team and externally to stakeholders. Partner with leadership and Architects to ensure that service quality meets and exceeds stakeholder expectations. Ideal candidate: Should have experience in complex enterprise-class High Performance Computing (HPC) Required Qualifications: Bachelors Degree in Computer Science or in STEM Majors (Science, Technology, Engineering and Math.) Significant professional experience in working with complex enterprise-class High Performance Computing (HPC) and/or Simulation Based Engineering and Science (SBES) systems. Demonstrable and quantifiable experience providing technical HPC operational support as well as building, operating and managing distributed end-to-end technical computing and Linux-based cluster solutions (compute, storage, visualization.) Willingness to work out of an office located in Bangalore, India with occasional weekend and off-hours availability required to execute on projects and respond to critical operational needs. Desired Qualifications: Technical Expertise: Demonstrated ability to design, develop and maintain creative solutions to complex software, engineering toolchain and application problems. Mission critical systems management experience with enterprise-class compute, storage, network, virtualization and cloud service technologies. Experience working directly with, administering, maintaining and securing global Linux compute clusters (RHEL 7.x/8.x), Lustre-based storage services, related VMware visualization (ESXi and vCenter) and data management systems. Experience working directly with cloud providers/infrastructure (Amazon, Microsoft, etc.), database technologies, operating systems (Windows, Linux) and orchestration tools (Chef, Puppet, etc.) Experience implementing, scaling, managing and administering infrastructure monitoring and management tools (i.e. NAGIOS, Splunk, HP Openview, Oracle Enterprise Manager, OpsView or equivalent.) Experience working with emerging HPC GPU technologies, Machine Learning frameworks, Slurm, Hadoop/Big Data frameworks, Docker/Singularity containerization, etc. Experience with engineering software aligned to finite element analysis, computational fluid dynamics and lifing analytics or equivalent. Experience delivering complex technical projects focused on infrastructure management and operations. Business Acumen: Strong analysis and problem-solving skills. Ability to interact at all levels of the organization and across cultural, geographic and business boundaries. Strong understanding of software governance and compliance/regulatory requirements. Leadership: Ability to work with cross-functional teams to build effective processes while applying FLIGHT DECK tools, mindset and methodology. Relentless drive and desire for continuous improvement.Challenges the status quo and pursues opportunities to drive service evolution, quality and efficiency while removing waste and non-value added work from team activities. Proactively identifies and removes project obstacles or barriers on behalf of the team. Articulates the story; uses two-way communication and influences outcomes and on-going results. Strong oral and written communication skills including executive level presentation skills. Self-motivated. Demonstrated ability to deliver on commitments to stakeholders. Humble: respectful, receptive, agile, eager to learn Transparent: shares critical information, speaks with candor, contributes constructively Focused: quick learner, strategically prioritizes work, committed Leadership ability: strong communicator, decision-maker, collaborative Problem solver: analytical-minded, challenges existing processes, critical thinker. Personal Attributes: Demonstrated interpersonal and global teaming abilities with ability to build consensus and navigate through debate, conflict and constructive disagreement. Works well in a fast paced, agile, adaptive environment. Wiling to adapt to change, and learn new tools, technologies and processes as needed. Proactively engages with cross-functional teams to resolve issues and design solutions using critical thinking and analytical skills and best practices. Ability to prioritize and manage multiple complex, competing priorities simultaneously. Additional Information Relocation Assistance Provided: Yes

Job Title: Senior / Lead Data Engineer Company: Synechron Technologies Locations: Pune or Chennai Experience: 5 to 12 years : Synechron Technologies is seeking an accomplished Senior or Lead Data Engineer with expertise in Java and Big Data technologies. The ideal candidate will have a strong background in Java Spark, with extensive experience working with big data frameworks such as Spark, Hadoop, HBase, Couchbase, and Phoenix. You will lead the design and development of scalable data solutions, ensuring efficient data processing and deployment in a modern technology environment. Key Responsibilities: Lead the development and optimization of large-scale data pipelines using Java and Spark. Design, implement, and maintain data infrastructure leveraging Spark, Hadoop, HBase, Couchbase, and Phoenix. Collaborate with cross-functional teams to gather requirements and develop robust data solutions. Lead deployment automation and management using CI/CD tools including Jenkins, Bitbucket, GIT, Docker, and OpenShift. Ensure the performance, security, and reliability of data processing systems. Provide technical guidance to team members and participate in code reviews. Stay updated on emerging technologies and leverage best practices in data engineering. Qualifications & Skills: 5 to 14 years of experience as a Data Engineer or similar role. Strong expertise in Java programming and Apache Spark. Proven experience with Big Data technologiesSpark, Hadoop, HBase, Couchbase, and Phoenix. Hands-on experience with CI/CD toolsJenkins, Bitbucket, GIT, Docker, OpenShift. Solid understanding of data modeling, ETL workflows, and data architecture. Excellent problem-solving, communication, and leadership skills. S YNECHRONS DIVERSITY & INCLUSION STATEMENT Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more. All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicants gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law . Candidate Application Notice

We are seeking a skilled Data Engineer who can function as a Data Architect, designing scalable data pipelines, table structures, and ETL workflows. The ideal candidate will be responsible for recommending cost-effective and high-performance data architecture solutions, collaborating with cross-functional teams to enable efficient analytics and data science initiatives. Key Responsibilities: Design and implement ETL workflows, data pipelines, and table structures to support business analytics and data science. Optimize data storage, retrieval, and processing for cost-efficiency and high performance. Collaborate with Analytics and Data Science teams for feature engineering and KPI computations. Develop and maintain data models for structured and unstructured data. Ensure data quality, integrity, and security across systems. Work with cloud platforms (AWS/ Azure/ GCP) to design and manage scalable data architectures. Technical Skills Required: SQL & Python Strong proficiency in writing optimized queries and scripts. PySpark Hands-on experience with distributed data processing. Cloud Technologies (AWS/ Azure/ GCP) Experience with cloud-based data solutions. Spark & Airflow Experience with big data frameworks and workflow orchestration. Gen AI (Preferred) Exposure to generative AI applications is a plus. Preferred Qualifications: Experience in data modeling, ETL optimization, and performance tuning. Strong problem-solving skills and ability to work in a fast-paced environment. Prior experience working with large-scale data processing.