11 Pipeline Orchestration Jobs

About the Role: You will be working on ensuring scaling of ML products and credit risk policy (incl. models) within pre-approved/ pre-qualified campaigns and live digital journeys. It will entail ensuring APIs operate smoothly at scale, identify ways to ensure simplified and low risk customer journey. We are looking for someone with entrepreneurial mindset, cloud-based tools/ packages for automation and encryption and APIs. About the team: You will be part of ML Engineering Team within Digital Banking Unit of the bank. You will work in a fast-paced environment where new ideas are encouraged. This will involve liaising with teams within Digital and those across the bank e.g. data engineers, d...

Description The Data Engineering Lead is a senior-level position within our Data Engineering team, responsible for spearheading the design, implementation, and optimization of advanced data architectures using Databricks. This role involves leading a team of data engineers, collaborating with business and data analysts, and business stakeholders, and delivering scalable, secure data solutions tailored to the needs of commercial analytics in the pharmaceutical industry. Responsibilities 1. Architecture Design Develop and maintain scalable data architectures using Databricks, including data lakes, data warehouses, and real-time processing systems. Create detailed blueprints for data processes ...

About the Role TheData Platform teamis part of the Platform org which is based in Bangalore and is responsible for building Uber's Data Lake for the rest of the company. The team owns end to end charters for multiple batch data areas around Pipeline Orchestration and Authoring, Metadata Platform, Lineage, Data Quality, Data Observability, Data Efficiency and Data Engineering. We have an eclectic team with a diverse set of backgrounds from companies such as Google, Cloudera, Amazon, Microsoft, LinkedIn, Twitter, other startups and college grads from the top schools. The team is proud to be a part of the open source community in innovating and shaping such exciting technologies as we move forw...

As a Senior Technical Lead Associate Architect specializing in Microsoft Fabric, your role will involve the following key responsibilities: - **Architect & Design Solutions**: Lead the end-to-end architecture of data platforms using Microsoft Fabric, ensuring alignment with business goals and technical standards. - **Data Engineering Leadership**: Guide the data engineering team in building robust, scalable, and efficient data pipelines and semantic models. - **Migration Strategy**: Define and implement validation rules, reconciliation mechanisms, and ensure zero data loss during migration from legacy systems to Fabric. - **Reporting & Visualization**: Architect and optimize reporting layers...

As a BI Architect in Gen AI, your role involves architecting a scalable and performant BI platform that serves as a single source of truth for key business metrics across functions such as Sales, Marketing, Supply Chain, and Finance. You will be advising the BI Lead by aligning architecture decisions with long-term strategy and business priorities. Additionally, you will design a GenAI-powered natural language interface on top of BI dashboards to enable business users to query KPIs, trends, and anomalies conversationally. Your responsibilities also include selecting and implementing appropriate architectures like RAG using vector stores and Knowledge Graphs to support intelligent, context-aw...

In this role at American Express, as a Senior Data Engineer on the Infrastructure Data & Analytics team, you will play a crucial part in unifying FinOps, Data Science, and Business Intelligence to enhance Technology cost transparency, optimize infrastructure performance, and improve commercial efficiency through high-quality data and predictive analytics. Your responsibility will be to lead the design, development, and enhancement of the data model and architecture, ensuring data quality meets the highest standards for users. Key Responsibilities: - **Data Architecture**: Perform technical aspects of data architecture and database management, develop data pipelines, new database structures, ...

Job Description At American Express, our culture is built on a 175-year history of innovation, shared and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. As part of our diverse tech team, you can architect, code and ship software that makes us an essential part of our customers digital lives.Here, you can w...

Hiring GCP Data Engineers (3–5 yrs) & Senior GCP Data Engineers (5–8 yrs). Skills: Python, SQL, BigQuery, Dataflow, Pub/Sub, Composer. Location: Remote/Hybrid/Onsite.

You are a Data Science Specialist with 4-5 years of experience in designing and implementing advanced analytical solutions. You should have a strong foundation in statistics and expertise in solving real-world business problems using Machine Learning and Data Science. Your track record should include building and deploying data products, and any exposure to NLP and Generative AI will be considered an advantage. Your key responsibilities will include collaborating with cross-functional teams to translate business problems into data science use cases, designing, developing, and deploying data science models, building and productionizing data products for measurable business impact, performing ...

Key Skills: Data engineering, Apache Airflow, GCP, BigQuery, GCS, SQL, ETL/ELT, Docker, Kubernetes, data governance, Agile, CI/CD, DevOps, pipeline orchestration, technical leadership. Roles & Responsibilities: Evaluate and provide scalable technical solutions to address complex and interdependent data processes. Ensure data quality and accuracy by implementing data quality checks, data contracts, and governance processes. Collaborate with software development teams and business analysts to understand data requirements and deliver fit-for-purpose data solutions. Lead the team in delivering end-to-end data engineering solutions. Design, develop, and maintain complex applications to support da...

Key responsibilities include: Data Architecture: Perform all technical aspects of data architecture and database management for ID&A, including developing data pipelines, new database structures and APIs as applicable Data Design: Translate logical data architectures into physical data designs, ensuring alignment with data modeling best practices and standards Data Process and Monitoring: Ensure proper data ingestion, validation, testing, and monitoring for ID&A data lake Support database and data platform administration for initiatives building, enhancing, or maintaining databases, data warehouses and data pipelines Data Migration: Design and support migration to a data model that is indepe...