12 Data Orchestration Jobs

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period). The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations. Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks. The candidate will implement CI/CD best practices for DBT, manage automated deployments, troubleshoot pipeline issues, and collaborate cross-functionally to deliver cloud-based real-time and batch data solutions. Strong SQL, scripting, API integrations, and AWS experience are essential.

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period) The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks The candidate will implement CI/CD best practices for DBT, manage automated deployments, troubleshoot pipeline issues, and collaborate cross-functionally to deliver cloud-based real-time and batch data solutions Strong SQL, scripting, API integrations, and AWS experience are essential

We are seeking a highly skilled Data Engineer with 5+ years of experience for our Bengaluru location (max 30 days notice period). The ideal candidate will have strong expertise in designing, developing, and maintaining robust data ingestion frameworks, scalable pipelines, and DBT-based transformations. Responsibilities include building and optimizing DBT models, architecting ELT pipelines with orchestration tools like Airflow/Prefect, integrating workflows with AWS services (S3, Lambda, Glue, RDS), and ensuring performance optimization on platforms like Snowflake, Redshift, and Databricks. The candidate will implement CI/CD best practices for DBT, manage automated deployments, troubleshoot pipeline issues, and collaborate cross-functionally to deliver cloud-based real-time and batch data solutions. Strong SQL, scripting, API integrations, and AWS experience are essential.

You have over 8 years of experience and are located in Balewadi, Pune. You possess a strong understanding of Data Architecture and have led data-driven projects. Your expertise includes knowledge of Data Modelling paradigms like Kimball, Inmon, Data Marts, Data Vault, Medallion, etc. Experience with Cloud Based data strategies, particularly AWS, is preferred. Designing data pipelines for ETL with expert knowledge on ingestion, transformation, and data quality is a must, along with hands-on experience in SQL. In-depth understanding of PostGreSQL development, query optimization, and designing indexes is a key requirement. Proficiency in Postgres PL/SQL for complex warehouse workflows is necessary. You should be able to manipulate intermediate to complex SQL and use advanced SQL concepts like RANK, DENSE_RANK, and apply advanced statistical concepts through SQL. Working experience with PostGres SQL extensions like PostGIS is desired. Expertise in writing ETL pipelines combining Python + SQL is required, as well as understanding of data manipulation libraries in Python like Pandas, Polars, DuckDB. Experience in designing Data visualization with tools such as Tableau and PowerBI is desirable. Your responsibilities include participation in designing and developing features in the existing Data Warehouse, providing leadership in establishing connections between Engineering, product, and analytics/data scientists team. Designing, implementing, and updating existing/new batch ETL pipelines, defining and implementing data architecture, and working with various data orchestration tools like Apache Airflow, Dagster, Prefect, and others. Collaboration with engineers and data analysts to build reliable datasets that can be trusted and used by the company is essential. You should be comfortable in a fast-paced start-up environment, passionate about your job, and enjoy a dynamic international working environment. Background or experience in the telecom industry is a plus, though not mandatory. You should have a penchant for automating tasks and enjoy monitoring processes.,

Do you have a passion for tackling global challenges like feeding the growing population and addressing climate change AGCO is dedicated to being part of the solution, and as an AI Platform Architect, you can play a crucial role in shaping the architecture of AGCO's AI platform to enable efficient, secure, and confident delivery of AI solutions. Your responsibilities will involve defining the reference architecture for AGCO's AI platform, including AI/ML data pipeline platforms, model training infrastructure, CI/CD for ML, observability, and developer tools. You will design core platform services such as containerized training environments, model registries, and integration interfaces to support AI delivery teams in consuming platform capabilities effectively. Collaboration with Enterprise Architecture, AI PODs, and Product Engineering teams will be key to ensuring interoperability across systems and supporting model deployment across various environments, including cloud, internal APIs, dashboards, and agricultural machinery. To excel in this role, you should have over 10 years of experience in Software, ML infrastructure, or Platform engineering, with a minimum of 3 years in AI platform architecture. Deep expertise in cloud-native technologies like GCP, CI/CD for ML, containerization, and model lifecycle management is essential. Strong systems thinking and architectural design skills are required to design for modularity, scalability, and maintainability. At AGCO, we value diversity, innovation, and personal growth. Benefits include health care and wellness plans, flexible work options, and the opportunity to work with cutting-edge technologies in a globally diverse and inclusive workplace. If you are ready to make a positive impact, contribute to innovative technologies, and help shape the future of agriculture, apply now to join our team at AGCO! Please note that AGCO is an Equal Opportunity Employer, committed to building a diverse workforce that values inclusion and innovation.,

Develop comprehensive High-Level Technical Design and Data Mapping documents to meet specific business integration requirements. Own the data integration and ingestion solutions throughout the project lifecycle, delivering key artifacts such as data flow diagrams and source system inventories. Provide end-to-end delivery ownership for assigned data pipelines, performing cleansing, processing, and validation on the data to ensure its quality. Define and implement robust Test Strategies and Test Plans, ensuring end-to-end accountability for middleware testing and evidence management. Collaborate with the Solutions Architecture and Business analyst teams to analyze system requirements and prototype innovative integration methods. Exhibit a hands-on leadership approach, ready to engage in coding, debugging, and all necessary actions to ensure the delivery of high-quality, scalable products. Influence and drive cross-product teams and collaboration while coordinating the execution of complex, technology-driven initiatives within distributed and remote teams. Work closely with various platforms and competencies to enrich the purpose of Enterprise Integration and guide their roadmaps to address current and emerging data integration and ingestion capabilities. Design ETL/ELT solutions, lead comprehensive system and integration testing, and outline standards and architectural toolkits to underpin our data integration efforts. Analyze data requirements and translate them into technical specifications for ETL processes. Develop and maintain ETL workflows, ensuring optimal performance and error handling mechanisms are in place. Monitor and troubleshoot ETL processes to ensure timely and successful data delivery. Collaborate with data analyst and other stakeholders to ensure alignment between data architecture and integration strategies. Document integration processes, data mappings, and ETL workflows to maintain clear communication and ensure knowledge transfer. What should you have: Bachelor s degree in information technology, Computer Science or any Technology stream 8+ years of working experience with enterprise data integration technologies - Informatica PowerCenter, Informatica Intelligent Data Management Cloud Services (CDI, CAI, Mass Ingest, Orchestration) 5+ years of integration experience utilizing REST and Custom API integration 8+ Years of working Experiences in Relational Database technologies and Cloud Data stores from AWS, GCP & Azure 2+ years of work experience utilizing AWS cloud well architecture framework, deployment & integration and data engineering. Preferred experience with CI/CD processes and related tools including- Terraform, GitHub Actions, Artifactory etc. Proven expertise in Python and Shell scripting, with a strong focus on leveraging these languages for data integration and orchestration to optimize workflows and enhance data processing efficiency Extensive Experience in design of reusable integration pattern using the cloud native technologies Extensive Experience Process orchestration and Scheduling Integration Jobs in Autosys, Airflow. Experience in Agile development methodologies and release management techniques Excellent analytical and problem-solving skills Good Understanding of data modeling and data architecture principles

As a valued member of Infosys Consulting, you will play a crucial role in supporting large Oil & Gas/Utilities prospects by showcasing Infosys" unique value proposition through practical use cases across the value chain. Your responsibilities will include gathering, identifying, and documenting business requirements, as well as creating functional specifications for new systems and processes. Utilizing your expertise in assessing current processes, conducting gap analyses, and designing future processes, you will recommend changes and drive continuous improvement using methodologies such as Six Sigma and Lean. In your role, you will be involved in Technology Project Management, which includes overseeing technology vendors and client stakeholders. You will also manage large projects and programs in a multi-vendor, globally distributed team environment, leveraging Agile principles and DevOps capabilities. Collaboration with the IT Project Management Office will be essential as you support the implementation of client-specific digital solutions, from business case development to IT strategy and tool/software selection. Your expertise in designing and implementing scalable data pipelines, ETL/ELT workflows, and optimized data models across cloud data warehouses and lakes will enable reliable access to high-quality data for business insights and strategic decision-making. You will also be responsible for building and maintaining dashboards, reports, and visualizations using tools like Power BI and Tableau, while conducting deep-dive analyses to evaluate business performance and identify opportunities. Collaboration with business stakeholders to translate strategic objectives into data-driven solutions, defining KPIs, and enabling self-service analytics will be a key aspect of your role. Additionally, you will work closely with client IT teams and business stakeholders to uncover opportunities and derive actionable insights. Participation in internal firm-building activities and supporting sales efforts for new and existing clients through proposal creation and sales presentation facilitation will also be part of your responsibilities. To qualify for this position, you should have at least 3-5 years of experience in data engineering, ideally within the Oil & Gas or Utilities sector. Strong communication skills, both written and verbal, are essential, along with a proven track record in business analysis, product design, or project management. A Bachelor's degree or Full-time MBA/PGDM from Tier 1/Tier 2 B-Schools in India or a foreign equivalent is required. Preferred qualifications include knowledge of digital technologies and agile development practices, as well as the ability to work effectively in a cross-cultural team environment. Strong teamwork, communication skills, and the ability to interact with mid-level managers of client organizations are highly valued. This position is preferred to be located in Electronic City, Bengaluru, but other locations such as Hyderabad, Chennai, Pune, Gurgaon, and Chandigarh are also considered based on business needs. Please note that the job may require extended periods of computer work and communication via telephone, email, or face-to-face interactions.,

As a Data Architect Vice President based in Chennai, you will play a crucial role in designing, developing, and implementing solutions to solve complex business problems. Your primary responsibility will be collaborating with stakeholders to understand their needs and requirements, and designing and implementing solutions that meet those needs while creating solutions that balance technology risks against business delivery. You will be driving consistency in data architecture and platform design, ensuring they align with policy and technical data standards. Your role will involve translating business/use case requirements into logical and physical data models, serving as the foundation for data engineers to build data products. This includes capturing requirements from business teams, translating them into data models while considering performance implications, and testing models with data engineers. Continuous monitoring and optimization of the performance of these models will be essential to ensure efficient data retrieval and processing. You will collaborate with the CDA team to design data product solutions, covering data architecture, platform design, and integration patterns. Additionally, you will work with the technical product lead on data governance requirements, including data ownership of data assets and data quality lineage and standards. Partnering with business stakeholders to understand their data needs and desired functionality for the data product will also be a key aspect of your role. To be successful in this role, you should have experience with cloud platform expertise (specifically AWS), big data technologies such as Hadoop and data warehousing analytics like Teradata and Snowflake processes, SQL/scripting, data governance, and quality. It is crucial to have the ability to engage with business stakeholders, tech teams, and data engineers to define requirements, align data strategies, and deliver high-value solutions. Proven experience leading cross-functional teams to execute complex data architectures is also required. Some additional skills that would be highly valued include advanced cloud services familiarity, data orchestration and automation, performance tuning and optimization, and data visualization. You may be assessed on key critical skills relevant for success in this role, such as risk and controls, change and transformation, business acumen, strategic thinking, and digital and technology, in addition to job-specific technical skills. The purpose of this role is to design, develop, and implement solutions to complex business problems, collaborating with stakeholders to understand their needs and requirements, and design and implement solutions that meet those needs while creating solutions that balance technology risks against business delivery, driving consistency. Your accountabilities will include designing and developing solutions as products that can evolve to meet business requirements aligned with modern software engineering practices and automated delivery tooling. You will need to apply targeted design activities that maximise the benefit of cloud capabilities and adopt standardised solutions where they fit, feeding into their ongoing evolution where appropriate. Additionally, you will provide fault finding and performance issue support to operational support teams, among other responsibilities. As a Vice President, you are expected to contribute or set strategy, drive requirements, and make recommendations for change. If you have leadership responsibilities, you should demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. For individual contributors, you are expected to be a subject matter expert within your own discipline and guide technical direction, leading collaborative multi-year assignments and coaching less experienced specialists. Overall, you are expected to demonstrate the Barclays Values of Respect, Integrity, Service, Excellence, and Stewardship, along with the Barclays Mindset of Empower, Challenge, and Drive in your day-to-day work.,

Strong Integration (CPI) experience. Setup and configuration of CPI (Administration of the BTP environment is assumed to be client responsibility). Independently handles CPI, VCP and all the CPQ-VC integration middleware for S/4 HANA and Infor connectivity. Having worked on SAP C4C and CPQ Implementation projects. Manage a team of 2-3 CPI/BTP consultants. Customer facing , communication good.

As a Data Engineer at WAC, you will be responsible for ensuring the availability, reliability, and scalability of the data infrastructure. Your role will involve collaborating closely with cross-functional teams to support data-driven initiatives, enabling data scientists, analysts, and business stakeholders to access high-quality data for critical decision-making. You will be involved in designing, developing, and maintaining efficient ETL processes and data pipelines to collect, process, and store data from various sources. Additionally, you will create and manage data warehouses and data lakes, optimizing storage and query performance for both structured and unstructured data. Implementing data quality checks, validation processes, and error handling will be crucial in ensuring data accuracy and consistency. Administering and optimizing relational and NoSQL databases to ensure data integrity and high availability will also be part of your responsibilities. Identifying and addressing performance bottlenecks in data pipelines and databases to improve overall system efficiency is another key aspect of the role. Furthermore, implementing data security measures and access controls to protect sensitive data assets will be essential. Collaboration with data scientists, analysts, and stakeholders to understand their data needs and provide support for analytics and reporting projects is an integral part of the job. Maintaining clear and comprehensive documentation for data processes, pipelines, and infrastructure will also be required. Monitoring data pipelines and databases, proactively identifying issues, and troubleshooting and resolving data-related problems in a timely manner are vital aspects of the position. To qualify for this role, you should have a Bachelor's degree in Computer Science, Information Technology, or a related field, with at least 4 years of experience in data engineering roles. Proficiency in programming languages such as Python, Java, or Scala is necessary. Experience with data warehousing solutions and database systems, as well as a strong knowledge of ETL processes, data integration, and data modeling, are also required. Familiarity with data orchestration and workflow management tools, an understanding of data security best practices and data governance principles, excellent problem-solving skills, and the ability to work in a fast-paced, collaborative environment are essential. Strong communication skills and the ability to explain complex technical concepts to non-technical team members are also important for this role. Thank you for your interest in joining the team at Webandcrafts. We look forward to learning more about your candidacy through this application.,

As a Data Engineer at Perch Energy, you will be a key player in the design, development, and maintenance of our data infrastructure and pipelines. Your collaboration with the Data and Analytics Engineering team, as well as engineering and operations teams, will ensure the smooth flow and availability of high-quality data for analysis and reporting purposes. Your expertise will be crucial in optimizing data workflows, ensuring data integrity, and scaling our data infrastructure to support the company's growth. You will have the opportunity to engage with cutting-edge technology, influence the development of a world-class data ecosystem, and work in a fast-paced environment as part of a small, high-impact team. The core data stack at Perch Energy includes Snowflake and dbt Core, orchestrated in Prefect and Argo within our AWS-based ecosystem. Data from a wide range of sources is loaded using Fivetran or Segment, with custom Python utilized when necessary. Your responsibilities will include designing, developing, and maintaining scalable and efficient data pipelines in an AWS environment, focusing on the Snowflake instance and utilizing tools such as Fivetran, Prefect, Argo, and dbt. Collaboration with business analysts, analytics engineers, and software engineers to understand data requirements and deliver reliable solutions will be essential. Additionally, you will design, build, and maintain tooling that facilitates interaction with the data platform, including CI/CD pipelines, testing frameworks, and command-line tools. To succeed in this role, you should have at least 3 years of experience as a Data Engineer, data-adjacent Software Engineer, or member of a small data team, with a strong focus on building and maintaining data pipelines. Proficiency in Python, SQL, database management, and design is required, along with familiarity with data integration patterns, ETL/ELT processes, and data warehousing concepts. Experience with data orchestration tools like Argo, Prefect, or Airflow is a must, along with excellent problem-solving skills and attention to detail. While not mandatory, an undergraduate or graduate degree in a technical field, experience with AWS, Snowflake, Fivetran, Argo, Prefect, dbt, and Github Actions, as well as DevOps practices, would be advantageous. Previous experience in managing enterprise-level data pipelines and working with large datasets or knowledge of the energy sector would also be beneficial. Perch Energy offers competitive compensation, a remote-first policy, flexible leave policy, medical insurance, annual performance cycle, team engagement activities, L&D programs, and a supportive engineering culture that values diversity, empathy, teamwork, trust, and efficiency. Perch Energy is committed to providing reasonable accommodations for individuals with disabilities throughout the job application, interview process, and employment tenure.,

Job Category Customer Success Job Details About Salesforce We're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good - you've come to the right place. The Data Excellence Data Architect is a demonstrated expert in technical and/or functional aspects of customer and partner engagements that lead to the successful delivery of data management projects. The Data Architect plays a critical role for setting customers up for success by prescriptively helping to shape and then execute in the Salesforce data space. This role also provides subject matter expertise related to the data management solutions and ensures successful project delivery. This includes helping identify and proactively manage risk areas, and ensuring issues are seen through to complete resolution as it relates to implementations. Will have the ability to configure and drive solutions to meet the customer's business and technical requirements. Additionally, this role will include helping align on the development of client-specific implementation proposals, SOWs, and staffing plans, engaging with SMEs across the organization to gain consensus on an acceptable proposal, developing best practices within the data excellence community, developing of shared assets. Responsibilities Serve as the Subject Matter Expert for Salesforce data excellence practice Recognized as a valuable and trusted advisor by our customers and other members of Salesforce community and continue to build a reputation for excellence in professional services Lead development of multi-year Data platform capabilities roadmaps for internal business units like Marketing, Sales, Services, and Finance. Facilitate enterprise information & data strategy development, opportunity identification, business cases, technology adoption opportunities, operating model development, and innovation opportunities. Maximize value derived from data & analytics by leveraging data assets through data exploitation, envisioning data-enabled strategies as well as enabling business outcomes through analytics, data & analytics governance, and enterprise information policy. Translating business requirements into technical specifications, including data streams, integrations, transformations, databases, and data warehouses Defining the data architecture framework, standards and principles, including modeling, metadata, security, reference data such as product codes and client categories, and master data such as clients, vendors, materials, and employees Defining data flows, i.e., which parts of the organization generate data, which require data to function, how data flows are managed, and how data changes in transition Design and implement effective data solutions and models to store and retrieve data from different data sources Prepare accurate dataset, architecture, and identity mapping design for execution and management purposes. Examine and identify data structural necessities by evaluating client operations, applications, and programming. Research and properly evaluate new sources of information and new technologies to determine possible solutions and limitations in reliability or usability Assess data implementation procedures to ensure they comply with internal and external regulations. Lead or participate in the architecture governance, compliance, and security activities (architectural reviews, technology sourcing) to ensure technology solutions are consistent with the target state architecture. Partner with stakeholders early in the project lifecycle to identify business, information, technical, and security architecture issues and act as a strategic consultant throughout the technology lifecycle. Oversee the migration of data from legacy systems to new solutions. Preferred Qualifications and Skills: BA/BS degree or foreign equivalent Overall 10+ years of experience in Marketing data & Data management space. Minimum 1 year of hands-on full lifecycle CDP implementation experience on platforms like Salesforce CDP(formerly 360 Audiences), Tealium AudienceStream, Adobe AEP, Segment, Arm Treasure Data, BlueShift, SessionM, RedPoint, etc. 5+ years of experience with data management, data transformation, ETL, preferably using cloud-based tools/infrastructure Experience with Data architecture (ideally with marketing data) using batch and/or real-time ingestion Relevant Salesforce experience in Sales & Service Cloud as well as Marketing Cloud, related certifications is a plus (Marketing Cloud Consultant, Administrator, Advanced Administrator, Service Cloud Consultant, Sales Cloud Consultant, etc.) Experience with Technologies and Processes for Marketing, Personalization, and Data Orchestration. Experience with master data management (MDM), data governance, data security, data quality and related tools desired. Demonstrate deep data integration and/or migration experience with Salesforce.com and other cloud-enabled tools Demonstrate expertise in complex SQL statements and RDBMS systems such as Oracle, Microsoft SQL Server, PostGres Demonstrate experience with complex coding through ETL tools such as Informatica, SSIS, Pentaho, and Talend Knowledge of Data Governance and Data Privacy concepts and regulations a plus Required Skills Ability to work independently and be a self-starter Comfort and ability to learn new technologies quickly & thoroughly Specializes in gathering and analyzing information related to data integration, subscriber management, and identify resolution Excellent analytical & problem-solving skills Demonstrated ability to influence a group audience, facilitate solutions and lead discussions such as implementation methodology, Road-mapping, Enterprise Transformation strategy, and executive-level requirement gathering sessions Travel to client site (up to 50%) Accommodations If you require assistance due to a disability applying for open positions please submit a request via this . Posting Statement Salesforce is an equal opportunity employer and maintains a policy of non-discrimination with all employees and applicants for employment. What does that mean exactly It means that at Salesforce, we believe in equality for all. And we believe we can lead the path to equality in part by creating a workplace that's inclusive, and free from discrimination. Any employee or potential employee will be assessed on the basis of merit, competence and qualifications - without regard to race, religion, color, national origin, sex, sexual orientation, gender expression or identity, transgender status, age, disability, veteran or marital status, political viewpoint, or other classifications protected by law. This policy applies to current and prospective employees, no matter where they are in their Salesforce employment journey. It also applies to recruiting, hiring, job assignment, compensation, promotion, benefits, training, assessment of job performance, discipline, termination, and everything in between. Recruiting, hiring, and promotion decisions at Salesforce are fair and based on merit. The same goes for compensation, benefits, promotions, transfers, reduction in workforce, recall, training, and education.