Who we are looking for
We are seeking a highly motivated and detail-oriented individual to lead the QA and Support function for the AI Center of Excellence (CoE) shared services. The ideal candidate will bring a blend of technical depth, hands-on quality assurance expertise, and operational support experience. You will play a critical role in validating the reliability, performance, and compliance of AI solutions and ensuring smooth operations and ongoing support across the AI CoE platform and services.
This role can be performed in a hybrid model, where you can balance work from home and office to match your needs and role requirements.
What you will be responsible for
As Artificial Intelligence Developer you will:
As Lead – AI COE Shared Service QA and Support, you will:
-
Lead the end-to-end QA strategy, framework, and execution for enterprise AI services including Prompt Hub, RAG-as-a-Service, Fine-tuning Studio, and Agents Marketplace.
-
Design and maintain test automation suites and validation pipelines for GenAI-based microservices, APIs, and data pipelines.
-
Establish procedures for model output validation, performance benchmarking, and guardrail enforcement (e.g., hallucination detection, grounding checks, bias tests).
-
Own the L2/L3 support processes for GenAI services; manage ticketing queues, triage incidents, and drive root-cause analysis with Engineering.
-
Coordinate UAT cycles, release readiness, and production rollouts with product managers, platform teams, and DevOps.
-
Develop and maintain monitoring dashboards, observability metrics, and alerting mechanisms to ensure production stability and SLA adherence.
-
Act as the primary interface for audit, compliance, and InfoSec queries related to QA artifacts and operational support.
-
Maintain comprehensive documentation and knowledge base for support workflows, known issues, and remediation steps.
What we value
These skills will help you succeed in this role:
-
Hands-on experience with QA practices and tools across software and ML/AI pipelines (e.g., pytest, unittest, Postman, JMeter, LangSmith, MLflow).
-
Strong working knowledge of GenAI lifecycle components – prompt testing, model evaluation, RAG, embedding validation, etc.
-
Familiarity with RESTful APIs, container orchestration (Docker/Kubernetes), and cloud environments (Azure, AWS).
-
Experience setting up automated CI/CD pipelines for test execution, using tools such as GitLab CI, Jenkins, or Azure DevOps.
-
Understanding of governance frameworks for AI, including Responsible AI principles and audit requirements.
-
Strong analytical and communication skills; ability to work cross-functionally with engineers, product owners, and risk managers.
-
Proven leadership in handling production incidents, managing on-call schedules, and delivering continuous improvements.
Education & Preferred Qualifications
- B.Tech/M.Tech/MCA or equivalent in Computer Science, Engineering, or related fields.
- 17 + years of experience in QA and/or production support, preferably in data/AI/ML platforms.
- Experience with ITIL or equivalent service management framework is a plus.
- GenAI/ML certification or experience working in a Center of Excellence setup preferred
Additional requirements
Are you the right candidate? Yes!
We truly believe in the power that comes from the diverse backgrounds and experiences our employees bring with them. Although each vacancy details what we are looking for, we don’t necessarily need you to fulfil all of them when applying. If you like change and innovation, seek to see the bigger picture, make data driven decisions and are a good team player, you could be a great fit.
Why this role is important to us
Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients. We’re driving the company’s digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation.
We offer a collaborative environment where technology skills and innovation are valued in a global organization. We’re looking for top technical talent to join our team and deliver creative technology solutions that help us become an end-to-end, next-generation financial services company.
Join us if you want to grow your technical skills, solve real problems and make your mark on our industry.
