AI - QA Engineer

About the job

What makes Techjays an inspiring place to work

At Techjays, we are driving the future of artificial intelligence with a bold mission to empower businesses worldwide by helping them build AI solutions that transform industries. As an established leader in the AI space, we combine deep expertise with a collaborative, agile approach to deliver impactful technology that drives meaningful change.

Our global team consists of professionals who have honed their skills at leading companies such as Google, Akamai, NetApp, ADP, Cognizant Consulting, and Capgemini. With engineering teams across the globe, we deliver tailored AI software and services to clients ranging from startups to large-scale enterprises.

Be part of a company that’s pushing the boundaries of digital transformation. At Techjays, you’ll work on exciting projects that redefine industries, innovate with the latest technologies, and contribute to solutions that make a real-world impact. Join us on our journey to shape the future with AI.

AI QA Engineer

Primary Skills:

Secondary Skills:

Experience:

Work Location:

Must-Have Skills:

Foundational QA Skills

• Strong knowledge of

  test design, defect management, and QA lifecycle

  .
• Experience with

  risk-based testing

  and QA strategy.

AI/ML Knowledge

• Basic understanding of

  machine learning workflows

  , training/inference cycles.
• Awareness of

  AI quality challenges

  : bias, fairness, transparency.
• Familiarity with AI evaluation metrics:

  accuracy, precision, recall, F1-score

  .
• Hands-on with

  prompt testing

  ,

  synthetic data generation

  , and non-deterministic behavior validation.

Technical Capabilities

• Python programming

  for test automation and data validation.
• Hands-on experience with

  API testing

  tools (Postman, Swagger, REST clients).
• Knowledge of test automation tools (e.g.,

  PyTest

  , Playwright, Selenium).
• Familiarity with

  Git

  and version control best practices.
• Understanding of

  CI/CD pipelines

  and integration testing.

Tooling (Preferred)

• Tools like

  Diffblue, Testim, Applitools, Kolena, Galileo, MLFlow, Weights & Biases

  .
• Basic understanding of

  cloud-based AI platforms

  (AWS Sagemaker, Azure ML, GCP Vertex AI).

Soft Skills

• Excellent

  analytical thinking

  and attention to detail.
• Strong

  collaboration and communication

  skills to work across cross-functional teams.
• Proactive and

  pull-mode work ethic

  —self-starter who takes ownership.
• Passion for learning new technologies and contributing to AI quality practices.

Roles & Responsibilities:

• Design, write, and execute

  test plans

  and

  test cases

  for AI/ML-based applications.
• Collaborate with data scientists, ML engineers, and developers to understand model behavior and expected outcomes.
• Perform

  functional, regression, and exploratory testing

  on AI components and APIs.
• Validate model outputs for

  accuracy, fairness, bias, and explainability

  .
• Implement and run

  adversarial testing

  , edge cases, and out-of-distribution data scenarios.
• Conduct

  prompt testing and evaluation

  for LLM (Large Language Model)-based applications.
• Use

  LLM-as-a-Judge

  and AI tools to automate evaluation of AI responses where possible.
• Validate

  data pipelines

  , datasets, and ETL workflows.
• Track model performance metrics such as

  precision, recall, F1-score

  , and flag potential degradation.
• Document defects, inconsistencies, and raise risks proactively with the team.

What we offer:

• Best in packages
• Paid holidays and flexible paid time away
• Casual dress code & flexible working environment
• Medical Insurance covering self & family up to 4 lakhs per person.
• Work in an engaging, fast-paced environment with ample opportunities for professional development.
• Diverse and multicultural work environment
• Be part of an innovation-driven culture that provides the support and resources needed to succeed.