AI Engineer ( Quality ,Performance)

Job Title: AI Engineer Quality & Performance

Job Description: We are seeking a highly skilled AI Engineer - Quality & Performance to

ensure the accuracy, robustness, and ethical integrity of AI-generated responses. In this

role, you will leverage AI validation frameworks, adversarial testing tools, and

responsible AI principles to assess and enhance AI systems. If you are passionate about

AI security, performance evaluation, and responsible AI practices, wed love to hear

from you!

Key Responsibilities:

• AI Response Validation: Evaluate AI-generated responses for accuracy,

coherence, relevance, and alignment with defined standards.

• AI Red Teaming: Utilize tools such as Pyrit, NVIDIA Garak, Giskard AI, Attest, and

TruLens to conduct adversarial testing and identify vulnerabilities, biases, or

security risks in AI models.

• AI Ground Truth Check: Develop and refine validation datasets using frameworks

like PromptFlow, Cerberus, CheckList, and FairBench to ensure factual integrity

and reliability of AI outputs.

• AI Performance Evaluation: Implement benchmarking strategies using tools such

as OpenAI Eval, DeepEval, Dynabench, and LM Harness to measure AI model

efficiency, robustness, and responsiveness.

• AI Quality Assurance: Apply automated validation frameworks such as

DeepChecks, Great Expectations, and Robustness Gym to maintain high

standards in AI response generation.

• Responsible AI Validation: Leverage tools like Fairlearn, Aequitas, and AI Fairness

360 to assess AI fairness, mitigate biases, and enforce ethical AI principles.

• Prompt Rewriting & Validation: Analyze and refine AI prompts using PromptFoo

and Helium to optimize response quality and consistency across diverse

applications.

Qualifications:

• Education: Bachelors or Masters degree in Computer Science, AI, Engineering,

or a related field (or equivalent practical experience).

• Experience:
• 2-8 years of experience in AI engineering, software development, or

automation testing.

• Minimum 1 year of experience working with red teaming frameworks

(e.g., Pyrite, Attest) and AI validation tools (e.g., Fairlearn, DeepChecks).

• Strong understanding of AI security, fairness principles, and adversarial

testing methodologies.

• Technical Skills:
• Proficiency in programming languages such as Python, JavaScript, or

relevant AI evaluation scripting.

• Experience in AI bias detection, robustness testing, and prompt

engineering.

• Familiarity with machine learning workflows, AI policy frameworks, and

ethical AI implementation.

Key Competencies:

• Strong analytical and problem-solving skills in AI validation and red teaming.
• Attention to detail when assessing AI-generated responses.
• Creativity and critical thinking in designing and refining AI prompts.
• Excellent written and verbal communication skills for reporting AI evaluation

results.

• Ability to work in a fast-paced, collaborative environment with multidisciplinary

teams.

Why Join Us?

• Work with cutting-edge AI validation frameworks to shape the future of

responsible AI.

• Engage in AI security, fairness assessments, and red teaming to enhance AI

resilience.

• Be part of an innovative and dynamic team at the forefront of AI model

evaluation.

• Competitive compensation package with strong career growth opportunities.