5 - 9 years
10 - 20 Lacs
Posted:1 week ago|
Platform:
Hybrid
Full Time
Design and implement end-to-end QA strategies for applications using Node.js, integrated with LLMs, retrieval-augmented generation (RAG), and Agentic AI workflows.
• Establish comprehensive benchmarks and quality metrics for GenAI components including accuracy, coherence, relevance, stability, and safety.• Develop structured evaluation datasheets for LLM behaviour validation: test prompts, expected responses, classification criteria, and scoring rubrics.• Perform data quality testing for RAG databases and ensure relevant, high-quality retrieval to minimize hallucinations and improve grounding.• Conduct A/B testing across model versions, prompt designs, and system configurations to measure and compare output quality.• Define methodologies and simulate non-deterministic behaviours using Agentic AI testing techniques.• Collaborate closely with developers, product owners, and AI engineers to test prompt engineering pipelines, function-calling interfaces, and fallback logic.• Build QA automation where applicable and integrate GenAI evaluations into CI/CD pipelines.• Lead internal capability development by mentoring QA peers on GenAI testing practices and helping evolve the organizations AI quality maturity.
6+ years of experience in software quality assurance, with at least 3+ years working in or around GenAI or LLM-based systems.
• Deep understanding of GenAI quality dimensions: response grounding, factual correctness, context awareness, and hallucination minimization.• Experience creating and maintaining LLM evaluation datasets and designing test cases for dynamic prompt behaviour.• Hands-on experience with tools and techniques for testing retrieval pipelines, embedding quality, and vector similarity results in RAG architectures.• Familiarity with non-deterministic testing strategies, agent loop evaluation, and multi-step LLM task validation.• Comfortable working with APIs, logs, test scripts, and tracing tools to validate both system and AI behaviour.• Strong analytical thinking and a methodical approach to identifying bugs, regressions, and inconsistencies in AI outputs.
Kongsberg Software And Services
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Salary: Not disclosed
10.0 - 20.0 Lacs P.A.
New Delhi, Gurugram
5.5 - 7.0 Lacs P.A.
14.0 - 24.0 Lacs P.A.
6.0 - 10.0 Lacs P.A.
8.0 - 13.0 Lacs P.A.
Experience: Not specified
4.0 - 7.0 Lacs P.A.
Bengaluru
9.0 - 10.0 Lacs P.A.
Ahmedabad
5.0 - 7.0 Lacs P.A.
Bengaluru
4.0 - 10.0 Lacs P.A.