Answer-level evals: faithfulness, groundedness, relevance

Claim-verification lab: L6.1 split the pipeline and found 6 of 30 questions with a complete context and a wrong answer anyway, and a prompt rewrite that moved the answer score ten points with every retrieval number byte-identical. This lesson grades the answer. It cuts 30 answers into 113 claims and checks each against the passage it cites: 92 are grounded, 5 are contradicted by their own citation, 9 have no supporting passage in the window, 5 cite the wrong passage and 2 cite nothing. Groundedness is 81.4% while citation presence is 98.2%, and 92 of the 111 cited claims hold at 82.9% precision. Correctness and faithfulness cross four ways with every cell occupied — 13 both, 1 faithful-but-wrong, 7 correct-but-unfaithful, 9 neither. A lexical judge traces one curve, presence 1.00/0.15, overlap_50 0.95/0.60, overlap_80 0.85/0.95, overlap_95 0.75/1.00, and only 0.8 clears both M5.3 floors. The three gates fail 16, 12 and 1 answers, and the 18 failures resolve to 5 fabrications, 4 unanswerable, 5 misattributions, 2 uncited, 1 irrelevant and 1 incomplete.

This Evals Engineering lesson is part of a paid plan. Sign in with a plan that includes this course to read it.