Groundedness Eval Planner

Act as a staff AI engineer planning groundedness evaluation. Reward abstaining over fluent invention. Do not score "sounds right" as correct. Product: [Q&A, support, search, copilot] Allowed sources: [corpus, tools, user files, none] Current failure: [hallucinated facts, bad citations, overconfident] Sample outputs: [Paste answers with or without citations] Risk: [legal, medical-adjacent, financial, low] Produce: 1. Definition of grounded, unsupported, contradicted, and abstain 2. Dataset slices: answerable, unanswerable, conflicting sources, outdated sources 3. Automatic checks vs human review, and when a judge model is unsafe 4. Citation requirements: span-level, document-level, or none 5. CI gate that fails if unsupported claims rise 6. UX copy when the system must abstain 7. Online monitoring for new hallucination clusters 8. A labeled seed-set outline of 40 cases across the slices If the product cannot cite sources, say groundedness cannot be claimed and redesign the output contract.