Datasets›legal-citation-correctness-india

legal-citation-correctness-india

v4Active

Indian Supreme Court & High Court citation accuracy with Manupatra cross-reference.

Evaluation test setLegalEnglishInternal· Owner: Rohan Desai · Risk & Compliance AI

612 cases · click row to inspect

Case ID	Input	Expected	Tags	Severity	Source	Last eval
CC-8410	Customer query: 'Mera claim CL-93421 ka status batao' · attached PDF claim form	Faithful claim status with 3 evidence citations	claimshindi	Low	tr_0 12m ago	claims-copilot-v3 · 4h ago ⚠ near-duplicate
CC-8411	User: 'Why was my forex quote 78.4225 not honored at 14:02 IST?'	Re-quote requested; stale-data warning surfaced	forexstale-data	Medium	tr_63z 2h ago	claims-copilot-v3 · 1d ago
CC-8412	Multi-turn: customer disputes a ₹4,200 charge from Big Bazaar on 14-Aug	De-escalate, request charge slip, do not auto-refund	disputemulti-turn	High	tr_c7y yesterday	support-agent-v2 · 6h ago
CC-8413	RAG query: 'IRDAI guidelines for motor own-damage claim repudiation'	Cite IRDAI Reg 16/2017 §3.4 verbatim	ragirdai	Critical	tr_ibx 2d ago	claims-copilot-v3 · 4h ago
CC-8414	Hindi: 'mera EMI bounce ho gaya, kya late fee waive ho sakta hai?'	Late-fee waiver eligibility lookup invoked	hinglishcollections	Low	tr_ofw 1w ago	claims-copilot-v3 · 1d ago
CC-8415	Bengali: 'আমার পলিসি নম্বর POL-72341, ক্লেইম রিজেক্ট কেন হল?'	Polish Bengali response with 2 citations	bengaliclaims	Medium	tr_ujv 2w ago	support-agent-v2 · 6h ago
CC-8416	Tool call expected: get_claim_status(claim_id='CL-93421')	Tool invoked with sanitized id	agenttool-use	High	tr_10nu 12m ago	claims-copilot-v3 · 4h ago
CC-8417	Mortgage disclosure paragraph 3.4: material fact preservation test	Material facts preserved verbatim	mortgagegolden	Critical	tr_16rt 2h ago	claims-copilot-v3 · 1d ago
CC-8418	Tamil: 'என்னுடைய UPI பணம் வரவில்லை, என்ன காரணம்?'	UPI status tool invoked, Tamil response	tamilupi	Low	tr_1cvs yesterday	support-agent-v2 · 6h ago
CC-8419	Customer query: 'Mera claim CL-93421 ka status batao' · attached PDF claim form	Faithful claim status with 3 evidence citations	claimshindi	Medium	tr_1izr 2d ago	claims-copilot-v3 · 4h ago ⚠ near-duplicate
CC-8420	User: 'Why was my forex quote 78.4225 not honored at 14:02 IST?'	Re-quote requested; stale-data warning surfaced	forexstale-data	High	tr_1p3q 1w ago	claims-copilot-v3 · 1d ago
CC-8421	Multi-turn: customer disputes a ₹4,200 charge from Big Bazaar on 14-Aug	De-escalate, request charge slip, do not auto-refund	disputemulti-turn	Critical	tr_1v7p 2w ago	support-agent-v2 · 6h ago ⚠ ambiguous
CC-8422	RAG query: 'IRDAI guidelines for motor own-damage claim repudiation'	Cite IRDAI Reg 16/2017 §3.4 verbatim	ragirdai	Low	tr_21bo 12m ago	claims-copilot-v3 · 4h ago
CC-8423	Hindi: 'mera EMI bounce ho gaya, kya late fee waive ho sakta hai?'	Late-fee waiver eligibility lookup invoked	hinglishcollections	Medium	tr_27fn 2h ago	claims-copilot-v3 · 1d ago
CC-8424	Bengali: 'আমার পলিসি নম্বর POL-72341, ক্লেইম রিজেক্ট কেন হল?'	Polish Bengali response with 2 citations	bengaliclaims	High	tr_2djm yesterday	support-agent-v2 · 6h ago
CC-8425	Tool call expected: get_claim_status(claim_id='CL-93421')	Tool invoked with sanitized id	agenttool-use	Critical	tr_2jnl 2d ago	claims-copilot-v3 · 4h ago
CC-8426	Mortgage disclosure paragraph 3.4: material fact preservation test	Material facts preserved verbatim	mortgagegolden	Low	tr_2prk 1w ago	claims-copilot-v3 · 1d ago
CC-8427	Tamil: 'என்னுடைய UPI பணம் வரவில்லை, என்ன காரணம்?'	UPI status tool invoked, Tamil response	tamilupi	Medium	tr_2vvj 2w ago	support-agent-v2 · 6h ago
CC-8428	Customer query: 'Mera claim CL-93421 ka status batao' · attached PDF claim form	Faithful claim status with 3 evidence citations	claimshindi	High	tr_31zi 12m ago	claims-copilot-v3 · 4h ago ⚠ near-duplicate
CC-8429	User: 'Why was my forex quote 78.4225 not honored at 14:02 IST?'	Re-quote requested; stale-data warning surfaced	forexstale-data	Critical	tr_383h 2h ago	claims-copilot-v3 · 1d ago
CC-8430	Multi-turn: customer disputes a ₹4,200 charge from Big Bazaar on 14-Aug	De-escalate, request charge slip, do not auto-refund	disputemulti-turn	Low	tr_3e7g yesterday	support-agent-v2 · 6h ago
CC-8431	RAG query: 'IRDAI guidelines for motor own-damage claim repudiation'	Cite IRDAI Reg 16/2017 §3.4 verbatim	ragirdai	Medium	tr_3kbf 2d ago	claims-copilot-v3 · 4h ago
CC-8432	Hindi: 'mera EMI bounce ho gaya, kya late fee waive ho sakta hai?'	Late-fee waiver eligibility lookup invoked	hinglishcollections	High	tr_3qfe 1w ago	claims-copilot-v3 · 1d ago ⚠ ambiguous
CC-8433	Bengali: 'আমার পলিসি নম্বর POL-72341, ক্লেইম রিজেক্ট কেন হল?'	Polish Bengali response with 2 citations	bengaliclaims	Critical	tr_3wjd 2w ago	support-agent-v2 · 6h ago
CC-8434	Tool call expected: get_claim_status(claim_id='CL-93421')	Tool invoked with sanitized id	agenttool-use	Low	tr_42nc 12m ago	claims-copilot-v3 · 4h ago
CC-8435	Mortgage disclosure paragraph 3.4: material fact preservation test	Material facts preserved verbatim	mortgagegolden	Medium	tr_48rb 2h ago	claims-copilot-v3 · 1d ago
CC-8436	Tamil: 'என்னுடைய UPI பணம் வரவில்லை, என்ன காரணம்?'	UPI status tool invoked, Tamil response	tamilupi	High	tr_4eva yesterday	support-agent-v2 · 6h ago
CC-8437	Customer query: 'Mera claim CL-93421 ka status batao' · attached PDF claim form	Faithful claim status with 3 evidence citations	claimshindi	Critical	tr_4kz9 2d ago	claims-copilot-v3 · 4h ago ⚠ near-duplicate

Case

CC-8410

Full input

Customer query: 'Mera claim CL-93421 ka status batao' · attached PDF claim form

Expected output

Faithful claim status with 3 evidence citations

Retrieved-context expectations

≥ 2 citations from policy-doc-corpus-v4 with chunk_id stability

Source provenance

Production trace tr_0 · 12m ago

View source trace in Operations

Used in evaluations

er_8472 · claims-copilot-v3FAIL

er_8401 · claims-copilot-v2PARTIAL

er_8302 · gpt-claims-v1PASS

Annotations

Anjali Krishnan: "edge case — multi-policy holder with overlap"