RAG Evaluation Dashboard

Benchmark metrics comparing system answers against curated expected output.

📊

Live System Benchmark Evaluation

The metrics shown below are calculated directly by executing standard benchmark Q&A pairs against the active database index. Report log: data/benchmark/results_sample.json. Last evaluated: Fri, 03 Jul 2026 16:08:18 GMT.

Benchmark Questions

questions.json

Top-1 Source Hit Rate

20.0%

target doc in rank 1

Top-3 Source Hit Rate

35.0%

target doc in top 3

Asset Hit Rate

100.0%

target tag resolved

Average Latency

100.6ms

embedding + retrieval

Failed Questions

below context threshold

Benchmark Run Items

Click on any row to inspect expected vs. actual answers

ID	Category	Target Doc	Top 1	Top 3	Asset Hit	Latency	Status
Q01	equipment spec	pump_oem_manual.pdf	❌	❌	✅	300ms	failed
Q02	spare parts	pump_oem_manual.pdf	❌	✅	✅	96ms	passed
Q03	spare parts	pump_oem_manual.pdf	❌	✅	✅	95ms	passed
Q04	failure mode	pump_oem_manual.pdf	❌	❌	✅	97ms	failed
Q05	procedure	sop_pump_startup.pdf	❌	❌	✅	97ms	failed
Q06	procedure	sop_pump_startup.pdf	❌	❌	✅	108ms	failed
Q07	inspection finding	inspection_report_q1_2025.pdf	❌	❌	✅	94ms	failed
Q08	compliance gap	inspection_report_q1_2025.pdf	✅	✅	✅	94ms	passed
Q09	compliance gap	inspection_report_q1_2025.pdf	❌	❌	✅	97ms	failed
Q10	compliance gap	compliance_checklist_2025.pdf	✅	✅	✅	96ms	passed
Q11	regulatory	compliance_checklist_2025.pdf	❌	❌	✅	92ms	failed
Q12	maintenance history	work_orders_clean.csv	✅	✅	✅	91ms	passed
Q13	asset timeline	work_orders_clean.csv	❌	✅	✅	96ms	passed
Q14	asset lookup	sop_compressor_operations.pdf	❌	❌	✅	95ms	failed
Q15	knowledge graph_relationship	compliance_checklist_2025.pdf	❌	✅	✅	93ms	passed
Q16	failure mode	pump_oem_manual.pdf	❌	❌	✅	94ms	failed
Q17	equipment spec	compressor_datasheet.xlsx	❌	❌	✅	94ms	failed
Q18	maintenance history	work_orders_clean.csv	❌	✅	✅	91ms	passed
Q19	failure/RCA	rca_findings_2025.csv	❌	❌	✅	94ms	failed
Q20	regulatory	compliance_checklist_2025.pdf	❌	❌	✅	94ms	failed
Q21	knowledge graph_relationship	work_orders_clean.csv	✅	✅	✅	98ms	passed
Q22	citation accuracy	compliance_checklist_2025.pdf	❌	❌	✅	100ms	failed
Q23	compliance gap	inspection_report_q1_2025.pdf	❌	❌	✅	106ms	failed
Q24	procedure	sop_compressor_operations.pdf	❌	❌	✅	93ms	failed
Q25	regulatory	compliance_checklist_2025.pdf	❌	❌	✅	92ms	failed
Q26	equipment spec	compressor_datasheet.xlsx	❌	❌	✅	95ms	failed
Q27	asset timeline	inspection_report_q1_2025.pdf	❌	❌	✅	91ms	failed
Q28	equipment spec	pump_oem_manual.pdf	✅	✅	✅	90ms	passed
Q29	procedure	sop_pump_startup.pdf	❌	❌	✅	91ms	failed
Q30	multi-document reasoning	compliance_checklist_2025.pdf	❌	✅	✅	91ms	passed
Q31	multi-document reasoning	compressor_datasheet.xlsx	❌	❌	✅	93ms	failed
Q32	citation accuracy	pump_oem_manual.pdf	✅	✅	✅	90ms	passed
Q33	spare parts	compressor_datasheet.xlsx	❌	❌	✅	91ms	failed
Q34	asset timeline	inspection_report_q1_2025.pdf	❌	❌	✅	90ms	failed
Q35	asset lookup	work_orders_clean.csv	✅	✅	✅	92ms	passed
Q36	failure/RCA	rca_findings_2025.csv	❌	❌	✅	99ms	failed
Q37	equipment spec	pump_oem_manual.pdf	✅	✅	✅	98ms	passed
Q38	procedure	sop_pump_startup.pdf	❌	❌	✅	94ms	failed
Q39	knowledge graph_relationship	inspection_report_q1_2025.pdf	❌	❌	✅	97ms	failed
Q40	compliance gap	compliance_checklist_2025.pdf	❌	❌	✅	107ms	failed

Run Inspector

Detailed retrieval comparison

📋

Select any question row on the left to inspect expected answers, extracted entities, and actual grounding text results.