2026 · Shipped
Can This LLM Judge Be Trusted?
A small Python kit that checks whether an LLM judge can be trusted before it grades anything. It judges every pair twice, once in each order, and compares the verdicts with human annotators. Judged on agreement alone, I'd have picked the wrong judge.