Evaluation framework for LLM knowledge inputs — prompts, RAG corpora, skills, agent workflows. Fix the model, vary the artifact. Built-in statistical rigor: bootstrap CI, Krippendorff α, length-debias, saturation curves.

A reliability and DIF report card for LLM-judge and human-rater scoring instruments.

Tool-agnostic inter-coder reliability (Krippendorff alpha, Cohen/Fleiss kappa) and disagreement adjudication for qualitative coding

Evaluation toolkit for multi-annotator human annotation research with agreement metrics, disagreement analysis, reports, and plots.

Browser-based inter-rater reliability calculator for systematic literature reviews. Computes Krippendorff's Alpha using a pooled coincidence matrix across all (Paper, RQ, Field) units. No installation required — single HTML file, fully client-side. Built for the GenAI Evidence Hub at Learning Data Insights.

Classificazione dei commenti YouTube di Breaking Italy e La Repubblica tramite il modello User Needs (Shishkin & SmartOcto, 2021). Tesi magistrale in Comunicazione, ICT e Media — UniTO.

Measure how much your LLM judges actually agree. Inter-judge agreement metrics for LLM-as-a-judge evaluations.