A safeguarding safety layer for LLM apps that serve under-18s. Wrap your chat handler, and ward screens each message for genuine safeguarding disclosures, separates them from ordinary bad conduct, and routes the real ones to a designated human on a clock. It is grounded in UK statutory safeguarding practice, not generic content moderation, and it is tuned to keep false alarms low so the humans on the other end keep trusting the alerts.
from ward import Ward, Message
from ward.classifiers import ClaudeClassifier
from ward.escalation import CallbackSink
ward = Ward(
classifier=ClaudeClassifier(), # your Claude key, low temperature
escalation=CallbackSink(page_the_dsl), # where a disclosure goes
)
decision = ward.screen(Message(text=user_message, age_band="13_15"))
if decision.is_escalation:
hand_conversation_to_a_human(decision) # ward routed it; your app respondsMore and more AI reaches children directly: tutors, homework helpers, companion apps. When a child is being hurt, they sometimes say so to the thing that feels safe to tell, and right now that is often a chatbot. In a UK school, recognising and referring that disclosure is a statutory duty with a named process and a named person. Most AI products that reach children have nothing equivalent.
The naive fix is a keyword filter. It does not work, and the way it fails is the whole problem. Match on "suicide" or "county lines" and you escalate the history essay, the novel draft, the song lyric, and "this homework is killing me", while missing the real disclosure that never used a trigger word. Page a Designated Safeguarding Lead on every false alarm and they learn to ignore the alerts, which is worse than having none. ward is built around that precision problem.
ward has two layers, one for each direction of the conversation.
Detection (inbound). For each message from a young person, ward returns one of three decisions:
- clear: ordinary use, including talking about dark topics for legitimate reasons (homework, news, fiction, debating, hyperbole). A heavy word is not a disclosure.
- flag: a conduct concern only (rudeness, bullying language, trying to cheat, oversharing personal data). Worth logging or reviewing. It is not a child-protection matter and does not page anyone.
- escalate: a genuine safeguarding disclosure. ward routes it to your designated human with a category, a confidence, a deadline, and a redacted excerpt for the audit trail.
Separating conduct (flag) from disclosure (escalate) is the core idea.
Optimising escalate for precision, without dropping recall on genuine
disclosures, is what keeps the layer trustworthy.
Response guardrails (outbound). When your app replies to a young person who
has just disclosed, ResponseGuard checks that the reply stays inside safe
safeguarding behaviour: it does not promise confidentiality, does not
interrogate, does not counsel or give medical advice, does not dismiss or
minimise, does not echo harmful method detail, and does signpost real help. ward
checks the reply; it never writes it. See DESIGN.md.
from ward import ResponseGuard
guard = ResponseGuard()
verdict = guard.check(assistant_reply) # before you send it to the child
if not verdict.safe:
... # your app rewrites or withholds; verdict.violations says whypip install -e . # from a clone; PyPI packaging is set up in pyproject
python -m ward.cli demo # tutor-chat demo, no API key needed
python -m ward.cli eval # detection eval, prints real metrics
python -m ward.cli eval-responses # response-guardrail eval, deterministic, no keyThe demo and eval default to a deterministic keyword baseline so they run with no API key. That baseline exists to be beaten (see the numbers below). To use the real classifier:
pip install -e ".[claude]"
export ANTHROPIC_API_KEY=sk-ant-...
python -m ward.cli demo --claude
python -m ward.cli eval --claudeWiring ward into your own app is three lines: build a ClaudeClassifier, choose
an escalation sink (CallbackSink, WebhookSink, or LogSink), and call
ward.screen() on each inbound message. ward returns the decision; your app
decides how to reply. ward never replies for you, never counsels, and never
promises confidentiality.
The detection taxonomy is drawn from UK statutory safeguarding practice, principally Keeping Children Safe in Education (KCSIE), the statutory framework for schools and colleges in England. The categories are the ones that matter for a student-facing AI: self-harm and suicide; the four categories of abuse (physical, emotional, sexual, neglect); child sexual exploitation (CSE); child criminal exploitation and county lines (CCE); radicalisation (the Prevent duty); domestic abuse; and online harms. The escalation model mirrors the KCSIE shape: recognise, respond, record, refer, routing to a Designated Safeguarding Lead. ward automates the recognise and refer steps. The responding and recording stay with the human, on a clock.
The default classifier is an LLM-as-judge: it sends one message plus an
inspectable rubric (prompts.py) to Claude at low
temperature, constrained to a strict JSON schema so it cannot invent a category
or an action. The rubric is deliberately readable and editable. It encodes the
conduct-versus-disclosure split and, above all, the instruction to weigh context
rather than keywords: is this a first-person indication that this child is
actually at risk, or a mention of a heavy topic?
The model defaults to Sonnet, a deliberate choice for a real-time per-message screen where latency and cost matter. Switching to Haiku (cheaper, for very high volume) or Opus (highest recall on genuinely ambiguous disclosures) is a one-argument change. The tradeoff is discussed in DESIGN.md.
The proof lives inside the product. ward ships two synthetic, labelled eval sets and runners that report precision, recall, and the safety-critical miss rate. Every number below comes from actually running the commands. None is hand-written.
python -m ward.cli eval --set {core,challenge,all}, positive class escalate.
The core set (65 cases) is the development set: direct and indirect
disclosures across every category, plain traps, conduct, and benign chat. The
challenge set (34 cases) is held out and adversarial. It was written after the
rubric, to break it: obfuscated disclosures in the euphemisms young people use to
dodge filters ("unaliving", "kms", "sewerslide"), disclosures buried in an
otherwise cheerful message, third-party reports, and traps that name a heavy topic
for a legitimate reason (coursework fiction, song lyrics, past-tense recovery,
academic method questions, figures of speech). See METHODOLOGY.md
for how the sets are built and where the labels are contestable.
Every number here comes from running the commands. None is hand-written.
| keyword baseline | Claude judge (Sonnet 4.6) | |
|---|---|---|
| core recall | 50.0% | 90.0% |
| core precision | 83.3% | 100.0% |
| core false-positive rate | 8.6% | 0.0% |
| challenge recall | 6.2% | 100.0% |
| challenge precision | 20.0% | 100.0% |
| challenge false-positive rate | 22.2% | 0.0% |
Two things to read here, and the second is the one I did not expect.
First, the challenge set does to the keyword baseline exactly what it was built to do. Recall falls from 50 percent to 6.2 percent: of sixteen obfuscated disclosures it catches one. Keyword matching has no idea that "unaliving myself" or "said my goodbyes" is a disclosure, and it still fires on the academic and fiction traps it does recognise. On the euphemistic self-harm cases its per-category recall is about 9 percent. This is the realistic distribution, and it is where a filter dies.
Second, the challenge set did not break the judge. It scored 100 percent on it. That was not what I expected. A frontier model reads "i've started wearing long sleeves so nobody asks about my arms" as a disclosure and "i'd 'kill' for a nap" as a figure of speech, and the lexical disguise that destroys a keyword filter does not slow it down. The judge's only misses are three cases, all in the core set, and all genuinely arguable: teenage extremist rhetoric, a bomb request framed as schoolwork, and a child reporting a friend's bruises. Its failures have nothing to do with disguised language. They are the few cases where the safeguarding policy itself is contestable, and where the labels in this set are too.
That 100 percent is more a caveat about the eval than a claim about the product. A single author writing adversarial cases can make them lexically hard but struggles to make them semantically hard enough to fool a strong model, so 100 percent on 34 cases says as much about the ceiling of one-author case-writing as it does about the judge. The sets are small (a 16-case category has wide error bars), synthetic, and single-author. The next step is a larger set with cases labelled by more than one person. Reproduce any of this:
python -m ward.cli eval --set challenge # keyword baseline
ANTHROPIC_API_KEY=... python -m ward.cli eval --set challenge --claude # the judgeAdd --json for per-category recall. A 65-case judge run costs about $0.30 on
Sonnet, $0.10 on Haiku.
python -m ward.cli eval-responses, positive class unsafe. Each case is a
synthetic assistant reply to a disclosure, labelled safe or unsafe. The harder
cases include a reply that signposts help but still promises secrecy, one that
signposts but interrogates, and one that advises confronting the abuser.
recall (unsafe) 100.0% [18/18 unsafe replies caught]
precision 100.0% [18/18 flags were really unsafe]
accuracy 100.0%
Read this honestly: a deterministic rule set scoring 100% on a 30-case set whose author also wrote the rules proves the rules fire on the cases they were designed for. It is not evidence of generalisation, and the number will drop on cases with harms the rules do not enumerate (a novel minimiser, an unusual piece of bad advice). It is here because it is real and reproducible, and because the one case that first failed was instructive: the guard wrongly flagged the correct safeguarding line "I can't keep this between us" until it was taught to respect the negation. That is the same disclosure-versus-mention brittleness the whole project is about, showing up one layer down. The honest next step is an LLM judge on the response layer and a larger held-out set.
Every number above is about escalations, and an escalation is a late signal. By the time a child writes the disclosure down, the thing being disclosed has already happened, and ward's job has narrowed to getting a human there quickly. That is worth a lot. It is not early.
Conduct is earlier. Swearing at the bot, one-word answers from someone who used
to write paragraphs, testing what it will say, oversharing: cheap to observe,
awkward to fake, and it moves before a disclosure does. That is why flag has
its own sink and never pages anyone. One flag on its own is close to meaningless
and should not interrupt a DSL's afternoon. A user_ref whose flag rate climbs
over a fortnight with no escalation in it is the signal you actually wanted, and
you only ever see it if something was counting.
ward = Ward(
classifier=ClaudeClassifier(),
escalation=CallbackSink(page_the_dsl), # interrupts a human
flag_sink=CallbackSink(write_a_row), # interrupts nobody; count these
)ward does not do the counting. It is stateless per message, with no dashboard
and no database by design. It stamps user_ref and session_id on flags so the
integrator can.
The same split applies to the humans on the other end. Alert fatigue is the
failure this whole design is built against, and "the DSL stopped trusting ward"
is something you find out late, usually from an escalation nobody opened. The
early version of it is countable: how long acknowledgement takes measured
against the respond_by stamp, and what share of escalations get closed with no
action. Both are computable from the payload the sink already receives. Neither
is something ward does for you.
ward supersedes and absorbs the safeguarding work from an earlier repo, claude-eval-toolkit. That repo was a broad Claude evaluation framework with a UK education safeguarding module. ward takes the parts that mattered (the KCSIE grounding, the response-level safeguarding cases and their must-signpost / must-not-promise-secrecy methodology) and rebuilds them as a focused, typed, tested product on the current KCSIE 2025, dropping the generic framework surface. If you are looking at both, this is the one that is maintained.
Everything in this repository is synthetic. There is no real student data, no real disclosure, and no data from any real service anywhere in the eval set or the examples. The realism comes from the taxonomy and from the shape of the false-positive traps, not from real children's words. This is a public safeguarding tool, and it is built so that it never has to touch a real child's disclosure to prove that it works. ward also stores no message content by default. An escalation carries a redacted excerpt and a category, not a transcript, and integrators configure their own retention for any audit trail they keep.
- ward routes to humans. It is not a clinician. It recognises and refers. It does not assess, diagnose, counsel, or treat, and it must not be used as a substitute for a safeguarding professional or a crisis service.
- Redaction is best-effort, not a guarantee. ward masks obvious identifiers (emails, phone numbers, long digit runs) in excerpts. It does not catch names or context that could identify a child. Handling real disclosures brings data-protection duties that are the integrator's responsibility.
- The judge is a model, and models are wrong sometimes. ward fails safe: on a model or parse error it routes for manual review rather than silently clearing. Recall on genuine disclosures is the metric that must not regress, and the eval set exists so you can measure it before and after any change.
- Statutory guidance changes. KCSIE is revised roughly annually. ward cites frameworks by name and marks anything version-specific for review. It is not legal advice, and the taxonomy should be checked against the current edition before you rely on it.
- A larger eval set with cases labelled by more than one person (the current limit, see METHODOLOGY). Semantically harder challenge cases, not just lexically harder ones.
- An LLM judge for the response layer, alongside the deterministic guard.
- A pluggable rules layer so integrators can add local policy alongside the judge.
- Configurable per-category severity and deadlines.
- A worked example of the flag-trend and acknowledgement-latency side: ward emits the fields, counts none of them, and an integrator should not have to invent the queries.
- Prompt caching on the judge to cut the repeated system-prompt cost.
MIT. See LICENSE.