Requirement→DAG: recognition-not-generation planning, sinks formalization, live HF Space - #26
Merged
Conversation
…-> Burr Standalone demo (no deps, no model): a closed typed action vocabulary, a plan validator (data deps + guardrails, checked BEFORE execution), Burr ApplicationBuilder codegen, and a mini-Burr runner with an audit trace. Demo 1 shows a naive black-box refund plan rejected statically; demo 2 shows an eligibility-gated plan validating, executing, and emitting the reviewable Burr source. plan_from_nl is the marked stub the probe work replaces.
Generation ladder (SmolLM2-1.7B / Qwen2.5-1.5B / Phi-4-mini-3.8B, N=12): whole-DAG generation is capacity-sensitive (1-2/12 at 1.5-1.7B, 6/12 Phi) and prompt-fragile (a branchy one-shot contaminates: Phi 2->6/12 from a disciplined two-shot). --mode regress replaces generation with symbolic goal-regression over the typed signatures (validity-by-construction, glue insertion solved) but disarms the dep-validator and exposes the abstraction gradient: every model's sinks are WORSE when asked for the goal list than for the full plan. Sink-F1/SEM-correct metrics score the demanded outcomes (per the sinks formalization) instead of glue.
v2, sinks-framed: 201 label-by-construction requirements (54 conditional, half neg-polarity incl. antonym negations), held-out = unseen wordings. 13 logistic heads on mean-pooled mid-stack states, layer picked by wording-transfer (in-distribution CV picks a lexical layer that collapses on synonyms). Held-out: keys set-exact .49/.57 (1.7B/360M) vs lexical .08; check identity .94/.89; polarity 9/9 vs neg-regex 0/9 on the regex-blind antonym subset; decision-table exact .45/.52 with 360M >= 1.7B — recognition is capacity-flat where generation was capacity-gated.
A plan's goals are exactly its sinks (writes no other consumer reads; fork guards consume their check's boolean). Checked mechanically: regress(sinks + checks) == plan and sink-minimality on 12/12 gold plans; corpus-label <-> sink-rule agreement 201/201. Consequences: sinks are the unique minimal generator (the goal list is a normal form, not a compression); goalness is relational (action x plan); polarity is the NL-bound residue; sink-F1 is the honest generation metric.
Live at https://huggingface.co/spaces/jdonaldson/requirement-dag. build_space_probe.py exports the SmolLM2-360M heads as a 254KB npz (deployment heads fit on the full corpus; held-out numbers reported separately) and evaluates clause-span arm assignment: the SAME requirement-level key heads re-scored on span-pooled features resolve which demanded key runs in which fork world 54/54 — the heads are span-composable, no retraining. agentforce_space/ is the standalone Gradio app: one 360M forward on CPU -> numpy heads -> regress/toposort -> clickable solarized SVG DAG (nodes link to verified Salesforce Agentforce docs) -> emitted Burr source, with a calibrated abstain path. Zero tokens generated end to end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Can a small model translate an NL requirement into a typed plan-DAG (Agentforce/Burr framing)?
Includes the merge of main's frame-family arc (both frontier entries kept).