Add workflow admission cards before similarity-based template reuse

[sunghunkwag]

Motivation

Mimosa-AI already has a strong Darwinian workflow loop and several open issues around mutation, evaluation, pattern memory, and program-based evaluators. This issue is intentionally narrower: add an explicit admission record before past workflows are reused as templates for new tasks.

The relevant path I see is:

• sources/core/workflow_selection.py discovers valid workflow folders, loads code and state, then selects reusable workflows by task similarity and overall_score.
• WorkflowSelector.select_best_workflows(...) filters by similarity and score.
• sources/core/dgm.py::select_workflow_template(...) selects the top candidate workflow when template_uuid is not provided.
• sources/core/dgm.py::self_improve_workflow(...) feeds the selected best workflow back into the next improvement prompt.
• sources/core/improvement_validator.py checks reward improvement magnitude, but not whether a selected workflow is transferable outside the local task family.

That means the workflow library can become a high-leverage memory substrate. The risk is not simply bad scoring. The sharper risk is that a workflow with high local score and high embedding similarity gets reused as a template even though its reusable claims have not been separated from task-specific residue.

Failure Mode

A prior workflow can be locally successful but globally misleading:

• it solved one scientific task because of incidental dataset structure;
• its generated code contains assumptions that do not transfer across similar-looking tasks;
• its overall_score is high, but the improvement came from a narrow formatting or judge-specific pattern;
• similarity search retrieves it as a template for a near-miss task where it should not apply.

This is especially important because Mimosa is explicitly designed to learn a library of proven workflows and start future tasks from stronger baselines. The library needs admission/provenance metadata, not only score and similarity.

Proposal: Workflow Admission Card

Before a workflow is eligible for template reuse, attach a compact admission card:

@dataclass
class WorkflowAdmissionCard:
    uuid: str
    task_family: str
    admitted_for_reuse: bool
    source_score: float
    similarity_floor: float
    positive_transfer_checks: int
    negative_control_checks: int
    unsupported_probe_hits: int
    reusable_claims: list[str]
    quarantined_claims: list[str]
    evaluator_type: Literal["generic_judge", "scenario_rubric", "programmatic", "mixed"]
    lineage_hash: str

Then WorkflowSelector.select_best_workflows(...) could prefer:

1. workflows with valid admission cards;
2. workflows that passed positive transfer checks;
3. workflows with zero unsupported-probe hits;
4. workflows whose reusable claims match the new task family.

Workflows without an admission card could still be used, but marked as exploratory rather than trusted template material.

Local Prototype Pattern

A local self-improvement prototype I use applies a similar gate:

{
  "claim_boundary": "heterogeneous feature-level self-improvement over list/string/grid/record tasks; not unbounded real-world general intelligence",
  "train_validated": 5,
  "cold_frontier": {"validated": 0, "total": 8},
  "warm_frontier": {"validated": 8, "total": 8},
  "unsupported_probe": {"validated": 0, "total": 2},
  "meta_gate": {"accepted": true, "delta": 8}
}

The important pattern is not the toy domains. It is the admission rule:

learned artifact
  -> positive transfer frontier improves
  -> unsupported probes remain rejected
  -> bounded claim is recorded
  -> only then promote as reusable material

For Mimosa, the same idea would mean that a workflow can be high-scoring without automatically becoming trusted reusable workflow memory.

Minimal Implementation Path

• Store admission_card.json inside each workflow folder after evaluation.
• Add optional fields to WorkflowInfo for admitted_for_reuse, reusable_claims, and unsupported_probe_hits.
• Update WorkflowSelector.sort_workflows_by_score(...) or select_best_workflows(...) to prefer admitted workflows when available.
• Add a simple negative-control scenario type: a near-miss task where the workflow should decline or not be selected.
• Add one test where two workflows have similar embedding scores, but only the admitted workflow is selected as template material.

This would complement issues like #105, #112, #120, and #146 by adding a concrete boundary between "successful workflow output" and "trusted reusable workflow knowledge."

[Fosowl]

Thanks for the detailed write-up. May I ask if you had a look at the V2 ? It's on both dev and mimosa_v2 (latest update). The main branch is several months behind and we are still undergoing benchmark evaluation before making a social post/paper.

Two clarifications because part of this targets V1 paths that no longer exist, and part of it is a real gap in V2.

What's changed since the code you're referencing: dgm.py::select_workflow_template / self_improve_workflow and the single-incumbent loop are V1. In V2 the loop is the EvolutionEngine + QD archive, and several of your failure modes are already addressed there:

"high overall_score from a narrow formatting or judge-specific pattern" the default evaluator is now a multi-source per-claim verifier that writes deterministic Python programs recomputing each claim from workspace files, rather than a generic LLM judge.

Additionally, only an abstracted prompt gradient flows back to the mutator, so the rubric vocabulary can't be optimized against.

Admission gate archive admission is already gated (validate_survivor(): improvement over baseline or qd_score > admit_threshold), with lineage tracking and lowest-qd_score eviction.

So source_score, evaluator_type, and lineage_hash from your card are effectively in place.

What's still open and where your proposal is on point:

The cold-start path in WorkflowSelector still selects prior workflows by task-embedding cosine (≥ 0.8) + score. There is currently no positive-transfer evidence, no negative-control check (near-miss task where the workflow should not be retrieved), and no separation of reusable claims from task-specific residue. This could be interesting future work.

Thank you again! And please have a look at the V2.

Note: the v2 needs https://github.com/HolobiomicsLab/Perspicacite-AI server running for new judge to work properly

[sunghunkwag]

Thanks, I reviewed the V2 direction more closely. You are right that the original issue mixed in V1 paths, so I would narrow the proposal to the remaining V2 cold-start boundary rather than the archive-driven steady-state loop.

A more precise fix shape would be:

• keep EvolutionEngine + QD archive + validate_survivor() as the steady-state admission mechanism;
• add a lightweight cold-start transfer guard to WorkflowSelector, specifically the disk-scan fallback that currently selects prior workflows by task-embedding similarity plus score;
• let WorkflowInfo optionally load a transfer_card.json from each workflow folder;
• in default/backward-compatible mode, prefer workflows with positive-transfer evidence and zero negative-control / unsupported-probe hits, while still allowing workflows without cards as exploratory candidates;
• in stricter mode, require a valid transfer card before a workflow can seed cold-start reuse.

The test case I have in mind is two disk workflows with similar embedding scores: one has a higher overall_score but a negative-control hit, while the other has lower score but positive-transfer evidence and zero unsupported-probe hits. Cold-start selection should prefer the latter.

So I agree the admission-card idea should not duplicate V2 archive admission. It should be scoped to cold-start reusable-workflow retrieval: separating reusable claims from task-specific residue before a prior workflow becomes template material for a new task.