refactor: self-contained plugin (vendor engine into matilde_plugin) for HSM install

CONTRIBUTING.md

@@ -26,10 +26,10 @@ the key in HSM or their `.env`. That is a runtime concern, not a code change.
 | You want to…          | Where it goes                                    | How it ships                                     |
 |-----------------------|--------------------------------------------------|--------------------------------------------------|
 | Add an **API key**    | operator's HSM env or `.env`                     | injected at runtime — **not a code change**      |
-| Add a **tool/skill**  | `hermes-plugin/` or `hermes-skill/`              | merged here → instances pull + restart           |
+| Add a **tool/skill**  | `matilde_plugin/` or `hermes-skill/`              | merged here → instances pull + restart           |
 | Add a **system binary** (`{{EXAMPLE_TOOL}}`, …) | `docker/Dockerfile.{{PACKAGE_NAME}}` | image rebuild + redeploy (operator action) |
 
-When in doubt, reach for `hermes-plugin/` over the Dockerfile. An image rebuild
+When in doubt, reach for `matilde_plugin/` over the Dockerfile. An image rebuild
 requires a coordinated operator action across all deployed instances; a plugin pull is
 just `git pull` + restart.
 
@@ -45,7 +45,7 @@ just `git pull` + restart.
 
 ## Sanitization
 
-Every PR that touches `hermes-skill/`, `hermes-plugin/`, `docker/SOUL*`, `docs/`, or
+Every PR that touches `hermes-skill/`, `matilde_plugin/`, `docker/SOUL*`, `docs/`, or
 top-level markdown files is scanned automatically by `scripts/check_sanitization.py`.
 The scanner has two layers:
 
@@ -96,7 +96,7 @@ it gets added as a plugin (see below) — but the key value itself is never comm
 
 ### New tool/skill
 
-Create the tool in `hermes-plugin/` following the existing plugin conventions, or add
+Create the tool in `matilde_plugin/` following the existing plugin conventions, or add
 a skill document to `hermes-skill/`. Open a PR. Once merged, operators pull the update
 and restart their agent. No image rebuild required.
 

README.md

@@ -63,7 +63,7 @@ polite pools.
 ## Try it
 
 ```python
-from engine.citations import Reference, verify_reference
+from matilde_plugin.engine.citations import Reference, verify_reference
 
 ref = Reference(title="Attention Is All You Need",
                 authors=["Vaswani", "Shazeer"], year=2017,
@@ -75,10 +75,10 @@ print(result.verdict, result.score)   # -> e.g. "verified" 1.0
 ### Command line — verify a whole bibliography
 
 ```bash
-python3 -m engine.cli refs.bib                  # verify a BibTeX file
-python3 -m engine.cli dois.txt                  # or a list of DOIs (one per line)
-python3 -m engine.cli --doi 10.1038/171737a0    # or a single DOI
-python3 -m engine.cli refs.bib --json           # machine-readable
+python3 -m matilde_plugin.engine.cli refs.bib                  # verify a BibTeX file
+python3 -m matilde_plugin.engine.cli dois.txt                  # or a list of DOIs (one per line)
+python3 -m matilde_plugin.engine.cli --doi 10.1038/171737a0    # or a single DOI
+python3 -m matilde_plugin.engine.cli refs.bib --json           # machine-readable
 ```
 
 Output for a mixed bibliography:
@@ -121,16 +121,18 @@ Matilde grows outward from citations toward a full scientific research assistant
 
 ## Architecture & layout
 
-Matilde is built on the Hermes use-case-package template: a plugin + a skill + a soul
-+ an engine, runnable standalone or managed via HSM. See the template's docs for the
-privacy model, sanitization gate, and promotion flow.
+Matilde is built on the Hermes use-case-package template: a plugin + a skill + a soul,
+runnable standalone or managed via HSM. The engine lives **inside** the plugin
+(`matilde_plugin/engine/`) so the plugin directory is a self-contained, copyable
+artifact. See the template's docs for the privacy model, sanitization gate, and
+promotion flow.
 
 | Path | What it is |
 |------|-----------|
-| `engine/citations.py` | The verifiable-citations engine (the core; the part we may open-source standalone) |
-| `engine/openneuro.py` | Read-only OpenNeuro/BIDS client — discovery, metadata, files (stdlib-only) |
-| `engine/parsing.py` · `engine/cli.py` | BibTeX/DOI ingestion + the `matilde` verify CLI |
-| `hermes-plugin/` | Hermes tool definitions exposing the engine to the agent |
+| `matilde_plugin/` | Self-contained Hermes plugin — tool definitions **plus** the bundled engine |
+| `matilde_plugin/engine/citations.py` | The verifiable-citations engine (the core; the part we may open-source standalone) |
+| `matilde_plugin/engine/openneuro.py` | Read-only OpenNeuro/BIDS client — discovery, metadata, files (stdlib-only) |
+| `matilde_plugin/engine/parsing.py` · `matilde_plugin/engine/cli.py` | BibTeX/DOI ingestion + the `matilde` verify CLI (`python3 -m matilde_plugin.engine.cli`) |
 | `hermes-skill/SKILL.md` | The agent's research methodology |
 | `docker/SOUL.Matilde.md` | The research-assistant identity |
 | `tests/` | Offline unit suite + live API integration tests |

docs/onboarding.md

@@ -25,7 +25,7 @@ bind-mounted into the container at `/opt/data` (`HERMES_HOME`).
    │  {{IMAGE}}                    │   ───▶  │  = /opt/data — a directory you own    │
    │                               │         │                                       │
    │  • Hermes runtime             │         │  • {{PACKAGE_NAME}}/  ← git clone of  │
-   │  • System dependencies /      │         │      THIS repo (engine/, plugin,      │
+   │  • System dependencies /      │         │      THIS repo (plugin+engine,        │
    │    domain-specific binaries   │         │      collectors/, skill, docs)        │
    │    ({{DOMAIN_BINARIES}})      │         │                                       │
    │                               │         │  • skills/       ← operator skills   │
@@ -110,7 +110,7 @@ tells you if it is missing.
 **What's in `.env`** — your API keys for any external services this package uses. The
 `.env` file is git-ignored; never commit it. The list of required keys is in
 `docker/instance-setup.sh` and in the `requires_env` block of
-`hermes-plugin/plugin.yaml`. See `.env.example` for the full set with descriptions.
+`matilde_plugin/plugin.yaml`. See `.env.example` for the full set with descriptions.
 
 ---
 
@@ -124,7 +124,7 @@ conflicting state.
 
 The data layout is identical to standalone. What changes is the orchestrator:
 
-- **Enable the plugin** via the HSM UI or API. HSM reads `hermes-plugin/plugin.yaml`
+- **Enable the plugin** via the HSM UI or API. HSM reads `matilde_plugin/plugin.yaml`
   and surfaces the capability toggle.
 - **Environment / API keys** go into the HSM env store (encrypted). They are injected
   at container start. The `requires_env` keys declared in `plugin.yaml` are what HSM

hermes-plugin/__init__.py → matilde_plugin/__init__.py

@@ -1,32 +1,49 @@
 """matilde plugin — registers Matilde's verifiable-citation tools with Hermes.
 
-Wraps the ``engine.citations`` verifier as Hermes tools. Each tool maps to a core
+Wraps the bundled ``.engine.citations`` verifier as Hermes tools. Each tool maps to a core
 operation: verify one citation, verify a whole bibliography, or quick-check a
 retraction. The engine is imported at tool-invocation time (via _check_available),
 so the plugin registers even if imports have issues — the gate handles execution.
 """
 
 from __future__ import annotations
 
+import importlib
 import importlib.util
 import os
 import sys
 
 # ---------------------------------------------------------------------------
-# Path setup — package root (engine/) is one level above this file.
+# Self-contained loading.
+#
+# The plugin is a self-contained package: ``engine/`` lives INSIDE this dir, and
+# ``tools.py`` imports it via relative imports (``from .engine.citations import …``).
+# Copying ONLY this directory into another location still imports cleanly.
+#
+# Hermes (and the test harness) load this ``__init__.py`` directly by file path
+# under the name ``matilde_plugin``. For ``tools.py``'s relative imports to
+# resolve, this module must be registered in ``sys.modules`` as a package (with a
+# search path) before we import ``tools``. We ensure that here, then import
+# ``tools`` as a real submodule.
 # ---------------------------------------------------------------------------
 _PLUGIN_DIR = os.path.dirname(os.path.realpath(__file__))
-_PACKAGE_ROOT = os.path.normpath(os.path.join(_PLUGIN_DIR, ".."))
-if _PACKAGE_ROOT not in sys.path:
-    sys.path.insert(0, _PACKAGE_ROOT)
+_PKG = __name__ if __name__ != "__main__" else "matilde_plugin"
 
-# ---------------------------------------------------------------------------
-# Load tools.py from this directory (import-path agnostic).
-# ---------------------------------------------------------------------------
-_tools_path = os.path.join(_PLUGIN_DIR, "tools.py")
-_spec = importlib.util.spec_from_file_location("_matilde_tools", _tools_path)
-_tools_mod = importlib.util.module_from_spec(_spec)
-_spec.loader.exec_module(_tools_mod)
+# Register THIS module as a package in sys.modules so relative imports resolve,
+# even when we were loaded by file path (spec_from_file_location does not always
+# pre-register the parent package or give it a search path).
+_self = sys.modules.get(_PKG)
+if _self is None:
+    _self = sys.modules[__name__]
+    sys.modules[_PKG] = _self
+if getattr(_self, "__path__", None) is None:
+    _self.__path__ = [_PLUGIN_DIR]  # mark as a package
+if getattr(_self, "__package__", None) in (None, ""):
+    _self.__package__ = _PKG
+
+# Import tools.py as the submodule ``<pkg>.tools`` so its ``from .engine…``
+# relative imports resolve against this package.
+_tools_mod = importlib.import_module(f"{_PKG}.tools")
 
 # ---------------------------------------------------------------------------
 # Re-export schemas, handlers, and the availability gate.

engine/__init__.py → matilde_plugin/engine/__init__.py

@@ -2,15 +2,15 @@
 
 The engine is the package's core: it verifies a citation along four independent
 axes (existence, metadata-match, retraction, URL-liveness) and produces a
-composite verifiability score and verdict. See :mod:`engine.citations`.
+composite verifiability score and verdict. See :mod:`matilde_plugin.engine.citations`.
 
 This is *verifiable*, not *provably correct* — axes 1-3 are near-deterministic;
 claim-support grounding (does the cited passage substantiate the claim?) is the
 probabilistic v2 axis and is not yet implemented.
 """
 from __future__ import annotations
 
-from engine.citations import (
+from .citations import (
     AxisResult,
     Reference,
     VerificationResult,

engine/cli.py → matilde_plugin/engine/cli.py

@@ -2,11 +2,11 @@
 
 Usage::
 
-    python3 -m engine.cli refs.bib                 # verify a BibTeX file
-    python3 -m engine.cli dois.txt                 # verify a list of DOIs
-    python3 -m engine.cli --doi 10.1038/171737a0   # verify one DOI
-    python3 -m engine.cli refs.bib --json          # machine-readable output
-    python3 -m engine.cli refs.bib --email you@example.org   # polite-pool contact
+    python3 -m matilde_plugin.engine.cli refs.bib                 # verify a BibTeX file
+    python3 -m matilde_plugin.engine.cli dois.txt                 # verify a list of DOIs
+    python3 -m matilde_plugin.engine.cli --doi 10.1038/171737a0   # verify one DOI
+    python3 -m matilde_plugin.engine.cli refs.bib --json          # machine-readable output
+    python3 -m matilde_plugin.engine.cli refs.bib --email you@example.org   # polite-pool contact
 
 Exit codes: 0 = all references verified / only warnings; 1 = at least one
 ``not_found`` or ``retracted`` reference (useful as a pre-commit / CI gate on a
@@ -20,8 +20,8 @@
 import sys
 from typing import Callable, Optional
 
-from engine.citations import Reference, verify_reference
-from engine.parsing import parse_bibtex, parse_dois
+from .citations import Reference, verify_reference
+from .parsing import parse_bibtex, parse_dois
 
 _VERDICT_MARK = {
     "verified": "OK ",

engine/parsing.py → matilde_plugin/engine/parsing.py

@@ -10,7 +10,7 @@
 
 import re
 
-from engine.citations import Reference, _normalize_doi
+from .citations import Reference, _normalize_doi
 
 
 def _strip_value(raw: str) -> str:

hermes-plugin/plugin.yaml → matilde_plugin/plugin.yaml

@@ -1,8 +1,11 @@
 # plugin.yaml — Hermes plugin manifest for Matilde (verifiable citations)
 #
 # Register the plugin by symlinking it into the agent's plugin dir:
-#   ~/.hermes-{agent-name}/plugins/matilde -> /path/to/Matilde/hermes-plugin/
+#   ~/.hermes-{agent-name}/plugins/matilde -> /path/to/Matilde/matilde_plugin/
 # Then restart the agent (or `hermes plugin reload` if supported).
+#
+# This directory is self-contained: the citation/OpenNeuro engine lives inside it
+# (matilde_plugin/engine/), so copying ONLY this dir is enough to run the plugin.
 
 name: "matilde"
 

hermes-plugin/tools.py → matilde_plugin/tools.py

@@ -1,6 +1,6 @@
 """Matilde tools for Hermes — verifiable-citation checking.
 
-These tools expose the ``engine.citations`` verifier to the agent. A citation is
+These tools expose the bundled ``.engine.citations`` verifier to the agent. A citation is
 checked along four axes (existence, metadata-match, retraction, URL-liveness) and
 scored. This is *verifiable*, not *provably correct* — the score is a confidence,
 not a proof. Claim-support grounding (does the cited passage back the claim?) is a
@@ -17,17 +17,12 @@
 from __future__ import annotations
 
 import json
-import os
-import sys
 from typing import Any
 
-# ---------------------------------------------------------------------------
-# Path setup — make the package root importable (engine/ lives one level up).
-# ---------------------------------------------------------------------------
-_PLUGIN_DIR = os.path.dirname(os.path.abspath(__file__))
-_PACKAGE_ROOT = os.path.normpath(os.path.join(_PLUGIN_DIR, ".."))
-if _PACKAGE_ROOT not in sys.path:
-    sys.path.insert(0, _PACKAGE_ROOT)
+# The engine lives INSIDE this plugin package (matilde_plugin/engine/), so it is
+# imported via relative imports below — no sys.path manipulation needed. Imports
+# are done lazily at call time so the plugin still registers (the _check_available
+# gate guards execution) even if an import would fail.
 
 
 # ---------------------------------------------------------------------------
@@ -58,7 +53,7 @@ def _check_available() -> bool:
     so the only real prerequisite is that the engine module loads.
     """
     try:
-        import engine.citations  # noqa: F401
+        from .engine import citations  # noqa: F401
         return True
     except Exception:
         return False
@@ -69,7 +64,7 @@ def _check_available() -> bool:
 # ---------------------------------------------------------------------------
 
 def _reference_from_args(d: dict) -> Any:
-    from engine.citations import Reference
+    from .engine.citations import Reference
     authors = d.get("authors") or []
     if isinstance(authors, str):
         # accept "Vaswani; Shazeer" or "Vaswani, Shazeer"
@@ -126,7 +121,7 @@ def _handle_verify_citation(args: dict, **kwargs: Any) -> str:
     if not (args.get("doi") or args.get("title")):
         return _tool_error("Provide at least a 'doi' or a 'title' to verify.")
     try:
-        from engine.citations import verify_reference
+        from .engine.citations import verify_reference
         ref = _reference_from_args(args)
         result = verify_reference(ref)
         out = result.to_dict()
@@ -187,7 +182,7 @@ def _handle_verify_bibliography(args: dict, **kwargs: Any) -> str:
     if not isinstance(refs, list) or not refs:
         return _tool_error("'references' must be a non-empty list of citation objects.")
     try:
-        from engine.citations import verify_reference
+        from .engine.citations import verify_reference
         results, summary = [], {}
         flagged = []
         for i, item in enumerate(refs):
@@ -244,7 +239,7 @@ def _handle_check_retraction(args: dict, **kwargs: Any) -> str:
     if not doi:
         return _tool_error("'doi' is required.")
     try:
-        from engine.citations import check_retraction, default_fetch
+        from .engine.citations import check_retraction, default_fetch
         res = check_retraction(doi, fetch=default_fetch)
         return _tool_result(
             doi=doi,
@@ -285,7 +280,7 @@ def _handle_openneuro_dataset_info(args: dict, **kwargs: Any) -> str:
     if not dsid:
         return _tool_error("'dataset_id' is required (e.g. 'ds000246').")
     try:
-        from engine.openneuro import get_dataset, OpenNeuroError
+        from .engine.openneuro import get_dataset, OpenNeuroError
         try:
             ds = get_dataset(dsid)
         except OpenNeuroError as exc:
@@ -323,7 +318,7 @@ def _handle_openneuro_dataset_info(args: dict, **kwargs: Any) -> str:
 
 def _handle_openneuro_search(args: dict, **kwargs: Any) -> str:
     try:
-        from engine.openneuro import list_datasets
+        from .engine.openneuro import list_datasets
         limit = args.get("limit") or 20
         try:
             limit = max(1, min(int(limit), 100))
@@ -363,7 +358,7 @@ def _handle_openneuro_list_files(args: dict, **kwargs: Any) -> str:
     if not dsid:
         return _tool_error("'dataset_id' is required (e.g. 'ds000246').")
     try:
-        from engine.openneuro import list_files, OpenNeuroError
+        from .engine.openneuro import list_files, OpenNeuroError
         tag = str(args.get("tag", "")).strip() or None
         try:
             files = list_files(dsid, tag=tag)

sanitize.config.json

@@ -6,7 +6,7 @@
 
   "sensitive_prefixes": [
     "hermes-skill/",
-    "hermes-plugin/",
+    "matilde_plugin/",
     "docker/SOUL",
     "docs/"
   ],

scripts/check_sanitization.py

@@ -39,7 +39,7 @@ def load_config(root="."):
         # Sensible defaults so the gate still runs on a fresh template.
         return {
             "package_kind": "SHARED, public agent package",
-            "sensitive_prefixes": ["hermes-skill/", "hermes-plugin/", "docker/SOUL", "docs/"],
+            "sensitive_prefixes": ["hermes-skill/", "matilde_plugin/", "docker/SOUL", "docs/"],
             "semantic": {"domain_noun": "operational engagement", "flag_examples": [],
                          "do_not_flag_examples": [], "model": "claude-opus-4-8"},
             "deterministic": {"enabled": True, "allow_substrings": []},

tests/test_citations.py

@@ -17,7 +17,7 @@
 
 sys.path.insert(0, os.path.normpath(os.path.join(os.path.dirname(__file__), "..")))
 
-from engine.citations import (  # noqa: E402
+from matilde_plugin.engine.citations import (  # noqa: E402
     AxisResult,
     Reference,
     VerificationResult,
@@ -309,7 +309,7 @@ def test_openalex_to_record_normalizes_to_crossref_shape():
 def test_retraction_via_openalex_is_retracted_flag():
     retracted_oa = dict(OPENALEX_ATTENTION, is_retracted=True)
     rec = _openalex_to_record(retracted_oa)
-    from engine.citations import _retraction_signal
+    from matilde_plugin.engine.citations import _retraction_signal
     assert _retraction_signal(rec) is not None
 
 

tests/test_citations_integration.py

@@ -17,7 +17,7 @@
 
 sys.path.insert(0, os.path.normpath(os.path.join(os.path.dirname(__file__), "..")))
 
-from engine.citations import (  # noqa: E402
+from matilde_plugin.engine.citations import (  # noqa: E402
     Reference,
     check_retraction,
     default_fetch,

tests/test_cli.py

@@ -12,8 +12,8 @@
 
 sys.path.insert(0, os.path.normpath(os.path.join(os.path.dirname(__file__), "..")))
 
-from engine import cli  # noqa: E402
-from engine.citations import AxisResult, Reference, VerificationResult  # noqa: E402
+from matilde_plugin.engine import cli  # noqa: E402
+from matilde_plugin.engine.citations import AxisResult, Reference, VerificationResult  # noqa: E402
 
 
 def _result(ref: Reference, verdict: str = "verified", score: float = 1.0) -> VerificationResult: