diff --git a/docs/modeldescr/actions.rst b/docs/modeldescr/actions.rst index e8c1976e..2c578e5c 100644 --- a/docs/modeldescr/actions.rst +++ b/docs/modeldescr/actions.rst @@ -26,6 +26,13 @@ Actions are predefined batches of specific acts that are yielding the state of an entity, based on its constraints. Actions are binding data to modules. +An action normally participates in the internal execution graph of the model. In other words, +the mere presence of an action definition does not automatically mean that the action should be +presented as a public query target. When the model author wishes to expose an action explicitly, +that action may be listed in the model's top-level ``interface.actions`` collection. This keeps +the distinction clear between the implementation of the model and the public surface that callers +are expected to use. + .. important:: The following rules are applied to an action: @@ -455,4 +462,4 @@ that checks if the file is really there. require a valid constraints attached to the corresponding action! Likewise chain conditions can be used for consistency check: if a specific device is working -as expected, no additional checks are needed (as an example). \ No newline at end of file +as expected, no additional checks are needed (as an example). diff --git a/docs/modeldescr/layout.rst b/docs/modeldescr/layout.rst index f6b21b39..ae5e8238 100644 --- a/docs/modeldescr/layout.rst +++ b/docs/modeldescr/layout.rst @@ -52,6 +52,7 @@ Model description index file has the following structure: This is a description of this model that gives you more idea what it is etc. maintainer: John Smith + interface: null checkbook: null config: null @@ -75,8 +76,37 @@ The following fields are supported: ``config`` - Global configuration section. It is applied to the whole session, globally. However - different model can have a different configuration. + Global configuration section. It is applied to the whole session, globally. However + different model can have a different configuration. + +``interface`` + + The ``interface`` section is optional. Its purpose is to define the public callable surface + of the model explicitly, rather than relying on the full internal structure of the model. + This is especially useful when a model contains helper entities or internal gating actions that + should participate in evaluation, but should not be shown as operator-facing targets. + + The syntax is intentionally simple and typed. It accepts three optional lists named + ``checkbook``, ``entities`` and ``actions``. + + .. code-block:: yaml + + interface: + checkbook: + - main-audit + entities: + - all + - summary + actions: + - python-proof + + In this form, ``main-audit`` is a public checkbook label, ``all`` and ``summary`` are public + entity entrypoints, and ``python-proof`` is declared as a public direct action entrypoint. + The declaration itself does not change the internal execution graph of the model. It simply + states what should be treated as public by tools that inspect the model. + + If ``interface`` is not present at all, SysInspect keeps the historic behaviour. In that case, + every inferred entrypoint is considered public. ``checkbook`` diff --git a/docs/modeldescr/overview.rst b/docs/modeldescr/overview.rst index d18978a4..bb2359ca 100644 --- a/docs/modeldescr/overview.rst +++ b/docs/modeldescr/overview.rst @@ -55,6 +55,14 @@ The declarative approach is much easier to comprehend and use, because it focuse system should do, rather than how to achieve it. In this way more readable and maintainable configurations are achieved. Maintenance is easier because the configurations remain simple and predictable. +Recent versions of the model description also allow the author to declare an explicit public +interface for the model. This is useful when a model contains helper entities, gating actions, +or intermediate implementation details that are necessary for evaluation but are not meant to be + presented as first-class query targets to operators. In that situation the model may declare an +``interface`` section and list only those entrypoints that are intended to be public. If the +section is omitted, the legacy behaviour remains in effect and every inferred entrypoint is +considered public. + .. important:: In a nutshell, declarative configurations are easier for teams to understand, update, and share diff --git a/examples/demos/nopython/README.md b/examples/demos/nopython/README.md new file mode 100644 index 00000000..b82d2af6 --- /dev/null +++ b/examples/demos/nopython/README.md @@ -0,0 +1,239 @@ +# Python Without Host Python + +This demo shows how SysInspect can execute Python-based runtime payloads on a +target that does not provide a system `python3` binary. + +The execution path is intentionally split into two layers: + +- native SysInspect modules collect host facts and drive the decision logic +- the embedded `runtime.py3` runtime executes Python only when the model DSL + determines that the target matches the intended profile + +This keeps the Python payload focused on proof of execution rather than policy +or branching logic. + +## Purpose + +The demo is designed for compact or appliance-style targets where host Python +should not be treated as a deployment prerequisite. + +It demonstrates that SysInspect can: + +- identify the target environment using native modules +- evaluate model constraints before runtime dispatch +- execute Python code through the embedded runtime when the target matches +- deliver Python helper libraries as repository payloads rather than operating + system packages + +The model is fully local at execution time and does not depend on outbound +network access from the minion. + +## Model Behavior + +The model evaluates four host-side facts: + +- target is Alpine +- target is Fedora +- host `python3` is absent +- host `python3` is present + +Based on those facts, the model selects one applicable branch: + +- Alpine without host Python: + - run `py3.nopython` +- Fedora: + - return a verification message from YAML +- non-Fedora hosts with host Python present: + - return a verification message from YAML +- non-Alpine, non-Fedora hosts without host Python: + - return a generic verification message from YAML + +The Python payload itself returns runtime proof only, including interpreter +identity, version, selected runtime values, and helper-import evidence. + +## Repository Contents + +Files in this directory: + +- `model.cfg` +- `lib/runtime/python3/nopython.py` +- `lib/runtime/python3/nopyproof.py` +- `lib/runtime/python3/site-packages/nopykit/__init__.py` + +## Scope And Entities + +Install the model under the `nopython` scope. + +This demo also declares an explicit public interface: + +```yaml +interface: + entities: + - all + actions: + - python-proof +``` + +The `interface` section is optional. + +- if `interface` is present, only listed entity and action entrypoints are + considered part of the model's public surface +- if `interface` is absent, the current legacy behavior remains in effect and + all inferred entrypoints are public + +The interface section does not alter internal execution semantics. It describes +which entrypoints are intended to be exposed to callers. + +Entities exposed by this model: + +- `all` +- `python-proof` +- `verify-fedora` +- `verify-python` +- `verify-other` + +## Prerequisites + +The master must provide: + +- the `runtime.py3` dispatcher module +- the `nopython` model scope +- the runtime Python payload tree from this directory + +The minion does not need a system `python3` package. + +## Install The Model + +Copy `model.cfg` into the master's models root: + +```text +$MASTER/data/models/nopython/model.cfg +``` + +Export the scope from the master configuration: + +```yaml +config: + master: + fileserver.models: + - nopython +``` + +## Install The Embedded Python Runtime + +Build and register `runtime.py3`: + +```bash +make all-devel +sysinspect module -A --path ./target/debug/runtime/py3-runtime --name runtime.py3 --descr "Python 3 runtime" +``` + +## Install The Python Runtime Payload Tree + +This demo ships its payloads under `lib/` so the runtime directory structure is +preserved during publication. + +From `examples/demos/nopython`, publish the `lib` tree itself: + +```bash +sysinspect module -A --path ./lib -l +``` + +Then sync the cluster: + +```bash +sysinspect --sync +``` + +Important: + +- select or publish `examples/demos/nopython/lib` +- do not publish the parent directory `examples/demos/nopython` + +The runtime expects the published tree to land under: + +- `lib/runtime/python3/` +- `lib/runtime/python3/site-packages/` + +## Python Runtime Layout Rules + +The embedded Python runtime expects the following repository layout: + +- executable Python modules: + - `lib/runtime/python3/` +- helper libraries: + - `lib/runtime/python3/site-packages/` + +For this demo, that means: + +- `lib/runtime/python3/nopython.py` +- `lib/runtime/python3/nopyproof.py` +- `lib/runtime/python3/site-packages/nopykit/__init__.py` + +Each runtime Python module should export: + +- `run(req)` as the entrypoint +- optional documentation as either: + - `doc = {...}` + - `def doc(): return {...}` + +The documentation object must be the payload itself, not wrapped as +`{"doc": ...}`. + +## Verify Installation + +Inspect the published runtime payloads: + +```bash +sysinspect module -Ll +``` + +Expected library entries include: + +- `runtime/python3/nopython.py` +- `runtime/python3/nopyproof.py` +- `runtime/python3/site-packages/nopykit/__init__.py` + +## Run The Demo + +Run the full model: + +```bash +sysinspect "nopython/all" '*' +``` + +Or run one branch-oriented entity directly: + +```bash +sysinspect "nopython/python-proof" '*' +sysinspect "nopython/verify-fedora" '*' +sysinspect "nopython/verify-python" '*' +sysinspect "nopython/verify-other" '*' +``` + +## Expected Outcomes + +When the Python proof branch is selected, the action should return structured +runtime evidence such as: + +- Python runtime implementation name +- Python runtime version +- Python platform and byte order +- selected `sys` values +- helper module import evidence + +The branch decision itself is made by the model DSL, not by the Python payload. + +Typical outcomes: + +- Alpine without host Python: + - `python-proof` is applicable +- Fedora: + - `verify-fedora` is applicable +- non-Fedora with host Python present: + - `verify-python` is applicable +- non-Alpine, non-Fedora without host Python: + - `verify-other` is applicable + +Non-selected branches are reported as `Not Applicable`, not as execution +failures. diff --git a/examples/demos/nopython/lib/runtime/python3/nopyproof.py b/examples/demos/nopython/lib/runtime/python3/nopyproof.py new file mode 100644 index 00000000..da953bae --- /dev/null +++ b/examples/demos/nopython/lib/runtime/python3/nopyproof.py @@ -0,0 +1,34 @@ +import sys + +from nopykit import proof_value + + +doc = { + "name": "nopyproof", + "version": "0.1.0", + "author": "SysInspect Demo", + "description": "Proof module that imports a helper package from runtime site-packages.", + "arguments": [ + {"name": "a", "type": "number", "required": False, "description": "First input"}, + {"name": "b", "type": "number", "required": False, "description": "Second input"}, + ], + "returns": { + "description": "Structured proof that helper imports worked.", + "sample": {"import_ok": True, "proof": 12, "python_runtime": "rustpython"}, + }, +} + + +def run(req): + args = req.get("args", {}) + a = args.get("a", 2) + b = args.get("b", 5) + impl = getattr(getattr(sys, "implementation", None), "name", "python") + return { + "import_ok": True, + "a": a, + "b": b, + "proof": proof_value(a, b), + "python_runtime": impl, + "message": f"Python helper import succeeded inside {impl} without requiring host python3.", + } diff --git a/examples/demos/nopython/lib/runtime/python3/nopython.py b/examples/demos/nopython/lib/runtime/python3/nopython.py new file mode 100644 index 00000000..0ba345d4 --- /dev/null +++ b/examples/demos/nopython/lib/runtime/python3/nopython.py @@ -0,0 +1,100 @@ +import sys + +doc = { + "name": "nopython", + "version": "0.1.0", + "author": "SysInspect Demo", + "description": "Return proof that Python code executed inside the embedded runtime.", + "arguments": [], + "options": [ + {"name": "rt.logs", "description": "Forward Python-side logs into SysInspect runtime logs"} + ], + "returns": { + "description": "Structured proof that Python executed and imported a runtime helper package.", + "sample": { + "python_runtime": "rustpython", + "python_version": "3.x", + "python_platform": "linux", + "python_byteorder": "little", + "lambda_type": "function", + "eval_result": 10, + "import_ok": True, + "message": "Python proof: impl=rustpython, ver=3.x, platform=linux, byteorder=little, stdlib=3, helper=nopykit.", + }, + }, +} + + +def runtime_identity(): + impl = getattr(sys, "implementation", None) + impl_name = getattr(impl, "name", "python") if impl is not None else "python" + version_info = getattr(sys, "version_info", None) + if version_info is not None: + version_short = f"{version_info.major}.{version_info.minor}.{version_info.micro}" + else: + version_short = sys.version.split()[0] if getattr(sys, "version", "") else "unknown" + + return { + "implementation": impl_name, + "version": getattr(sys, "version", version_short), + "version_short": version_short, + } + + +def run(_req): + runtime = runtime_identity() + runtime_seed = sum(ord(ch) for ch in runtime["implementation"]) + sys.version_info.major + sys.version_info.minor + sys.version_info.micro + stdlib_imports = { + "json": __import__("json").__name__, + "math": __import__("math").__name__, + "hashlib": __import__("hashlib").__name__, + } + lambda_type = type(lambda: 1).__name__ + comprehension_values = [n * n for n in range(sys.version_info.major + 2)] + comprehension_sum = sum(comprehension_values) + eval_expr = f"{sys.version_info.major} + {sys.version_info.minor} + {sys.version_info.micro}" + eval_result = eval(compile(eval_expr, "", "eval")) + generator_result = list(x + sys.version_info.major for x in range(3)) + helper_module = __import__("nopykit") + try: + int(f"{runtime['implementation']}-{sys.version_info.major}") + except Exception as exc: + exception_type = type(exc).__name__ + + message = ( + f"Python proof: impl={runtime['implementation']}, ver={runtime['version_short']}, " + f"platform={sys.platform}, byteorder={sys.byteorder}, stdlib={len(stdlib_imports)}, helper={helper_module.__name__}." + ) + + log.info("Collected embedded Python runtime proof") + log.info(message) + + return { + "python_runtime": runtime["implementation"], + "python_version": runtime["version_short"], + "python_full_version": runtime["version"], + "python_platform": sys.platform, + "python_byteorder": sys.byteorder, + "python_version_info": { + "major": sys.version_info.major, + "minor": sys.version_info.minor, + "micro": sys.version_info.micro, + }, + "sys_argv": list(sys.argv), + "sys_path_head": list(sys.path[:5]), + "runtime_seed": runtime_seed, + "lambda_type": lambda_type, + "lambda_result": (lambda x: x + sys.version_info.minor)(sys.version_info.major), + "comprehension_values": comprehension_values, + "comprehension_sum": comprehension_sum, + "eval_expression": eval_expr, + "eval_result": eval_result, + "generator_result": generator_result, + "exception_type": exception_type, + "stdlib_imports": stdlib_imports, + "import_ok": True, + "helper_module": helper_module.__name__, + "helper_module_file": getattr(helper_module, "__file__", None), + "helper_module_keys": sorted([key for key in dir(helper_module) if not key.startswith("_")])[:8], + "message": message, + } diff --git a/examples/demos/nopython/lib/runtime/python3/site-packages/nopykit/__init__.py b/examples/demos/nopython/lib/runtime/python3/site-packages/nopykit/__init__.py new file mode 100644 index 00000000..92881cf5 --- /dev/null +++ b/examples/demos/nopython/lib/runtime/python3/site-packages/nopykit/__init__.py @@ -0,0 +1,70 @@ +import os + + +def _read_lines(path): + with open(path, "r", encoding="utf-8") as fh: + return fh.readlines() + + +def os_release_summary(): + out = {} + for line in _read_lines("/etc/os-release"): + if "=" not in line: + continue + key, value = line.split("=", 1) + out[key.strip()] = value.strip().strip('"') + return { + "id": out.get("ID"), + "version_id": out.get("VERSION_ID"), + "pretty_name": out.get("PRETTY_NAME"), + } + + +def detect_python3(): + for base in ["/usr/bin", "/usr/local/bin", "/bin"]: + if os.path.exists(base + "/python3"): + return True + return False + + +def meminfo_summary(): + data = {} + for line in _read_lines("/proc/meminfo"): + if ":" not in line: + continue + key, value = line.split(":", 1) + data[key.strip()] = value.strip() + + def _kb_to_mb(name): + raw = data.get(name, "0 kB").split()[0] + return int(raw) // 1024 + + return { + "mem_total_mb": _kb_to_mb("MemTotal"), + "mem_available_mb": _kb_to_mb("MemAvailable"), + } + + +def disk_free_mb(path): + stat = os.statvfs(path) + return int((stat.f_bavail * stat.f_frsize) / (1024 * 1024)) + + +def readiness_verdict(python3_present, meminfo, root_free_mb): + reasons = [] + if not python3_present: + reasons.append("system python3 absent") + if meminfo.get("mem_available_mb", 0) < 64: + reasons.append("low available memory") + if root_free_mb < 128: + reasons.append("low free disk on /") + + verdict = "ready" + if meminfo.get("mem_available_mb", 0) < 64 or root_free_mb < 128: + verdict = "warning" + + return verdict, reasons + + +def proof_value(a, b): + return a + (a * b) diff --git a/examples/demos/nopython/model.cfg b/examples/demos/nopython/model.cfg new file mode 100644 index 00000000..32e386cb --- /dev/null +++ b/examples/demos/nopython/model.cfg @@ -0,0 +1,189 @@ +name: "Python Without Python" +version: "0.1" +description: | + Demo for minimal Alpine-like systems where Python is not installed. + + The model uses native actions and DSL constraints to decide whether the + target matches the intended profile. Only when it does, the embedded Python + runtime is invoked to prove that Python code still runs without host Python. + + If the target does not match, the model returns troll messages from YAML + actions instead of burying the logic inside the Python script. + +maintainer: SysInspect Demo +interface: + entities: + - all + actions: + - python-proof +checkbook: +relations: +entities: + - all + - python-proof + - verify-fedora + - verify-python + - verify-other + +actions: + target-alpine: + descr: "Detect Alpine targets for the demo gate" + module: sys.run + bind: + - all + state: + $: + args: + cmd: >- + awk -F= '$1=="ID"{gsub(/"/,"",$2); printf "%s", $2}' /etc/os-release + + target-fedora: + descr: "Detect Fedora targets for the demo gate" + module: sys.run + bind: + - all + state: + $: + args: + cmd: >- + awk -F= '$1=="ID"{gsub(/"/,"",$2); printf "%s", $2}' /etc/os-release + + host-python-absent: + descr: "Detect whether host python3 is absent" + module: sys.pkg + bind: + - all + state: + $: + opts: + - check + args: + name: python3 + + host-python-present: + descr: "Detect whether host python3 is already present" + module: sys.pkg + bind: + - all + state: + $: + opts: + - check + args: + name: python3 + + python-proof: + descr: "Run one embedded Python proof action only on Alpine without host python3" + module: py3.nopython + if-true: + - target-alpine + - host-python-absent + if-false: + - target-fedora + - host-python-present + bind: + - all + - python-proof + state: + $: + opts: + - rt.logs + + verify-fedora: + descr: "Return a Fedora-specific message" + module: sys.run + if-true: + - target-fedora + bind: + - all + - verify-fedora + state: + $: + args: + cmd: >- + printf 'This is Fedora. Remove Python first to speed up Ansible :-P' + + verify-python: + descr: "Return a message when the host already provides python3" + module: sys.run + if-false: + - target-alpine + - target-fedora + if-true: + - host-python-present + bind: + - all + - verify-python + state: + $: + args: + cmd: >- + printf 'Host python3 detected, nope.' + + verify-other: + descr: "Return a generic message for non-Alpine minimal systems" + module: sys.run + if-false: + - target-alpine + - target-fedora + - host-python-present + bind: + - all + - verify-other + state: + $: + args: + cmd: >- + printf 'Not Alpine. Cute host, wrong demo.' + +constraints: + target-alpine: + descr: Target OS must be Alpine + entities: + - all + all: + $: + - fact: stdout + equals: alpine + + target-fedora: + descr: Target OS is Fedora + entities: + - all + all: + $: + - fact: stdout + equals: fedora + + host-python-absent: + descr: Host must not provide python3 + entities: + - all + all: + $: + - fact: installed + equals: false + + host-python-present: + descr: Host already provides python3 + entities: + - all + all: + $: + - fact: installed + equals: true + +events: + $|$|$|$: + handlers: + - console-logger + - outcome-logger + + console-logger: + concise: false + prefix: No Python Demo + + outcome-logger: + prefix: No Python Demo + +telemetry: {} diff --git a/libmodpak/src/lib.rs b/libmodpak/src/lib.rs index b5e84239..11219738 100644 --- a/libmodpak/src/lib.rs +++ b/libmodpak/src/lib.rs @@ -765,6 +765,17 @@ impl SysInspectModPak { }) } + fn normalize_library_source(path: &Path) -> PathBuf { + let nested = path.join(DEFAULT_MODULES_LIB_DIR); + if path.is_dir() && nested.is_dir() { nested } else { path.to_path_buf() } + } + + fn rebuild_library_index_from_repo(&mut self, repo_lib_root: &Path) -> Result<(), SysinspectError> { + self.idx.library = IndexMap::new(); + self.idx.index_library(repo_lib_root)?; + Ok(()) + } + /// Heuristic to determine the OS label of an ELF file, since EI_OSABI is often unreliable. fn get_os_label(elf: &goblin::elf::Elf) -> &'static str { // Check section names - BSDs put their identity here @@ -795,23 +806,30 @@ impl SysInspectModPak { /// Adds a library to the repository. pub fn add_library(&mut self, p: PathBuf) -> Result<(), SysinspectError> { + let src = Self::normalize_library_source(&p); let path = self.root.join("lib"); if !path.exists() { log::info!("Creating module repository at {}", path.display()); std::fs::create_dir_all(&path)?; } + let malformed_nested = path.join(DEFAULT_MODULES_LIB_DIR); + if malformed_nested.exists() { + log::warn!("Removing malformed nested library tree at {} before rebuilding the library index", malformed_nested.display()); + std::fs::remove_dir_all(&malformed_nested)?; + } + let mut options = CopyOptions::new(); options.overwrite = true; // Overwrite existing files if necessary - options.copy_inside = true; // Copy the contents inside `p` instead of the directory itself + options.copy_inside = true; // Copy the contents inside `src` instead of the directory itself options.content_only = true; // Copy only the contents of the directory - log::info!("Copying library from {} to {}", p.display(), path.display()); - fs_extra::dir::copy(&p, &path, &options).map_err(|e| SysinspectError::MasterGeneralError(format!("Failed to copy library: {e}")))?; - self.idx.index_library(&path)?; + log::info!("Copying library from {} to {}", src.display(), path.display()); + fs_extra::dir::copy(&src, &path, &options).map_err(|e| SysinspectError::MasterGeneralError(format!("Failed to copy library: {e}")))?; + self.rebuild_library_index_from_repo(&path)?; log::debug!("Writing index to {}", self.root.join(REPO_MOD_INDEX).display().to_string().bright_yellow()); fs::write(self.root.join(REPO_MOD_INDEX), self.idx.to_yaml()?)?; // XXX: needs flock - log::info!("Library {} added to index", p.display().to_string().bright_yellow()); + log::info!("Library {} added to index", src.display().to_string().bright_yellow()); Ok(()) } @@ -1025,7 +1043,12 @@ impl SysInspectModPak { .idx .library() .into_iter() - .filter(|(name, _)| profile.libraries().iter().any(|expr| glob::Pattern::new(expr).is_ok_and(|pattern| pattern.matches(name)))) + .filter(|(name, _)| { + profile.libraries().iter().any(|expr| { + glob::Pattern::new(expr) + .is_ok_and(|pattern| pattern.matches(name) || name.strip_prefix("lib/").is_some_and(|rel| pattern.matches(rel))) + }) + }) .collect::>(); libraries.sort_by(|(a, _), (b, _)| a.cmp(b)); diff --git a/libmodpak/src/lib_ut.rs b/libmodpak/src/lib_ut.rs index c255830c..cc623e8e 100644 --- a/libmodpak/src/lib_ut.rs +++ b/libmodpak/src/lib_ut.rs @@ -81,11 +81,11 @@ mod tests { fn remove_library_supports_exact_names() { let (root, mut repo) = seeded_repo(); - repo.remove_library(vec!["lib/lua/baz.lua".to_string()]).expect("exact library removal should succeed"); + repo.remove_library(vec!["lua/baz.lua".to_string()]).expect("exact library removal should succeed"); - assert!(!root.path().join("lib/lib/lua/baz.lua").exists()); - assert!(root.path().join("lib/lib/library/foo.py").exists()); - assert!(root.path().join("lib/lib/library/bar.py").exists()); + assert!(!root.path().join("lib/lua/baz.lua").exists()); + assert!(root.path().join("lib/library/foo.py").exists()); + assert!(root.path().join("lib/library/bar.py").exists()); } #[test] @@ -94,9 +94,9 @@ mod tests { repo.remove_library(vec!["library/*".to_string()]).expect("glob library removal should succeed"); - assert!(!root.path().join("lib/lib/library/foo.py").exists()); - assert!(!root.path().join("lib/lib/library/bar.py").exists()); - assert!(root.path().join("lib/lib/lua/baz.lua").exists()); + assert!(!root.path().join("lib/library/foo.py").exists()); + assert!(!root.path().join("lib/library/bar.py").exists()); + assert!(root.path().join("lib/lua/baz.lua").exists()); } #[test] @@ -213,7 +213,7 @@ mod tests { repo.add_library(src.path().to_path_buf()).expect("library tree should be indexed"); let library = repo.idx.library(); - let entry = library.get("lib/runtime/wasm/demo.wasm").expect("wasm library entry should exist"); + let entry = library.get("runtime/wasm/demo.wasm").expect("wasm library entry should exist"); assert_eq!(entry.kind(), "wasm"); } @@ -229,10 +229,33 @@ mod tests { repo.add_library(src.path().to_path_buf()).expect("library tree should be indexed"); let library = repo.idx.library(); - let entry = library.get("lib/runtime/native/demo").expect("binary library entry should exist"); + let entry = library.get("runtime/native/demo").expect("binary library entry should exist"); assert_eq!(entry.kind(), "binary"); } + #[test] + fn add_library_normalizes_nested_lib_root_and_purges_legacy_nested_tree() { + let root = tempfile::tempdir().expect("repo tempdir should be created"); + let src = tempfile::tempdir().expect("src tempdir should be created"); + let payload = src.path().join("lib/runtime/python3"); + fs::create_dir_all(&payload).expect("payload dir should be created"); + fs::write(payload.join("demo.py"), b"print('demo')").expect("payload file should be written"); + + let legacy = root.path().join("lib/lib/runtime/python3"); + fs::create_dir_all(&legacy).expect("legacy nested dir should be created"); + fs::write(legacy.join("stale.py"), b"print('stale')").expect("legacy stale file should be written"); + + let mut repo = SysInspectModPak::new(root.path().to_path_buf()).expect("repo should be created"); + repo.add_library(src.path().to_path_buf()).expect("library tree should be normalized and indexed"); + + assert!(!root.path().join("lib/lib").exists(), "legacy nested lib tree should be removed"); + assert!(root.path().join("lib/runtime/python3/demo.py").exists(), "normalized runtime payload should exist"); + + let library = repo.idx.library(); + assert!(library.contains_key("runtime/python3/demo.py")); + assert!(!library.contains_key("lib/runtime/python3/stale.py")); + } + #[test] fn add_module_installs_binary_under_namespace_path_not_source_filename() { let root = tempfile::tempdir().expect("repo tempdir should be created"); @@ -424,7 +447,7 @@ mod tests { write_module(&mut repo, "netbsd", "noarch", "runtime.lua", "runtime/lua"); repo.new_profile("toto").expect("profile should be created"); repo.add_profile_matches("toto", vec!["runtime.lua".to_string()], false).expect("module selector should be added"); - repo.add_profile_matches("toto", vec!["lib/runtime/lua/*.lua".to_string()], true).expect("library selector should be added"); + repo.add_profile_matches("toto", vec!["runtime/lua/*.lua".to_string()], true).expect("library selector should be added"); let rendered = repo.show_profile("toto").expect("profile should render"); let module_pos = rendered.find("runtime.lua").expect("module row should exist"); diff --git a/libmodpak/tests/profile_sync.rs b/libmodpak/tests/profile_sync.rs index a5295249..9db97a27 100644 --- a/libmodpak/tests/profile_sync.rs +++ b/libmodpak/tests/profile_sync.rs @@ -92,9 +92,9 @@ async fn narrow_profile_syncs_only_allowed_artifacts_and_removes_old_ones() { repo.new_profile("Alpha").expect("Alpha should be created"); repo.new_profile("Beta").expect("Beta should be created"); repo.add_profile_matches("Alpha", vec!["alpha.demo".to_string()], false).expect("Alpha module selector should be added"); - repo.add_profile_matches("Alpha", vec!["lib/runtime/lua/alpha.lua".to_string()], true).expect("Alpha library selector should be added"); + repo.add_profile_matches("Alpha", vec!["runtime/lua/alpha.lua".to_string()], true).expect("Alpha library selector should be added"); repo.add_profile_matches("Beta", vec!["beta.demo".to_string()], false).expect("Beta module selector should be added"); - repo.add_profile_matches("Beta", vec!["lib/runtime/lua/beta.lua".to_string()], true).expect("Beta library selector should be added"); + repo.add_profile_matches("Beta", vec!["runtime/lua/beta.lua".to_string()], true).expect("Beta library selector should be added"); let (port, server) = start_fileserver(master.path().join("data")).await; let minion = tempfile::tempdir().expect("minion tempdir should be created"); @@ -110,8 +110,8 @@ async fn narrow_profile_syncs_only_allowed_artifacts_and_removes_old_ones() { SysInspectModPakMinion::new(cfg.clone()).sync().await.expect("first sync should work"); assert!(share.path().join("modules/alpha/demo").exists()); assert!(!share.path().join("modules/beta/demo").exists()); - assert!(share.path().join("lib/lib/runtime/lua/alpha.lua").exists()); - assert!(!share.path().join("lib/lib/runtime/lua/beta.lua").exists()); + assert!(share.path().join("lib/runtime/lua/alpha.lua").exists()); + assert!(!share.path().join("lib/runtime/lua/beta.lua").exists()); TraitUpdateRequest::from_context(r#"{"op":"set","traits":{"minion.profile":["Beta"]}}"#) .expect("set request should parse") @@ -120,8 +120,8 @@ async fn narrow_profile_syncs_only_allowed_artifacts_and_removes_old_ones() { SysInspectModPakMinion::new(cfg).sync().await.expect("second sync should work"); assert!(!share.path().join("modules/alpha/demo").exists()); assert!(share.path().join("modules/beta/demo").exists()); - assert!(!share.path().join("lib/lib/runtime/lua/alpha.lua").exists()); - assert!(share.path().join("lib/lib/runtime/lua/beta.lua").exists()); + assert!(!share.path().join("lib/runtime/lua/alpha.lua").exists()); + assert!(share.path().join("lib/runtime/lua/beta.lua").exists()); server.abort(); } diff --git a/libsysinspect/src/cfg/mmconf.rs b/libsysinspect/src/cfg/mmconf.rs index e94a0e55..b18b9260 100644 --- a/libsysinspect/src/cfg/mmconf.rs +++ b/libsysinspect/src/cfg/mmconf.rs @@ -789,7 +789,8 @@ impl MinionConfig { } } - PathBuf::from(DEFAULT_MINION_MACHINE_ID) + let default = PathBuf::from(DEFAULT_MINION_MACHINE_ID); + if default.exists() { default } else { self.root_dir().join(DEFAULT_MINION_MACHINE_ID_REL) } } /// Return sharelib path @@ -1307,7 +1308,7 @@ pub struct HopstartConfig { impl HopstartConfig { pub fn batch(&self) -> usize { - self.batch.unwrap_or(10) + self.batch.unwrap_or(20) } pub fn network_forward(&self) -> bool { diff --git a/libsysinspect/src/cfg/mmconf_ut.rs b/libsysinspect/src/cfg/mmconf_ut.rs index 05228eea..dd95a5a3 100644 --- a/libsysinspect/src/cfg/mmconf_ut.rs +++ b/libsysinspect/src/cfg/mmconf_ut.rs @@ -67,7 +67,7 @@ fn master_cmdb_update_accepts_humantime_override() { fn master_hopstart_defaults_are_used_when_not_configured() { let cfg = MasterConfig::new(write_master_cfg("config:\n master:\n fileserver.models: []\n")).unwrap(); - assert_eq!(cfg.hopstart().batch(), 10); + assert_eq!(cfg.hopstart().batch(), 20); assert!(!cfg.hopstart().network_forward()); assert!(!cfg.hopstart().on_start()); } diff --git a/libsysinspect/src/console/mod.rs b/libsysinspect/src/console/mod.rs index 2e114349..4d8cf7d7 100644 --- a/libsysinspect/src/console/mod.rs +++ b/libsysinspect/src/console/mod.rs @@ -476,6 +476,15 @@ pub struct ConsoleModelRow { /// Entrypoint kind for each entry in `entrypoints`: "checkbook" or "entity". #[serde(default)] pub entrypoint_kinds: Vec, + /// Public entrypoint ids after applying the optional `interface:` section. + #[serde(default)] + pub public_entrypoints: Vec, + /// Entrypoint kind for each entry in `public_entrypoints`. + #[serde(default)] + pub public_entrypoint_kinds: Vec, + /// Public direct action entrypoints exported through `interface.actions`. + #[serde(default)] + pub public_actions: Vec, /// Declared state names across all actions. #[serde(default)] pub states: Vec, diff --git a/libsysinspect/src/inspector.rs b/libsysinspect/src/inspector.rs index e607a1f1..b55d58f4 100644 --- a/libsysinspect/src/inspector.rs +++ b/libsysinspect/src/inspector.rs @@ -197,7 +197,11 @@ impl SysInspectRunner { Err(err) => return Err(err), } } else { - log::warn!("Action {} skipped due to dependencies results mismatch", ac.id()) + log::warn!("Action {} skipped due to dependencies results mismatch", ac.id()); + if let Some(response) = ac.skipped_response(&self.cstr_s, &self.cstr_f) { + log::trace!("Synthetic skipped action response for '{}': {:#?}", ac.id(), response); + evtproc.lock().await.receiver().register(response.eid().to_owned(), response); + } } } Err(err) => return Err(err), diff --git a/libsysinspect/src/intp/actions.rs b/libsysinspect/src/intp/actions.rs index df1e5f57..c3e0ff5e 100644 --- a/libsysinspect/src/intp/actions.rs +++ b/libsysinspect/src/intp/actions.rs @@ -1,5 +1,8 @@ use super::{ - actproc::{modfinder::ModCall, response::ActionResponse}, + actproc::{ + modfinder::ModCall, + response::{ActionModResponse, ActionOutcome, ActionResponse, ConstraintResponse}, + }, constraints::Expression, functions, inspector::SysInspector, @@ -9,6 +12,7 @@ use colored::Colorize; use indexmap::IndexMap; use libcommon::SysinspectError; use serde::{Deserialize, Serialize}; +use serde_json::json; use serde_yaml::Value; use std::fmt::Display; @@ -234,6 +238,32 @@ impl Action { Ok(r_opt) } + pub fn skipped_response(&self, passed_constraints: &[String], failed_constraints: &[String]) -> Option { + let call = self.call.as_ref()?; + let if_true = self.if_true(); + let if_false = self.if_false(); + let missing_true = if_true.iter().filter(|c| !passed_constraints.contains(*c)).cloned().collect::>(); + let missing_false = if_false.iter().filter(|c| !failed_constraints.contains(*c)).cloned().collect::>(); + + let mut response = ActionModResponse::with_retcode(0); + response.set_outcome(ActionOutcome::NotApplicable); + response.set_message("Branch not selected by DSL constraints".to_string()); + response.set_data(json!({ + "not_applicable": true, + "reason": "dependencies_mismatch", + "required_true": if_true, + "required_false": if_false, + "missing_true": missing_true, + "missing_false": missing_false, + "matched_true": passed_constraints, + "matched_false": failed_constraints, + "module": self.module(), + "description": self.descr(), + })); + + Some(ActionResponse::new(call.eid().to_string(), call.aid().to_string(), call.state(), response, ConstraintResponse::default())) + } + /// Forward logs value (string/array/anything) to internal logger. fn forward_logs(logs_val: &serde_json::Value) { match logs_val { diff --git a/libsysinspect/src/intp/actproc/modfinder.rs b/libsysinspect/src/intp/actproc/modfinder.rs index 6008dc4a..ca420940 100644 --- a/libsysinspect/src/intp/actproc/modfinder.rs +++ b/libsysinspect/src/intp/actproc/modfinder.rs @@ -565,6 +565,14 @@ impl ModCall { self.state.to_owned() } + pub fn aid(&self) -> &str { + &self.aid + } + + pub fn eid(&self) -> &str { + &self.eid + } + /// Get state ref pub fn with_state(&self, state: String) -> bool { self.state == state diff --git a/libsysinspect/src/intp/actproc/response.rs b/libsysinspect/src/intp/actproc/response.rs index 16d11b71..5eb8d3f4 100644 --- a/libsysinspect/src/intp/actproc/response.rs +++ b/libsysinspect/src/intp/actproc/response.rs @@ -6,6 +6,15 @@ use chrono::{DateTime, Utc}; use serde::{Deserialize, Serialize}; use serde_json::{Map, Value}; +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)] +#[serde(rename_all = "snake_case")] +pub enum ActionOutcome { + #[default] + Success, + Error, + NotApplicable, +} + /// This struct is a future carrier of tracability. /// Currently only a single string log message. #[derive(Debug, Clone, Serialize, Deserialize)] @@ -118,6 +127,9 @@ pub struct ActionModResponse { // Return code retcode: i32, + #[serde(default)] + outcome: ActionOutcome, + // Warnings collection warning: Option>, @@ -131,7 +143,13 @@ pub struct ActionModResponse { impl ActionModResponse { /// Create new with return code pub fn with_retcode(retcode: i32) -> Self { - Self { retcode, warning: None, message: String::new(), data: None } + Self { + retcode, + outcome: if retcode == 0 { ActionOutcome::Success } else { ActionOutcome::Error }, + warning: None, + message: String::new(), + data: None, + } } /// Get a return code @@ -139,6 +157,26 @@ impl ActionModResponse { self.retcode } + pub fn outcome(&self) -> ActionOutcome { + if self.outcome == ActionOutcome::Success && self.retcode > 0 { ActionOutcome::Error } else { self.outcome } + } + + pub fn set_outcome(&mut self, outcome: ActionOutcome) { + self.outcome = outcome; + } + + pub fn is_success(&self) -> bool { + self.outcome() == ActionOutcome::Success + } + + pub fn is_error(&self) -> bool { + self.outcome() == ActionOutcome::Error + } + + pub fn is_not_applicable(&self) -> bool { + self.outcome() == ActionOutcome::NotApplicable + } + /// Return collected warnings pub fn warnings(&self) -> Vec { if let Some(w) = &self.warning { diff --git a/libsysinspect/src/mdescr/browse_types.rs b/libsysinspect/src/mdescr/browse_types.rs index ca426588..8bad1d1d 100644 --- a/libsysinspect/src/mdescr/browse_types.rs +++ b/libsysinspect/src/mdescr/browse_types.rs @@ -70,6 +70,8 @@ pub struct BrowsedModel { pub entities: Vec, pub relations: Vec, pub entrypoints: Vec, + pub public_entrypoints: Vec, + pub public_actions: Vec, pub actions: Vec, /// Deduplicated list of all declared action state keys. pub states: Vec, diff --git a/libsysinspect/src/mdescr/browser.rs b/libsysinspect/src/mdescr/browser.rs index 8412cbae..6b5ac8f8 100644 --- a/libsysinspect/src/mdescr/browser.rs +++ b/libsysinspect/src/mdescr/browser.rs @@ -11,11 +11,13 @@ static CTX_FN_RE: LazyLock = LazyLock::new(|| Regex::new(r"context\((\w+) use crate::{ cfg::mmconf::MinionConfig, intp::{actions::Action, entities::Entity, relations::Relation}, - mdescr::{DSL_DIR_ACTIONS, DSL_DIR_ENTITIES, DSL_DIR_RELATIONS, DSL_IDX_CHECKBOOK}, + mdescr::{DSL_DIR_ACTIONS, DSL_DIR_ENTITIES, DSL_DIR_INTERFACE, DSL_DIR_RELATIONS, DSL_IDX_CHECKBOOK}, }; use super::{browse_types::*, mspec, mspecdef::ModelSpec}; +type InterfaceLists = (Option>, Option>, Option>, Vec); + /// Read-only browser for one model. /// /// Built on `ModelSpec` loaded by `mspec::load()`. The browser @@ -28,6 +30,50 @@ pub struct ModelBrowser { } impl ModelBrowser { + fn interface_lists(&self) -> InterfaceLists { + let mut diagnostics = Vec::new(); + let Some(section) = self.spec.top(DSL_DIR_INTERFACE) else { + return (None, None, None, diagnostics); + }; + let Some(mapping) = section.as_mapping() else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: "interface section is not a mapping and will be ignored".to_string(), + path: Some("interface".to_string()), + }); + return (None, None, None, diagnostics); + }; + + let parse_list = |key: &str, diagnostics: &mut Vec| -> Option> { + let raw = mapping.get(serde_yaml::Value::String(key.to_string()))?; + let Some(seq) = raw.as_sequence() else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: format!("interface.{key} is not a list and will be ignored"), + path: Some(format!("interface.{key}")), + }); + return None; + }; + let mut items = Vec::new(); + for item in seq { + if let Some(name) = item.as_str() { + if !items.iter().any(|existing| existing == name) { + items.push(name.to_string()); + } + } else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: format!("interface.{key} contains a non-string entry and it was ignored"), + path: Some(format!("interface.{key}")), + }); + } + } + Some(items) + }; + + (parse_list("checkbook", &mut diagnostics), parse_list("entities", &mut diagnostics), parse_list("actions", &mut diagnostics), diagnostics) + } + /// Load a model from the given directory path. /// /// Internally calls `mspec::load()` to walk the directory, @@ -417,6 +463,64 @@ impl ModelBrowser { let (actions, a_diags) = self.actions(); diagnostics.extend(a_diags); + let (interface_checkbooks, interface_entities, interface_actions, interface_diags) = self.interface_lists(); + diagnostics.extend(interface_diags); + + let public_entrypoints = if interface_checkbooks.is_none() && interface_entities.is_none() { + entrypoints.clone() + } else { + let mut out = Vec::new(); + if let Some(labels) = interface_checkbooks { + for label in labels { + if let Some(ep) = + entrypoints.iter().find(|ep| matches!(ep, BrowsedEntrypoint::CheckbookLabel { label: existing, .. } if existing == &label)) + { + out.push(ep.clone()); + } else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: format!("interface.checkbook references unknown checkbook \"{label}\""), + path: Some("interface.checkbook".to_string()), + }); + } + } + } + if let Some(ids) = interface_entities { + for id in ids { + if let Some(ep) = entrypoints.iter().find(|ep| matches!(ep, BrowsedEntrypoint::Entity { id: existing, .. } if existing == &id)) { + out.push(ep.clone()); + } else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: format!("interface.entities references unknown entity \"{id}\""), + path: Some("interface.entities".to_string()), + }); + } + } + } + out + }; + + let public_actions = if let Some(interface_actions) = interface_actions { + let mut out = Vec::new(); + for aid in interface_actions { + if actions.iter().any(|action| action.action_id == aid) { + if !out.iter().any(|existing| existing == &aid) { + out.push(aid); + } + } else { + diagnostics.push(ModelBrowseDiagnostic { + level: ModelBrowseDiagnosticLevel::Warning, + message: format!("interface.actions references unknown action \"{aid}\""), + path: Some("interface.actions".to_string()), + }); + } + } + out + } else { + Vec::new() + }; + let states = self.states(); // Deduplicate: keep only the first occurrence of each (level, message, path) tuple. @@ -426,7 +530,17 @@ impl ModelBrowser { seen.insert(key) }); - Ok(BrowsedModel { metadata: self.metadata(), entities, relations, entrypoints, actions, states, diagnostics }) + Ok(BrowsedModel { + metadata: self.metadata(), + entities, + relations, + entrypoints, + public_entrypoints, + public_actions, + actions, + states, + diagnostics, + }) } /// Build entrypoints from already-extracted entities and relations, diff --git a/libsysinspect/src/mdescr/browser_ut.rs b/libsysinspect/src/mdescr/browser_ut.rs index b56b5e15..44e0c49f 100644 --- a/libsysinspect/src/mdescr/browser_ut.rs +++ b/libsysinspect/src/mdescr/browser_ut.rs @@ -1011,6 +1011,103 @@ actions: // No diagnostics expected for this valid model assert!(summary.diagnostics.is_empty()); + assert_eq!(summary.public_entrypoints.len(), summary.entrypoints.len()); + assert!(summary.public_actions.is_empty()); +} + +#[test] +fn interface_filters_public_entrypoints_and_actions() { + let td = tempfile::TempDir::new().unwrap(); + write_model( + &td, + r#" +name: Interface Test +version: "0.1" +description: Explicit public interface. +maintainer: tester + +interface: + entities: + - all + actions: + - python-proof + +entities: + all: + descr: Main public entrypoint + helper: + descr: Internal helper + +actions: + python-proof: + descr: Public action entrypoint + module: sys.run + bind: [all] + state: + $: + args: + cmd: echo proof + + helper-action: + descr: Internal action + module: sys.run + bind: [helper] + state: + $: + args: + cmd: echo helper +"#, + ); + + let browser = ModelBrowser::load(Arc::new(MinionConfig::default()), td.path()).expect("load should succeed"); + let summary = browser.summarize().expect("summarize should succeed"); + + assert!(summary.public_entrypoints.iter().any(|ep| matches!(ep, BrowsedEntrypoint::Entity { id, .. } if id == "all"))); + assert!(!summary.public_entrypoints.iter().any(|ep| matches!(ep, BrowsedEntrypoint::Entity { id, .. } if id == "helper"))); + assert_eq!(summary.public_actions, vec!["python-proof".to_string()]); +} + +#[test] +fn interface_unknown_members_emit_diagnostics() { + let td = tempfile::TempDir::new().unwrap(); + write_model( + &td, + r#" +name: Interface Diagnostics +version: "0.1" +description: Bad interface references. +maintainer: tester + +interface: + checkbook: + - main-check + entities: + - ghost-entity + actions: + - ghost-action + +entities: + all: + descr: Main entity + +actions: + real-action: + descr: Internal action + module: sys.run + bind: [all] + state: + $: + args: + cmd: echo ok +"#, + ); + + let browser = ModelBrowser::load(Arc::new(MinionConfig::default()), td.path()).expect("load should succeed"); + let summary = browser.summarize().expect("summarize should succeed"); + + assert!(summary.diagnostics.iter().any(|d| d.message.contains("interface.checkbook") && d.message.contains("main-check"))); + assert!(summary.diagnostics.iter().any(|d| d.message.contains("interface.entities") && d.message.contains("ghost-entity"))); + assert!(summary.diagnostics.iter().any(|d| d.message.contains("interface.actions") && d.message.contains("ghost-action"))); } // Smoke test against real example diff --git a/libsysinspect/src/mdescr/mod.rs b/libsysinspect/src/mdescr/mod.rs index 9ec7116a..84431748 100644 --- a/libsysinspect/src/mdescr/mod.rs +++ b/libsysinspect/src/mdescr/mod.rs @@ -14,6 +14,7 @@ mod catalog_ut; /// DSL directives pub static DSL_DIR_ENTITIES: &str = "entities"; pub static DSL_DIR_ACTIONS: &str = "actions"; +pub static DSL_DIR_INTERFACE: &str = "interface"; pub static DSL_DIR_RELATIONS: &str = "relations"; pub static DSL_DIR_CONSTRAINTS: &str = "constraints"; diff --git a/libsysinspect/src/reactor/handlers/cstr_stdhdl.rs b/libsysinspect/src/reactor/handlers/cstr_stdhdl.rs index 0948a27d..59f11764 100644 --- a/libsysinspect/src/reactor/handlers/cstr_stdhdl.rs +++ b/libsysinspect/src/reactor/handlers/cstr_stdhdl.rs @@ -50,6 +50,10 @@ impl EventHandler for ConstraintHandler { return; } + if evt.response.is_not_applicable() { + return; + } + let prefix = self.get_prefix(); if evt.constraints.is_info() { diff --git a/libsysinspect/src/reactor/handlers/stdhdl.rs b/libsysinspect/src/reactor/handlers/stdhdl.rs index 964730ed..9d7a976a 100644 --- a/libsysinspect/src/reactor/handlers/stdhdl.rs +++ b/libsysinspect/src/reactor/handlers/stdhdl.rs @@ -41,7 +41,7 @@ impl EventHandler for StdoutEventHandler { } } - if evt.response.retcode() == 0 { + if evt.response.is_success() { log::info!("{}{}/{} - {}", prefix, evt.eid().bright_cyan(), evt.aid().bright_cyan(), evt.response.message()); if verbose && let Some(data) = evt.response.data() { log::info!( @@ -52,6 +52,17 @@ impl EventHandler for StdoutEventHandler { KeyValueFormatter::new(data).format() ); } + } else if evt.response.is_not_applicable() { + log::info!("{}{}/{} (Not applicable) - {}", prefix, evt.eid().bright_cyan(), evt.aid().bright_cyan(), evt.response.message()); + if verbose && let Some(data) = evt.response.data() { + log::info!( + "{}{}/{} - Other data:\n{}", + prefix, + evt.eid().bright_cyan(), + evt.aid().bright_cyan(), + KeyValueFormatter::new(data).format() + ); + } } else { log::error!( "{}{}/{} (Error: {}) - {}", diff --git a/src/netadd/workflow.rs b/src/netadd/workflow.rs index f32f17e7..05e58e1b 100644 --- a/src/netadd/workflow.rs +++ b/src/netadd/workflow.rs @@ -615,7 +615,8 @@ impl HostSetup { } fn read_minion_id(&self, ssh: &SSHSession) -> Result, SysinspectError> { - let rsp = ssh.exec(&RemoteCommand::new(format!("cat {} 2>/dev/null || true", shell_quote(&self.target.layout.machine_id))))?; + let cmd = format!("{} -c {} --id", shell_quote(&self.target.layout.install_bin), shell_quote(&self.target.layout.config)); + let rsp = ssh.exec(&RemoteCommand::new(cmd))?; let mid = rsp.stdout.trim(); Ok((!mid.is_empty()).then(|| mid.to_string())) } diff --git a/src/ui/alert.rs b/src/ui/alert.rs index a739da40..1a02fbbf 100644 --- a/src/ui/alert.rs +++ b/src/ui/alert.rs @@ -390,6 +390,25 @@ impl SysInspectUX { Some((10.0, &[palette::GRAY_0, palette::BG_2] as &[Color])), ) } + 4 => Self::_popup_ex( + parent, + buf, + Some("Cluster Operation"), + "\nStart every offline minion\nin the cluster?", + None, + Alignment::Center, + self.cluster_confirm_choice.clone(), + AlertButtons::YesNo, + Some(0), + Some(palette::PROCESSING_PEAK), + None, + None, + Some(palette::WHITE), + None, + None, + None, + Some((10.0, &[palette::GRAY_0, palette::BG_2] as &[Color])), + ), _ => return, }; self.popup_button_rects.set(Some(rects)); @@ -471,6 +490,44 @@ impl SysInspectUX { Self::draw_popup_shadow(buf, canvas, height); } + pub fn dialog_cluster_start_progress(&self, parent: Rect, buf: &mut Buffer) { + if !self.cluster_start_progress.visible { + return; + } + + let text = Line::from(vec![Span::styled( + format!("{} {}", self.cluster_start_progress.spinner.view(), self.cluster_start_progress.message), + Style::default().fg(palette::FG), + )]); + let width = (UnicodeWidthStr::width(self.cluster_start_progress.message.as_str()) as u16 + 12).max(48); + let height = 5u16; + let x = parent.x + (parent.width.saturating_sub(width)) / 2; + let y = parent.y + (parent.height.saturating_sub(height)) / 2; + let canvas = Rect { x, y, width, height }; + + Clear.render(canvas, buf); + + let popup_block = Block::default() + .borders(Borders::ALL) + .border_type(ratatui::widgets::BorderType::Rounded) + .border_style(Style::default().fg(palette::PROCESSING_PEAK)) + .padding(Padding::horizontal(2)) + .style(Style::default().bg(palette::POPUP_BG_BASE)); + let popup_inner = popup_block.inner(canvas); + popup_block.render(canvas, buf); + + let [_, text_area, _]: [Rect; 3] = Layout::default() + .direction(Direction::Vertical) + .constraints([Constraint::Length(1), Constraint::Length(1), Constraint::Min(0)]) + .split(popup_inner) + .as_ref() + .try_into() + .unwrap(); + Paragraph::new(text).alignment(Alignment::Center).render(text_area, buf); + + Self::draw_popup_shadow(buf, canvas, height); + } + pub fn dialog_master_confirm(&self, parent: Rect, buf: &mut Buffer) { if !self.master_confirm_visible { return; diff --git a/src/ui/dslbrowser.rs b/src/ui/dslbrowser.rs index bccebc82..26c36d91 100644 --- a/src/ui/dslbrowser.rs +++ b/src/ui/dslbrowser.rs @@ -208,8 +208,13 @@ impl DslBrowser { if let Some(row) = self.resolved_model() { let mut checkbook: Vec = Vec::new(); let mut entities: Vec = Vec::new(); - for (i, entrypoint) in row.entrypoints.iter().enumerate() { - let kind = row.entrypoint_kinds.get(i).map(|s| s.as_str()).unwrap_or("entity"); + let (entrypoints, entrypoint_kinds) = if !row.public_entrypoints.is_empty() || !row.public_actions.is_empty() { + (&row.public_entrypoints, &row.public_entrypoint_kinds) + } else { + (&row.entrypoints, &row.entrypoint_kinds) + }; + for (i, entrypoint) in entrypoints.iter().enumerate() { + let kind = entrypoint_kinds.get(i).map(|s| s.as_str()).unwrap_or("entity"); if kind == "checkbook" { checkbook.push(entrypoint.clone()); } else { diff --git a/src/ui/elements.rs b/src/ui/elements.rs index 3e437a58..fdbeeed8 100644 --- a/src/ui/elements.rs +++ b/src/ui/elements.rs @@ -149,19 +149,31 @@ impl EventListItem { Cell::from(v).style(Style::default().fg(palette::ERROR).add_modifier(Modifier::BOLD)) } + fn outcome(&self) -> String { + self.event.get_response().get("outcome").and_then(|value| value.as_str()).map(str::to_string).unwrap_or_else(|| { + if as_int(self.event.get_response().get("retcode").cloned()) == 0 { "success".to_string() } else { "error".to_string() } + }) + } + pub fn get_aligned_line(&self, left_pad: usize) -> Line<'static> { let arrow = " \u{27A4} "; let t = self.title().replace(" with ", arrow); + let (text_style, arrow_style) = match self.outcome().as_str() { + "error" => { + (Style::default().fg(palette::ERROR).add_modifier(Modifier::BOLD), Style::default().fg(palette::ERROR).add_modifier(Modifier::BOLD)) + } + "not_applicable" => ( + Style::default().fg(palette::WARNING).add_modifier(Modifier::BOLD), + Style::default().fg(palette::WARNING).add_modifier(Modifier::BOLD), + ), + _ => (Style::default().fg(palette::FG), Style::default().fg(palette::PROCESSING).add_modifier(Modifier::BOLD)), + }; if let Some(pos) = t.find(arrow) { let left = right_pad(&t[..pos], left_pad); let after = &t[pos + arrow.len()..]; - Line::from(vec![ - Span::styled(left, Style::default().fg(palette::FG)), - Span::styled(arrow, Style::default().fg(palette::PROCESSING).add_modifier(Modifier::BOLD)), - Span::styled(after.to_string(), Style::default().fg(palette::FG)), - ]) + Line::from(vec![Span::styled(left, text_style), Span::styled(arrow, arrow_style), Span::styled(after.to_string(), text_style)]) } else { - Line::from(vec![Span::styled(t, Style::default().fg(palette::FG))]) + Line::from(vec![Span::styled(t, text_style)]) } } @@ -172,14 +184,15 @@ impl EventListItem { /// Get events data table pub fn get_event_table(&self, keywidth: usize) -> Vec> { + let outcome = self.outcome(); vec![ Row::new(vec![Self::yc("Info:".to_string(), keywidth), Self::gc(as_str(self.event.get_response().get("message").cloned()))]), Row::new(vec![ - Self::yc("Return code:".to_string(), keywidth), - if as_int(self.event.get_response().get("retcode").cloned()) == 0 { - Self::grc("Success".to_string()) - } else { - Self::rc(format!("Error - {}", as_int(self.event.get_response().get("retcode").cloned()))) + Self::yc("Outcome:".to_string(), keywidth), + match outcome.as_str() { + "error" => Self::rc(format!("Error - {}", as_int(self.event.get_response().get("retcode").cloned()))), + "not_applicable" => Cell::from("Not Applicable").style(Style::default().fg(palette::WARNING).add_modifier(Modifier::BOLD)), + _ => Self::grc("Success".to_string()), }, ]), Row::new(vec![Self::yc("Occurred:".to_string(), keywidth), Self::gc(self.event.get_timestamp())]), @@ -195,7 +208,26 @@ impl DbListItem for EventListItem { type EventType = EventData; fn title(&self) -> String { - as_str(self.event.get_constraints().get("descr").cloned()) + let descr = as_str(self.event.get_constraints().get("descr").cloned()); + if !descr.trim().is_empty() { + return descr; + } + + let action = self.event.get_action_id(); + let module = + self.event.get_response().get("data").and_then(|data| data.get("module")).map(|value| as_str(Some(value.clone()))).unwrap_or_default(); + + if !action.trim().is_empty() && !module.trim().is_empty() { + return format!("{action} with {module}"); + } + if !action.trim().is_empty() { + return action; + } + if !module.trim().is_empty() { + return module; + } + + "failed action".to_string() } /// Stub diff --git a/src/ui/macts.rs b/src/ui/macts.rs index 46ce9e0a..8e028d95 100644 --- a/src/ui/macts.rs +++ b/src/ui/macts.rs @@ -25,7 +25,7 @@ const MENU_SECTIONS: &[MenuSection] = &[ }, MenuSection { title: "Cluster Operations", - items: &[("Shutdown everything", "^X"), ("Reconnect all minions", "^A"), ("Register a new minion", "INS")], + items: &[("Start all minions", "^H"), ("Shutdown everything", "^X"), ("Reconnect all minions", "^A"), ("Register a new minion", "INS")], }, ]; diff --git a/src/ui/minreg.rs b/src/ui/minreg.rs index 70495c3d..c39fbe9a 100644 --- a/src/ui/minreg.rs +++ b/src/ui/minreg.rs @@ -461,7 +461,7 @@ pub fn render_progress(progress: &RegistrationProgress, parent: Rect, buf: &mut /// Handle keyboard input for the progress popup. pub fn handle_progress_key(key: KeyEvent, progress: &mut RegistrationProgress) -> bool { - if !progress.visible || progress.done { + if !progress.visible || (progress.done && progress.error.is_none()) { if matches!(key.code, KeyCode::Esc | KeyCode::Enter) || key.code == KeyCode::Char(' ') { return true; } diff --git a/src/ui/mod.rs b/src/ui/mod.rs index a9fa6dbc..0b951b0a 100644 --- a/src/ui/mod.rs +++ b/src/ui/mod.rs @@ -22,10 +22,10 @@ use libsysinspect::{ use libsysproto::query::{ SCHEME_COMMAND, commands::{ - CLUSTER_CONFIG_RELOAD, CLUSTER_LIBRARY_INDEX, CLUSTER_MARK_UPGRADE_REQUIRED, CLUSTER_MASTER_LOGS, CLUSTER_MINION_HOPSTART, + CLUSTER_CONFIG_RELOAD, CLUSTER_HOPSTART, CLUSTER_LIBRARY_INDEX, CLUSTER_MARK_UPGRADE_REQUIRED, CLUSTER_MASTER_LOGS, CLUSTER_MINION_HOPSTART, CLUSTER_MINION_INFO, CLUSTER_MINION_LOGS, CLUSTER_MINION_PROCESS_SIGNAL, CLUSTER_MINION_RECONNECT, CLUSTER_MINION_SHUTDOWN, CLUSTER_MINION_TOP, CLUSTER_MODELS, CLUSTER_MODULE_INDEX, CLUSTER_ONLINE_MINIONS, CLUSTER_PROFILE, CLUSTER_RECONNECT, CLUSTER_REMOVE_MINION, - CLUSTER_SHUTDOWN, CLUSTER_TRAITS_UPDATE, CLUSTER_UPGRADE_MINIONS, CLUSTER_UPGRADE_STATUS, + CLUSTER_SHUTDOWN, CLUSTER_SYNC, CLUSTER_TRAITS_UPDATE, CLUSTER_UPGRADE_MINIONS, CLUSTER_UPGRADE_STATUS, }, }; use ratatui::{ @@ -149,6 +149,25 @@ impl Default for ClusterUpgradeProgressState { type ClusterUpgradeTaskResult = Result<(usize, usize, usize, usize, usize, Vec), String>; +type ClusterStartTaskResult = Result<(usize, Vec), String>; + +#[derive(Debug)] +pub struct ClusterStartProgressState { + pub visible: bool, + pub message: String, + pub spinner: spinner::Model, + pub last_tick: Instant, +} + +impl Default for ClusterStartProgressState { + fn default() -> Self { + let mut spinner_model = spinner::Model::new(); + spinner_model.spinner = spinner::Spinner::mini_dot(); + spinner_model.style = Style::default().fg(palette::PROCESSING_PEAK); + Self { visible: false, message: String::new(), spinner: spinner_model, last_tick: Instant::now() } + } +} + #[derive(Debug)] pub struct SysInspectUX { exit: bool, @@ -288,6 +307,8 @@ pub struct SysInspectUX { pub cluster_upgrade_unreachable_count: usize, pub cluster_upgrade_pending_count: usize, pub cluster_upgrade_check_message: Option, + pub cluster_start_progress: ClusterStartProgressState, + pub cluster_start_task: Option>, // Exit-after-popup state (for setup config-written notice) pub pending_exit: bool, @@ -413,6 +434,8 @@ impl Default for SysInspectUX { cluster_upgrade_unreachable_count: 0, cluster_upgrade_pending_count: 0, cluster_upgrade_check_message: None, + cluster_start_progress: ClusterStartProgressState::default(), + cluster_start_task: None, pending_exit: false, pending_exit_message: None, @@ -793,6 +816,7 @@ impl SysInspectUX { || self.registration_progress.lock().unwrap().visible || self.delete_progress.visible || self.cluster_upgrade_progress.visible + || self.cluster_start_progress.visible { Duration::from_millis(50) } else { @@ -812,8 +836,18 @@ impl SysInspectUX { } } else { if !self.offline { + let selected_sid = if self.cycles_buf.is_empty() { None } else { Some(self.get_selected_cycle().event().sid().to_string()) }; match self.get_cycles() { - Ok(cycles) => self.cycles_buf = cycles, + Ok(cycles) => { + self.cycles_buf = cycles; + if let Some(selected_sid) = selected_sid + && let Some(idx) = self.cycles_buf.iter().position(|cycle| cycle.event().sid() == selected_sid) + { + self.selected_cycle = idx; + } else if self.selected_cycle >= self.cycles_buf.len() { + self.selected_cycle = self.cycles_buf.len().saturating_sub(1); + } + } Err(_) => { self.offline = true; self.evtipc = None; @@ -824,6 +858,7 @@ impl SysInspectUX { if self.minions_visible { self.refresh_minions(); } + self.refresh_selected_cycle_contents_preserve_selection(); if self.minion_logs_visible && self.minion_logs_polling && self.minion_logs_last_fetch.elapsed() >= Duration::from_secs(3) { match self.load_selected_minion_logs() { Ok(()) => self.minion_logs_online = true, @@ -854,6 +889,11 @@ impl SysInspectUX { self.cluster_upgrade_progress.spinner.update(tick); self.cluster_upgrade_progress.last_tick = Instant::now(); } + if self.cluster_start_progress.visible && self.cluster_start_progress.last_tick.elapsed() >= self.cluster_start_progress.spinner.spinner.fps { + let tick = self.cluster_start_progress.spinner.tick(); + self.cluster_start_progress.spinner.update(tick); + self.cluster_start_progress.last_tick = Instant::now(); + } if self.delete_progress.visible && self.delete_task.as_ref().is_some_and(|task| task.is_finished()) && let Some(task) = self.delete_task.take() @@ -914,6 +954,37 @@ impl SysInspectUX { } } } + if self.cluster_start_progress.visible + && self.cluster_start_task.as_ref().is_some_and(|task| task.is_finished()) + && let Some(task) = self.cluster_start_task.take() + { + let result = tokio::task::block_in_place(|| tokio::runtime::Handle::current().block_on(task)); + self.cluster_start_progress.visible = false; + self.cluster_start_progress.message.clear(); + self.restore_status(); + self.status_at_minions_browser(); + match result { + Ok(Ok((count, items))) => { + let failed = items.len(); + let ok = count.saturating_sub(failed); + if failed == 0 { + self.info_alert_message = format!("All {ok} minion(s) started successfully"); + } else { + self.info_alert_message = format!("{ok}/{count} minions started"); + } + self.info_alert_title = "Cluster Start".to_string(); + self.info_alert_visible = true; + } + Ok(Err(err)) => { + self.error_alert_visible = true; + self.error_alert_message = err; + } + Err(err) => { + self.error_alert_visible = true; + self.error_alert_message = format!("Cluster start task failed: {err}"); + } + } + } // Process file picker result for repo manager if self.repo_manager.visible && let Some(path) = self.file_picker.selected.take() @@ -937,7 +1008,7 @@ impl SysInspectUX { if self.repo_manager.active_tab == 4 { let _ = self.load_platforms(); } - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); } } else { // Track that a reload is needed when progress finishes @@ -1360,10 +1431,12 @@ impl SysInspectUX { } else if self.minions_menu_sel == 6 { self.open_cluster_confirm(3); } else if self.minions_menu_sel == 7 { - self.open_cluster_confirm(1); + self.open_cluster_confirm(4); } else if self.minions_menu_sel == 8 { - self.open_cluster_confirm(2); + self.open_cluster_confirm(1); } else if self.minions_menu_sel == 9 { + self.open_cluster_confirm(2); + } else if self.minions_menu_sel == 10 { self.registration_form.visible = true; } } @@ -1583,6 +1656,7 @@ impl SysInspectUX { || self.registration_progress.lock().unwrap().visible || self.delete_progress.visible || self.cluster_upgrade_progress.visible + || self.cluster_start_progress.visible } fn sync_main_focus_for_overlays(&mut self) { @@ -1661,6 +1735,10 @@ impl SysInspectUX { } } + fn mark_repo_sync_pending(&mut self) { + self.repo_manager.pending_cluster_upgrade = true; + } + fn start_cluster_upgrade(&mut self) { self.cluster_upgrade_progress.visible = true; self.cluster_upgrade_progress.message = "Applying repository updates across the cluster...".to_string(); @@ -1683,6 +1761,38 @@ impl SysInspectUX { })); } + fn start_cluster_sync(&mut self) { + match tokio::task::block_in_place(|| { + tokio::runtime::Handle::current() + .block_on(async { call_master_console(&self.cfg, &format!("{SCHEME_COMMAND}{CLUSTER_SYNC}"), "*", None, None, None).await }) + }) { + Ok(resp) => { + self.refresh_cluster_upgrade_status(); + self.refresh_minions(); + match resp.payload { + ConsolePayload::Ack { items, .. } => { + self.info_alert_visible = true; + self.info_alert_title = "Cluster Sync".to_string(); + self.info_alert_styled = None; + self.info_alert_message = if items.is_empty() { + "Cluster sync dispatched".to_string() + } else { + format!("Cluster sync dispatched\n\n{}", items.join("\n")) + }; + } + other => { + self.error_alert_visible = true; + self.error_alert_message = format!("Unexpected console payload for cluster sync: {other:?}"); + } + } + } + Err(err) => { + self.error_alert_visible = true; + self.error_alert_message = format!("Failed to run cluster sync: {err}"); + } + } + } + fn load_selected_minion_info(&mut self) { if self.minion_traits_modified { return; @@ -1835,6 +1945,10 @@ impl SysInspectUX { self.open_cluster_confirm(1); true } + KeyCode::Char('h') if e.modifiers.contains(KeyModifiers::CONTROL) => { + self.open_cluster_confirm(4); + true + } KeyCode::Char('a') if e.modifiers.contains(KeyModifiers::CONTROL) => { self.open_cluster_confirm(2); true @@ -1909,6 +2023,7 @@ impl SysInspectUX { 1 => self.do_cluster_shutdown(), 2 => self.do_cluster_reconnect(), 3 => self.do_minion_delete(self.delete_force_remove), + 4 => self.do_cluster_hopstart(), _ => {} } } @@ -1954,6 +2069,26 @@ impl SysInspectUX { } } + fn do_cluster_hopstart(&mut self) { + self.cluster_start_progress.visible = true; + self.cluster_start_progress.message = "Booting cluster, please wait...".to_string(); + self.cluster_start_progress.last_tick = Instant::now(); + self.status_text = Line::from(vec![ + Span::styled(" Esc ", Style::default().fg(palette::FG)), + Span::styled("wait for completion", Style::default().fg(palette::FAINT)), + ]); + let cfg = self.cfg.clone(); + self.cluster_start_task = Some(tokio::spawn(async move { + call_master_console(&cfg, &format!("{SCHEME_COMMAND}{CLUSTER_HOPSTART}"), "*", None, None, None) + .await + .map_err(|err| err.to_string()) + .and_then(|resp| match resp.payload { + ConsolePayload::Ack { count, items, .. } => Ok((count, items)), + _ => Err("Unexpected console payload for cluster start".to_string()), + }) + })); + } + fn do_cluster_reconnect(&mut self) { match tokio::task::block_in_place(|| { tokio::runtime::Handle::current() @@ -2515,12 +2650,26 @@ impl SysInspectUX { self.error_alert_visible = true; self.error_alert_message = "No items selected".to_string(); } else { + let staging_mode = self.repo_manager.staging_mode; + let cross_platform_delete = self.repo_manager.cross_platform_delete; self.repo_manager.exit_staging(); - if self.repo_manager.cross_platform_delete { - let names: Vec = checked.iter().map(|m| m.name.clone()).collect(); - self.bulk_delete_modules(&names); - } else { - self.bulk_delete_single_platform(&checked); + match staging_mode { + repomanager::StagingMode::LibraryDelete => { + let names: Vec = checked.iter().map(|m| m.name.clone()).collect(); + self.bulk_delete_libraries(&names); + } + repomanager::StagingMode::ModuleDelete => { + if cross_platform_delete { + let names: Vec = checked.iter().map(|m| m.name.clone()).collect(); + self.bulk_delete_modules(&names); + } else { + self.bulk_delete_single_platform(&checked); + } + } + _ => { + let names: Vec = checked.iter().map(|m| m.name.clone()).collect(); + self.bulk_delete_modules(&names); + } } } } @@ -2704,9 +2853,14 @@ impl SysInspectUX { } self.repo_manager.models_dirty = false; } + let start_repo_sync = self.repo_manager.pending_cluster_upgrade; + self.repo_manager.pending_cluster_upgrade = false; self.repo_manager.exit_staging(); self.repo_manager.visible = false; self.status_at_cycles(); + if start_repo_sync { + self.start_cluster_sync(); + } } KeyCode::Left => { self.repo_manager.active_tab = self.repo_manager.active_tab.saturating_sub(1); @@ -2883,6 +3037,7 @@ impl SysInspectUX { } } else if self.repo_manager.active_tab == 1 && !self.repo_manager.lib_rows.is_empty() { self.repo_manager.delete_mode = true; + self.repo_manager.cross_platform_delete = false; self.repo_manager.staged = self .repo_manager .lib_rows @@ -2897,6 +3052,7 @@ impl SysInspectUX { arch: None, }) .collect(); + self.repo_manager.staging_mode = repomanager::StagingMode::LibraryDelete; self.repo_manager.staging = true; self.repo_manager.staging_cursor = 0; self.repo_manager.staging_focus = repomanager::StagingFocus::List; @@ -3104,7 +3260,7 @@ impl SysInspectUX { let ctx = serde_json::json!({"op": "new", "name": name}).to_string(); self.call_profile_rpc(&ctx)?; self.load_profile_list()?; - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); Ok(()) } @@ -3112,21 +3268,21 @@ impl SysInspectUX { let ctx = serde_json::json!({"op": "delete", "name": name}).to_string(); self.call_profile_rpc(&ctx)?; self.load_profile_list()?; - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); Ok(()) } fn do_profile_add_matches(&mut self, name: &str, matches: Vec, library: bool) -> Result<(), String> { let ctx = serde_json::json!({"op": "add", "name": name, "matches": matches, "library": library}).to_string(); self.call_profile_rpc(&ctx)?; - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); Ok(()) } fn do_profile_remove_match(&mut self, name: &str, selector: &str, library: bool) -> Result<(), String> { let ctx = serde_json::json!({"op": "remove", "name": name, "matches": [selector], "library": library}).to_string(); self.call_profile_rpc(&ctx)?; - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); Ok(()) } @@ -3284,6 +3440,8 @@ impl SysInspectUX { .map(|m| { let mut entrypoints: Vec = Vec::new(); let mut entrypoint_kinds: Vec = Vec::new(); + let mut public_entrypoints: Vec = Vec::new(); + let mut public_entrypoint_kinds: Vec = Vec::new(); #[allow(clippy::type_complexity)] let mut target_actions: Vec<(String, Vec<(String, Vec, Vec<(String, String, bool)>)>)> = Vec::new(); @@ -3324,6 +3482,19 @@ impl SysInspectUX { } } + for ep in &m.public_entrypoints { + match ep { + libsysinspect::mdescr::browse_types::BrowsedEntrypoint::CheckbookLabel { label, .. } => { + public_entrypoints.push(label.clone()); + public_entrypoint_kinds.push("checkbook".to_string()); + } + libsysinspect::mdescr::browse_types::BrowsedEntrypoint::Entity { id, .. } => { + public_entrypoints.push(id.clone()); + public_entrypoint_kinds.push("entity".to_string()); + } + } + } + ConsoleModelRow { id: m.metadata.id.clone(), enabled: enabled.contains(&m.metadata.id), @@ -3332,6 +3503,9 @@ impl SysInspectUX { description: m.metadata.description.clone(), entrypoints, entrypoint_kinds, + public_entrypoints, + public_entrypoint_kinds, + public_actions: m.public_actions.clone(), states: m.states.clone(), target_actions, } @@ -3364,7 +3538,6 @@ impl SysInspectUX { self.write_enabled_models_dropin(ids)?; self.refresh_local_model_rows(&self.enabled_model_ids_with(model_id, enabled))?; self.repo_manager.models_dirty = true; - self.mark_cluster_upgrade_required(); Ok(()) } @@ -3413,7 +3586,6 @@ impl SysInspectUX { { Ok(()) => { self.repo_manager.models_dirty = true; - self.mark_cluster_upgrade_required(); self.info_alert_visible = true; self.info_alert_title = "Model Import".to_string(); self.info_alert_styled = None; @@ -3437,7 +3609,6 @@ impl SysInspectUX { self.write_enabled_models_dropin(enabled_ids.clone())?; self.refresh_local_model_rows(&enabled_ids)?; self.repo_manager.models_dirty = true; - self.mark_cluster_upgrade_required(); Ok(()) } @@ -3608,7 +3779,7 @@ impl SysInspectUX { self.error_alert_message = format!("Cannot remove modules: {e}"); } else { let _ = self.load_module_index(); - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); } } @@ -3632,7 +3803,26 @@ impl SysInspectUX { } } let _ = self.load_module_index(); - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); + } + + fn bulk_delete_libraries(&mut self, names: &[String]) { + let repo_root = self.cfg.fileserver_root().join("repo"); + let mut repo = match SysInspectModPak::new(repo_root) { + Ok(r) => r, + Err(e) => { + self.error_alert_visible = true; + self.error_alert_message = format!("Cannot open repository: {e}"); + return; + } + }; + if let Err(e) = repo.remove_library(names.to_vec()) { + self.error_alert_visible = true; + self.error_alert_message = format!("Cannot remove libraries: {e}"); + } else { + let _ = self.load_library_index(); + self.mark_repo_sync_pending(); + } } fn process_library_add(&mut self, path: &std::path::Path) { @@ -3651,7 +3841,7 @@ impl SysInspectUX { self.error_alert_message = format!("Cannot add library: {e}"); } else { self.load_library_index().ok(); - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); } } else { // Single file: wrap in temp dir, use add_library @@ -3665,7 +3855,7 @@ impl SysInspectUX { self.error_alert_message = format!("Cannot add library file: {e}"); } else { self.load_library_index().ok(); - self.mark_cluster_upgrade_required(); + self.mark_repo_sync_pending(); } let _ = std::fs::remove_dir_all(&tmp); } @@ -4285,6 +4475,68 @@ impl SysInspectUX { } } + fn refresh_selected_cycle_contents_preserve_selection(&mut self) { + if self.cycles_buf.is_empty() { + self.li_minions.clear(); + self.li_events.clear(); + self.event_data.clear(); + self.selected_minion = 0; + self.selected_event = 0; + return; + } + + let sid = self.get_selected_cycle().event().sid().to_string(); + let selected_mid = self.get_selected_minion().map(|mli| mli.event().id().to_string()); + let selected_event = self.get_selected_event().map(|evt| { + let event = evt.event(); + (event.get_action_id(), event.get_entity_id(), event.get_status_id(), event.get_timestamp()) + }); + + match self.get_minions(&sid) { + Ok(minions) => { + self.li_minions = minions; + if let Some(selected_mid) = selected_mid + && let Some(idx) = self.li_minions.iter().position(|mli| mli.event().id() == selected_mid) + { + self.selected_minion = idx; + } else if self.selected_minion >= self.li_minions.len() { + self.selected_minion = self.li_minions.len().saturating_sub(1); + } + } + Err(_) => return, + } + + let Some(mli) = self.get_selected_minion() else { + self.li_events.clear(); + self.event_data.clear(); + self.selected_event = 0; + return; + }; + + match self.get_events(&sid, mli.event().id()) { + Ok(events) => { + self.li_events = events; + if let Some((aid, eid, sid, ts)) = selected_event + && let Some(idx) = self.li_events.iter().position(|evt| { + let event = evt.event(); + event.get_action_id() == aid && event.get_entity_id() == eid && event.get_status_id() == sid && event.get_timestamp() == ts + }) + { + self.selected_event = idx; + } else if self.selected_event >= self.li_events.len() { + self.selected_event = self.li_events.len().saturating_sub(1); + } + } + Err(_) => return, + } + + if let Some(evt) = self.get_selected_event() { + self.event_data = evt.event().flatten(); + } else { + self.event_data.clear(); + } + } + fn on_mouse_move(&mut self, me: MouseEvent) { let rects = match self.popup_button_rects.get() { Some(r) => r, @@ -4444,6 +4696,10 @@ impl SysInspectUX { return; } + if self.cluster_start_progress.visible { + return; + } + // Master operations menu is modal if self.on_master_menu(e) { return; diff --git a/src/ui/online.rs b/src/ui/online.rs index c03c43ce..f18e8cec 100644 --- a/src/ui/online.rs +++ b/src/ui/online.rs @@ -12,7 +12,7 @@ use ratatui::{ prelude::{Buffer, Rect}, style::{Modifier, Style}, text::{Line, Span}, - widgets::{Block, BorderType, Borders, Clear, Paragraph, Row, Scrollbar, ScrollbarState, StatefulWidget, Table, Widget}, + widgets::{Block, BorderType, Borders, Cell, Clear, Paragraph, Row, Scrollbar, ScrollbarState, StatefulWidget, Table, Widget}, }; use ratatui_cheese::{ input::{Input, InputState}, @@ -123,13 +123,11 @@ impl SysInspectUX { .try_into() .unwrap(); - let max_w = 10u16; - let online_selected = self.minions_online_sel.min(online_filtered.len().saturating_sub(1)); let offline_selected = self.minions_offline_sel.min(offline_filtered.len().saturating_sub(1)); - Self::_render_pane(self, "Online", &online_filtered, online_pane, buf, focus_enabled && self.minions_focus == 1, online_selected, max_w); - Self::_render_pane(self, "Offline", &offline_filtered, offline_pane, buf, focus_enabled && self.minions_focus == 2, offline_selected, max_w); + Self::_render_pane(self, "Online", &online_filtered, online_pane, buf, focus_enabled && self.minions_focus == 1, online_selected); + Self::_render_pane(self, "Offline", &offline_filtered, offline_pane, buf, focus_enabled && self.minions_focus == 2, offline_selected); } fn _render_filter(area: Rect, buf: &mut Buffer, focused: bool, filter_state: &InputState) { @@ -155,9 +153,7 @@ impl SysInspectUX { } #[allow(clippy::too_many_arguments)] - fn _render_pane( - &self, title: &str, filtered: &[&&ConsoleOnlineMinionRow], area: Rect, buf: &mut Buffer, active: bool, selected: usize, max_w: u16, - ) { + fn _render_pane(&self, title: &str, filtered: &[&&ConsoleOnlineMinionRow], area: Rect, buf: &mut Buffer, active: bool, selected: usize) { let popup_bg = palette::BG_1; let t = format!(" {title} ({}) ", filtered.len()); let block = if active { @@ -195,50 +191,98 @@ impl SysInspectUX { let ip_data: Vec = filtered.iter().map(|r| if r.upgrade_unreachable { format!("📦 {}", Self::_fmt_ip(&r.ip)) } else { Self::_fmt_ip(&r.ip) }).collect(); - let host_data: Vec = filtered.iter().map(|r| Self::_trunc_ellipsis(&Self::online_host(r), max_w as usize)).collect(); - let ver_data: Vec = filtered.iter().map(|r| Self::_trunc_ellipsis(&Self::_fmt_version(r), max_w as usize)).collect(); - let id_data: Vec = filtered.iter().map(|r| Self::shorten_mid(&r.minion_id, 4)).collect(); - let os_data: Vec = filtered + let host_data: Vec = filtered.iter().map(|r| Self::online_host(r)).collect(); + let ver_from_data: Vec = filtered.iter().map(|r| if r.version.is_empty() { "-".to_string() } else { r.version.clone() }).collect(); + let ver_to_data: Vec = filtered .iter() - .map(|r| { - let name = if r.os_name.is_empty() { "-" } else { r.os_name.as_str() }; - let dist = if r.os_distribution.is_empty() { "-" } else { r.os_distribution.as_str() }; - Self::_trunc_ellipsis(&format!("{name}/{dist}"), max_w as usize) - }) + .map(|r| if r.outdated && !r.version.is_empty() && !r.target_version.is_empty() { r.target_version.clone() } else { String::new() }) .collect(); - let osv_data: Vec = filtered.iter().map(|r| Self::_trunc_ellipsis(&r.os_version, max_w as usize)).collect(); - let ker_data: Vec = filtered.iter().map(|r| r.kernel.clone()).collect(); + let dist_name_data: Vec = + filtered.iter().map(|r| if r.os_distribution.is_empty() { "-".to_string() } else { r.os_distribution.clone() }).collect(); + let dist_ver_data: Vec = + filtered.iter().map(|r| if r.os_version.is_empty() { "-".to_string() } else { r.os_version.clone() }).collect(); let ip_w = ip_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(2).max(2); let host_w = host_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(4).max(4); - let ver_w = ver_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(7).min(max_w); - let id_w = id_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(2).max(2); - let os_w = os_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(2).max(2); - let osv_w = osv_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(2).max(2); - let ker_w = ker_data.iter().map(|s| UnicodeWidthStr::width(s.as_str()) as u16).max().unwrap_or(2).max(2); - - let base_w: Vec = vec![ip_w, host_w, ver_w, id_w, os_w, osv_w, ker_w]; - let mut cols: Vec = base_w.into_iter().map(Constraint::Length).collect(); - cols.push(Constraint::Min(1)); + let ver_w = ver_from_data + .iter() + .zip(ver_to_data.iter()) + .map(|(f, t)| { + let fw = UnicodeWidthStr::width(f.as_str()); + if t.is_empty() { fw as u16 } else { (fw + 1 + UnicodeWidthStr::width(t.as_str())) as u16 } + }) + .max() + .unwrap_or(2) + .max(2); + let dist_w = dist_name_data + .iter() + .zip(dist_ver_data.iter()) + .map(|(n, v)| (UnicodeWidthStr::width(n.as_str()) + 1 + UnicodeWidthStr::width(v.as_str())) as u16) + .max() + .unwrap_or(3) + .max(3); + + let col_spacing: u16 = 1; + let fixed_w = ip_w + host_w + ver_w + dist_w + 5 * col_spacing + 1; + let ker_avail = inner.width.saturating_sub(fixed_w) as usize; + let ker_data: Vec = filtered.iter().map(|r| Self::_trunc_ellipsis(&r.kernel, ker_avail)).collect(); + + let cols: Vec = vec![ + Constraint::Length(ip_w), + Constraint::Length(host_w), + Constraint::Length(ver_w), + Constraint::Length(dist_w), + Constraint::Length(ker_avail as u16), + Constraint::Length(1), + ]; let sel_style = if active { Style::default().fg(palette::BLACK).bg(palette::HIGHLIGHT) } else { Style::default().fg(palette::SECONDARY) }; let norm_style = Style::default().fg(palette::FG).bg(popup_bg); + let ip_style = Style::default().fg(palette::GRAY_1).bg(popup_bg); + let host_style = Style::default().fg(palette::PROCESSING_PEAK).bg(popup_bg); let rows: Vec = filtered .iter() .enumerate() .map(|(idx, _)| { - let sty = if idx == selected { sel_style } else { norm_style }; - Row::new(vec![ - ip_data[idx].as_str(), - host_data[idx].as_str(), - ver_data[idx].as_str(), - id_data[idx].as_str(), - os_data[idx].as_str(), - osv_data[idx].as_str(), - ker_data[idx].as_str(), - "", - ]) - .style(sty) + if idx == selected { + let ver_cell = if ver_to_data[idx].is_empty() { + Cell::from(ver_from_data[idx].as_str()) + } else { + Cell::from(Line::from(vec![Span::raw(ver_from_data[idx].as_str()), Span::raw("→"), Span::raw(ver_to_data[idx].as_str())])) + }; + Row::new(vec![ + Cell::from(ip_data[idx].as_str()), + Cell::from(host_data[idx].as_str()), + ver_cell, + Cell::from(Line::from(vec![Span::raw(dist_name_data[idx].as_str()), Span::raw("/"), Span::raw(dist_ver_data[idx].as_str())])), + Cell::from(ker_data[idx].as_str()), + Cell::from(""), + ]) + .style(sel_style) + } else { + let ver_cell = if ver_to_data[idx].is_empty() { + Cell::from(ver_from_data[idx].as_str()).style(Style::default().fg(palette::PROCESSING_GLOW).bg(popup_bg)) + } else { + Cell::from(Line::from(vec![ + Span::styled(ver_from_data[idx].as_str(), Style::default().fg(palette::PROCESSING_GLOW)), + Span::styled("→", Style::default().fg(palette::ERROR)), + Span::styled(ver_to_data[idx].as_str(), Style::default().fg(palette::WARNING_HEAT)), + ])) + }; + Row::new(vec![ + Cell::from(ip_data[idx].as_str()).style(ip_style), + Cell::from(host_data[idx].as_str()).style(host_style), + ver_cell, + Cell::from(Line::from(vec![ + Span::styled(dist_name_data[idx].as_str(), Style::default().fg(palette::PROCESSING_PEAK)), + Span::styled("/", Style::default().fg(palette::FG)), + Span::styled(dist_ver_data[idx].as_str(), Style::default().fg(palette::PRIMARY)), + ])), + Cell::from(ker_data[idx].as_str()).style(Style::default().fg(palette::PROCESSING_GLOW).bg(popup_bg)), + Cell::from(""), + ]) + .style(norm_style) + } }) .collect(); @@ -308,6 +352,10 @@ impl SysInspectUX { if ip.is_empty() { return "unknown".to_string(); } + let parts: Vec<&str> = ip.split('.').collect(); + if parts.len() == 4 && parts.iter().all(|p| p.parse::().is_ok()) { + return format!("{:>3}.{:>3}.{:>3}.{:>3}", parts[0], parts[1], parts[2], parts[3]); + } if ip.len() > 15 { return ip.chars().take(15).collect::() + "…"; } @@ -333,7 +381,7 @@ impl SysInspectUX { let start = if selected < vis_rows { 0 } else { (selected + 1).saturating_sub(vis_rows) }; let end = (start + vis_rows).min(rows.len()); let displayed: Vec = if start < rows.len() { rows[start..end].to_vec() } else { vec![] }; - Widget::render(Table::new(displayed, cols).column_spacing(2), area, buf); + Widget::render(Table::new(displayed, cols).column_spacing(1), area, buf); let scroller_area = Rect { x: area.right().saturating_sub(1), y: area.y, width: 1, height: area.height }; let mut scroller = ScrollbarState::default().content_length(rows.len()).position(selected); Scrollbar::default() diff --git a/src/ui/repomanager.rs b/src/ui/repomanager.rs index 924799e3..ea3ab5ca 100644 --- a/src/ui/repomanager.rs +++ b/src/ui/repomanager.rs @@ -43,6 +43,7 @@ pub enum StagingFocus { pub enum StagingMode { ModuleAdd, ModuleDelete, + LibraryDelete, ProfileModuleAdd, ProfileLibraryAdd, } @@ -80,6 +81,7 @@ pub struct RepoManager { pub bulk_add_triggered: bool, pub bulk_delete_triggered: bool, pub needs_reload: bool, + pub pending_cluster_upgrade: bool, // Filter pub filter: InputState, @@ -136,6 +138,7 @@ impl Default for RepoManager { bulk_add_triggered: false, bulk_delete_triggered: false, needs_reload: false, + pending_cluster_upgrade: false, filter: InputState::new(), filter_focus: false, info_visible: false, @@ -271,7 +274,7 @@ impl RepoManager { self.staging_focus = match self.staging_focus { List => AddSelected, AddSelected => { - if self.delete_mode { + if self.delete_mode && matches!(self.staging_mode, StagingMode::ModuleDelete) { CrossPlatformDelete } else { Cancel @@ -287,7 +290,7 @@ impl RepoManager { List => Cancel, AddSelected => List, Cancel => { - if self.delete_mode { + if self.delete_mode && matches!(self.staging_mode, StagingMode::ModuleDelete) { CrossPlatformDelete } else { AddSelected @@ -552,7 +555,7 @@ impl RepoManager { } // Cross-platform delete checkbox - if self.delete_mode { + if self.delete_mode && matches!(self.staging_mode, StagingMode::ModuleDelete) { let chk_y = inner.y + list_height + 1; let (chk, chk_style) = if self.cross_platform_delete { ("â–£", Style::default().fg(palette::SUCCESS)) @@ -574,7 +577,7 @@ impl RepoManager { } // Buttons - let btn_y = inner.y + list_height + (if self.delete_mode { 2 } else { 1 }); + let btn_y = inner.y + list_height + (if self.delete_mode && matches!(self.staging_mode, StagingMode::ModuleDelete) { 2 } else { 1 }); let action_label = if self.delete_mode { "[ Delete ]" } else { "[ Add Selected ]" }; let cancel_label = "[ Cancel ]"; let action_w = action_label.len() as u16; diff --git a/src/ui/wgt.rs b/src/ui/wgt.rs index e193f952..91d27db2 100644 --- a/src/ui/wgt.rs +++ b/src/ui/wgt.rs @@ -453,6 +453,7 @@ impl Widget for &SysInspectUX { self.dialog_cluster_confirm(area, buf); self.dialog_delete_progress(area, buf); self.dialog_cluster_upgrade_progress(area, buf); + self.dialog_cluster_start_progress(area, buf); self.dialog_master_confirm(area, buf); self.master_actions_menu(area, buf); self.repo_manager.render(area, buf); diff --git a/sysmaster/src/console.rs b/sysmaster/src/console.rs index 8f2eec4c..9e6507b1 100644 --- a/sysmaster/src/console.rs +++ b/sysmaster/src/console.rs @@ -287,6 +287,7 @@ impl SysMaster { let current_version = minion.get_traits().get("minion.version").and_then(|v| v.as_str()).unwrap_or_default().to_string(); let current_sha = minion.get_traits().get("minion.binary.sha256").and_then(|v| v.as_str()).unwrap_or_default().to_string(); let os_dist = minion.get_traits().get("system.os.name").and_then(|v| v.as_str()).unwrap_or_default().to_lowercase(); + let os_distribution = minion.get_traits().get("system.os.distribution").and_then(|v| v.as_str()).unwrap_or_default().to_string(); let arch = minion.get_traits().get("system.arch").and_then(|v| v.as_str()).unwrap_or_default().to_string(); let target_sha = repo_checksums.get(&(os_dist.clone(), arch.clone())).cloned().unwrap_or_default(); let target_version = repo_versions.get(&(os_dist.clone(), arch)).cloned().unwrap_or_default(); @@ -305,7 +306,7 @@ impl SysMaster { upgrade_unreachable: upgrade_marker.as_ref().is_some_and(|marker| marker.unreachable), version: current_version, target_version, - os_distribution: os_dist, + os_distribution, os_name, os_version, kernel, @@ -511,6 +512,8 @@ impl SysMaster { .map(|m| { let mut entrypoints: Vec = Vec::new(); let mut entrypoint_kinds: Vec = Vec::new(); + let mut public_entrypoints: Vec = Vec::new(); + let mut public_entrypoint_kinds: Vec = Vec::new(); #[allow(clippy::type_complexity)] let mut target_actions: Vec<(String, Vec<(String, Vec, Vec<(String, String, bool)>)>)> = Vec::new(); @@ -551,6 +554,19 @@ impl SysMaster { } } + for ep in &m.public_entrypoints { + match ep { + libsysinspect::mdescr::browse_types::BrowsedEntrypoint::CheckbookLabel { label, .. } => { + public_entrypoints.push(label.clone()); + public_entrypoint_kinds.push("checkbook".to_string()); + } + libsysinspect::mdescr::browse_types::BrowsedEntrypoint::Entity { id, .. } => { + public_entrypoints.push(id.clone()); + public_entrypoint_kinds.push("entity".to_string()); + } + } + } + ConsoleModelRow { id: m.metadata.id.clone(), enabled: enabled_models.contains(&m.metadata.id), @@ -559,6 +575,9 @@ impl SysMaster { description: m.metadata.description.clone(), entrypoints, entrypoint_kinds, + public_entrypoints, + public_entrypoint_kinds, + public_actions: m.public_actions.clone(), states: m.states.clone(), target_actions, } @@ -888,13 +907,13 @@ impl SysMaster { } } - HopStarter::new(self.cfg.hopstart()).issue(targets.clone()).await; + let failed = HopStarter::new(self.cfg.hopstart()).issue(targets.clone()).await; Ok(ConsoleResponse::ok(ConsolePayload::Ack { action: "hopstart_issued".to_string(), target: String::new(), count: targets.len(), - items: vec![], + items: failed, })) } @@ -932,9 +951,9 @@ impl SysMaster { HopStartTarget::new(host.to_string(), root.to_string(), user.to_string(), bin.to_string(), config.to_string()) }; - HopStarter::new(master.lock().await.cfg.hopstart()).issue(vec![target]).await; + let failed = HopStarter::new(master.lock().await.cfg.hopstart()).issue(vec![target]).await; - Ok(ConsoleResponse::ok(ConsolePayload::Ack { action: "minion_start".to_string(), target: minion_id, count: 1, items: vec![] })) + Ok(ConsoleResponse::ok(ConsolePayload::Ack { action: "minion_start".to_string(), target: minion_id, count: 1, items: failed })) } async fn minion_shutdown(master: Arc>, query: &str, traits: &str, mid: &str) -> Result { diff --git a/sysmaster/src/hopstart.rs b/sysmaster/src/hopstart.rs index de747c8a..afdb987c 100644 --- a/sysmaster/src/hopstart.rs +++ b/sysmaster/src/hopstart.rs @@ -2,7 +2,8 @@ use colored::Colorize; use libcommon::SysinspectError; use libsysinspect::cfg::mmconf::HopstartConfig; use std::sync::{Arc, OnceLock}; -use tokio::{process::Command, sync::Semaphore}; +use std::time::Duration; +use tokio::{net::TcpStream, process::Command, sync::Semaphore}; /// Shared semaphore capping concurrent hopstart SSH calls across all callers. pub(crate) static HOPSTART_SEMAPHORE: OnceLock> = OnceLock::new(); @@ -44,6 +45,10 @@ impl HopStartTarget { } pub(crate) async fn issue(&self) -> Result<(), SysinspectError> { + if !is_port_available(&self.host, 22, Duration::from_secs(5)).await { + return Err(SysinspectError::MasterGeneralError(format!("Hop-start aborted: host {} is unreachable via SSH", self.host))); + } + let status = Command::new("ssh").arg(self.ssh_target()).arg(self.remote_command()).status().await?; if status.success() { @@ -67,32 +72,54 @@ impl HopStarter { Self { cfg } } - pub(crate) async fn issue(&self, targets: Vec) { + pub(crate) async fn issue(&self, targets: Vec) -> Vec { let limit = hopstart_semaphore(); - let mut tasks = Vec::with_capacity(targets.len()); + let total = targets.len(); + let mut tasks = Vec::with_capacity(total); for target in targets { tasks.push(tokio::spawn({ let limit = Arc::clone(&limit); async move { - if let Ok(_permit) = limit.acquire_owned().await { - target.log_issue(); - if let Err(err) = target.issue().await { + let _permit = limit.acquire_owned().await.ok()?; + target.log_issue(); + Some(match target.issue().await { + Ok(()) => Ok(target.host), + Err(err) => { log::error!("{err}"); + Err(target.host) } - } + }) } })); } + let mut failed = Vec::new(); + let mut ok = 0usize; for task in tasks { - if let Err(err) = task.await { - log::error!("Hop-start task failed: {err}"); + match task.await { + Ok(Some(Ok(_))) => ok += 1, + Ok(Some(Err(host))) => failed.push(host), + _ => {} } } + + let fail_count = failed.len(); + if fail_count == 0 { + log::info!("Hop-start complete: all {total} minion(s) started"); + } else { + log::warn!("Hop-start finished: {ok} started, {fail_count} failed out of {total}"); + } + + failed } } +async fn is_port_available(host: &str, port: u16, timeout: Duration) -> bool { + let addr = format!("{host}:{port}"); + tokio::time::timeout(timeout, TcpStream::connect(&addr)).await.map(|r| r.is_ok()).unwrap_or(false) +} + pub(crate) fn shell_quote(value: &str) -> String { format!("'{}'", value.replace('\'', "'\"'\"'")) } diff --git a/sysminion/src/clidef.rs b/sysminion/src/clidef.rs index 60255958..6bfd8c51 100644 --- a/sysminion/src/clidef.rs +++ b/sysminion/src/clidef.rs @@ -61,6 +61,12 @@ pub fn cli(version: &'static str, appname: &'static str) -> Command { .action(ArgAction::SetTrue) .help("Display minion info") ) + .arg( + Arg::new("id") + .long("id") + .action(ArgAction::SetTrue) + .help("Print minion system ID to stdout and exit") + ) .next_help_heading("Minion") .subcommand(Command::new("setup").about("Minion local setup").styles(styles.clone()).disable_help_flag(true) diff --git a/sysminion/src/main.rs b/sysminion/src/main.rs index d0fa5044..59327412 100644 --- a/sysminion/src/main.rs +++ b/sysminion/src/main.rs @@ -279,6 +279,17 @@ fn main() -> std::io::Result<()> { std::process::exit(0); } + if params.get_flag("id") { + let cfg = get_config(¶ms); + let p = cfg.machine_id_path(); + if p.exists() + && let Ok(id) = std::fs::read_to_string(&p) + { + print!("{}", id.trim()); + } + std::process::exit(0); + } + // Setup logger if let Err(err) = log::set_boxed_logger(Box::new(RingBufferLogger { nocolor: params.get_flag("no-color") })).map(|()| { log::set_max_level(match params.get_count("debug") { diff --git a/sysminion/src/minion.rs b/sysminion/src/minion.rs index c62d3153..e21ac488 100644 --- a/sysminion/src/minion.rs +++ b/sysminion/src/minion.rs @@ -2446,7 +2446,14 @@ pub(crate) fn setup(args: &ArgMatches) -> Result<(), SysinspectError> { } } - libsetup::mnsetup::MinionSetup::new().set_config(get_minion_config(None)?).set_alt_dir(dir.to_str().unwrap_or_default().to_string()).setup() + let cfg = get_minion_config(None)?; + libsetup::mnsetup::MinionSetup::new().set_config(cfg.clone()).set_alt_dir(dir.to_str().unwrap_or_default().to_string()).setup()?; + + if !cfg.machine_id_path().exists() { + util::write_machine_id(Some(cfg.machine_id_path()))?; + } + + Ok(()) } pub(crate) fn setup_master_addr(addr: Option<&str>, ssh_ip: Option) -> Result<(String, u32), SysinspectError> { diff --git a/sysminion/src/minion_ut.rs b/sysminion/src/minion_ut.rs index cfe7a0fa..1b355f38 100644 --- a/sysminion/src/minion_ut.rs +++ b/sysminion/src/minion_ut.rs @@ -71,6 +71,62 @@ mod tests { Duration::from_secs(15) } + #[cfg(target_os = "linux")] + fn async_settle_wait() -> Duration { + Duration::from_millis(500) + } + + #[cfg(not(target_os = "linux"))] + fn async_settle_wait() -> Duration { + Duration::from_secs(1) + } + + #[cfg(target_os = "linux")] + fn signal_timeout() -> Duration { + Duration::from_secs(5) + } + + #[cfg(not(target_os = "linux"))] + fn signal_timeout() -> Duration { + Duration::from_secs(10) + } + + #[cfg(target_os = "linux")] + fn watchdog_ping_timeout() -> Duration { + Duration::from_millis(750) + } + + #[cfg(not(target_os = "linux"))] + fn watchdog_ping_timeout() -> Duration { + Duration::from_millis(1500) + } + + #[cfg(target_os = "linux")] + fn watchdog_assert_timeout() -> Duration { + Duration::from_secs(5) + } + + #[cfg(not(target_os = "linux"))] + fn watchdog_assert_timeout() -> Duration { + Duration::from_secs(10) + } + + async fn wait_until(timeout_window: Duration, mut predicate: F) + where + F: FnMut() -> bool, + { + timeout(timeout_window, async { + loop { + if predicate() { + break; + } + tokio::time::sleep(Duration::from_millis(25)).await; + } + }) + .await + .expect("condition was not met before timeout"); + } + fn secure_state(master_pbk: &RsaPublicKey, minion_pbk: &RsaPublicKey) -> TransportPeerState { let mut state = TransportPeerState::new( "mid-1".to_string(), @@ -206,10 +262,12 @@ mod tests { let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); let addr = listener.local_addr().unwrap(); let (shutdown_tx, shutdown_rx) = oneshot::channel(); + let (accepted_tx, accepted_rx) = oneshot::channel(); // Accept connection and just sit there tokio::spawn(async move { let (sock, _peer) = listener.accept().await.unwrap(); + let _ = accepted_tx.send(()); let _ = shutdown_rx.await; drop(sock); }); @@ -225,8 +283,10 @@ mod tests { let h = tokio::spawn(async move { _minion_instance(cfg, Some("fp-test".to_string()), dpq).await }); - // Let it start - tokio::time::sleep(reconnect_boot_wait()).await; + timeout(reconnect_accept_timeout(), accepted_rx) + .await + .expect("fake master never accepted initial connection") + .expect("fake master accept signal dropped"); // Trigger reconnect let _ = CONNECTION_TX.send(()); @@ -279,9 +339,11 @@ mod tests { let _guard = TEST_LOCK.lock().await; let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); let addr = listener.local_addr().unwrap(); + let (accepted_tx, accepted_rx) = oneshot::channel(); tokio::spawn(async move { let (sock, _) = listener.accept().await.unwrap(); + let _ = accepted_tx.send(()); drop(sock); // immediate EOF for minion }); @@ -302,10 +364,12 @@ mod tests { let mut rx = CONNECTION_TX.subscribe(); minion.as_ptr().do_proto().await.unwrap(); - // let proto loop actually start - tokio::time::sleep(Duration::from_millis(200)).await; + timeout(reconnect_accept_timeout(), accepted_rx) + .await + .expect("fake master never accepted proto connection") + .expect("fake master accept signal dropped"); - let got = tokio::time::timeout(Duration::from_secs(2), rx.recv()).await; + let got = tokio::time::timeout(signal_timeout(), rx.recv()).await; assert!(got.is_ok(), "expected reconnect signal on EOF"); match got { Ok(Ok(_)) => { @@ -828,13 +892,13 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let mut minion = Arc::try_unwrap(SysMinion::new(cfg, None, dpq).await.unwrap()).ok().unwrap(); - minion.set_ping_timeout(Duration::from_millis(300)); + minion.set_ping_timeout(watchdog_ping_timeout()); let minion = Arc::new(minion); let mut rx = CONNECTION_TX.subscribe(); minion.as_ptr().do_ping_update(Arc::new(ExitState::new())).await.unwrap(); - let res = timeout(Duration::from_secs(2), rx.recv()).await; + let res = timeout(watchdog_assert_timeout(), rx.recv()).await; assert!(res.is_ok(), "watchdog did not trigger reconnect"); } @@ -857,15 +921,15 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let mut minion = Arc::try_unwrap(SysMinion::new(cfg, None, dpq).await.unwrap()).ok().unwrap(); - minion.set_ping_timeout(Duration::from_millis(300)); + minion.set_ping_timeout(watchdog_ping_timeout()); let minion = Arc::new(minion); minion.update_ping().await; - tokio::time::sleep(Duration::from_millis(150)).await; + tokio::time::sleep(async_settle_wait() / 2).await; minion.update_ping().await; let elapsed = minion.last_ping.lock().await.elapsed(); - assert!(elapsed < Duration::from_millis(200)); + assert!(elapsed < async_settle_wait()); } #[tokio::test] @@ -935,9 +999,11 @@ mod tests { let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); let addr = listener.local_addr().unwrap(); let (shutdown_tx, shutdown_rx) = oneshot::channel(); + let (accepted_tx, accepted_rx) = oneshot::channel(); tokio::spawn(async move { let (sock, _) = listener.accept().await.unwrap(); + let _ = accepted_tx.send(()); let _ = shutdown_rx.await; drop(sock); }); @@ -947,13 +1013,16 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let handle = tokio::spawn(async move { _minion_instance(cfg, Some("fp-test".to_string()), dpq).await }); - tokio::time::sleep(Duration::from_millis(200)).await; + timeout(reconnect_accept_timeout(), accepted_rx) + .await + .expect("fake master never accepted reconnect-storm test connection") + .expect("fake master accept signal dropped"); let _ = CONNECTION_TX.send(()); let _ = CONNECTION_TX.send(()); let _ = CONNECTION_TX.send(()); let _ = shutdown_tx.send(()); - assert!(timeout(Duration::from_secs(2), handle).await.is_ok(), "instance did not exit under reconnect storm"); + assert!(timeout(reconnect_exit_timeout(), handle).await.is_ok(), "instance did not exit under reconnect storm"); } // ── Phase 8: independent mode behaviour proofs ─────────────────────── @@ -983,7 +1052,7 @@ mod tests { let mut rx = CONNECTION_TX.subscribe(); let _ = minion.try_request(b"durable payload".to_vec(), OutboundMessageClass::DurableData).await; - assert!(timeout(Duration::from_millis(500), rx.recv()).await.is_err(), "durable send failure must not trigger reconnect in independent mode"); + assert!(timeout(signal_timeout(), rx.recv()).await.is_err(), "durable send failure must not trigger reconnect in independent mode"); } #[tokio::test] @@ -1011,7 +1080,7 @@ mod tests { let mut rx = CONNECTION_TX.subscribe(); let _ = minion.try_request(b"durable payload".to_vec(), OutboundMessageClass::DurableData).await; - assert!(timeout(Duration::from_millis(500), rx.recv()).await.is_ok(), "durable send failure must trigger reconnect in follow mode"); + assert!(timeout(signal_timeout(), rx.recv()).await.is_ok(), "durable send failure must trigger reconnect in follow mode"); } #[tokio::test] @@ -1039,7 +1108,7 @@ mod tests { let mut rx = CONNECTION_TX.subscribe(); let _ = minion.try_request(b"session ctl".to_vec(), OutboundMessageClass::SessionControl).await; - assert!(timeout(Duration::from_millis(500), rx.recv()).await.is_ok(), "session control delivery failure must always trigger reconnect"); + assert!(timeout(signal_timeout(), rx.recv()).await.is_ok(), "session control delivery failure must always trigger reconnect"); } #[tokio::test] @@ -1063,14 +1132,14 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let mut minion = Arc::try_unwrap(SysMinion::new(cfg, None, dpq).await.unwrap()).ok().unwrap(); - minion.set_ping_timeout(Duration::from_millis(300)); + minion.set_ping_timeout(watchdog_ping_timeout()); let minion = Arc::new(minion); let state = Arc::new(ExitState::new()); let mut rx = CONNECTION_TX.subscribe(); // subscribe BEFORE starting watchdog minion.as_ptr().do_ping_update(state.clone()).await.unwrap(); - let res = timeout(Duration::from_secs(2), rx.recv()).await; + let res = timeout(watchdog_assert_timeout(), rx.recv()).await; assert!(res.is_ok(), "watchdog must fire reconnect signal in independent mode"); assert!(!state.exit.load(std::sync::atomic::Ordering::Relaxed), "watchdog must not stop execution in independent mode"); } @@ -1096,19 +1165,13 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let mut minion = Arc::try_unwrap(SysMinion::new(cfg, None, dpq).await.unwrap()).ok().unwrap(); - minion.set_ping_timeout(Duration::from_millis(300)); + minion.set_ping_timeout(watchdog_ping_timeout()); let minion = Arc::new(minion); let state = Arc::new(ExitState::new()); minion.as_ptr().do_ping_update(state.clone()).await.unwrap(); - let res = timeout(Duration::from_secs(2), async { - while !state.exit.load(std::sync::atomic::Ordering::Relaxed) { - tokio::time::sleep(Duration::from_millis(10)).await; - } - }) - .await; - assert!(res.is_ok(), "watchdog must stop execution in follow mode"); + wait_until(watchdog_assert_timeout(), || state.exit.load(std::sync::atomic::Ordering::Relaxed)).await; assert!(state.exit.load(std::sync::atomic::Ordering::Relaxed), "exit flag must be set in follow mode"); } @@ -1175,6 +1238,7 @@ mod tests { let tmp = tempfile::tempdir().unwrap(); let cfg = mk_cfg(format!("{addr}"), "127.0.0.1:1".to_string(), tmp.path()); let master_prk = seed_managed_transport_with_master(&cfg, tmp.path()); + let (saw_ehlo_tx, saw_ehlo_rx) = oneshot::channel(); let server = tokio::spawn({ let root = tmp.path().to_path_buf(); @@ -1184,6 +1248,8 @@ mod tests { let mut master_channel = accept_secure_minion(&mut sock, &cfg, &root, &master_prk).await; let ehlo = master_channel.open_bytes(&read_frame(&mut sock).await).unwrap(); let _: MinionMessage = serde_json::from_slice(&ehlo).unwrap(); + let _ = saw_ehlo_tx.send(()); + tokio::time::sleep(async_settle_wait()).await; drop(sock); } }); @@ -1191,12 +1257,25 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let handle = tokio::spawn(async move { _minion_instance(cfg, None, dpq).await }); - tokio::time::sleep(Duration::from_secs(2)).await; - assert!(!handle.is_finished(), "independent instance should stay alive after proto transport loss"); + timeout(reconnect_accept_timeout(), saw_ehlo_rx) + .await + .expect("server never observed EHLO before transport drop") + .expect("EHLO readiness signal dropped"); + let _ = server.await; + + assert!( + timeout(async_settle_wait(), async { + while !handle.is_finished() { + tokio::time::sleep(Duration::from_millis(25)).await; + } + }) + .await + .is_err(), + "independent instance should stay alive after proto transport loss" + ); handle.abort(); let _ = handle.await; - let _ = server.await; } #[tokio::test] @@ -1209,6 +1288,7 @@ mod tests { let mut cfg = mk_cfg(format!("{addr}"), "127.0.0.1:1".to_string(), tmp.path()); cfg.set_offline(MinionOfflineMode::Follow); let master_prk = seed_managed_transport_with_master(&cfg, tmp.path()); + let (saw_ehlo_tx, saw_ehlo_rx) = oneshot::channel(); let server = tokio::spawn({ let root = tmp.path().to_path_buf(); @@ -1218,6 +1298,8 @@ mod tests { let mut master_channel = accept_secure_minion(&mut sock, &cfg, &root, &master_prk).await; let ehlo = master_channel.open_bytes(&read_frame(&mut sock).await).unwrap(); let _: MinionMessage = serde_json::from_slice(&ehlo).unwrap(); + let _ = saw_ehlo_tx.send(()); + tokio::time::sleep(async_settle_wait()).await; drop(sock); } }); @@ -1225,7 +1307,12 @@ mod tests { let dpq = Arc::new(DiskPersistentQueue::open(tmp.path().join("pending-tasks")).unwrap()); let handle = tokio::spawn(async move { _minion_instance(cfg, None, dpq).await }); - let res = timeout(Duration::from_secs(5), handle).await; + timeout(reconnect_accept_timeout(), saw_ehlo_rx) + .await + .expect("server never observed EHLO before follow-mode transport drop") + .expect("EHLO readiness signal dropped"); + + let res = timeout(reconnect_exit_timeout(), handle).await; assert!(res.is_ok(), "follow instance should exit after proto transport loss"); let _ = server.await; } @@ -1287,6 +1374,9 @@ mod tests { let ack = libsysproto::MasterMessage::new(RequestType::CycleAck, serde_json::json!({"cycle_id":"c1"})); let sealed = master_channel.seal_bytes(&ack.sendable().unwrap()).unwrap(); write_frame(&mut sock2, &sealed).await; + // Give the minion proto loop a chance to process the ack before + // the mock master drops the recovered socket on slower CI runners. + tokio::time::sleep(async_settle_wait()).await; break true; } RequestType::SensorsSyncRequest => {} @@ -1301,17 +1391,8 @@ mod tests { minion.reconnect_transport().await.unwrap(); - timeout(Duration::from_secs(5), async { - loop { - if minion.journal.stats().unwrap().pending_entries == 0 { - break; - } - tokio::time::sleep(Duration::from_millis(50)).await; - } - }) - .await - .expect("cycle ack never cleared replayed backlog"); + wait_until(Duration::from_secs(30), || minion.journal.stats().unwrap().pending_entries == 0).await; - server.await.unwrap(); + timeout(Duration::from_secs(30), server).await.expect("mock master never completed replay/ack flow").unwrap(); } }