diff --git a/.gitignore b/.gitignore index 4b0667c..24ffb5c 100644 --- a/.gitignore +++ b/.gitignore @@ -137,3 +137,6 @@ scripts/audit-modules-prompt.md # Go engine build artifacts go-engine/endstate.exe + +# OMC operational state (runtime artifacts, never committed) +.omc/ diff --git a/docs/contracts/cli-json-contract.md b/docs/contracts/cli-json-contract.md index 2bae329..dd67fcc 100644 --- a/docs/contracts/cli-json-contract.md +++ b/docs/contracts/cli-json-contract.md @@ -101,7 +101,7 @@ When `success` is `false`, the `error` field contains: | `ROLLBACK_FAILED` | The backend rollback failed (non-systemic). Raw backend text is confined to `error.detail`. Additive in schema 1.x. | | `CONVERGENCE_UNSUPPORTED` | `apply --prune` was requested on a backend that cannot safely remove installed-but-undeclared packages (the winget driver, or any host with no realizer). Nothing is removed. Additive in schema 1.x. | | `CONFIRMATION_REQUIRED` | `rebuild` was invoked for a live run (restore on, not `--dry-run`) without `--confirm`. Raised before any mutation, so the refusal has no side effects. Additive in schema 1.x. | -| `NOT_SUPPORTED` | The requested operation is not supported on the current platform (e.g. `schedule enable` on non-Windows), or the input mode is unsupported (e.g. `rebuild --from `). Additive in schema 1.x. | +| `NOT_SUPPORTED` | The requested operation is not supported on the current platform (e.g. `schedule enable` on non-Windows), or the input mode is unsupported (e.g. `rebuild --from `, or `import --from ` for an unrecognised source). Additive in schema 1.x. | | `TASK_REGISTRATION_FAILED` | `schedule enable` could not register the Windows Scheduled Task via `schtasks.exe`. Additive in schema 1.x. | #### Hosted Backup error codes @@ -533,6 +533,62 @@ endstate rebuild --from ./machine.jsonc --no-restore --json --- +## Command: `import` + +Imports an external package list into an Endstate manifest. The only supported source in v0 is `unigetui` — a [UniGetUI](https://github.com/marticliment/UniGetUI) `.ubundle` backup/bundle (JSON, `export_version: 3`). Winget-source packages become manifest apps; every other package (chocolatey, scoop, pip, ...) and every `incompatible_packages` entry is reported, never silently dropped. Import is a **pure transform**: no installs, no network access, deterministic (byte-identical) output. It imports the package list only — UniGetUI's own settings are not imported; config restore comes from Endstate's module catalog on a later `plan`/`apply`. + +```powershell +endstate import --from unigetui --path "backup.ubundle" --json +endstate import --from unigetui --path "backup.ubundle" --pin --out ./machine.jsonc --json +``` + +### Flags + +| Flag | Behavior | +|------|----------| +| `--from ` | Required. The source format. Only `unigetui` is supported; any other value returns `NOT_SUPPORTED` with remediation. An empty value returns `MANIFEST_VALIDATION_ERROR`. | +| `--path ` | Required. The input bundle file. An empty value returns `MANIFEST_VALIDATION_ERROR`; a missing path returns `MANIFEST_NOT_FOUND`; a file that is not valid JSON returns `MANIFEST_PARSE_ERROR`; a well-formed JSON file that is not a UniGetUI bundle (no `export_version` and no `packages`) returns `MANIFEST_VALIDATION_ERROR`. | +| `--out ` | Output manifest path. Defaults to `manifests/local/imported-unigetui.jsonc` (gitignored); outside a repo checkout the default lands next to the input bundle instead. The emitted JSONC must load through the manifest loader before it is written; a load failure returns `MANIFEST_VALIDATION_ERROR` and writes nothing. An existing file at the output path is replaced (the write is atomic: staged to a temp file, then renamed into place). A write failure returns `MANIFEST_WRITE_FAILED`. | +| `--pin` | Record a `version` on each imported app: the package's `InstallationOptions.Version` pin when present (authored intent), otherwise the observed `Version`. Off by default (no version written). | +| `--json` | Emit the standard envelope to stdout. | + +### Response + +```json +{ + "schemaVersion": "1.0", + "cliVersion": "0.1.0", + "command": "import", + "runId": "20241220-143052", + "timestampUtc": "2024-12-20T14:30:52Z", + "success": true, + "data": { + "source": "unigetui", + "input": "C:\\Users\\me\\backup.ubundle", + "output": "C:\\repo\\manifests\\local\\imported-unigetui.jsonc", + "exportVersion": 3, + "pinned": false, + "counts": { "imported": 3, "skipped": 3, "incompatible": 1 }, + "imported": [ + { "id": "visualstudiocode", "ref": "Microsoft.VisualStudioCode", "displayName": "Microsoft Visual Studio Code" } + ], + "skipped": [ + { "id": "nodejs", "name": "Node.js", "manager": "Chocolatey", "reason": "unsupported package manager (Endstate installs via winget on Windows)" } + ], + "incompatible": [ + { "id": "Contoso.LocalOnlyApp", "name": "Contoso Local Only App", "version": "1.0.0", "source": "Local PC" } + ] + }, + "error": null +} +``` + +- `imported`, `skipped`, and `incompatible` together account for every entry in the bundle — nothing is silently dropped. `counts` is their length summary. +- `imported[].version` is present only under `--pin`. `warnings` carries the parser's version-mismatch note (a non-3 `export_version`) and any app-id collision de-duplication; it is omitted when empty. +- Additive in schema 1.x; no schema bump. + +--- + ## Command: `verify` Verifies current machine state against manifest. diff --git a/go-engine/cmd/endstate/main.go b/go-engine/cmd/endstate/main.go index 8d6c0c0..886d8d9 100644 --- a/go-engine/cmd/endstate/main.go +++ b/go-engine/cmd/endstate/main.go @@ -27,6 +27,7 @@ Commands: capabilities Report CLI capabilities (GUI handshake) apply Execute provisioning plan rebuild Rebuild a machine from a bundle or manifest (install + restore + verify) + import Import an external package list into a manifest (--from unigetui) verify Verify machine state against manifest capture Capture current machine state plan Generate execution plan @@ -67,6 +68,8 @@ Per-command flags: --export Export directory path (restore, export-config, validate-export) --restore-filter Filter restore entries by module ID (restore, apply) --from Bundle (.zip) or manifest (.jsonc) to rebuild from (rebuild) + --from External package-list source to import (import; e.g. unigetui) + --path Source bundle file to import (import) --no-restore Install without restoring configuration (rebuild) --latest Most recent run (report) --last Last N runs (report) @@ -117,9 +120,12 @@ type parsedArgs struct { restoreFilter string // --restore-filter // Rebuild flags - from string // rebuild --from + from string // rebuild --from ; import --from noRestore bool // rebuild --no-restore: install without restoring configuration + // Import flags + path string // import --path : source file to import + // Capture flags out string name string @@ -308,6 +314,11 @@ func parseArgs(args []string) parsedArgs { p.from = args[i+1] i++ } + case "--path": + if i+1 < len(args) { + p.path = args[i+1] + i++ + } case "--email": if i+1 < len(args) { p.email = args[i+1] @@ -371,6 +382,8 @@ func commandUsage(cmd string) string { return "Usage: endstate apply [--manifest ] [--dry-run] [--enable-restore] [--only ] [--prune] [--repin] [--confirm] [--bootstrap-backends] [--no-bootstrap] [--json] [--events jsonl]\n\nExecute provisioning plan. With --only, limit the run to the comma-separated list of manifest app ids (filtering happens before planning so only the selected apps are installed, restored, and verified). With --prune, converge the engine-managed set to exactly the manifest by removing installed-but-undeclared packages (realizer backends only, e.g. Nix on Linux/macOS). With --repin, reinstall a declared app version when the installed version has drifted from it (winget only). --prune and --repin both require --confirm to execute; use --dry-run to preview what would change. --only and --prune cannot be combined. On macOS/Linux, when a needed package backend is absent, --bootstrap-backends authorizes the engine to install it via its official installer; --no-bootstrap forces skipping it. Without either flag the engine skips the lane and requests consent.\n" case "rebuild": return "Usage: endstate rebuild --from [--dry-run] [--confirm] [--no-restore] [--json] [--events jsonl]\n\nRebuild a machine from a capture bundle (.zip) or a bare manifest (.jsonc): install the declared apps, restore configuration, then verify. Restore is ON by default, so a live run (not --dry-run, not --no-restore) requires --confirm. Use --dry-run to preview the plan without changing anything, or --no-restore to install and verify without touching configuration. Overwritten files are backed up first and can be undone with 'endstate revert'. Local file input only — URL input is not supported.\n" + case "import": + return "Usage: endstate import --from unigetui --path [--out ] [--pin] [--json]\n\nImport an external package list into an Endstate manifest. The only supported source is 'unigetui' (a UniGetUI .ubundle backup/bundle, JSON). Winget-source packages become manifest apps; every non-winget package (chocolatey, scoop, pip, ...) and every incompatible entry is reported, never silently dropped. With --pin, the app version is recorded (a package's InstallationOptions.Version pin wins over the observed version). Output defaults to manifests/local/imported-unigetui.jsonc (gitignored). This is a pure transform: no installs, no network. Scope: the package list only — UniGetUI's own settings are not imported; Endstate's module catalog restores app config on plan/apply.\n" case "verify": return "Usage: endstate verify [--manifest ] [--json] [--events jsonl]\n\nVerify machine state against manifest.\n" case "capture": @@ -512,6 +525,14 @@ func dispatch(p parsedArgs) (interface{}, *envelope.Error) { Events: p.events, }) + case "import": + return commands.RunImport(commands.ImportFlags{ + From: p.from, + Path: p.path, + Out: p.out, + Pin: p.pin, + }) + case "verify": return commands.RunVerify(commands.VerifyFlags{ Manifest: p.manifest, diff --git a/go-engine/internal/commands/capabilities.go b/go-engine/internal/commands/capabilities.go index 458df34..7af6b1f 100644 --- a/go-engine/internal/commands/capabilities.go +++ b/go-engine/internal/commands/capabilities.go @@ -165,6 +165,10 @@ func RunCapabilities() (interface{}, *envelope.Error) { Supported: true, Flags: []string{"--from", "--dry-run", "--confirm", "--no-restore", "--json", "--events"}, }, + "import": { + Supported: true, + Flags: []string{"--from", "--path", "--out", "--pin", "--json"}, + }, }, Features: FeaturesInfo{ Streaming: false, diff --git a/go-engine/internal/commands/import.go b/go-engine/internal/commands/import.go new file mode 100644 index 0000000..9dafc31 --- /dev/null +++ b/go-engine/internal/commands/import.go @@ -0,0 +1,309 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +package commands + +import ( + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/Artexis10/endstate/go-engine/internal/config" + "github.com/Artexis10/endstate/go-engine/internal/envelope" + "github.com/Artexis10/endstate/go-engine/internal/importer" + "github.com/Artexis10/endstate/go-engine/internal/manifest" +) + +// defaultImportOut is the default output path for a UniGetUI import, relative to +// the repo root. manifests/local/ is gitignored per repo conventions. +const defaultImportOutRel = "manifests/local/imported-unigetui.jsonc" + +// ImportFlags holds the parsed CLI flags for the import command. +type ImportFlags struct { + // From is the source format keyword. Only "unigetui" is supported in v0. + From string + // Path is the input file to import (a UniGetUI .ubundle). + Path string + // Out is the output manifest path. Empty defaults to + // manifests/local/imported-unigetui.jsonc (gitignored). + Out string + // Pin records versions on imported apps (InstallationOptions.Version pin + // beats the observed Version). Off by default. + Pin bool +} + +// ImportCounts summarises the outcome by category. +type ImportCounts struct { + Imported int `json:"imported"` + Skipped int `json:"skipped"` + Incompatible int `json:"incompatible"` +} + +// ImportData is the data payload for the import command JSON envelope. imported, +// skipped and incompatible together account for every package in the bundle — +// skip transparency is a first-class output. +type ImportData struct { + Source string `json:"source"` + Input string `json:"input"` + Output string `json:"output"` + ExportVersion float64 `json:"exportVersion"` + Pinned bool `json:"pinned"` + Counts ImportCounts `json:"counts"` + Imported []importer.ImportedApp `json:"imported"` + Skipped []importer.SkippedPackage `json:"skipped"` + Incompatible []importer.IncompatibleEntry `json:"incompatible"` + Warnings []string `json:"warnings,omitempty"` +} + +// RunImport reads a UniGetUI bundle, maps its winget-source packages onto a +// manifest, gates the emitted JSONC through the manifest loader, then writes it. +// Import is pure: no network access, no package operations, deterministic output. +func RunImport(flags ImportFlags) (interface{}, *envelope.Error) { + // --- 1. Validate --from (source format) --- + from := strings.ToLower(strings.TrimSpace(flags.From)) + if from == "" { + return nil, envelope.NewError( + envelope.ErrManifestValidationError, + "--from is required"). + WithRemediation("Specify the source format, e.g. --from unigetui.") + } + if from != "unigetui" { + return nil, envelope.NewError( + envelope.ErrNotSupported, + fmt.Sprintf("import source %q is not supported", flags.From)). + WithDetail(map[string]string{"from": flags.From}). + WithRemediation("The only supported source is 'unigetui'. Re-run with --from unigetui.") + } + + // --- 2. Validate --path (input file) --- + path := strings.TrimSpace(flags.Path) + if path == "" { + return nil, envelope.NewError( + envelope.ErrManifestValidationError, + "--path is required"). + WithRemediation("Provide the UniGetUI bundle to import, e.g. --path backup.ubundle.") + } + if _, statErr := os.Stat(path); errors.Is(statErr, os.ErrNotExist) { + return nil, envelope.NewError( + envelope.ErrManifestNotFound, + "The specified bundle file does not exist."). + WithDetail(map[string]string{"path": path}). + WithRemediation("Check the file path and ensure the UniGetUI bundle exists.") + } + + // Resolve the input to an absolute path so both the envelope's input and the + // out-of-repo default output are reported as absolute, resolved paths. + absInputPath, inAbsErr := filepath.Abs(path) + if inAbsErr != nil { + absInputPath = path + } + + // --- 3. Parse the bundle (pure; no network) --- + f, openErr := os.Open(path) + if openErr != nil { + return nil, envelope.NewError( + envelope.ErrManifestParseError, + "Failed to open the UniGetUI bundle."). + WithDetail(map[string]string{"path": path, "error": openErr.Error()}) + } + defer f.Close() + + bundle, warnings, parseErr := importer.ParseUniGetUI(f) + if parseErr != nil { + // A well-formed JSON file that simply is not a UniGetUI bundle is a + // validation problem (wrong --path), not a JSON parse failure. + if errors.Is(parseErr, importer.ErrNotUniGetUIBundle) { + return nil, envelope.NewError( + envelope.ErrManifestValidationError, + "The file does not look like a UniGetUI bundle."). + WithDetail(map[string]string{"path": path, "error": parseErr.Error()}). + WithRemediation("Point --path at a UniGetUI .ubundle export (JSON with export_version and packages).") + } + return nil, envelope.NewError( + envelope.ErrManifestParseError, + "Failed to parse the UniGetUI bundle."). + WithDetail(map[string]string{"path": path, "error": parseErr.Error()}). + WithRemediation("Ensure --path points to a valid UniGetUI .ubundle (JSON).") + } + + // --- 4. Map winget-source packages onto app entries --- + mapRes := importer.MapBundle(bundle, importer.MapOptions{Pin: flags.Pin}) + + // --- 5. Resolve output path and build the JSONC manifest --- + out := strings.TrimSpace(flags.Out) + if out == "" { + out = defaultImportOut(absInputPath) + } + outAbs, absErr := filepath.Abs(out) + if absErr != nil { + outAbs = out + } + + cliVersion := config.ReadVersion(resolveRepoRootFn()) + jsoncBytes := buildImportManifest(path, bundle.ExportVersion, cliVersion, flags.Pin, mapRes) + + // --- 6. Round-trip validity gate: the emitted bytes MUST load through the + // manifest loader BEFORE anything is written to the output path. --- + if gateErr := importRoundTripGateFn(jsoncBytes); gateErr != nil { + return nil, gateErr + } + + // --- 7. Write the manifest --- + if writeErr := writeImportManifest(outAbs, jsoncBytes); writeErr != nil { + return nil, envelope.NewError( + envelope.ErrManifestWriteFailed, + "Failed to write the imported manifest."). + WithDetail(map[string]string{"path": outAbs, "error": writeErr.Error()}). + WithRemediation("Ensure the output directory is writable.") + } + + // --- 8. Assemble the result. Collisions surface as warnings alongside the + // parser's version warning — every non-imported package is already in + // skipped/incompatible. --- + allWarnings := append([]string{}, warnings...) + allWarnings = append(allWarnings, mapRes.Collisions...) + + return &ImportData{ + Source: "unigetui", + Input: absInputPath, + Output: outAbs, + ExportVersion: bundle.ExportVersion, + Pinned: flags.Pin, + Counts: ImportCounts{ + Imported: len(mapRes.Imported), + Skipped: len(mapRes.Skipped), + Incompatible: len(mapRes.Incompatible), + }, + Imported: mapRes.Imported, + Skipped: mapRes.Skipped, + Incompatible: mapRes.Incompatible, + Warnings: allWarnings, + }, nil +} + +// defaultImportOut resolves the output path when --out is omitted. Inside a repo +// checkout it lands at manifests/local/imported-unigetui.jsonc (gitignored). +// Outside a repo (no root marker resolved) there is no manifests/local to target, +// so it lands next to the input bundle instead of polluting the working directory. +func defaultImportOut(absInputPath string) string { + root := resolveRepoRootFn() + if root == "" { + return filepath.Join(filepath.Dir(absInputPath), "imported-unigetui.jsonc") + } + return filepath.Join(root, filepath.FromSlash(defaultImportOutRel)) +} + +// buildImportManifest renders the JSONC manifest bytes deterministically: an +// ordered manifest struct marshalled with indentation, prefixed by a generated +// header comment naming the source file and tool version. No timestamp is +// emitted so identical input yields byte-identical output. +func buildImportManifest(sourcePath string, exportVersion float64, cliVersion string, pinned bool, res *importer.MapResult) []byte { + type outApp struct { + ID string `json:"id"` + Refs map[string]string `json:"refs"` + DisplayName string `json:"displayName,omitempty"` + Version string `json:"version,omitempty"` + } + type outManifest struct { + Version int `json:"version"` + Name string `json:"name"` + Apps []outApp `json:"apps"` + } + + apps := make([]outApp, 0, len(res.Imported)) + for _, a := range res.Imported { + apps = append(apps, outApp{ + ID: a.ID, + Refs: map[string]string{"windows": a.Ref}, + DisplayName: a.DisplayName, + Version: a.Version, + }) + } + + body, _ := json.MarshalIndent(outManifest{ + Version: 1, + Name: "imported-unigetui", + Apps: apps, + }, "", " ") + + pinNote := "" + if pinned { + pinNote = " (--pin: versions recorded)" + } + + var b strings.Builder + b.WriteString("// Generated by \"endstate import --from unigetui\".\n") + b.WriteString(fmt.Sprintf("// Source: %s (UniGetUI export_version %s)\n", sourcePath, importer.FormatExportVersion(exportVersion))) + b.WriteString(fmt.Sprintf("// Tool: endstate %s%s\n", cliVersion, pinNote)) + b.WriteString("// Scope: package list only. UniGetUI's own settings are NOT imported;\n") + b.WriteString("// Endstate's module catalog restores app config on plan/apply.\n") + b.Write(body) + b.WriteString("\n") + return []byte(b.String()) +} + +// importRoundTripGateFn gates the emitted bytes through the manifest loader +// before anything is written. It is a package-level var so tests can force the +// abort branch and assert that no output file is produced. +var importRoundTripGateFn = importRoundTripGate + +// importRoundTripGate writes the emitted bytes to a temp file and loads them +// through the manifest loader. A load failure aborts the import (nothing is +// written to the output path) with a validation error. +func importRoundTripGate(jsoncBytes []byte) *envelope.Error { + tmp, err := os.CreateTemp("", "endstate-import-*.jsonc") + if err != nil { + return envelope.NewError( + envelope.ErrInternalError, + "Failed to create a temporary file for the manifest validity check."). + WithDetail(map[string]string{"error": err.Error()}) + } + tmpPath := tmp.Name() + defer os.Remove(tmpPath) + + if _, err := tmp.Write(jsoncBytes); err != nil { + tmp.Close() + return envelope.NewError( + envelope.ErrInternalError, + "Failed to stage the manifest for the validity check."). + WithDetail(map[string]string{"error": err.Error()}) + } + if err := tmp.Close(); err != nil { + return envelope.NewError( + envelope.ErrInternalError, + "Failed to stage the manifest for the validity check."). + WithDetail(map[string]string{"error": err.Error()}) + } + + if _, err := manifest.LoadManifest(tmpPath); err != nil { + return envelope.NewError( + envelope.ErrManifestValidationError, + "The generated manifest failed to load and was not written."). + WithDetail(map[string]string{"error": err.Error()}). + WithRemediation("This is unexpected — please report the source bundle so the mapping can be fixed.") + } + return nil +} + +// writeImportManifest creates the output directory and writes the manifest bytes +// atomically: the bytes go to a sibling temp file first, then os.Rename swaps it +// into place (mirroring the state package's temp+rename pattern). On Windows +// os.Rename replaces an existing target, so an existing file at the output path +// is overwritten in a single step rather than truncated in place. +func writeImportManifest(path string, jsoncBytes []byte) error { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + tmpPath := path + ".tmp" + if err := os.WriteFile(tmpPath, jsoncBytes, 0o644); err != nil { + return err + } + if err := os.Rename(tmpPath, path); err != nil { + os.Remove(tmpPath) // best-effort cleanup of the staged temp file + return err + } + return nil +} diff --git a/go-engine/internal/commands/import_test.go b/go-engine/internal/commands/import_test.go new file mode 100644 index 0000000..5ff852d --- /dev/null +++ b/go-engine/internal/commands/import_test.go @@ -0,0 +1,400 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +package commands + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/Artexis10/endstate/go-engine/internal/envelope" + "github.com/Artexis10/endstate/go-engine/internal/manifest" +) + +const sampleBundlePath = "testdata/unigetui-sample.ubundle" + +// runImportOK runs RunImport against the sample fixture, fails the test on an +// envelope error, and returns the typed data payload. +func runImportOK(t *testing.T, flags ImportFlags) *ImportData { + t.Helper() + res, err := RunImport(flags) + if err != nil { + t.Fatalf("RunImport returned error: %+v", err) + } + data, ok := res.(*ImportData) + if !ok { + t.Fatalf("expected *ImportData, got %T", res) + } + return data +} + +// End-to-end: the sample fixture imports to a manifest that loads through the +// engine's manifest loader, and its apps match the winget-source packages. +func TestRunImport_FixtureRoundTrips(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + data := runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + + // The fixture has 3 winget packages (VS Code, Git, Firefox), 3 non-winget + // (choco, scoop, pip), and 1 incompatible. + if data.Counts.Imported != 3 { + t.Errorf("imported = %d, want 3", data.Counts.Imported) + } + if data.Counts.Skipped != 3 { + t.Errorf("skipped = %d, want 3", data.Counts.Skipped) + } + if data.Counts.Incompatible != 1 { + t.Errorf("incompatible = %d, want 1", data.Counts.Incompatible) + } + if data.ExportVersion != 3 { + t.Errorf("exportVersion = %v, want 3", data.ExportVersion) + } + + // The written file loads through the manifest loader (the same gate the + // command applies before writing). + mf, loadErr := manifest.LoadManifest(out) + if loadErr != nil { + t.Fatalf("emitted manifest failed to load: %v", loadErr) + } + if len(mf.Apps) != 3 { + t.Fatalf("loaded manifest apps = %d, want 3", len(mf.Apps)) + } + + byRef := map[string]manifest.App{} + for _, a := range mf.Apps { + byRef[a.Refs["windows"]] = a + } + vscode, ok := byRef["Microsoft.VisualStudioCode"] + if !ok { + t.Fatalf("expected an app with refs.windows=Microsoft.VisualStudioCode, apps=%+v", mf.Apps) + } + if vscode.ID != "visualstudiocode" || vscode.DisplayName != "Microsoft Visual Studio Code" { + t.Errorf("vscode app = %+v", vscode) + } + // No --pin: no app carries a version. + for _, a := range mf.Apps { + if a.Version != "" { + t.Errorf("app %q has version %q without --pin", a.ID, a.Version) + } + } +} + +// --pin records versions; the InstallationOptions.Version pin wins over the +// observed Version (VS Code: observed 1.85.1, pinned 1.85.0). +func TestRunImport_PinRecordsVersions(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out, Pin: true}) + + mf, loadErr := manifest.LoadManifest(out) + if loadErr != nil { + t.Fatalf("emitted manifest failed to load: %v", loadErr) + } + versions := map[string]string{} + for _, a := range mf.Apps { + versions[a.Refs["windows"]] = a.Version + } + if versions["Microsoft.VisualStudioCode"] != "1.85.0" { + t.Errorf("VS Code version = %q, want 1.85.0 (pin beats observed)", versions["Microsoft.VisualStudioCode"]) + } + if versions["Git.Git"] != "2.43.0" { + t.Errorf("Git version = %q, want 2.43.0 (observed)", versions["Git.Git"]) + } + // Firefox is version-less in the fixture: even with --pin it has no version. + if versions["Mozilla.Firefox"] != "" { + t.Errorf("Firefox version = %q, want empty (version-less package)", versions["Mozilla.Firefox"]) + } +} + +// Skip transparency: every non-winget manager is named and the incompatible +// entry passes through. +func TestRunImport_SkipTransparency(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + data := runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + + managers := map[string]bool{} + for _, s := range data.Skipped { + managers[s.Manager] = true + if s.Reason == "" { + t.Errorf("skipped %q missing a reason", s.ID) + } + } + for _, want := range []string{"Chocolatey", "Scoop", "Pip"} { + if !managers[want] { + t.Errorf("expected manager %q in skipped list, got %v", want, managers) + } + } + if len(data.Incompatible) != 1 || data.Incompatible[0].ID != "Contoso.LocalOnlyApp" { + t.Errorf("incompatible passthrough failed: %+v", data.Incompatible) + } +} + +// Import is deterministic: the emitted file bytes are identical across two runs. +func TestRunImport_DeterministicOutput(t *testing.T) { + out1 := filepath.Join(t.TempDir(), "a.jsonc") + out2 := filepath.Join(t.TempDir(), "b.jsonc") + runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out1, Pin: true}) + runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out2, Pin: true}) + + b1, _ := os.ReadFile(out1) + b2, _ := os.ReadFile(out2) + if string(b1) != string(b2) { + t.Errorf("import output is not byte-identical across runs:\n--- run1 ---\n%s\n--- run2 ---\n%s", b1, b2) + } +} + +// The default output path resolves under the repo root (ENDSTATE_ROOT here) at +// manifests/local/imported-unigetui.jsonc. +func TestRunImport_DefaultOutputPath(t *testing.T) { + root := t.TempDir() + t.Setenv("ENDSTATE_ROOT", root) + + data := runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath}) + + want := filepath.Join(root, "manifests", "local", "imported-unigetui.jsonc") + if data.Output != want { + t.Errorf("output = %q, want %q", data.Output, want) + } + if _, err := os.Stat(want); err != nil { + t.Errorf("expected the manifest at the default path, stat err = %v", err) + } +} + +// Outside a repo checkout (repo-root resolution empty), the default output lands +// next to the input bundle rather than in a cwd-relative manifests/local path. +func TestRunImport_DefaultOutputOutsideRepo(t *testing.T) { + orig := resolveRepoRootFn + resolveRepoRootFn = func() string { return "" } + defer func() { resolveRepoRootFn = orig }() + + // Copy the sample bundle into a temp dir so the default output lands beside it + // (and never pollutes the testdata directory). + dir := t.TempDir() + in := filepath.Join(dir, "backup.ubundle") + srcBytes, err := os.ReadFile(sampleBundlePath) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(in, srcBytes, 0o644); err != nil { + t.Fatal(err) + } + + data := runImportOK(t, ImportFlags{From: "unigetui", Path: in}) + + want := filepath.Join(dir, "imported-unigetui.jsonc") + if data.Output != want { + t.Errorf("output = %q, want %q (beside the input bundle when outside a repo)", data.Output, want) + } + if _, statErr := os.Stat(want); statErr != nil { + t.Errorf("expected the manifest beside the input bundle, stat err = %v", statErr) + } +} + +// The reported input path is absolute even when --path is relative, matching the +// absolute output path. +func TestRunImport_InputAbsolutized(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + data := runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + if !filepath.IsAbs(data.Input) { + t.Errorf("input = %q, want an absolute path", data.Input) + } + wantAbs, _ := filepath.Abs(sampleBundlePath) + if data.Input != wantAbs { + t.Errorf("input = %q, want %q", data.Input, wantAbs) + } +} + +// A well-formed JSON file that is not a UniGetUI bundle (e.g. a package.json) is +// a validation error pointing back at --path; nothing is written. +func TestRunImport_WrongFileShapeValidation(t *testing.T) { + notBundle := filepath.Join(t.TempDir(), "package.json") + if err := os.WriteFile(notBundle, []byte(`{"name": "x", "version": "1.0.0"}`), 0o644); err != nil { + t.Fatal(err) + } + out := filepath.Join(t.TempDir(), "imported.jsonc") + _, err := RunImport(ImportFlags{From: "unigetui", Path: notBundle, Out: out}) + if err == nil || err.Code != envelope.ErrManifestValidationError { + t.Fatalf("expected MANIFEST_VALIDATION_ERROR for a non-bundle JSON file, got %+v", err) + } + if err.Remediation == "" || !strings.Contains(err.Remediation, "--path") { + t.Errorf("remediation should mention --path, got %q", err.Remediation) + } + if _, statErr := os.Stat(out); !os.IsNotExist(statErr) { + t.Errorf("nothing should be written for a non-bundle file, stat err = %v", statErr) + } +} + +// An empty but versioned bundle imports as empty (zero apps, no error). +func TestRunImport_EmptyVersionedBundle(t *testing.T) { + in := filepath.Join(t.TempDir(), "empty.ubundle") + if err := os.WriteFile(in, []byte(`{"export_version": 3, "packages": []}`), 0o644); err != nil { + t.Fatal(err) + } + out := filepath.Join(t.TempDir(), "imported.jsonc") + data := runImportOK(t, ImportFlags{From: "unigetui", Path: in, Out: out}) + if data.Counts.Imported != 0 || data.Counts.Skipped != 0 || data.Counts.Incompatible != 0 { + t.Errorf("empty bundle counts = %+v, want all zero", data.Counts) + } +} + +// The round-trip gate rejects bytes that fail to load as a manifest (white-box). +func TestImportRoundTripGate_RejectsInvalid(t *testing.T) { + gateErr := importRoundTripGate([]byte("{ this is not a manifest")) + if gateErr == nil { + t.Fatal("expected the round-trip gate to reject invalid manifest bytes") + } + if gateErr.Code != envelope.ErrManifestValidationError { + t.Errorf("gate error code = %q, want MANIFEST_VALIDATION_ERROR", gateErr.Code) + } +} + +// When the gate aborts, RunImport returns the gate error and writes no output. +func TestRunImport_GateFailureWritesNothing(t *testing.T) { + orig := importRoundTripGateFn + importRoundTripGateFn = func(_ []byte) *envelope.Error { + return envelope.NewError(envelope.ErrManifestValidationError, "forced gate failure") + } + defer func() { importRoundTripGateFn = orig }() + + out := filepath.Join(t.TempDir(), "imported.jsonc") + _, err := RunImport(ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + if err == nil || err.Code != envelope.ErrManifestValidationError { + t.Fatalf("expected the gate's MANIFEST_VALIDATION_ERROR, got %+v", err) + } + if _, statErr := os.Stat(out); !os.IsNotExist(statErr) { + t.Errorf("no manifest should be written when the gate fails, stat err = %v", statErr) + } +} + +// An existing file at the output path is replaced (atomic overwrite). +func TestRunImport_OverwritesExistingOutput(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + if err := os.WriteFile(out, []byte("STALE CONTENT"), 0o644); err != nil { + t.Fatal(err) + } + runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + + b, _ := os.ReadFile(out) + if string(b) == "STALE CONTENT" { + t.Error("expected the existing output file to be replaced, but it was unchanged") + } + if _, err := manifest.LoadManifest(out); err != nil { + t.Errorf("the replaced file must load as a manifest, got: %v", err) + } + // No temp file lingers. + if _, statErr := os.Stat(out + ".tmp"); !os.IsNotExist(statErr) { + t.Errorf("expected the .tmp staging file to be removed, stat err = %v", statErr) + } +} + +// The emitted manifest carries a JSONC header comment yet still loads (the +// header must round-trip through StripJsoncComments). +func TestRunImport_EmitsJsoncHeader(t *testing.T) { + out := filepath.Join(t.TempDir(), "imported.jsonc") + runImportOK(t, ImportFlags{From: "unigetui", Path: sampleBundlePath, Out: out}) + + b, _ := os.ReadFile(out) + if len(b) == 0 || b[0] != '/' { + t.Fatalf("expected the file to start with a // header comment, got: %.40q", string(b)) + } + if _, err := manifest.LoadManifest(out); err != nil { + t.Errorf("a header-commented manifest must load via the JSONC loader, got: %v", err) + } +} + +// An unknown --from source is NOT_SUPPORTED with remediation. +func TestRunImport_UnknownSourceNotSupported(t *testing.T) { + _, err := RunImport(ImportFlags{From: "dsc", Path: sampleBundlePath}) + if err == nil || err.Code != envelope.ErrNotSupported { + t.Fatalf("expected NOT_SUPPORTED, got %+v", err) + } + if err.Remediation == "" { + t.Error("NOT_SUPPORTED should carry remediation") + } +} + +// An empty --from is a validation error. +func TestRunImport_EmptyFromValidation(t *testing.T) { + _, err := RunImport(ImportFlags{From: "", Path: sampleBundlePath}) + if err == nil || err.Code != envelope.ErrManifestValidationError { + t.Fatalf("expected MANIFEST_VALIDATION_ERROR, got %+v", err) + } +} + +// An empty --path is a validation error. +func TestRunImport_EmptyPathValidation(t *testing.T) { + _, err := RunImport(ImportFlags{From: "unigetui", Path: ""}) + if err == nil || err.Code != envelope.ErrManifestValidationError { + t.Fatalf("expected MANIFEST_VALIDATION_ERROR, got %+v", err) + } +} + +// A missing bundle path is MANIFEST_NOT_FOUND. +func TestRunImport_MissingPath(t *testing.T) { + missing := filepath.Join(t.TempDir(), "nope.ubundle") + _, err := RunImport(ImportFlags{From: "unigetui", Path: missing}) + if err == nil || err.Code != envelope.ErrManifestNotFound { + t.Fatalf("expected MANIFEST_NOT_FOUND, got %+v", err) + } +} + +// Malformed bundle JSON is MANIFEST_PARSE_ERROR; nothing is written. +func TestRunImport_MalformedBundle(t *testing.T) { + bad := filepath.Join(t.TempDir(), "bad.ubundle") + if err := os.WriteFile(bad, []byte("{not json"), 0o644); err != nil { + t.Fatal(err) + } + out := filepath.Join(t.TempDir(), "imported.jsonc") + _, err := RunImport(ImportFlags{From: "unigetui", Path: bad, Out: out}) + if err == nil || err.Code != envelope.ErrManifestParseError { + t.Fatalf("expected MANIFEST_PARSE_ERROR, got %+v", err) + } + if _, statErr := os.Stat(out); !os.IsNotExist(statErr) { + t.Errorf("no manifest should be written on a parse error, stat err = %v", statErr) + } +} + +// A future export_version imports with a warning surfaced in the result. +func TestRunImport_FutureVersionWarns(t *testing.T) { + src := `{"export_version": 4, "packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + in := filepath.Join(t.TempDir(), "v4.ubundle") + if err := os.WriteFile(in, []byte(src), 0o644); err != nil { + t.Fatal(err) + } + out := filepath.Join(t.TempDir(), "imported.jsonc") + data := runImportOK(t, ImportFlags{From: "unigetui", Path: in, Out: out}) + if len(data.Warnings) == 0 { + t.Error("expected a version warning in the result") + } + if data.Counts.Imported != 1 { + t.Errorf("imported = %d, want 1 under a future version", data.Counts.Imported) + } +} + +// Capabilities advertises the import command with --from and --path. +func TestRunCapabilities_ImportFlags(t *testing.T) { + result, err := RunCapabilities() + if err != nil { + t.Fatalf("RunCapabilities returned error: %v", err) + } + data := result.(CapabilitiesData) + importCmd, ok := data.Commands["import"] + if !ok { + t.Fatal("import command not found in capabilities") + } + if !importCmd.Supported { + t.Error("expected commands.import.supported = true") + } + for _, want := range []string{"--from", "--path", "--out", "--pin", "--json"} { + found := false + for _, f := range importCmd.Flags { + if f == want { + found = true + break + } + } + if !found { + t.Errorf("commands.import.flags missing %q; got %v", want, importCmd.Flags) + } + } +} diff --git a/go-engine/internal/commands/testdata/unigetui-sample.ubundle b/go-engine/internal/commands/testdata/unigetui-sample.ubundle new file mode 100644 index 0000000..0ebce32 --- /dev/null +++ b/go-engine/internal/commands/testdata/unigetui-sample.ubundle @@ -0,0 +1,64 @@ +{ + "export_version": 3, + "packages": [ + { + "Id": "Microsoft.VisualStudioCode", + "Name": "Microsoft Visual Studio Code", + "Version": "1.85.1", + "Source": "winget", + "ManagerName": "WinGet", + "InstallationOptions": { + "OverridesNextLevelOpts": true, + "Version": "1.85.0" + } + }, + { + "Id": "Git.Git", + "Name": "Git", + "Version": "2.43.0", + "Source": "winget", + "ManagerName": "WinGet" + }, + { + "Id": "Mozilla.Firefox", + "Name": "Mozilla Firefox", + "Version": "", + "Source": "winget", + "ManagerName": "WinGet" + }, + { + "Id": "nodejs", + "Name": "Node.js", + "Version": "20.10.0", + "Source": "chocolatey", + "ManagerName": "Chocolatey" + }, + { + "Id": "neovim", + "Name": "Neovim", + "Version": "0.9.5", + "Source": "main", + "ManagerName": "Scoop" + }, + { + "Id": "black", + "Name": "black", + "Version": "23.12.1", + "Source": "pip", + "ManagerName": "Pip", + "Updates": { + "UpdatesIgnored": true, + "IgnoredVersion": "24.0.0" + } + } + ], + "incompatible_packages_info": "Incompatible packages cannot be installed from UniGetUI, either because they came from a local source (for example Local PC) or because the package manager was unavailable. Nevertheless, they have been listed here for logging purposes.", + "incompatible_packages": [ + { + "Id": "Contoso.LocalOnlyApp", + "Name": "Contoso Local Only App", + "Version": "1.0.0", + "Source": "Local PC" + } + ] +} diff --git a/go-engine/internal/importer/mapper.go b/go-engine/internal/importer/mapper.go new file mode 100644 index 0000000..99a5775 --- /dev/null +++ b/go-engine/internal/importer/mapper.go @@ -0,0 +1,243 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +package importer + +import ( + "fmt" + "sort" + "strings" +) + +// MapOptions controls how a Bundle is mapped onto manifest app entries. +type MapOptions struct { + // Pin, when true, records a version on each imported app: the package's + // InstallationOptions.Version pin when present (authored intent), otherwise + // the bundle's observed Version. When false, no version is written. + Pin bool +} + +// ImportedApp is a winget-source package mapped to an Endstate manifest app. +// Ref is the winget package Id (the manifest's refs.windows value); ID is the +// deterministic, de-duplicated manifest app id. +type ImportedApp struct { + ID string `json:"id"` + Ref string `json:"ref"` + DisplayName string `json:"displayName,omitempty"` + Version string `json:"version,omitempty"` +} + +// SkippedPackage records a bundle package that was NOT imported, with the +// managing tool and the reason. Skip transparency is a first-class output: no +// package is silently dropped. +type SkippedPackage struct { + ID string `json:"id,omitempty"` + Name string `json:"name,omitempty"` + Manager string `json:"manager"` + Reason string `json:"reason"` +} + +// IncompatibleEntry is a bundle incompatible_packages entry passed through to +// the report verbatim. +type IncompatibleEntry struct { + ID string `json:"id,omitempty"` + Name string `json:"name,omitempty"` + Version string `json:"version,omitempty"` + Source string `json:"source,omitempty"` +} + +// MapResult is the deterministic outcome of mapping a Bundle. Imported, Skipped +// and Incompatible together account for every package and incompatible entry in +// the bundle. Collisions notes any app-id de-duplication that occurred. +type MapResult struct { + Imported []ImportedApp + Skipped []SkippedPackage + Incompatible []IncompatibleEntry + Collisions []string +} + +// MapBundle maps a UniGetUI Bundle onto manifest app entries. Winget-source +// packages become ImportedApp entries (Id → refs.windows, Name → displayName, +// a deterministic slug → app id); every other package is Skipped with its +// manager and a reason; incompatible_packages pass through. Output ordering is +// canonicalised — winget packages are sorted by Id (case-insensitive) before +// slug/de-dup assignment, and skipped/incompatible are sorted before returning — +// so two exports of the same machine that differ only in package order yield +// identical output. +func MapBundle(b *Bundle, opts MapOptions) *MapResult { + res := &MapResult{ + Imported: []ImportedApp{}, + Skipped: []SkippedPackage{}, + Incompatible: []IncompatibleEntry{}, + } + + // Partition first: route non-winget and Id-less packages straight to skipped + // (order-independent, they are sorted at the end), and collect the winget + // candidates for a stable, order-independent mapping pass. + winget := make([]Package, 0, len(b.Packages)) + for _, pkg := range b.Packages { + if !isWingetPackage(pkg.Source, pkg.ManagerName) { + res.Skipped = append(res.Skipped, SkippedPackage{ + ID: strings.TrimSpace(pkg.ID), + Name: strings.TrimSpace(pkg.Name), + Manager: managerLabel(pkg.Source, pkg.ManagerName), + Reason: "unsupported package manager (Endstate installs via winget on Windows)", + }) + continue + } + if strings.TrimSpace(pkg.ID) == "" { + res.Skipped = append(res.Skipped, SkippedPackage{ + Name: strings.TrimSpace(pkg.Name), + Manager: "winget", + Reason: "winget package has no Id", + }) + continue + } + winget = append(winget, pkg) + } + + // Sort winget candidates by Id (case-insensitive) so slug assignment and + // collision suffixes are stable regardless of the bundle's package order. + sort.SliceStable(winget, func(i, j int) bool { + return strings.ToLower(strings.TrimSpace(winget[i].ID)) < + strings.ToLower(strings.TrimSpace(winget[j].ID)) + }) + + // seen tracks every app id already assigned so de-duplication is stable and + // collision-safe even against a synthesized "-N" suffix. slugByRef records + // the slug assigned to each winget Id (case-insensitive) so a repeated Id is + // reported as a duplicate of the first occurrence rather than mapped twice. + seen := make(map[string]bool) + slugByRef := make(map[string]string) + + for _, pkg := range winget { + id := strings.TrimSpace(pkg.ID) + if firstSlug, dup := slugByRef[strings.ToLower(id)]; dup { + res.Skipped = append(res.Skipped, SkippedPackage{ + ID: id, + Name: strings.TrimSpace(pkg.Name), + Manager: "winget", + Reason: fmt.Sprintf("duplicate of %s", firstSlug), + }) + continue + } + + base := slugForID(id) + slug := uniqueSlug(base, seen) + if slug != base { + res.Collisions = append(res.Collisions, fmt.Sprintf( + "app id %q collided; imported as %q (winget id %q)", base, slug, id)) + } + slugByRef[strings.ToLower(id)] = slug + + app := ImportedApp{ + ID: slug, + Ref: id, + DisplayName: strings.TrimSpace(pkg.Name), + } + if opts.Pin { + v := strings.TrimSpace(pkg.InstallationOptions.Version) + if v == "" { + v = strings.TrimSpace(pkg.Version) + } + app.Version = v + } + res.Imported = append(res.Imported, app) + } + + for _, inc := range b.IncompatiblePackages { + res.Incompatible = append(res.Incompatible, IncompatibleEntry{ + ID: strings.TrimSpace(inc.ID), + Name: strings.TrimSpace(inc.Name), + Version: strings.TrimSpace(inc.Version), + Source: strings.TrimSpace(inc.Source), + }) + } + + // Canonical ordering for the transparency lists: skipped by manager then Id, + // incompatible by Id. + sort.SliceStable(res.Skipped, func(i, j int) bool { + if res.Skipped[i].Manager != res.Skipped[j].Manager { + return res.Skipped[i].Manager < res.Skipped[j].Manager + } + return res.Skipped[i].ID < res.Skipped[j].ID + }) + sort.SliceStable(res.Incompatible, func(i, j int) bool { + return res.Incompatible[i].ID < res.Incompatible[j].ID + }) + + return res +} + +// isWingetPackage reports whether a package should be treated as winget-source. +// Source is authoritative when present (case-insensitive "winget"); when Source +// is empty, ManagerName is the fallback discriminator. A package from another +// source (chocolatey, pip, a scoop bucket, msstore) is therefore not winget even +// if the WinGet manager brokered it. +func isWingetPackage(source, managerName string) bool { + s := strings.TrimSpace(source) + if s != "" { + return strings.EqualFold(s, "winget") + } + return strings.EqualFold(strings.TrimSpace(managerName), "winget") +} + +// managerLabel returns the human-facing managing tool for a skip report. It +// prefers ManagerName (the manager, e.g. "Scoop") over Source (which may be a +// sub-source or bucket, e.g. "main"), falling back to "unknown". +func managerLabel(source, managerName string) string { + if m := strings.TrimSpace(managerName); m != "" { + return m + } + if s := strings.TrimSpace(source); s != "" { + return s + } + return "unknown" +} + +// slugForID derives a manifest app id from a winget package Id: the lowercased +// last dot-segment, sanitized to [a-z0-9-]. "Microsoft.VisualStudioCode" → +// "visualstudiocode", "Git.Git" → "git". Falls back to "app" if nothing usable +// remains. +func slugForID(id string) string { + seg := id + if idx := strings.LastIndex(id, "."); idx >= 0 && idx < len(id)-1 { + seg = id[idx+1:] + } + seg = strings.ToLower(strings.TrimSpace(seg)) + + var b strings.Builder + for _, r := range seg { + switch { + case r >= 'a' && r <= 'z', r >= '0' && r <= '9': + b.WriteRune(r) + default: + b.WriteByte('-') + } + } + s := b.String() + for strings.Contains(s, "--") { + s = strings.ReplaceAll(s, "--", "-") + } + s = strings.Trim(s, "-") + if s == "" { + return "app" + } + return s +} + +// uniqueSlug returns base if unused, otherwise the first free "base-N" (N≥2), +// marking the returned slug as used. Deterministic given a stable call order. +func uniqueSlug(base string, seen map[string]bool) string { + if !seen[base] { + seen[base] = true + return base + } + for n := 2; ; n++ { + cand := fmt.Sprintf("%s-%d", base, n) + if !seen[cand] { + seen[cand] = true + return cand + } + } +} diff --git a/go-engine/internal/importer/mapper_test.go b/go-engine/internal/importer/mapper_test.go new file mode 100644 index 0000000..0b7d4d7 --- /dev/null +++ b/go-engine/internal/importer/mapper_test.go @@ -0,0 +1,263 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +package importer + +import ( + "reflect" + "testing" +) + +// Winget-source packages map to app entries: Id → Ref, Name → DisplayName, +// slugged last-segment → ID. Imported ordering is canonical (by Id, +// case-insensitive), so lookups are by Ref rather than input position. +func TestMapBundle_WingetMapping(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "Microsoft.VisualStudioCode", Name: "Microsoft Visual Studio Code", Source: "winget", ManagerName: "WinGet"}, + {ID: "Git.Git", Name: "Git", Source: "winget", ManagerName: "WinGet"}, + }} + + res := MapBundle(b, MapOptions{}) + if len(res.Imported) != 2 { + t.Fatalf("imported = %d, want 2", len(res.Imported)) + } + byRef := map[string]ImportedApp{} + for _, a := range res.Imported { + byRef[a.Ref] = a + } + vsc := byRef["Microsoft.VisualStudioCode"] + if vsc.ID != "visualstudiocode" || vsc.DisplayName != "Microsoft Visual Studio Code" { + t.Errorf("vscode imported = %+v", vsc) + } + if git := byRef["Git.Git"]; git.ID != "git" { + t.Errorf("git imported = %+v", git) + } + if vsc.Version != "" { + t.Errorf("no --pin: version should be empty, got %q", vsc.Version) + } + + // Canonical order: Git.Git sorts before Microsoft.VisualStudioCode. + if res.Imported[0].Ref != "Git.Git" || res.Imported[1].Ref != "Microsoft.VisualStudioCode" { + t.Errorf("imported order = [%s, %s], want [Git.Git, Microsoft.VisualStudioCode]", + res.Imported[0].Ref, res.Imported[1].Ref) + } +} + +// A repeated winget Id is imported once; the second occurrence is reported in +// skipped[] as a duplicate of the first slug — never silently dropped. +func TestMapBundle_DuplicateWingetId(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "Git.Git", Name: "Git", Source: "winget"}, + {ID: "Git.Git", Name: "Git (again)", Source: "winget"}, + }} + res := MapBundle(b, MapOptions{}) + + if len(res.Imported) != 1 { + t.Fatalf("a doubled winget Id must import once, got %d: %+v", len(res.Imported), res.Imported) + } + if res.Imported[0].Ref != "Git.Git" || res.Imported[0].ID != "git" { + t.Errorf("imported = %+v, want ref Git.Git / id git", res.Imported[0]) + } + + // refs.windows are unique across imported apps. + refs := map[string]int{} + for _, a := range res.Imported { + refs[a.Ref]++ + } + for ref, n := range refs { + if n != 1 { + t.Errorf("ref %q appears %d times in imported; refs.windows must be unique", ref, n) + } + } + + // The second occurrence surfaces as a skip with the duplicate reason. + var dup *SkippedPackage + for i := range res.Skipped { + if res.Skipped[i].ID == "Git.Git" { + dup = &res.Skipped[i] + } + } + if dup == nil { + t.Fatalf("expected the duplicate reported in skipped, got %+v", res.Skipped) + } + if dup.Reason != "duplicate of git" { + t.Errorf("duplicate skip reason = %q, want %q", dup.Reason, "duplicate of git") + } + if dup.Manager != "winget" { + t.Errorf("duplicate skip manager = %q, want winget", dup.Manager) + } +} + +// Two exports of the same machine that differ only in package order produce an +// identical MapResult (stable ordering across re-exports). +func TestMapBundle_StableAcrossReorder(t *testing.T) { + pkgs := []Package{ + {ID: "Microsoft.VisualStudioCode", Name: "VS Code", Source: "winget", Version: "1.0"}, + {ID: "Git.Git", Name: "Git", Source: "winget"}, + {ID: "Mozilla.Firefox", Name: "Firefox", Source: "winget"}, + {ID: "nodejs", Name: "Node.js", Source: "chocolatey", ManagerName: "Chocolatey"}, + {ID: "black", Name: "black", Source: "pip", ManagerName: "Pip"}, + } + reordered := []Package{pkgs[4], pkgs[0], pkgs[3], pkgs[2], pkgs[1]} + + a := MapBundle(&Bundle{Packages: pkgs}, MapOptions{Pin: true}) + c := MapBundle(&Bundle{Packages: reordered}, MapOptions{Pin: true}) + if !reflect.DeepEqual(a, c) { + t.Errorf("MapBundle is order-sensitive:\n original = %+v\n reordered = %+v", a, c) + } +} + +// The winget discriminator is case-insensitive, and ManagerName is only a +// fallback when Source is empty. +func TestMapBundle_WingetDiscriminator(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "A.One", Name: "One", Source: "WinGet"}, // case-insensitive source + {ID: "B.Two", Name: "Two", Source: "", ManagerName: "winget"}, // fallback via ManagerName + {ID: "C.Three", Name: "Three", Source: "msstore", ManagerName: "WinGet"}, // non-winget source wins over manager + }} + res := MapBundle(b, MapOptions{}) + if len(res.Imported) != 2 { + t.Fatalf("imported = %d, want 2 (msstore-source must not import)", len(res.Imported)) + } + if len(res.Skipped) != 1 || res.Skipped[0].ID != "C.Three" { + t.Fatalf("expected the msstore package skipped, got %+v", res.Skipped) + } +} + +// --pin: InstallationOptions.Version (authored intent) beats the observed Version. +func TestMapBundle_PinPrecedence(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "A.Pinned", Name: "Pinned", Source: "winget", Version: "1.2.3", + InstallationOptions: InstallOptions{Version: "1.2.0"}}, + {ID: "B.Observed", Name: "Observed", Source: "winget", Version: "9.9.9"}, + {ID: "C.Versionless", Name: "Versionless", Source: "winget"}, + }} + res := MapBundle(b, MapOptions{Pin: true}) + if res.Imported[0].Version != "1.2.0" { + t.Errorf("pin should win: version = %q, want 1.2.0", res.Imported[0].Version) + } + if res.Imported[1].Version != "9.9.9" { + t.Errorf("observed version should be used absent a pin: version = %q, want 9.9.9", res.Imported[1].Version) + } + if res.Imported[2].Version != "" { + t.Errorf("versionless package with --pin should have empty version, got %q", res.Imported[2].Version) + } +} + +// Without --pin, no imported app carries a version. +func TestMapBundle_NoPinDefault(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "A.One", Name: "One", Source: "winget", Version: "1.0.0", + InstallationOptions: InstallOptions{Version: "0.9.0"}}, + }} + res := MapBundle(b, MapOptions{}) + if res.Imported[0].Version != "" { + t.Errorf("default (no --pin): version must be empty, got %q", res.Imported[0].Version) + } +} + +// Skip transparency: every non-winget manager is reported with a count and a +// reason; incompatible packages pass through; nothing is silently dropped. +func TestMapBundle_SkipTransparency(t *testing.T) { + b := &Bundle{ + Packages: []Package{ + {ID: "Git.Git", Name: "Git", Source: "winget", ManagerName: "WinGet"}, + {ID: "nodejs", Name: "Node.js", Source: "chocolatey", ManagerName: "Chocolatey"}, + {ID: "neovim", Name: "Neovim", Source: "main", ManagerName: "Scoop"}, + {ID: "black", Name: "black", Source: "pip", ManagerName: "Pip"}, + }, + IncompatiblePackages: []IncompatiblePackage{ + {ID: "Contoso.Local", Name: "Local App", Version: "1.0.0", Source: "Local PC"}, + }, + } + res := MapBundle(b, MapOptions{}) + + // No package is unaccounted for. + if got := len(res.Imported) + len(res.Skipped); got != len(b.Packages) { + t.Errorf("imported+skipped = %d, want %d (no silent drops)", got, len(b.Packages)) + } + if len(res.Skipped) != 3 { + t.Fatalf("skipped = %d, want 3", len(res.Skipped)) + } + + // Each non-winget manager is named. + managers := map[string]int{} + for _, s := range res.Skipped { + managers[s.Manager]++ + if s.Reason == "" { + t.Errorf("skipped %q has no reason", s.ID) + } + } + for _, want := range []string{"Chocolatey", "Scoop", "Pip"} { + if managers[want] != 1 { + t.Errorf("expected manager %q reported once, counts = %v", want, managers) + } + } + + if len(res.Incompatible) != 1 || res.Incompatible[0].ID != "Contoso.Local" { + t.Errorf("incompatible passthrough failed: %+v", res.Incompatible) + } +} + +// A winget package with no Id is skipped (not imported) with a clear reason. +func TestMapBundle_MissingIdSkipped(t *testing.T) { + b := &Bundle{Packages: []Package{{ID: "", Name: "Nameless", Source: "winget"}}} + res := MapBundle(b, MapOptions{}) + if len(res.Imported) != 0 { + t.Errorf("a winget package with no Id must not import, got %+v", res.Imported) + } + if len(res.Skipped) != 1 || res.Skipped[0].Manager != "winget" { + t.Fatalf("expected one winget skip for the missing Id, got %+v", res.Skipped) + } +} + +// Slug collisions are de-duplicated deterministically and recorded. +func TestMapBundle_SlugCollisionDedup(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "Foo.Git", Name: "Foo Git", Source: "winget"}, + {ID: "Bar.Git", Name: "Bar Git", Source: "winget"}, + {ID: "Baz.Git", Name: "Baz Git", Source: "winget"}, + }} + res := MapBundle(b, MapOptions{}) + got := []string{res.Imported[0].ID, res.Imported[1].ID, res.Imported[2].ID} + want := []string{"git", "git-2", "git-3"} + if !reflect.DeepEqual(got, want) { + t.Errorf("collision de-dup ids = %v, want %v", got, want) + } + if len(res.Collisions) != 2 { + t.Errorf("expected 2 collision notes, got %d: %v", len(res.Collisions), res.Collisions) + } +} + +// Mapping is deterministic: the same input with the same options yields an +// identical result value. +func TestMapBundle_Deterministic(t *testing.T) { + b := &Bundle{Packages: []Package{ + {ID: "Microsoft.VisualStudioCode", Name: "VS Code", Source: "winget", Version: "1.0"}, + {ID: "Foo.Git", Name: "Foo Git", Source: "winget"}, + {ID: "Bar.Git", Name: "Bar Git", Source: "winget"}, + {ID: "nodejs", Name: "Node.js", Source: "chocolatey", ManagerName: "Chocolatey"}, + }} + a := MapBundle(b, MapOptions{Pin: true}) + c := MapBundle(b, MapOptions{Pin: true}) + if !reflect.DeepEqual(a, c) { + t.Errorf("MapBundle is not deterministic:\n a = %+v\n c = %+v", a, c) + } +} + +func TestSlugForID(t *testing.T) { + cases := map[string]string{ + "Microsoft.VisualStudioCode": "visualstudiocode", + "Git.Git": "git", + "Mozilla.Firefox": "firefox", + "vim": "vim", + "": "app", + "A.B.C": "c", + "Some.Weird!Id": "weird-id", + } + for in, want := range cases { + if got := slugForID(in); got != want { + t.Errorf("slugForID(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/go-engine/internal/importer/ubundle.go b/go-engine/internal/importer/ubundle.go new file mode 100644 index 0000000..29df425 --- /dev/null +++ b/go-engine/internal/importer/ubundle.go @@ -0,0 +1,166 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +// Package importer parses external package-list formats (currently UniGetUI +// `.ubundle` backups/bundles) and maps them onto Endstate manifest app entries. +// It is a pure, hermetic transform: no network access, no package operations, +// and byte-identical output for identical input. +package importer + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "io" + "strconv" + "strings" +) + +// ExpectedExportVersion is the UniGetUI SerializableBundle version this parser +// was written against (matches SerializableBundle.ExpectedVersion upstream). A +// bundle declaring a different version parses anyway but yields a warning — the +// format is additive, so hard-failing would break on UniGetUI's next release. +const ExpectedExportVersion = 3 + +// ErrNotUniGetUIBundle is returned when the input is valid JSON but carries +// neither an export_version nor a packages key — i.e. it parses as JSON but is +// not a UniGetUI bundle (a mistakenly-passed package.json, for example). The +// command surfaces this as a validation error pointing back at --path. +var ErrNotUniGetUIBundle = errors.New( + "importer: does not look like a UniGetUI bundle (missing export_version and packages)") + +// Bundle is a UniGetUI SerializableBundle. The top-level field names are +// snake_case in UniGetUI's serialization while the package-level fields are +// PascalCase — that mixed casing is faithful to upstream (System.Text.Json +// serializes each model with its own property names). Unknown fields (e.g. +// incompatible_packages_info) are ignored by Go's decoder. +type Bundle struct { + // ExportVersion is UniGetUI's export_version (a C# double). It is decoded + // out of band (see coerceExportVersion) rather than via struct tags so a + // non-numeric encoding degrades to a warning instead of failing the parse; + // the json:"-" tag keeps the strict struct decode from choking on a string. + ExportVersion float64 `json:"-"` + Packages []Package `json:"packages"` + IncompatiblePackages []IncompatiblePackage `json:"incompatible_packages"` +} + +// Package is a UniGetUI SerializablePackage. Only the fields Endstate maps are +// modelled; the rest of the upstream shape (Updates, the full InstallOptions +// surface) is ignored. +type Package struct { + ID string `json:"Id"` + Name string `json:"Name"` + Version string `json:"Version"` + Source string `json:"Source"` + ManagerName string `json:"ManagerName"` + InstallationOptions InstallOptions `json:"InstallationOptions"` +} + +// InstallOptions is a UniGetUI InstallOptions object. Only the Version pin is +// modelled — it is the sole field Endstate consumes (authored install intent). +// Upstream omits this object entirely when it holds only default values, so it +// is frequently absent; the zero value (empty Version) is correct in that case. +type InstallOptions struct { + Version string `json:"Version"` +} + +// IncompatiblePackage is a UniGetUI SerializableIncompatiblePackage: an entry +// UniGetUI itself could not install (a local source such as "Local PC", or an +// unavailable manager). It carries no ManagerName. Endstate passes these +// through to the import report verbatim — never silently dropped. +type IncompatiblePackage struct { + ID string `json:"Id"` + Name string `json:"Name"` + Version string `json:"Version"` + Source string `json:"Source"` +} + +// utf8BOM is the UTF-8 byte-order mark some editors prepend; it must be stripped +// before json.Unmarshal, which rejects a leading BOM. +var utf8BOM = []byte{0xEF, 0xBB, 0xBF} + +// ParseUniGetUI reads a UniGetUI `.ubundle` (plain JSON, not JSONC) from r and +// returns the parsed Bundle. It hard-fails only on malformed JSON or a payload +// that is not a UniGetUI bundle at all (neither export_version nor packages — +// see ErrNotUniGetUIBundle). Otherwise it is forward-compatible: an +// export_version other than ExpectedExportVersion (or a non-numeric encoding) +// yields a warning (returned in the warnings slice), not an error, and unknown +// JSON fields are ignored. +func ParseUniGetUI(r io.Reader) (*Bundle, []string, error) { + data, err := io.ReadAll(r) + if err != nil { + return nil, nil, fmt.Errorf("importer: cannot read bundle: %w", err) + } + data = bytes.TrimPrefix(data, utf8BOM) + + // Probe the top-level object first: this validates that the payload is a JSON + // object (malformed JSON hard-fails here) and lets us check for the bundle's + // signature keys and coerce export_version tolerantly. + var probe map[string]json.RawMessage + if err := json.Unmarshal(data, &probe); err != nil { + return nil, nil, fmt.Errorf("importer: invalid UniGetUI bundle JSON: %w", err) + } + + // Wrong-file shape guard: a UniGetUI bundle always carries at least one of + // export_version or packages. Neither present means this is some other JSON + // file (e.g. a package.json), not a bundle. + _, hasVersion := probe["export_version"] + _, hasPackages := probe["packages"] + if !hasVersion && !hasPackages { + return nil, nil, ErrNotUniGetUIBundle + } + + var b Bundle + if err := json.Unmarshal(data, &b); err != nil { + return nil, nil, fmt.Errorf("importer: invalid UniGetUI bundle JSON: %w", err) + } + + // Decode export_version out of band: a number (or quoted number) is honoured, + // anything else degrades to a warning and an unknown version. + version, recognized, coerceWarn := coerceExportVersion(probe["export_version"]) + b.ExportVersion = version + + var warnings []string + if coerceWarn != "" { + warnings = append(warnings, coerceWarn) + } + // Only flag a version mismatch when we actually recognized a version — an + // unrecognized encoding already carries its own warning. + if recognized && version != ExpectedExportVersion { + warnings = append(warnings, fmt.Sprintf( + "bundle export_version is %s; this build was written for version %d — proceeding, but the format may have changed", + FormatExportVersion(version), ExpectedExportVersion)) + } + + return &b, warnings, nil +} + +// coerceExportVersion tolerantly interprets the raw export_version value. A JSON +// number (integer or C# double) or a quoted number (e.g. "3") is honoured and +// reported as recognized. An absent key is treated as a recognized version 0 so +// the caller's mismatch warning fires (mirroring UniGetUI backups that omit it). +// Any other encoding (a non-numeric string, an object, an array, a bool) yields +// an unknown version and a warning — never a parse failure. +func coerceExportVersion(raw json.RawMessage) (value float64, recognized bool, warning string) { + if raw == nil { + return 0, true, "" + } + var f float64 + if err := json.Unmarshal(raw, &f); err == nil { + return f, true, "" + } + var s string + if err := json.Unmarshal(raw, &s); err == nil { + if parsed, perr := strconv.ParseFloat(strings.TrimSpace(s), 64); perr == nil { + return parsed, true, "" + } + } + return 0, false, "unrecognized export_version — treating the bundle as an unknown version and proceeding" +} + +// FormatExportVersion renders the export_version number without a trailing ".0" +// so a message reads "4" rather than "4.000000". +func FormatExportVersion(v float64) string { + return strconv.FormatFloat(v, 'g', -1, 64) +} diff --git a/go-engine/internal/importer/ubundle_test.go b/go-engine/internal/importer/ubundle_test.go new file mode 100644 index 0000000..47e5e13 --- /dev/null +++ b/go-engine/internal/importer/ubundle_test.go @@ -0,0 +1,218 @@ +// Copyright 2025 Substrate Systems OÜ +// SPDX-License-Identifier: Apache-2.0 + +package importer + +import ( + "errors" + "strings" + "testing" +) + +// A valid export_version 3 bundle parses every package with all mapped fields. +func TestParseUniGetUI_ValidV3(t *testing.T) { + src := `{ + "export_version": 3, + "packages": [ + {"Id": "Microsoft.VisualStudioCode", "Name": "Microsoft Visual Studio Code", "Version": "1.85.1", "Source": "winget", "ManagerName": "WinGet", + "InstallationOptions": {"Version": "1.85.0"}}, + {"Id": "Git.Git", "Name": "Git", "Version": "2.43.0", "Source": "winget", "ManagerName": "WinGet"} + ], + "incompatible_packages": [ + {"Id": "Contoso.Local", "Name": "Local App", "Version": "1.0.0", "Source": "Local PC"} + ] + }` + + b, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("ParseUniGetUI returned error: %v", err) + } + if len(warnings) != 0 { + t.Errorf("expected no warnings for v3, got %v", warnings) + } + if b.ExportVersion != 3 { + t.Errorf("export_version = %v, want 3", b.ExportVersion) + } + if len(b.Packages) != 2 { + t.Fatalf("packages = %d, want 2", len(b.Packages)) + } + p0 := b.Packages[0] + if p0.ID != "Microsoft.VisualStudioCode" || p0.Name != "Microsoft Visual Studio Code" || + p0.Version != "1.85.1" || p0.Source != "winget" || p0.ManagerName != "WinGet" { + t.Errorf("package[0] fields not parsed as expected: %+v", p0) + } + if p0.InstallationOptions.Version != "1.85.0" { + t.Errorf("InstallationOptions.Version = %q, want 1.85.0", p0.InstallationOptions.Version) + } + if len(b.IncompatiblePackages) != 1 || b.IncompatiblePackages[0].ID != "Contoso.Local" { + t.Errorf("incompatible_packages not parsed: %+v", b.IncompatiblePackages) + } +} + +// A future export_version parses but returns a warning (forward compatibility). +func TestParseUniGetUI_FutureVersionWarns(t *testing.T) { + src := `{"export_version": 4, "packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + + b, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("expected forward-compatible parse, got error: %v", err) + } + if len(warnings) == 0 { + t.Fatal("expected a version warning for export_version 4, got none") + } + if !strings.Contains(warnings[0], "4") { + t.Errorf("warning should mention the version, got %q", warnings[0]) + } + if len(b.Packages) != 1 { + t.Errorf("packages should still parse under a future version, got %d", len(b.Packages)) + } +} + +// export_version encoded as a decimal (C# double) still compares equal to 3. +func TestParseUniGetUI_DecimalVersionNoWarn(t *testing.T) { + src := `{"export_version": 3.0, "packages": []}` + _, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("ParseUniGetUI returned error: %v", err) + } + if len(warnings) != 0 { + t.Errorf("3.0 should equal 3 with no warning, got %v", warnings) + } +} + +// A missing export_version (decodes to 0) is treated as a version mismatch and warns. +func TestParseUniGetUI_MissingVersionWarns(t *testing.T) { + src := `{"packages": []}` + _, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("ParseUniGetUI returned error: %v", err) + } + if len(warnings) == 0 { + t.Error("expected a warning when export_version is absent") + } +} + +// Malformed JSON is a hard error. +func TestParseUniGetUI_MalformedJSON(t *testing.T) { + _, _, err := ParseUniGetUI(strings.NewReader(`{not valid json`)) + if err == nil { + t.Fatal("expected an error for malformed JSON, got nil") + } +} + +// A quoted export_version ("3") coerces to the number 3 — no version warning. +func TestParseUniGetUI_StringVersionCoerces(t *testing.T) { + src := `{"export_version": "3", "packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + b, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("a quoted numeric export_version must parse, got error: %v", err) + } + if b.ExportVersion != 3 { + t.Errorf("export_version = %v, want 3 (coerced from \"3\")", b.ExportVersion) + } + if len(warnings) != 0 { + t.Errorf("\"3\" equals 3 with no warning, got %v", warnings) + } + if len(b.Packages) != 1 { + t.Errorf("packages should still parse, got %d", len(b.Packages)) + } +} + +// A non-numeric export_version ("abc") degrades to a warning, not a parse error. +func TestParseUniGetUI_GarbageVersionWarns(t *testing.T) { + src := `{"export_version": "abc", "packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + b, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("an unrecognized export_version must degrade to a warning, got error: %v", err) + } + if len(warnings) == 0 { + t.Fatal("expected a warning for an unrecognized export_version, got none") + } + if !strings.Contains(warnings[0], "unrecognized export_version") { + t.Errorf("warning should name the unrecognized version, got %q", warnings[0]) + } + if len(b.Packages) != 1 { + t.Errorf("packages should still parse under an unknown version, got %d", len(b.Packages)) + } +} + +// A well-formed JSON file that is not a bundle (no export_version and no +// packages) is rejected with ErrNotUniGetUIBundle. +func TestParseUniGetUI_WrongFileShapeRejected(t *testing.T) { + // A package.json-shaped object: valid JSON, but not a UniGetUI bundle. + src := `{"name": "my-app", "version": "1.0.0", "dependencies": {"left-pad": "^1.3.0"}}` + _, _, err := ParseUniGetUI(strings.NewReader(src)) + if err == nil { + t.Fatal("expected a non-bundle JSON file to be rejected, got nil") + } + if !errors.Is(err, ErrNotUniGetUIBundle) { + t.Errorf("expected ErrNotUniGetUIBundle, got %v", err) + } +} + +// A legitimately empty but versioned bundle imports as empty (no error). +func TestParseUniGetUI_EmptyVersionedBundleOK(t *testing.T) { + src := `{"export_version": 3, "packages": []}` + b, warnings, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("an empty versioned bundle must parse, got error: %v", err) + } + if len(warnings) != 0 { + t.Errorf("expected no warnings for an empty v3 bundle, got %v", warnings) + } + if len(b.Packages) != 0 { + t.Errorf("expected zero packages, got %d", len(b.Packages)) + } +} + +// A bundle carrying only packages (no export_version key) is still a bundle — the +// shape guard requires just one of the two signature keys. +func TestParseUniGetUI_PackagesOnlyIsBundle(t *testing.T) { + src := `{"packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + _, _, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("a packages-only payload is a bundle, got error: %v", err) + } +} + +// Unknown fields (e.g. incompatible_packages_info, per-package Updates) are +// tolerated and ignored — the known fields still parse. +func TestParseUniGetUI_ToleratesUnknownFields(t *testing.T) { + src := `{ + "export_version": 3, + "incompatible_packages_info": "some message", + "future_top_level_field": {"nested": true}, + "packages": [ + {"Id": "Git.Git", "Name": "Git", "Source": "winget", "ManagerName": "WinGet", + "Updates": {"UpdatesIgnored": true}, "SomeFutureField": 123} + ] + }` + b, _, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("unknown fields should be ignored, got error: %v", err) + } + if len(b.Packages) != 1 || b.Packages[0].ID != "Git.Git" { + t.Errorf("known fields should still parse alongside unknown ones: %+v", b.Packages) + } +} + +// A missing InstallationOptions object leaves the zero value (empty Version). +func TestParseUniGetUI_MissingInstallOptions(t *testing.T) { + src := `{"export_version": 3, "packages": [{"Id": "Git.Git", "Name": "Git", "Source": "winget"}]}` + b, _, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("ParseUniGetUI returned error: %v", err) + } + if b.Packages[0].InstallationOptions.Version != "" { + t.Errorf("absent InstallationOptions should yield empty Version, got %q", b.Packages[0].InstallationOptions.Version) + } +} + +// A leading UTF-8 BOM is tolerated. +func TestParseUniGetUI_ToleratesBOM(t *testing.T) { + src := "\xEF\xBB\xBF" + `{"export_version": 3, "packages": []}` + _, _, err := ParseUniGetUI(strings.NewReader(src)) + if err != nil { + t.Fatalf("leading BOM should be tolerated, got error: %v", err) + } +} diff --git a/openspec/changes/unigetui-import/.openspec.yaml b/openspec/changes/unigetui-import/.openspec.yaml new file mode 100644 index 0000000..eb5fa80 --- /dev/null +++ b/openspec/changes/unigetui-import/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-10 diff --git a/openspec/changes/unigetui-import/design.md b/openspec/changes/unigetui-import/design.md new file mode 100644 index 0000000..07735be --- /dev/null +++ b/openspec/changes/unigetui-import/design.md @@ -0,0 +1,42 @@ +# Design: unigetui-import + +## Context + +UniGetUI serializes backups and bundles with the same C# models (`SerializableBundle`: `export_version` (currently 3), `packages[]`, `incompatible_packages_info`, `incompatible_packages[]`; `SerializablePackage`: `Id`, `Name`, `Version`, `Source`, `ManagerName`, optional `InstallationOptions` incl. a `Version` pin, optional `Updates`). Files are JSON with a `.ubundle` extension; local periodic backups are named ` installed packages .ubundle`; the v3.3.0 cloud backup syncs the same JSON to a private GitHub Gist. Real backups show `winget` as the dominant source with chocolatey/npm/pip/PSGallery entries alongside, and Microsoft Store / "Local PC" apps relegated to `incompatible_packages`. + +Endstate has no import/convert command today; manifests are JSONC loaded via `manifest.LoadManifest`. The engine's app entry shape (`App{ID, Refs, Version, DisplayName}`) maps cleanly onto winget-source packages. + +## Goals / Non-Goals + +**Goals:** +- One command turns a `.ubundle` into a valid, human-readable JSONC manifest a user can immediately `plan`/`apply`, with module-catalog matching then lighting up config restore for known apps. +- Total transparency about fidelity: every package that doesn't map is listed with a reason. +- Zero side effects: import never installs, never touches the network, and is deterministic. + +**Non-Goals (v0):** +- No import of UniGetUI's own settings; no chocolatey/scoop install lanes (winget is Endstate's Windows driver); no `.ubundle` export; no direct Gist download (users export/download the file themselves); no YAML/XML bundle variants (UniGetUI's canonical interchange is JSON). + +## Decisions + +1. **Command shape `import --from unigetui --path [--out ] [--pin] [--json]`** — a generic `import` verb with a `--from` source keeps the door open for `--from dsc` later under the same `manifest-import` capability. *Alternative:* `import-unigetui` dedicated verb — rejected; verb proliferation in a hand-rolled dispatcher. +2. **Strict-but-forward-compatible parsing.** `export_version` other than 3 produces a warning, not a failure, provided required fields parse; unknown JSON fields are ignored (Go default). Rationale: UniGetUI bumps the version on additive changes; hard-failing would break on their next release. +3. **Mapping policy.** `Source == "winget"` (case-insensitive; `ManagerName` as fallback discriminator) → app entry: slugged `ID` (lowercased last segment of the winget Id, deduplicated), `refs.windows = Id`, `displayName = Name`. Version pinning is opt-in via `--pin` (bundle `Version` is "installed at backup time", which self-updating apps churn); a per-package `InstallationOptions.Version` pin is authored intent and therefore wins over the observed `Version` when `--pin` is set. *Alternative:* always pin — rejected; contradicts the version-drift-noise posture established for `capture` pinning. +4. **Skip transparency as a first-class output.** The command result (text and `--json` envelope) contains `imported[]`, `skipped[]` (each with manager + reason), and `incompatible[]` passed through from the bundle. No silent drops — this mirrors the "explicit about what is NOT moved" positioning rule from the market brief. +5. **Output is JSONC with a generated header comment** naming the source file and tool version, written to `--out` (default `manifests/local/imported-unigetui.jsonc` — gitignored per repo conventions). Output loads via `manifest.LoadManifest` round-trip as a validity gate before writing. +6. **Package layout**: parser and mapper in `internal/importer` (pure, hermetic, fixture-tested); command glue in `internal/commands/import.go`; dispatcher entry in `cmd/endstate/main.go` (protected area — explicit instruction 2026-07-10). + +## Risks / Trade-offs + +- [UniGetUI schema drift (v4+)] → warn-don't-fail on version mismatch; parser tolerates unknown fields; fixture from a real-world gist backup locks current behavior. +- [Winget IDs in bundle not present in winget catalog anymore] → import doesn't resolve against winget (pure transform); `plan`/`apply` surface unavailable packages through existing paths. +- [Slug collisions (two packages → same app id)] → deterministic de-dup suffixing; collision listed in the summary. +- [Users expect settings to import too] → the summary explicitly states that config restore comes from Endstate's module catalog matching, not from the bundle — managing exactly the expectation the UniGetUI thread will create. + +## Migration Plan + +Purely additive; rollback = remove the dispatcher entry. No schema or contract changes. + +## Open Questions + +- Whether `--from dsc` lands in the same release train (separate change; this design only reserves the flag shape). +- Whether to accept `unigetui://` deep-link payloads or Gist URLs later — deferred until the interop conversation with the UniGetUI project happens. diff --git a/openspec/changes/unigetui-import/proposal.md b/openspec/changes/unigetui-import/proposal.md new file mode 100644 index 0000000..d960df6 --- /dev/null +++ b/openspec/changes/unigetui-import/proposal.md @@ -0,0 +1,32 @@ +# Proposal: unigetui-import + +## Why + +UniGetUI (~25k stars, now under the Devolutions org) is the most popular package-manager UI on Windows, and its backup feature stores only the package list — by its own docs, "not your personal data or settings." Its community explicitly asks for the missing layer (Discussion #3443: app-config transfer was "95% of the effort" of a migration; maintainer: WinGet-Config support won't come "in the near future"; issues #2377, #3890 ask for settings backup/sync). Importing a UniGetUI backup as an Endstate manifest gives those users a one-step bridge into capture/restore/verify — and is the concrete substance behind the planned public interop proposal to the UniGetUI project (2026-07 market brief: borrow adjacency to an existing audience; the interop thread is the acquisition mechanism). + +## What Changes + +- New CLI command: `endstate import --from unigetui --path ` consuming a UniGetUI bundle/backup (`.ubundle`, JSON `export_version: 3`) and emitting a JSONC manifest. +- Mapping: winget-source packages become manifest apps (`Id` → `refs.windows`, `Name` → `displayName`); with `--pin`, the bundle's `Version` (or a per-package `InstallationOptions.Version` pin, which always wins) is written to the app's `version` field. +- Every non-imported package is reported with a reason — non-winget managers (chocolatey, npm, pip, PSGallery, scoop) and `incompatible_packages` are listed, never silently dropped. +- Import is a pure transform: no network calls, no installs, deterministic output. +- New dispatcher entry in `cmd/endstate/main.go` (protected area — explicit user instruction for this change was given 2026-07-10). + +## Capabilities + +### New Capabilities + +- `manifest-import`: importing external package-list formats into Endstate manifests — source-format parsing, mapping fidelity, skip transparency, and output-manifest validity. UniGetUI `.ubundle` is the first supported source; a future DSC-configuration import extends this same capability. + +### Modified Capabilities + + + +## Impact + +- **New code**: `go-engine/internal/importer/` (ubundle parser + winget mapper), `go-engine/internal/commands/import.go`. +- **Reused code**: manifest types and JSONC conventions (`internal/manifest`); envelope construction for `--json`. +- **Touched (protected)**: `go-engine/cmd/endstate/main.go` dispatcher. +- **Dependencies**: none added — `.ubundle` is plain JSON. +- **Contracts**: `--json` output follows the existing envelope; no event-schema changes. +- **Non-goals (v0)**: no UniGetUI settings import, no chocolatey/scoop driver support (report-only), no reverse export to `.ubundle`, no cloud-backup (Gist) fetching — file input only. diff --git a/openspec/changes/unigetui-import/specs/manifest-import/spec.md b/openspec/changes/unigetui-import/specs/manifest-import/spec.md new file mode 100644 index 0000000..4c60fed --- /dev/null +++ b/openspec/changes/unigetui-import/specs/manifest-import/spec.md @@ -0,0 +1,56 @@ +# manifest-import Specification (delta) + +## ADDED Requirements + +### Requirement: UniGetUI bundle parsing + +The engine SHALL parse UniGetUI bundle/backup JSON (`SerializableBundle`, `export_version` 3) including `packages[]` fields `Id`, `Name`, `Version`, `Source`, `ManagerName`, optional `InstallationOptions.Version`, and `incompatible_packages[]`. An `export_version` other than 3 SHALL produce a warning, not a failure, when required fields parse. + +#### Scenario: Valid v3 bundle parses +- **WHEN** `import --from unigetui --path backup.ubundle` runs on a valid `export_version: 3` file +- **THEN** all packages SHALL be parsed with their Id, Name, Version, Source, and ManagerName + +#### Scenario: Future version warns but proceeds +- **WHEN** the bundle declares `export_version: 4` and required fields parse +- **THEN** the command SHALL emit a version warning and continue + +### Requirement: Winget package mapping + +Packages with the winget source SHALL map to manifest app entries: winget `Id` to `refs.windows`, `Name` to `displayName`, and a deterministic slug as the app `id`. When `--pin` is passed, the app `version` SHALL be set from the package's `InstallationOptions.Version` pin when present, otherwise from the bundle's recorded `Version`. Without `--pin`, no version SHALL be written. + +#### Scenario: Winget package becomes an app entry +- **WHEN** the bundle contains `{"Id": "Microsoft.VisualStudioCode", "Name": "Microsoft Visual Studio Code", "Source": "winget"}` +- **THEN** the manifest SHALL contain an app with `refs.windows` = `Microsoft.VisualStudioCode` and `displayName` = `Microsoft Visual Studio Code` + +#### Scenario: Authored pin beats observed version +- **WHEN** `--pin` is passed and a package has `Version: "1.2.3"` and `InstallationOptions.Version: "1.2.0"` +- **THEN** the app entry's `version` SHALL be `1.2.0` + +#### Scenario: No pin by default +- **WHEN** `--pin` is not passed +- **THEN** no app entry SHALL contain a `version` field + +### Requirement: Skip transparency + +Every package that is not imported SHALL be reported with its manager and a reason; `incompatible_packages` SHALL be passed through to the report. The command SHALL NOT silently drop any bundle entry. + +#### Scenario: Non-winget manager is reported +- **WHEN** the bundle contains a chocolatey-sourced package +- **THEN** the import summary SHALL list it as skipped with reason indicating the unsupported manager + +#### Scenario: Incompatible packages surface +- **WHEN** the bundle contains `incompatible_packages` +- **THEN** the import summary SHALL list each of them + +### Requirement: Output manifest validity and purity + +The emitted manifest SHALL be JSONC that loads successfully through the engine's manifest loader before being written, and import SHALL be a pure transform: no network access, no package operations, and byte-identical output for identical input. + +#### Scenario: Round-trip validity gate +- **WHEN** import generates a manifest +- **THEN** the manifest SHALL load via the manifest loader before the file is written +- **AND** a load failure SHALL abort the write with an error + +#### Scenario: Deterministic output +- **WHEN** import runs twice on the same bundle with the same flags +- **THEN** the two outputs SHALL be byte-identical diff --git a/openspec/changes/unigetui-import/tasks.md b/openspec/changes/unigetui-import/tasks.md new file mode 100644 index 0000000..8b968ae --- /dev/null +++ b/openspec/changes/unigetui-import/tasks.md @@ -0,0 +1,29 @@ +# Tasks: unigetui-import + +## 1. Parser (internal/importer) + +- [ ] 1.1 Create `internal/importer` package with `.ubundle` types (`Bundle`, `Package`, `InstallOptions`, `IncompatiblePackage`) matching UniGetUI's SerializableBundle v3 JSON +- [ ] 1.2 Implement `ParseUniGetUI(r io.Reader)` — strict on required fields, tolerant of unknown fields, warning (not error) on `export_version != 3` +- [ ] 1.3 Add a real-world fixture `.ubundle` (anonymized, multi-manager: winget + chocolatey + pip + incompatible_packages) under `tests/fixtures/` +- [ ] 1.4 Parser unit tests: valid v3, future version warning, malformed JSON error, missing-fields behavior + +## 2. Mapper + +- [ ] 2.1 Implement winget-package → manifest-app mapping (Id → refs.windows, Name → displayName, deterministic slug for app id with collision de-dup) +- [ ] 2.2 Implement `--pin` policy: InstallationOptions.Version wins over observed Version; no version field without `--pin` +- [ ] 2.3 Implement skip report: non-winget managers with reasons, incompatible_packages pass-through, slug collisions noted +- [ ] 2.4 Mapper unit tests: mapping table, pin precedence, no-pin default, skip transparency (no silent drops), deterministic output byte-equality + +## 3. Command + wiring + +- [ ] 3.1 Implement `commands.RunImport(Flags)`: `--from unigetui`, `--path`, `--out` (default `manifests/local/imported-unigetui.jsonc`), `--pin`, `--json` envelope with imported/skipped/incompatible counts +- [ ] 3.2 JSONC emission with generated header comment (source file, tool version) and manifest-loader round-trip validity gate before write +- [ ] 3.3 Register `import` command in `cmd/endstate/main.go` dispatcher (protected area — explicit instruction 2026-07-10) +- [ ] 3.4 Command tests: end-to-end fixture → manifest loads via `manifest.LoadManifest`; abort-on-invalid; summary contents + +## 4. Verification + docs + +- [ ] 4.1 Run `cd go-engine && go test ./internal/importer/... ./internal/commands/...` and full `go test ./...` +- [ ] 4.2 Run `npm run openspec:validate` +- [ ] 4.3 Manual check: import a real UniGetUI backup, `endstate plan` against the result, confirm module catalog matching lights up config modules for known apps +- [ ] 4.4 README: short "Import from UniGetUI" section (feeds the interop Discussion post) diff --git a/readme.md b/readme.md index 2f41309..9a21a55 100644 --- a/readme.md +++ b/readme.md @@ -169,6 +169,22 @@ endstate rebuild --from MyProfile.zip --confirm Overwritten files are backed up first and can be undone with `endstate revert`. Use `--no-restore` to install and verify without touching configuration. +### Import from UniGetUI + +Already tracking your apps in [UniGetUI](https://github.com/marticliment/UniGetUI)? Export a bundle (or grab a local backup `.ubundle`) and turn it into an Endstate manifest: + +```bash +# Convert a UniGetUI backup into a manifest (pure transform: no installs, no network) +endstate import --from unigetui --path "MY-PC installed packages.ubundle" + +# Record versions too (a package's pinned version wins over the observed one) +endstate import --from unigetui --path backup.ubundle --pin +``` + +The default output is `manifests/local/imported-unigetui.jsonc` (gitignored); outside a repo checkout it lands next to the input bundle instead. Use `--out` to choose another path. From there, `endstate plan`/`apply` installs the apps and Endstate's module catalog lights up config restore for the apps it recognises. + +**Scope — the honest version:** import moves the **package list only**. UniGetUI's own settings are *not* imported (UniGetUI's backup doesn't include them either). Winget-source packages become manifest apps; everything else is reported, never silently dropped — non-winget managers (chocolatey, scoop, pip, ...) and UniGetUI's own incompatible entries are listed with a reason. Config restore comes from Endstate's module catalog after import, not from the bundle. + ### CLI Commands | Command | Description | @@ -178,6 +194,7 @@ Overwritten files are backed up first and can be undone with `endstate revert`. | `plan` | Generate execution plan from manifest without applying | | `apply` | Execute the plan (with optional `-DryRun`) | | `rebuild` | Rebuild a machine from a bundle/manifest in one step (install + restore + verify; live run needs `--confirm`) | +| `import` | Import an external package list into a manifest (`--from unigetui`; winget packages in, non-winget reported) | | `restore` | Restore configuration files from manifest (requires `-EnableRestore`) | | `export-config` | Export config files from system to export folder (inverse of restore) | | `validate-export` | Validate export integrity before restore |