Skip to content

Split deployment identity from datasource model: a single root org.yaml · One org.yaml per deployment, not per profile #142

Description

@vishalkalbi27

F14 (ACE-056/057) made org_id a deployment-scoped identity — one id shared by every profile in
an artifacts dir, because a company with several datasources is one tenant. But the id is stored
per profile, so today it is physically duplicated into every org.yaml:

agami-artifacts/
├── analytics/org.yaml org_id: 1d4e301f… ← same value
├── salesforce/org.yaml org_id: 1d4e301f… ← duplicated
└── servicenow/org.yaml org_id: 1d4e301f… ← duplicated

The duplication is a symptom. The real issue is that org.yaml conflates two different scopes:

Scope Keys Truly belongs to
Deployment (one per artifacts dir) org_id the deployment / tenant
Datasource (one per profile) organization, version, description, fiscal_year_start_month, storage_connections, subject_areas, cross_subject_area_relationships that datasource

Consequences today:

  • Drift is representable. Nothing structurally prevents two profiles in one dir holding different
    org_ids (e.g. a profile copied in from another machine). loader.deployment_org_id() has to scan
    and take the first match — deterministic, but arbitrary if they disagree. A single source of truth
    makes the invalid state unrepresentable.
  • Adoption logic exists only to paper over it. build.ensure_org_id() has a "check my siblings and
    adopt their id" branch purely because there is no deployment-level home for the value.
  • Resolution is a directory scan. tools.resolved_org_id() iterates profile dirs to answer a
    deployment-level question.

Proposal

Introduce a deployment-level manifest at the artifacts root, and keep per-datasource models beneath it:

agami-artifacts/
├── org.yaml ← NEW: deployment identity (org_id) + any org-wide config
├── analytics/ ← datasource model (no org_id)
├── salesforce/
└── servicenow/

  • org_id is written once, read once, and cannot drift.
  • ensure_org_id() loses its sibling-adoption branch — a new datasource simply inherits the root manifest.
  • resolved_org_id() becomes a single file read instead of a directory scan.

Open design questions

  1. What else is deployment-level? Candidates that are arguably org-wide rather than per-datasource:
    organization (the company name), fiscal_year_start_month, key_terminology, and the
    cross_subject_area_* collections. Does this issue move only org_id (minimal, safe) or take the
    opportunity to split the whole scope properly (cleaner, larger)?
  2. Naming. A root org.yaml alongside per-profile org.yaml files is ambiguous. Consider
    deployment.yaml / agami.yaml for the root, or rename the per-profile file to datasource.yaml.
  3. Per-profile file: keep or drop the key? Keep org_id mirrored (read-only, derived) for
    debuggability, or remove it entirely so there is exactly one writer?

Backward compatibility / migration

Existing deployments have per-profile org.yaml files carrying org_id. Needs a read path that:

  • prefers the root manifest when present;
  • else falls back to the current per-profile scan (so pre-migration deployments keep working);
  • and a one-time hoist (a sm subcommand, or automatic on next connect) that writes the root manifest
    from the existing id and stops re-writing it per profile.

Load-bearing constraint: the id must remain immutable through the migration. Hoisting must adopt
the existing id, never mint a new one — changing it would orphan every already-stamped DB row and break
idempotent import into hosted.

Code touchpoints

  • semantic_model/build.pywrite_tree() (stops emitting org_id per profile), ensure_org_id()
    (sibling-adoption branch removed)
  • semantic_model/loader.pyload_org_id(), deployment_org_id() (becomes a single root read)
  • semantic_model/models.pyOrganization.org_id, and any keys that move scope
  • semantic_model/cli.pyensure-org-id, plus a hoist/migrate subcommand
  • tools.pyresolved_org_id()
  • model_deploy.py_default_org()
  • plugins/agami/skills/agami-connect/SKILL.md — the sample-copy path

Acceptance criteria

  • A fresh agami-connect writes org_id exactly once, at the artifacts root
  • Connecting additional datasources adds no new org_id anywhere
  • An existing (pre-migration) deployment keeps resolving its same id — no re-mint, no orphaned rows
  • resolved_org_id() reads a single file; no directory scan
  • Deploy-time stamp and serve-time resolution still agree (the F14 invariant)
  • tests/test_privacy_no_network.py stays green
  • E2E covers: fresh deploy, add-a-datasource, and migrate-from-per-profile

Out of scope

  • Multi-tenant isolation (agami-core stays N=1)
  • The hosted import/lift (AH F5)

Context

Follow-up to #141 (F14 — portable org identity). Not a defect in that PR; it ships correct,
deployment-scoped behaviour. This issue removes the duplication that implementation had to work around.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions