Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
},
"metadata": {
"description": "The trust layer between AI and your data. A governed semantic model for AI-to-database access — every join approved, every metric signed off, every answer with a receipt. Local-first: no infra, no servers, no data leaves your machine.",
"version": "0.4.5",
"version": "0.5.0",
"pluginRoot": "./plugins",
"tags": ["data", "sql", "governance", "trust-layer", "semantic-model", "local-first", "fair-code"],
"repository": "https://github.com/AgamiAI/agami-core",
Expand All @@ -18,7 +18,7 @@
"name": "agami-core",
"source": "./plugins/agami",
"description": "A governed trust layer between AI and your database: every join FK-derived or human-approved, every metric signed off, every answer with a receipt (the SQL, the model version, who vouched for each definition). Runs entirely on your machine via Claude's Bash / Read / Write tools — no MCP server or Python required if you have psql/mysql or DuckDB (an optional local MCP server is available for use from Claude Desktop).",
"version": "0.4.5",
"version": "0.5.0",
"keywords": ["database", "governance", "trust-layer", "semantic-model", "sql", "postgres", "snowflake"],
"category": "data"
}
Expand Down
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,18 @@ is the source of truth a host installs against — bumping it is what invalidate
user's plugin cache (see [CONTRIBUTING.md](CONTRIBUTING.md)). Each released section
below corresponds to one such version.

## [0.5.0] — 2026-07-25

### Changed

- **Renamed the per-datasource model files** so their names say what they are, now that a
company-wide `organization.yaml` exists at the artifacts root. In each profile,
`org.yaml` → `datasource.yaml` and `ORGANIZATION.md` → `datasource.md`. The model root's
display-name field `organization:` is likewise `datasource:`, and the served per-datasource
memory row uses `kind='datasource'`. The company record (`organization.yaml`, `org_id`,
tenancy) is unchanged. No on-disk migration is provided — re-run `agami-connect` (or rename
the two files by hand) for any pre-existing profile.

## [0.4.5] — 2026-07-15

Hosted/self-hosted server hardening: a real-wheel packaging fix, a multi-tenancy seam, and an
Expand Down Expand Up @@ -293,7 +305,7 @@ result size, and concurrent load grow. Behaviour-preserving unless a note says o
- **Self-host deploy no longer crash-loops on artifact permissions.** The team
server runs as a non-root container user; the deploy now stages the model
**world-readable**, so the boot-time model load can't fail `Permission denied`
on `ORGANIZATION.md` under a mismatched host owner.
on `datasource.md` under a mismatched host owner.
- **claude.ai connects to a self-hosted server.** The `/mcp` endpoint no longer
answers the bare (no-trailing-slash) URL with a `307` redirect that the MCP
client won't follow — the server normalizes it internally, so `{base}/mcp`
Expand Down
18 changes: 9 additions & 9 deletions dev/render_previews.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,11 +124,11 @@ def write(name: str, html: str) -> None:
write("07-admin-activity.html", admin.activity_tab_html(model_store.list_sessions(_s), **CHROME))

# The read-only model explorer — rendered from the REAL builders over the SAME served tree the MCP
# tools read (load_organization), so the previews can't drift from production. Neutral demo data only.
from semantic_model.models import Organization # noqa: E402
# tools read (load_datasource), so the previews can't drift from production. Neutral demo data only.
from semantic_model.models import Datasource # noqa: E402

_MODEL_ORG = {
"organization": "acme",
_MODEL_DS = {
"datasource": "acme",
"version": 1,
"description": "Acme Commerce — the deployed semantic model.",
"fiscal_year_start_month": 1,
Expand Down Expand Up @@ -248,19 +248,19 @@ def write(name: str, html: str) -> None:
"from_subject_area": "Sales", "to_subject_area": "Catalog"},
],
}
_ORG_MD = (
_DS_MD = (
"# About Acme Commerce\n\n"
"Acme sells direct-to-consumer goods online. Every checkout writes an **orders** row.\n\n"
"## Key terms\n\n"
"- **Net revenue** excludes cancelled orders and nets refunds.\n"
"- A **member** has a row in the loyalty table — not just anyone who checked out.\n"
)
model_store.write_organization(_s, "SALES_DATA", Organization.model_validate(_MODEL_ORG))
model_store.write_organization(_s, "MARKETING", Organization.model_validate(_MODEL_ORG)) # → picker
model_store.write_memory(_s, "SALES_DATA", organization=_ORG_MD)
model_store.write_datasource(_s, "SALES_DATA", Datasource.model_validate(_MODEL_DS))
model_store.write_datasource(_s, "MARKETING", Datasource.model_validate(_MODEL_DS)) # → picker
model_store.write_memory(_s, "SALES_DATA", datasource_doc=_DS_MD)
model_store.write_model_version(_s, "SALES_DATA", "a1f4c39c0b2e", created_at="2026-06-27T09:00:00Z")
_dss = model_store.list_datasources(_s)
_org = model_store.load_organization(_s, "SALES_DATA")
_org = model_store.load_datasource(_s, "SALES_DATA")
_ver = model_store.newest_model_version(_s, "SALES_DATA")
_catalog = next(a for a in _org.subject_areas if a.name == "Catalog")
_products = next(t for t in _catalog.tables_defined if t.name == "products")
Expand Down
16 changes: 8 additions & 8 deletions dev/reset-yamls.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# hand-written docs, or the git audit trail.
#
# What gets nuked (default):
# <artifacts_dir>/<profile>/org.yaml (model root)
# <artifacts_dir>/<profile>/datasource.yaml (model root)
# <artifacts_dir>/<profile>/datasources/ (storage connections)
# <artifacts_dir>/<profile>/subject_areas/ (tables/entities/metrics/relationships)
# <artifacts_dir>/<profile>/prompt_examples/ (NL→SQL example library)
Expand All @@ -16,7 +16,7 @@
# <artifacts_dir>/local/credentials DB connection details
# <artifacts_dir>/local/.config reviewer email, role, etc.
# <artifacts_dir>/USER_MEMORY.md cross-DB preferences
# <artifacts_dir>/<profile>/ORGANIZATION.md domain context the user wrote
# <artifacts_dir>/<profile>/datasource.md domain context the user wrote
# <artifacts_dir>/<profile>/.git/ audit trail of past curator edits
# <artifacts_dir>/<profile>/.legacy_backup/ legacy (v1) model backed up on upgrade
# <artifacts_dir>/<profile>/curation_log.jsonl
Expand All @@ -29,7 +29,7 @@
# Usage:
# dev/reset-yamls.sh # default profile, soft reset
# dev/reset-yamls.sh acme # specific profile, soft reset
# dev/reset-yamls.sh acme --hard # also drop ORGANIZATION.md + .git/ + logs
# dev/reset-yamls.sh acme --hard # also drop datasource.md + .git/ + logs
# dev/reset-yamls.sh acme --clean-renders # also drop <artifacts_dir>/local/<kind>/acme/* (this profile only)
# dev/reset-yamls.sh acme --clean-renders-all # wipe legacy flat-layout files + every profile
# dev/reset-yamls.sh acme --hard --clean-renders # full per-profile wipe
Expand Down Expand Up @@ -94,7 +94,7 @@ action() {
}

echo "Resetting YAMLs in $PROFILE_DIR (profile: $PROFILE)"
echo " mode: $([[ $HARD -eq 1 ]] && echo 'hard (drops ORGANIZATION.md, .git/, logs)' || echo 'soft (keeps ORGANIZATION.md, .git/, logs)')"
echo " mode: $([[ $HARD -eq 1 ]] && echo 'hard (drops datasource.md, .git/, logs)' || echo 'soft (keeps datasource.md, .git/, logs)')"
if [[ $CLEAN_RENDERS_ALL -eq 1 ]]; then
echo " also cleaning <artifacts_dir>/local/{charts,review,examples-validation,model,exports}/ (ALL profiles + legacy)"
elif [[ $CLEAN_RENDERS -eq 1 ]]; then
Expand All @@ -109,7 +109,7 @@ if [[ -d "$PROFILE_DIR/.snapshots" ]]; then
action "chmod -R u+w '$PROFILE_DIR/.snapshots'"
fi

# 1. Top-level YAML files (org.yaml, etc.)
# 1. Top-level YAML files (datasource.yaml, etc.)
while IFS= read -r f; do
[[ -z "$f" ]] && continue
action "rm -f '$f'"
Expand All @@ -132,9 +132,9 @@ if [[ -d "$PROFILE_DIR/.snapshots" ]]; then
action "rm -rf '$PROFILE_DIR/.snapshots'"
fi

# 4. Hard reset extras: ORGANIZATION.md, the audit trail (.git/), and logs.
# 4. Hard reset extras: datasource.md, the audit trail (.git/), and logs.
if [[ $HARD -eq 1 ]]; then
action "rm -f '$PROFILE_DIR/ORGANIZATION.md'"
action "rm -f '$PROFILE_DIR/datasource.md'"
action "rm -rf '$PROFILE_DIR/.git'"
action "rm -f '$PROFILE_DIR/curation_log.jsonl'"
action "rm -f '$PROFILE_DIR/corrections.jsonl'"
Expand Down Expand Up @@ -186,7 +186,7 @@ fi
if [[ $HARD -eq 0 ]]; then
echo ""
echo "Preserved in soft mode (would be dropped on --hard):"
[[ -f "$PROFILE_DIR/ORGANIZATION.md" ]] && echo " $PROFILE_DIR/ORGANIZATION.md"
[[ -f "$PROFILE_DIR/datasource.md" ]] && echo " $PROFILE_DIR/datasource.md"
[[ -d "$PROFILE_DIR/.git" ]] && echo " $PROFILE_DIR/.git (audit trail)"
[[ -f "$PROFILE_DIR/curation_log.jsonl" ]] && echo " $PROFILE_DIR/curation_log.jsonl"
[[ -f "$PROFILE_DIR/corrections.jsonl" ]] && echo " $PROFILE_DIR/corrections.jsonl"
Expand Down
14 changes: 7 additions & 7 deletions docs/format-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,19 +23,19 @@ agami's state splits across two directories. See [plugins/agami/shared/file-layo
| File | Format | Owner |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `<artifacts_dir>/USER_MEMORY.md` | Free-form Markdown — cross-database preferences | Seeded by `agami-connect` Phase 0a, edited by user or appended by `agami-save-correction` |
| `<artifacts_dir>/<profile>/org.yaml` | Org description + storage-connection refs + subject-area refs + cross_subject_area_relationships | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/datasource.yaml` | Org description + storage-connection refs + subject-area refs + cross_subject_area_relationships | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/datasources/<conn>/storage.yaml` | Storage connection (physical): storage_type + storage_config (env-var refs, never secrets) | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/subject_areas/<area>/subject_area.yaml` | Subject area: description, default_time_window, TableRefs (with optional expose_column_groups) | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/subject_areas/<area>/tables/<table>.yaml` | Canonical Table: columns, grain, default_filters, caveats, column_groups, performance_hints | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/subject_areas/<area>/{entities,metrics}/<name>.yaml`, `relationships.yaml` | Entities, metrics, and the intra-area FK graph (join cardinality + trust block) | Skill-written, user-editable |
| `<artifacts_dir>/<profile>/prompt_examples/<area>/examples.yaml` | NL→SQL few-shot library (scope-tagged) | Skill-written (seeds) + append-only via `agami-save-correction` |
| `<artifacts_dir>/<profile>/ORGANIZATION.md` | Free-form Markdown — per-database domain context | Seeded by `agami-connect`, edited by user or appended by `agami-save-correction` |
| `<artifacts_dir>/<profile>/datasource.md` | Free-form Markdown — per-database domain context | Seeded by `agami-connect`, edited by user or appended by `agami-save-correction` |
| `<artifacts_dir>/local/cross_profile_relationships.yaml` | Agami-bespoke YAML — declared JOIN paths across profiles for federation. **Lives in `<artifacts_dir>/local/` because it spans profiles** and isn't tied to one team's repo | User-edited (optional) |


`USER_MEMORY.md` is **distinct** from Claude Code's auto-memory at `~/.claude/projects/<workspace>/memory/MEMORY.md`. The auto-memory is host-managed and project-scoped; `USER_MEMORY.md` is agami-managed, lives in the artifacts dir, and persists across Claude Code hosts (CLI / VS Code extension / Cursor extension) the same way the rest of `<artifacts_dir>/` does.

`USER_MEMORY.md` covers **user preferences that apply across every database** (default time windows, display preferences, exclude rules). The per-database **`ORGANIZATION.md`** at `<artifacts_dir>/<profile>/ORGANIZATION.md` covers **domain knowledge for that specific database** (terminology, key metrics, what the data represents). See [plugins/agami/shared/organization-context-format.md](../plugins/agami/shared/organization-context-format.md).
`USER_MEMORY.md` covers **user preferences that apply across every database** (default time windows, display preferences, exclude rules). The per-database **`datasource.md`** at `<artifacts_dir>/<profile>/datasource.md` covers **domain knowledge for that specific database** (terminology, key metrics, what the data represents). See [plugins/agami/shared/organization-context-format.md](../plugins/agami/shared/organization-context-format.md).

`<profile>` matches the section name in `<artifacts_dir>/local/credentials` (default: `default`). One *directory* per profile under `<artifacts_dir>/`. The `agami-connect` skill auto-migrates v1.0 (single-file) and v1.1 (under `<artifacts_dir>/local/`) installs on first run after upgrade.

Expand All @@ -44,7 +44,7 @@ agami's state splits across two directories. See [plugins/agami/shared/file-layo
Three concrete wins (full design in [shared/file-layout.md](../plugins/agami/shared/file-layout.md)):

1. **Zero credential-leak risk on commit.** `<artifacts_dir>/local/` is gitignored by default; `<artifacts_dir>/` is the only place anything goes when teams share.
2. **Team workflows just work.** `cd ~/code/myteam/data && git add agami/` commits everyone's tuned semantic model, examples, ORGANIZATION.md, and USER_MEMORY.md preferences.
2. **Team workflows just work.** `cd ~/code/myteam/data && git add agami/` commits everyone's tuned semantic model, examples, datasource.md, and USER_MEMORY.md preferences.
3. **Power users override per-environment.** Set `AGAMI_ARTIFACTS_DIR=/path/to/staging-models` for an experimental session.

---
Expand All @@ -70,7 +70,7 @@ password = mypassword
The model is a **provider-portable, standard-concepts hierarchy** that any LLM can traverse to build reliable SQL against any backend:

```
Organization (org.yaml)
Datasource (datasource.yaml)
├─ Storage Connections[] (physical: host/creds/dialect — datasources/<conn>/storage.yaml)
└─ Subject Areas[] (logical — the primary unit the LLM consumes; cap ~20-30 tables)
├─ tables[] (TableRefs into storage connections; expose_column_groups scopes wide tables)
Expand All @@ -88,8 +88,8 @@ Every write is gated by the validator at [`packages/agami-core/src/semantic_mode
### Worked example — a minimal model on disk

```yaml
# org.yaml
organization: shop
# datasource.yaml
datasource: shop
version: 1
description: E-commerce shop.
storage_connections:
Expand Down
2 changes: 1 addition & 1 deletion docs/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Four tools, mirroring the hosted connector so the client experience is identical
| Tool | What it does |
|---|---|
| `list_datasources` | Enumerate local profiles (credential sections) and whether each has a model. |
| `get_datasource_schema` | Return the semantic model: the subject-area index, full per-table detail for requested `dataset_names`, plus `ORGANIZATION.md` / `USER_MEMORY.md`. |
| `get_datasource_schema` | Return the semantic model: the subject-area index, full per-table detail for requested `dataset_names`, plus `datasource.md` / `USER_MEMORY.md`. |
| `get_prompt_examples` | Return the curated `examples.yaml` few-shot library. |
| `execute_sql` | Run **one read-only** `SELECT` / `WITH...SELECT` locally and return `{columns, rows, row_count, ...}`. DML/DDL/multi-statement are rejected. |

Expand Down
4 changes: 2 additions & 2 deletions docs/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ Every byte agami reads or writes stays on your machine:
- **Credentials** (`<artifacts_dir>/local/credentials`) — chmod 600
- **Auth files** (`<artifacts_dir>/local/.pgpass`, `.mysql.cnf`, `.snowsql.cnf`) — chmod 600, written by `setup_pgauth.py`
- **Config** (`<artifacts_dir>/local/.config`) — `active_profile`, `tool_paths`, `reviewer_email`, `reviewer_role` (the artifacts-dir location lives in the `~/.config/agami/path` pointer)
- **Semantic model** (`org.yaml` + the `subject_areas/<area>/` tree under `<artifacts_dir>/<profile>/`; default `<artifacts_dir>` is `~/agami-artifacts/`)
- **Semantic model** (`datasource.yaml` + the `subject_areas/<area>/` tree under `<artifacts_dir>/<profile>/`; default `<artifacts_dir>` is `~/agami-artifacts/`)
- **Examples library** (`<artifacts_dir>/<profile>/examples.yaml`)
- **Organization context** (`<artifacts_dir>/<profile>/ORGANIZATION.md`) — your description of what the database represents, domain terminology
- **Datasource context** (`<artifacts_dir>/<profile>/datasource.md`) — your description of what the database represents, domain terminology
- **User memory** (`<artifacts_dir>/USER_MEMORY.md`) — your cross-database preferences
- **Query results** (everything the assistant shows you)
- **Query log** (`<artifacts_dir>/local/query_log.jsonl`) — your personal record of every query you ran
Expand Down
2 changes: 1 addition & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ rm -rf ~/.claude/plugins/cache/agami-skills # only if you also installed our e
# 3. Remove your data (only if you're sure you don't want it back)
# Snapshot files are intentionally immutable — chmod first so rm can delete them.
chmod -R u+w ~/agami-artifacts 2>/dev/null
rm -rf ~/agami-artifacts # semantic model, examples, ORGANIZATION.md, USER_MEMORY.md, .snapshots/, .git/
rm -rf ~/agami-artifacts # semantic model, examples, datasource.md, USER_MEMORY.md, .snapshots/, .git/
rm -rf <artifacts_dir>/local # credentials, .config, charts, exports, review + examples-validation dashboards

# 4. Restart Claude Code (full quit, not just close window)
Expand Down
Loading
Loading