Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,10 @@ cython_debug/
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# you could uncomment the following to ignore the entire vscode folder
# .vscode/
.vscode/

# Generic IDE metadata
.idea/

# OS / editor
.DS_Store
Expand Down
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ Why `3.12`:
- Existing local Aim repositories on disk (read-only); run params are read from Aim run metadata attributes under `.aim` (004-run-params-query)
- Python 3.12 for development, runtime support `>=3.10,<3.13` + Python standard library, `numpy>=1.24`, `rich>=13.7`, `plotext>=5.3`, existing Aim SDK usage for owned trace commands; no new runtime dependency planned (005-distribution-trace-visual)
- Existing local Aim repositories on disk, read-only; distribution histogram points are read from Aim sequence data under `.aim` (005-distribution-trace-visual)
- Python 3.12 for development, runtime support `>=3.10,<3.13` + Python standard library, `numpy>=1.24`, `rich>=13.7`, `plotext>=5.3`, `textual-image>=0.12.0`, existing Aim SDK usage for owned query and trace commands; no new dependency planned (007-default-query-runtime)
- Existing local Aim repositories on disk, read-only; query/trace data and run timing metadata are read from `.aim` repositories without modification (007-default-query-runtime)

## Recent Changes
- 001-aim-command-passthrough: Added Python 3.12 for development, runtime support `>=3.10,<3.13` + Python standard library, native Aim CLI (external runtime prerequisite for delegated commands), pytest for test automation
46 changes: 28 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,17 +64,17 @@ When provided, `--repo` accepts either the repository root, such as `data`, or
the metadata directory itself, such as `data/.aim`.

```bash
# Summarize matching metrics
aimx query metrics "metric.name == 'loss'" --repo data
# Summarize metrics from all runs
aimx query metrics --repo data

# Preview matching images in supported terminals
aimx query images "images" --repo data
# Preview images in supported terminals
aimx query images --repo data

# Compare run parameters across matching runs
aimx query params "run.hash != ''" --repo data
# Compare run parameters and durations
aimx query params --repo data

# Plot a metric time series
aimx trace "metric.name == 'loss'" --repo data
# Plot metric time series from all runs
aimx trace --repo data
```

## Features
Expand Down Expand Up @@ -106,8 +106,9 @@ aimx trace "metric.name == 'loss'" --repo data
| `aimx query params` | Compare run-level parameters across matching runs. |
| `aimx trace` | Plot, tabulate, or export metric time series. |

Both `aimx query` and `aimx trace` accept **AimQL** expressions as their filter
argument. AimQL is Aim's native Python-like query language.
Both `aimx query` and `aimx trace` accept optional **AimQL** expressions as
their filter argument. When the expression is omitted or blank, `aimx` uses
`run.hash != ''`. AimQL is Aim's native Python-like query language.

```bash
aimx query metrics "metric.name == 'loss' and run.hparams.learning_rate > 0.001"
Expand All @@ -126,6 +127,9 @@ statistics such as step count, last value, min value, and max value.

```bash
# Rich table, colored in terminals by default
aimx query metrics --repo data

# Filter with AimQL
aimx query metrics "metric.name == 'loss'" --repo data

# Short run hashes are transparently expanded to full hashes
Expand Down Expand Up @@ -156,11 +160,11 @@ images directly in the terminal.

```bash
# Inline preview in modern terminals
aimx query images "images" --repo data
aimx query images --repo data

# Metadata output only
aimx query images "images" --repo data --plain
aimx query images "images" --repo data --json
aimx query images --repo data --plain
aimx query images --repo data --json

# Filter and sample matching image rows
aimx query images "images" --repo data --epochs 10:50 --head 10
Expand Down Expand Up @@ -193,7 +197,7 @@ matching runs.

```bash
# Compare discovered params across all matching runs
aimx query params "run.hash != ''" --repo data
aimx query params --repo data

# Select specific params
aimx query params "run.experiment == 'cloud-segmentation'" --repo data \
Expand All @@ -209,7 +213,9 @@ aimx query params "run.hparam.lr == 0.0001" --repo data --param hparam.lr
```

Missing selected params are displayed as `-` in terminal/plain output and listed
under `missing_params` in JSON.
under `missing_params` in JSON. Params output also includes run duration as a
table column, a plain-output field after run name, and a JSON `duration` object
with `seconds`, `status`, and `source`.

### Trace metrics

Expand All @@ -219,6 +225,9 @@ overlaid on the same plot.

```bash
# Plot all matching loss curves
aimx trace --repo data

# Filter with AimQL
aimx trace "metric.name == 'loss'" --repo data

# Plot one run by short hash
Expand Down Expand Up @@ -252,7 +261,7 @@ current-step histogram and step-by-bin heatmap. Use `--table`, `--csv`, or

```bash
# Show a web-like terminal visual for the first matched distribution
aimx trace distribution "distribution.name != ''" --repo data
aimx trace distribution --repo data

# Inspect a specific training step; nearest tracked step is used if needed
aimx trace distribution "distribution.name != ''" --repo data --step 12300
Expand Down Expand Up @@ -325,6 +334,7 @@ Useful local checks:
uv run aimx --help
uv run aimx version
uv run aimx doctor
uv run aimx query metrics "metric.name == 'loss'" --repo data
uv run aimx query images "images" --repo data/.aim --json
uv run aimx query metrics --repo data
uv run aimx query images --repo data/.aim --json
uv run aimx query params --repo data --json
```
37 changes: 37 additions & 0 deletions specs/006-default-query-runtime/checklists/requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Specification Quality Checklist: Default Query Expression And Run Duration

**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-06-14
**Feature**: [spec.md](../spec.md)

## Content Quality

- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed

## Requirement Completeness

- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified

## Feature Readiness

- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification

## Notes

- Validation iteration 1 passed. The spec uses user-facing command behavior and output contracts, includes no [NEEDS CLARIFICATION] markers, and bounds scope to read-only default-expression handling plus run duration in params results.
- Implementation verification on 2026-06-15 passed manual quickstart commands,
focused query/trace/params tests, passthrough regression tests, and full
suite validation (`347 passed, 15 skipped`).
146 changes: 146 additions & 0 deletions specs/006-default-query-runtime/contracts/cli-output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# CLI Output Contract: Default Query Expression And Run Duration

**Feature**: `006-default-query-runtime`

This contract defines observable CLI behavior for omitted query-language input
and run duration in `aimx query params`.

## Affected Command Shapes

The following owned command forms accept an omitted expression and use
`run.hash != ''` as the effective expression:

```text
aimx query metrics [<expression>] [--repo <path>] [--json] [--oneline | --plain]
[--steps start:end | --epochs start:end]
[--head N] [--tail N] [--every K]

aimx query images [<expression>] [--repo <path>] [--json] [--oneline | --plain]
[--steps start:end | --epochs start:end]
[--head N] [--tail N] [--every K] [--max-images N]

aimx query params [<expression>] [--repo <path>] [--json] [--oneline | --plain]
[--param <key>]...

aimx trace [<expression>] [--repo <path>] [--table | --csv | --json]
[--steps start:end] [--head N] [--tail N] [--every K]
[--width W] [--height H] [--no-color]

aimx trace distribution [<expression>] [--repo <path>]
[--steps start:end]
[--head N] [--tail N] [--every K]
[--step N]
[--width W] [--height H] [--no-color]
[--table | --csv | --json]
```

The command target is still required for `aimx query`. The `distribution`
subtarget is still required for distribution trace.

## Effective Expression Rules

| Input Shape | Effective Expression |
|-------------|----------------------|
| Expression omitted | `run.hash != ''` |
| Expression token is empty or whitespace-only | `run.hash != ''` |
| First token after target is an option such as `--repo` | `run.hash != ''`; option parsing starts at that token |
| Non-empty explicit expression | The explicit expression exactly as supplied |

Examples:

```bash
aimx query metrics --repo data
aimx query images --repo data --plain
aimx query params --repo data --json
aimx trace --repo data
aimx trace --json --repo data
aimx trace distribution --repo data --head 2
```

Each example above is equivalent to the same command with
`"run.hash != ''"` supplied as the expression.

## Query Params Duration Output

`aimx query params` includes run duration in every output mode.

### Human-Readable Output

The default params table includes a duration column near the run identity
columns:

```text
RUN EXPERIMENT NAME DURATION hparam.lr ...
eca37394 cloud-segmentation ucloudnet-pre-0503 1h20m13s 0.0001
```

If duration cannot be determined, the cell is an explicit marker such as
`unavailable` or `running`; it must not be blank or `0s` unless duration is
actually zero.

### Plain Output

Plain output emits one tab-separated row per matched run:

```text
<repo> <short_hash> <experiment> <run_name> <duration> <key=value>...
```

Missing duration uses the same explicit marker semantics as the rich output.

### JSON Output

JSON output keeps the existing top-level params envelope and adds a stable
`duration` object for every run:

```json
{
"target": "params",
"repo": "data",
"expression": "run.hash != ''",
"runs_count": 1,
"param_keys": ["hparam.lr"],
"runs": [
{
"hash": "eca37394eeb84f48a5d2d736",
"experiment": "cloud-segmentation",
"name": "ucloudnet-pre-0503",
"duration": {
"seconds": 4813.352312088013,
"status": "available",
"source": "duration"
},
"params": {
"hparam.lr": 0.0001
},
"missing_params": []
}
]
}
```

When duration cannot be calculated, `duration.seconds` is `null` and
`duration.status` is `unavailable` or `running`.

## Exit Status

| Condition | Exit Status | Output |
|-----------|-------------|--------|
| Omitted expression with valid repo and matches | `0` | Same mode-specific output as explicit `run.hash != ''` |
| Omitted expression with valid repo and zero matches | `0` | Existing no-results or empty structured output behavior |
| Explicit valid expression | `0` | Same output behavior as before this feature |
| Explicit invalid expression | `2` | Existing actionable evaluation error on stderr |
| Missing repository path | `2` | Existing actionable repository error on stderr |
| Missing option value | `2` | Existing actionable parser error on stderr |
| Missing duration metadata on a params run | `0` | Run remains visible with duration status marker |

## Non-Regression Requirements

- Explicit query and trace expressions keep existing match semantics.
- `query images` inline rendering and `--max-images` behavior remain unchanged.
- `query params --param` selection and missing-param behavior remain unchanged.
- Metric trace and distribution trace `--table`, `--csv`, and `--json` modes
remain parseable with their existing data shapes, except where an existing
output already reports an expression and must now report the effective
default.
- Commands outside owned `aimx` surfaces continue to delegate to native `aim`.
Loading
Loading