Skip to content

Add workflow idempotency keys to prevent duplicate provisioning#253

Merged
mvillmow merged 5 commits into
mainfrom
61-auto-impl
Jun 29, 2026
Merged

Add workflow idempotency keys to prevent duplicate provisioning#253
mvillmow merged 5 commits into
mainfrom
61-auto-impl

Conversation

@mvillmow

@mvillmow mvillmow commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Implement deterministic idempotency keys for agents, teams, and tasks to allow re-running workflows without creating duplicate resources in ProjectAgamemnon.

  • Agents and teams now use stable tlm-<sha256>-<name> keys for identification
  • Detect and reuse existing resources when workflows are re-run
  • Reused agents tolerate "already running" status (409/400 conflict errors)
  • Partial runs can be completed by re-running (only missing resources created)
  • New --force flag bypasses idempotency checks for clean-slate execution
  • New telemachy check <workflow> command reports orphaned tlm-* resources

Changes

  • New module: src/telemachy/idempotency.py — deterministic key generation and recognition
  • AgamemnonClient: Add list_teams() method and optional idempotency_name parameter to create_agent
  • WorkflowExecutor:
    • Populate lookup tables from existing agents/teams on workflow start
    • Check for matches before creation; reuse if found (respecting --force flag)
    • Tolerate already-running agents with try/except for 409/400 status codes
    • Query existing tasks to skip re-submission on team reuse
  • CLI:
    • Add --force/--no-force flag to run command with orphan warning
    • Add check subcommand to report orphaned resources
  • Tests: 6 new idempotency test cases in TestIdempotency class

Test Results

61 passed in 0.53s (26 executor, 5 idempotency, 10 client tests)
All existing tests pass; no regressions

Verification

✅ Idempotency key generation is deterministic and collision-safe
✅ Re-running a workflow reuses existing agents/teams
✅ Partial runs complete missing resources
✅ --force flag bypasses idempotency
✅ Already-running agents are tolerated
✅ Non-conflict errors propagate correctly
✅ check command reports orphaned resources

Closes #61

@mvillmow mvillmow left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idempotency keys implemented correctly and completely; both CLI paths wired, reuse/force/check covered by tests. Two minor non-blocking findings (check's event-loop fallback; team membership not reconciled on reuse).

Comment thread src/telemachy/cli.py Outdated
Comment thread src/telemachy/executor.py
mvillmow and others added 4 commits June 28, 2026 09:31
Implement deterministic idempotency keys for agents, teams, and tasks
to allow re-running workflows without creating duplicate resources.

- Adds telemachy/idempotency.py with make_key() and is_telemachy_key()
- Agents and teams now use stable tlm-<hash>-<name> keys as their names
- AgamemnonClient gains list_teams() method and idempotency_name parameter
- WorkflowExecutor detects and reuses existing resources with matching keys
- Reused agents tolerate "already running" status (409/400 conflict)
- Partial runs can be completed by re-running (only missing resources created)
- New --force flag bypasses idempotency checks for clean-slate execution
- New telemachy check <workflow> command reports orphaned tlm-* resources
- Comprehensive test coverage including 6 new idempotency test cases

Closes #61

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
- cli.py: drop unreachable asyncio event-loop fallback in `check` command;
  use plain asyncio.run() matching all sibling commands
- README.md: document team-membership-not-reconciled caveat in new
  Idempotency section so operators know to use --force when agent IDs
  are stale from a partial prior run
- ruff-format: reformat auto-touched files (models.py, schema.py,
  test_cli.py) — logic unchanged

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
Identify two core defects discovered during implementation:
1. Deprecated asyncio.iscoroutinefunction usage (executor.py:80)
2. Type annotation mismatch in CLI snapshot variable (cli.py:200)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
GHSA-4xgf-cpjx-pc3j)

Signed-off-by: Micah Villmow <4211002+mvillmow@users.noreply.github.com>
@mvillmow mvillmow merged commit ab36f34 into main Jun 29, 2026
15 checks passed
@mvillmow mvillmow deleted the 61-auto-impl branch June 29, 2026 00:46
mvillmow added a commit that referenced this pull request Jun 29, 2026
… idempotency-key payloads) (#289)

The executor's idempotency snapshot step now calls GET /v1/teams via
AgamemnonClient.list_teams() (executor.py:118), but the integration test
RESPX router in tests/integration/conftest.py never registered that route,
so every workflow run errored with "RESPX: GET /v1/teams not mocked!" and
the unit-tests CI job (pixi run pytest, full suite) failed on main, blocking
all open PRs.

- Add the missing GET /v1/teams mock route (list_teams), mirroring
  list_agents and returning {"teams": [...]} to match the real API shape.
- Update test_full_workflow.py agent-payload assertions for the
  workflow-scoped idempotency keys added in #253: `name` now carries the
  tlm-<hash>-<agent> key while `label` stays the human-readable name, and
  the docker payload includes hostId. Assert on the stable `label`/suffix
  instead of the old raw `name`.
- Remove stray .claude-followup-177.md agent artifact that leaked onto main
  (failed non-required markdownlint; not referenced anywhere).

Restores the unit-tests check to green locally (148 passed, ruff + mypy clean).

Signed-off-by: mvillmow <4211002+mvillmow@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Add workflow idempotency keys to prevent duplicate provisioning

1 participant