Skip to content

awf validate rejects all mcp_proxy.plugin_tools entries (UNKNOWN_PLUGIN) even when plugins are installed #354

@pocky

Description

@pocky

Summary

awf validate <workflow> always reports USER.MCP_PROXY.UNKNOWN_PLUGIN for any entry under mcp_proxy.plugin_tools[], even when the referenced plugin is installed and visible to awf plugin list and accepted at runtime by awf run.

This makes static validation of plugin_tools effectively unusable: any workflow declaring plugin tools cannot pass awf validate in CI, which defeats the purpose of having the USER.MCP_PROXY.UNKNOWN_PLUGIN / USER.MCP_PROXY.UNKNOWN_OPERATION checks introduced by F099.

Severity

Medium — does not affect runtime correctness, but:

  • Prevents using awf validate as a CI gate for workflows that legitimately use plugin_tools
  • Forces users onto awf run --dry-run as the only syntactic-check option
  • Creates an inconsistency between validate and run that is hard to debug for end users (the diagnostic message claims the plugin is missing while awf plugin list shows it)

Expected Behavior

awf validate <workflow> should accept mcp_proxy.plugin_tools entries whose plugin matches a plugin that is currently installed and discoverable by awf plugin list. It should only emit USER.MCP_PROXY.UNKNOWN_PLUGIN when the plugin is genuinely absent from the runtime environment.

Actual Behavior

awf validate rejects every plugin_tools[].plugin value with USER.MCP_PROXY.UNKNOWN_PLUGIN, regardless of whether the plugin is installed.

Reproduction

# Confirm awf-plugin-time is installed and visible
$ awf plugin list
NAME             TYPE      VERSION  STATUS   ENABLED  CAPABILITIES
awf-plugin-time  external  1.2.0    loaded   yes      operations
notify           builtin   ...      builtin  yes      operations
[...]

# Create a minimal workflow with plugin_tools
$ cat > .awf/workflows/repro.yaml << 'YAML'
name: repro
states:
  initial: step1
  step1:
    type: agent
    provider: openai_compatible
    prompt: "hi"
    mcp_proxy:
      enable: true
      intercept_builtins: false
      plugin_tools:
        - plugin: awf-plugin-time
          expose: [time]
    on_success: done
  done:
    type: terminal
    status: success
YAML

# validate fails
$ awf validate repro
[USER.MCP_PROXY.UNKNOWN_PLUGIN] plugin "awf-plugin-time" not found in operation registry
  Details:
    path: states.step1.mcp_proxy.plugin_tools[0].plugin
    plugin: awf-plugin-time
    step: step1
$ echo $?
1

# But the same workflow is accepted by run --dry-run
$ awf run repro --dry-run
[...]
OK No commands will be executed (dry-run mode).

The same UNKNOWN_PLUGIN error appears for notify (builtin), github (builtin), and every other installed plugin.

Root Cause

internal/interfaces/cli/validate.go:100-106:

svc := application.NewWorkflowService(repo, nil, nil, nil, validator)

// Inject an OperationProvider so that mcp_proxy.plugin_tools checks run.
// A CompositeOperationProvider with no sub-providers returns empty results,
// which causes UNKNOWN_PLUGIN errors for any plugin reference — the correct
// behavior when no plugins are installed in the current environment.
svc.SetPluginOperationProvider(infrastructurePlugin.NewCompositeOperationProvider())

The validate command instantiates a CompositeOperationProvider with zero sub-providers, so ListOperations() returns an empty slice. Downstream:

  • WorkflowService.buildKnownPluginSet() (internal/application/service.go:290-301) iterates that empty list and returns an empty map[string]bool.
  • WorkflowService.validateMCPProxyPluginTools() (service.go:332-350) then rejects every plugin_tools[].plugin because the lookup against the empty map fails.

The comment in validate.go describes the "correct behavior when no plugins are installed", but the validate command runs in the same environment as awf run — plugins are installed, they're simply not loaded into the validate command's service.

By contrast, internal/interfaces/cli/run.go:514 and run.go:593 construct WorkflowService with the full composite provider (builtins + external plugins discovered through the plugin manager), which is why awf run --dry-run works.

Proposed Fix

Wire the same operation provider into validate.go that run.go uses. Concretely:

  1. In internal/interfaces/cli/validate.go, load the plugin manager and builtin registry the same way run.go does.
  2. Build a CompositeOperationProvider containing the builtin providers (notify, github, http) plus any externally loaded plugins from the plugin manager.
  3. Pass that composite to svc.SetPluginOperationProvider(...).
  4. Remove the misleading comment at validate.go:103-105.

A minimal fix likely involves extracting the OperationProvider construction from run.go into a shared helper (e.g., internal/interfaces/cli/operations.go: buildCompositeOperationProvider(...)) and calling it from both run.go and validate.go. This keeps a single source of truth for plugin discovery across CLI commands.

Workaround

Until fixed, users must rely on awf run <workflow> --dry-run instead of awf validate <workflow> to check workflows that declare plugin_tools. awf validate remains usable for workflows that do not use plugin_tools.

Acceptance Criteria

  • awf validate <workflow> accepts mcp_proxy.plugin_tools[].plugin values that match any installed plugin (builtin or external).
  • awf validate still emits USER.MCP_PROXY.UNKNOWN_PLUGIN when the referenced plugin is genuinely absent.
  • awf validate still emits USER.MCP_PROXY.UNKNOWN_OPERATION when the operation is not exposed by the (now-resolved) plugin.
  • Behavior of awf validate and awf run --dry-run is consistent for any workflow that uses mcp_proxy.
  • A regression test under internal/interfaces/cli/validate_mcp_proxy_test.go (or a sibling file) covers the case where validate is invoked against a workflow declaring a plugin known to the composite provider.
  • The misleading "correct behavior when no plugins are installed" comment at validate.go:103-105 is removed or rewritten to reflect the real semantics.

References

  • Introduced by: F099 — MCP Proxy (PR F099: MCP Proxy — Interception et contrôle des tool calls #353 / branch feature/F099-mcp-proxy-...)
  • Related files:
    • internal/interfaces/cli/validate.go:100-106 (empty composite injected)
    • internal/interfaces/cli/run.go:514, 593 (full composite injected — reference implementation)
    • internal/application/service.go:290-301 (buildKnownPluginSet)
    • internal/application/service.go:332-350 (validateMCPProxyPluginTools)
  • Discovered while building .awf/workflows/test-mcp-proxy-additive.yaml for F099 manual testing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions