Register the connector-service (webapi) plugin in the Flyte 2 executor

[dejanzele]

The Flyte 2 executor doesn't register the connector-service (webapi) plugin, so it can't run connector tasks. The Python SDK already ships the connector interfaces (AsyncConnector and friends), but nothing on the executor side runs them: the plugin that delegates to a connector deployment is never registered.

I ran into this building a connector for Armada. The task type resolves to connector-service, but the executor has no plugin by that name, so the task never routes.

Registering the plugin isn't quite enough on its own. The webapi machinery also pulls a ResourceManager and a SecretManager from the plugin setup context, and the executor provides neither, so the allocation-token path nil-derefs.

I opened #7565 with the change. It registers connector-service in the executor and supplies a no-op ResourceManager and registrar so the allocation path doesn't panic. With it, a connector works end to end: https://github.com/dejanzele/armada-flyte runs plain @env.task functions as Armada jobs through it (fan-out, gang scheduling, a small ML pipeline).

A couple of questions for reviewers:

1. Is registering connector-service in the executor the intended path for v2, or is connector execution meant to be wired up somewhere else?
2. Is a no-op ResourceManager and SecretManager acceptable in the executor, or should it provide real implementations?

This isn't Armada-specific. Any connector backend (Databricks, Ray, etc.) needs the same plugin registered to run on v2.

[runllm]

To register the connector-service (webapi) plugin in the Flyte 2 executor, ensure that "connector-service" is listed under enabled-plugins in your Flyte deployment configuration. For example, in your Helm values or ConfigMap, include:

tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - k8s-array
      - connector-service
      - echo

This enables the connector-service plugin, allowing Flyte to delegate tasks to registered connectors. For more details, see Flyte deployment configuration and plugin registration. Would you like more detail on how to implement or test a custom connector?

Sources:

• Flyte deployment configuration
• plugin registration

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @Herald to ask follow-up questions. See runllm.com for more!

[dejanzele]

Thanks, but that part is already in place. The stock flyte-binary config already lists connector-service under enabled-plugins, and connector tasks still don't run.

enabled-plugins only selects from plugins that are registered in the executor binary, and the v2 executor never registers connector-service, so there's nothing for the config to enable. That's the gap here, and it's a code change rather than a config one: #7565 registers the plugin in executor/setup.go (plus a no-op ResourceManager so the allocation-token path doesn't nil-deref).

The values you linked are the v1 flyte-core chart, where the agent plugin is compiled in and self-registers, so config alone was enough there.
v2 is a different executor, and that registration step is what's missing.