refactor: run local subworkflows in process

[yohamta0]

Summary

• run local child workflows in process through the runtime agent instead of spawning the Dagu CLI
• remove the subprocess-backed local subworkflow runner and wire CLI, worker, tests, and embedded Engine to the in-process runner
• preserve child status, logs, artifacts, retries, cancellation, nested subworkflows, workspace bundles, and agent-store dependencies

Testing

• make test
• make lint

---

Summary by cubic

Run local subworkflows in-process via the runtime agent instead of spawning the Dagu CLI. This removes the executable requirement and reduces overhead while preserving child run behavior (status, logs, artifacts, retries, cancellation, nesting).

• Refactors
  • Added internal/subflow/local.go and local_helpers.go with a new subflow.NewLocal runner; removed local_cli.go.
  • Wired factories in internal/cmd/context.go, internal/engine/engine.go, internal/engine/run.go, internal/service/worker/remote_handler.go, and tests to build a router with subflow.NewLocal(...), passing stores, status/log/artifact handlers, worker ID, and run dirs; uses a self-referential factory to support nested subworkflows.
  • Updated in-process runner to honor workspace bundles, set artifact directories, filter inherited tool envs, and map cancel intents to signals; kept dag.run parity with the previous subprocess path.
  • Added tests to verify no dagu executable is required and artifacts are written for child runs; adjusted router tests to new ShouldRun rules (rejects worker selectors unless ForceLocal is set).

^{Written for commit 975a954. Summary will update on new commits.}

Summary by CodeRabbit

• New Features

  • Sub-workflows now execute in-process instead of through subprocess calls, improving performance and reducing external dependencies.

• Tests

  • Added test coverage for local sub-DAG execution, including artifact isolation and parameter propagation validation.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c04d6605-3d72-4549-ad53-fb337693d8a1

📥 Commits

Reviewing files that changed from the base of the PR and between dd88f53 and 975a954.

📒 Files selected for processing (11)

• internal/cmd/context.go
• internal/engine/engine.go
• internal/engine/run.go
• internal/runtime/agent/agent_test.go
• internal/service/worker/remote_handler.go
• internal/subflow/local.go
• internal/subflow/local_cli.go
• internal/subflow/local_helpers.go
• internal/subflow/router_test.go
• internal/subflow/runner.go
• internal/test/helper.go

💤 Files with no reviewable changes (1)

• internal/subflow/local_cli.go

---

📝 Walkthrough

Walkthrough

This PR replaces the subprocess-based LocalCLI sub-workflow runner with a fully in-process Local runner that executes child DAGs directly inside the parent process. The new runner integrates local stores, state management, and recursive sub-workflow support while eliminating the dependency on an external Dagu CLI executable.

Changes

In-process Local Sub-workflow Runner

Layer / File(s)	Summary
Core Local runner implementation internal/subflow/local.go	Introduces Local struct implementing executor.SubWorkflowRunner with full in-process execution: validates requests, loads child DAGs with optional workspace materialization, constructs and executes runtime agents with TriggerTypeSubDAG/retry modes, tracks active agents by runID for cancellation, and translates agent status into exec.RunStatus. Supports ShouldRun, Run, Retry, and Cancel methods with recursive sub-workflow runner factory injection.
Local sub-workflow helpers internal/subflow/local_helpers.go	Provides materializeLocalWorkspace (validates/extracts workspace archives with path escape protection), localCancelSignal (maps force-cancel to os.Kill), and environment filtering (inheritedEnvForLocalRunner, hasDAGToolsEnv, isDAGToolsEnvKey) to strip DAG-tools manifest keys when present.
Sub-workflow runner factory wiring internal/cmd/context.go, internal/engine/engine.go, internal/engine/run.go, internal/service/worker/remote_handler.go, internal/test/helper.go, internal/subflow/runner.go	Updates SubWorkflowRunnerFactory implementations in Context, Engine, and RemoteHandler to construct a runtime dispatcher and return a subflow.NewRouter backed by the new Local runner wired to local DAG run/state/queue/secret stores, service registry, and status/log/artifact side-effect handlers; adds sentinel error errNoRunDatabase; Engine.runLocal now passes stores argument to factory.
Test coverage updates internal/subflow/router_test.go, internal/runtime/agent/agent_test.go	Removes four LocalCLI-specific tests and replaces with three Local runner ShouldRun tests verifying valid in-process requests, WorkerSelector rejection, and DAG.ForceLocal override behavior; adds TestAgent_LocalSubDAGRunDoesNotRequireDaguExecutable (verifies sub-DAG completion without external executable and param propagation), and TestAgent_LocalSubDAGRunSetsArtifactDir (verifies artifact isolation under ArchiveDir).

Sequence Diagram(s)

sequenceDiagram
  participant Parent as Parent DAG Agent
  participant Local as Local Sub-workflow Runner
  participant DAGStore as DAG/State Stores
  participant Child as Child Runtime Agent
  Parent->>Local: Run/Retry/Cancel request
  Local->>Local: Validate (DAG, RunID, RootDAGRun)
  Local->>DAGStore: Load child DAG
  opt Workspace materialization
    Local->>Local: Extract & verify workspace
  end
  Local->>Child: Create agent (stores, config, trigger)
  Local->>Child: Execute in-process
  Child->>DAGStore: Update run status
  Child->>Local: Final status
  Local->>Parent: Return exec.RunStatus

Loading

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• dagucloud/dagu#2247: Introduces subflow.NewRouter used throughout this PR's factory implementations to combine dispatcher and local runner.
• dagucloud/dagu#2235: Refactors coordinator.NewRuntimeDispatcher construction used in all three factory contexts (Context, Engine, RemoteHandler).
• dagucloud/dagu#2244: Prior refactor of SubWorkflowRunnerFactory/runner-port patterns that this PR builds upon by replacing LocalCLI with the new in-process Local runner.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'refactor: run local subworkflows in process' directly describes the main change: shifting local subworkflow execution from subprocess-based (Dagu CLI) to in-process execution.
Description check	✅ Passed	The PR description includes a clear summary of changes, specific implementation details, testing steps, and covers the main refactoring goals. While it extends beyond the template format, it provides comprehensive information about the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/in-process-subworkflow-runner

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}