Skip to content

feat(coding-agent): add async bash() to IPython kernel - #1187

Open
samsja wants to merge 12 commits into
mainfrom
feat/async-bash-tool
Open

feat(coding-agent): add async bash() to IPython kernel#1187
samsja wants to merge 12 commits into
mainfrom
feat/async-bash-tool

Conversation

@samsja

@samsja samsja commented Aug 11, 2026

Copy link
Copy Markdown
Member

Summary

Adds an async bash() function to the IPython kernel bootstrap code so await bash("...") runs shell commands via asyncio.create_subprocess_exec without blocking the kernel event loop.

Motivation

%%bash cells block the IPython kernel until the command finishes — the kernel can't respond to interrupts or other messages while a shell command runs. This is the same anti-pattern as time.sleep() polling. await bash("...") uses asyncio.create_subprocess_exec so the kernel event loop stays responsive.

Changes

  • ipython.ts: Added _PrimeAgentBashResult class and async bash() function to RLM_BOOTSTRAP_BASE_CODE. Supports timeout (raises TimeoutError) and cwd parameters. Returns stdout, stderr, and returncode.
  • rlm.ts: Updated the IPython control prompt to prefer await bash("...") over %%bash cells.
  • system-prompt.test.ts: Updated exact-match block and added toContain assertion for the new prompt text.

Usage

result = await bash("echo hello")           # BashResult(stdout="hello\n", ...)
result = await bash("sleep 5", timeout=2)   # raises TimeoutError
print(result.stdout, result.stderr, result.returncode)

Testing

  • npx vitest run test/system-prompt.test.ts — 25/25 pass
  • npx vitest run test/ipython-bootstrap.test.ts — 6/6 pass (including real kernel tests)
  • Manually verified in a real IPython kernel: stdout/stderr capture, exit codes, and timeout all work correctly
  • npm run check — clean

Note

Medium Risk
Changes how shell commands run in the kernel (new process groups and aggressive descendant termination), which can differ from %%bash behavior for background jobs; scope is confined to the agent IPython runtime on POSIX.

Overview
Adds an async bash() helper to the IPython kernel bootstrap so agents can run await bash("...") instead of blocking %%bash cells, keeping the kernel event loop responsive to interrupts and cancellation.

The implementation spawns commands in a new POSIX session, streams stdout/stderr into tail-bounded buffers (default 32 KiB per stream, configurable via max_output_bytes), returns a structured result with truncation metadata, and tears down the process group on timeout, cancellation, or after the shell exits (including background descendants). Result objects render captured output in IPython when used as the final cell expression.

RLM / system prompts now tell the model to prefer await bash("...") over %%bash, while still documenting %%bash rules. Changelog and bootstrap / real-kernel tests cover output bounds, UTF-8 handling, descendant cleanup, and SIGTERM/SIGKILL escalation.

Reviewed by Cursor Bugbot for commit 02543c7. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add async bash() coroutine to the IPython kernel for non-blocking shell execution

  • Injects an async bash() function into the IPython kernel bootstrap so shell commands run without blocking the event loop, replacing the recommended use of %%bash magic cells.
  • Captures stdout/stderr into bounded buffers with a configurable byte cap, reporting truncation flags and total byte counts via a structured _PrimeAgentBashResult return value.
  • Handles timeouts and cancellation with robust process-group termination: POSIX uses killpg in a new session; Windows uses taskkill /T /F.
  • Updates the RLM system prompt in rlm.ts to prefer await bash("...") over %%bash, with guidance on responsiveness, cleanup, and output capture.
  • Risk: bash() spawns a new process group/session on POSIX, which changes the process tree structure relative to %%bash or !cmd shell escapes.

Changes since #1187 opened

  • Removed Windows support from bash() function in IPython kernel [91eae25]
  • Updated IPython bootstrap test expectations for Windows bash() unavailability [91eae25]
  • Modified _PrimeAgentBashResult.__repr__ method to return the captured command output (combined stdout/stderr and exit code) instead of a metadata summary [785b9b7]
  • Added test case validating that await bash() renders command output when used as the final expression in an IPython cell [785b9b7]
  • Reworked bash command process completion and cleanup logic in _prime_agent_bash_collect function to terminate background descendants and handle output draining [6b696f8]
  • Reduced default bash command output capture limit and added explicit truncation notices to output representation [6b696f8]
  • Added test verification for truncation notices in bash command output [6b696f8]
  • Added test coverage for background process cleanup after shell exit [6b696f8]
  • Added explicit boolean type rejection to max_output_bytes parameter validation in the bash() function within the IPython kernel bootstrap code [9a05080]
  • Changed decoding of bounded bash output from UTF-8 to Latin-1 in IPython kernel [66902ff]
  • Added test case for malformed byte output handling in bash execution [66902ff]
  • Updated IPython shell execution guidance wording in prompt constant [66902ff]
  • Replaced timeout-based SIGKILL escalation with process-group-existence-based escalation in _prime_agent_bash_collect and _prime_agent_bash_stop functions [af69851]
  • Added test validating SIGKILL escalation for descendants that ignore SIGTERM and close pipes [af69851]
  • Changed _PrimeAgentBoundedBytes.text to decode captured bytes as UTF-8 with replacement for malformed sequences, then re-encode and head-trim the UTF-8 payload when it exceeds the byte limit, setting a new render_truncated instance attribute to track head-trimming, and updated _PrimeAgentBoundedBytes.__init__ to initialize render_truncated to False [a15a9e0]
  • Updated _PrimeAgentBashResult.__str__ to compute and display truncation byte counts using UTF-8 encoding instead of latin-1 for both stdout and stderr [a15a9e0]
  • Modified bash async function to combine raw capture truncation flags with the new render_truncated flag from _PrimeAgentBoundedBytes using OR operation for both stdout and stderr [a15a9e0]
  • Added test case verifying UTF-8 command output preservation through bash and modified existing truncation test to use UTF-8 byte lengths for assertions [a15a9e0]
  • Added platform-specific shell command execution guidance to IPYTHON_CONTROL_PROMPT constant [dd88b46]
  • Changed truncation message format in _PrimeAgentBashResult.__str__ method for stdout and stderr [bdbd8ec]
  • Removed Windows platform restriction from bash() async function in IPython kernel bootstrap [02543c7]
  • Updated IPYTHON_CONTROL_PROMPT constant to provide platform-agnostic shell command guidance [02543c7]
  • Enabled Windows execution for IPython bash() integration tests [02543c7]
  • Updated buildRlmPrompt test assertions to expect platform-agnostic guidance [02543c7]

Macroscope summarized ff15d7b.

Comment thread packages/coding-agent/src/core/tools/ipython.ts
Comment thread packages/coding-agent/src/core/tools/ipython.ts
Comment thread packages/coding-agent/src/core/tools/ipython.ts Outdated

@sethkarten sethkarten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subprocess lifecycle clean-up and potential high memory usage needs addressing

Comment thread packages/coding-agent/src/core/prompts/rlm.ts Outdated
Comment thread packages/coding-agent/src/core/tools/ipython.ts
samsja and others added 2 commits August 12, 2026 13:00
Add an async bash() function to the RLM bootstrap code in ipython.ts
that uses asyncio.create_subprocess_exec to run shell commands without
blocking the kernel event loop. Unlike %%bash cells (which block the
kernel until the command finishes), await bash('...') keeps the kernel
responsive to interrupts and other messages while the process runs.

Supports optional timeout (raises TimeoutError) and cwd parameters.
Returns a _PrimeAgentBashResult with stdout, stderr, and returncode.

Update the RLM system prompt to prefer await bash('...') over %%bash
cells.
@sethkarten
sethkarten force-pushed the feat/async-bash-tool branch from e92f4b0 to ff15d7b Compare August 12, 2026 20:16
@sethkarten
sethkarten requested a review from eliebak August 12, 2026 20:24
Comment thread packages/coding-agent/src/core/tools/ipython.ts Outdated
Comment thread packages/coding-agent/src/core/tools/ipython.ts Outdated
Comment thread packages/coding-agent/src/core/tools/ipython.ts
Comment thread packages/coding-agent/src/core/tools/ipython.ts Outdated
Comment thread packages/coding-agent/src/core/prompts/rlm.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit dd88b46. Configure here.

Comment thread packages/coding-agent/src/core/tools/ipython.ts

@snimu snimu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a close look at the final state — the process-group cleanup and bounded capture are careful and the real-kernel tests cover the hard cases well. Three things I think are worth addressing before (or right after) merge, since they're all on the path the prompt now recommends:

"""
if isinstance(max_output_bytes, bool) or not isinstance(max_output_bytes, int) or max_output_bytes <= 0:
raise ValueError("max_output_bytes must be a positive integer")
shell = _prime_agent_shutil.which("bash") or "/bin/bash"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bash() hardcodes the shell and skips the user's shell settings. The %%bash path goes through applyShellSettingsToBashMagicCell(), which applies the configured commandPrefix and shellPath. Users who set those get them silently ignored on the now-preferred path. Suggest either applying the same settings here or noting in the prompt that bash() ignores them.

# pipes, even after the shell's returncode is set. Observe that state directly.
while proc.returncode is None:
await _prime_agent_asyncio.sleep(0.01)
# A successful shell may leave background descendants in its session. Stop

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kills background jobs even on success — by design, and the tests pin it. But that's a real behavior difference from %%bash: with bash() you can never start a long-lived background process (e.g. nohup server &). The prompt says "prefer await bash(...)" without mentioning this, so a model following the doctrine will launch a dev server and find it dead. One extra clause in the prompt line would prevent that.

"Do not assume IPython is the native runtime of the external thing being investigated. A repository, package, service, dataset, paper, website, benchmark, or API may have its own environment and normal interface. Evaluate external systems through their own interface, then use IPython to coordinate the process and analyze what comes back.",
"",
"When running shell commands from IPython, use `%%bash` cells. If you use `%%bash`, it must be the first line of the code cell: no comments, spaces, blank lines, imports, or Python statements before it. Avoid `!cmd` shell escapes for project commands so shell behavior is explicit and multi-line commands share one shell context.",
'When running shell commands from IPython, prefer `await bash("...")` over `%%bash` cells: it keeps the kernel event loop responsive to interrupts, cleans up its managed process group on interruption, and bounds captured stdout/stderr. `%%bash` cells block the kernel until the command finishes. If you do use `%%bash`, it must be the first line of the code cell: no comments, spaces, blank lines, imports, or Python statements before it. Avoid `!cmd` shell escapes for project commands so shell behavior is explicit and multi-line commands share one shell context.',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No streaming: %%bash shows output as it runs, bash() buffers everything until the command exits. For a long build the model and the user watching see nothing until the end — and since only the tail is kept, the beginning is gone. Fine for short commands, but the prompt recommends bash() unconditionally. Worth either mentioning "use %%bash for long-running commands where you want live output" here, or treating streaming as a follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants