Add async/await, non-blocking I/O, and networking stdlib by sb2bg · Pull Request #3 · sb2bg/walrus

sb2bg · 2026-03-05T17:37:48Z

Summary

Adds a complete async/await system to Walrus with non-blocking I/O, networking primitives, and a concurrent HTTP server example, bringing the language to near-JavaScript/Python level for async programming.

Async Runtime

Single-threaded event loop with cooperative multitasking — no race conditions, simple mental model
async fn / await syntax with full closure capture support
asyncx.spawn(fn, args) — schedule tasks for concurrent execution
asyncx.gather(tasks) — wait for all tasks (short-circuits on first failure, like Promise.all)
asyncx.race(tasks) — return first completed result (like Promise.race)
asyncx.all_settled(tasks) — wait for all tasks regardless of success/failure, returns [{status, value/error}] (like Promise.allSettled)
asyncx.sleep(ms) / asyncx.timeout(task, ms) — timers and deadlines
asyncx.cancel(task) / asyncx.status(task) — task lifecycle management
asyncx.yield() — cooperative yielding for round-robin scheduling
asyncx.channel() / asyncx.send() / asyncx.recv() — Go-style channels for task-to-task communication

Non-blocking I/O

Background worker threads for blocking syscalls (TCP accept, read, write)
Wakeup channel for efficient event loop notification on I/O completion
VM remains single-threaded — worker threads only handle I/O, never touch VM state

Networking & HTTP (`std/net`, `std/http`)

TCP primitives: tcp_bind, tcp_accept, tcp_connect, tcp_read, tcp_write, tcp_close
HTTP parsing: read_request, response, response_with_headers, match_route
Route matching: /users/:id, /users/:id/posts/:post_id, /assets/* wildcard

Standard Library Additions

std/json — json.encode() / json.decode() powered by serde_json
core.timestamp() — epoch milliseconds

Language Improvements

Struct fields as callables — self.handler(req) works when handler is a function stored in a field
try/catch no longer requires trailing semicolon
dict.get(key, default) — safe access with fallback for missing keys

HTTP Server Example (`examples/http_router_sim/`)

A real concurrent HTTP server written entirely in Walrus demonstrating Express-style routing:

app.get("/users/:id", : req {
    let id = req["params"]["id"];
    return router.json(200, json.encode({"id": id}));
});

app.get("/slow", : req {
    await asyncx.sleep(1000);  // doesn't block other requests
    return router.text(200, "done\n");
});

app.listen("127.0.0.1", 8080);

Features: middleware, route params, wildcards, custom 404/error handlers, keep-alive, concurrent connection handling via async tasks.

Support `async : args { body }` syntax for anonymous async functions, mirroring the existing lambda syntax with an `async` prefix.

race(tasks) resolves to the first completed task result, skipping failures unless all tasks fail. status(task) returns the task state as a string: pending, ready, failed, or cancelled.

Organize flat pass/fail fixture directories into semantic categories (async, basics, collections, control_flow, functions, modules, stdlib, strings) and update the test runner to recursively discover fixtures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

yield voluntarily gives other queued tasks a turn to run, then resumes. Works as a no-op when no tasks are queued. Enables round-robin style interleaving between concurrent async tasks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

yield is a scheduling hint, not a language construct — it belongs in std/async alongside spawn, sleep, cancel, etc. Removes Yield from AST, grammar, and opcode; adds AsyncYield native function with vm.run_queued_tasks(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Node.js-style architecture: blocking TCP and HTTP operations now run on worker threads, communicating results back to the single-threaded VM via mpsc channels. The event loop waits on a wakeup channel instead of sleeping, so it resumes immediately when I/O completes. Async operations: tcp_accept, tcp_connect, tcp_read, tcp_read_line, tcp_write, and http.read_request. Sync operations unchanged: tcp_bind, tcp_close, tcp_local_port, and HTTP response builders.

…ess-style HTTP server - Add std/json module with encode() and decode() (recursive descent parser) - Allow struct fields holding functions to be called via method syntax - Make try/catch trailing semicolon optional in grammar - Rewrite http_router_sim as full Express-style framework with middleware, route params, wildcards, error handling, keep-alive, and concurrent connections - Fix dict safe access in router (use .get() with default for missing headers) - Add regressions for all changes: json encode/decode, struct field callable, try/catch no semicolon, core.timestamp(), dict.get() default

- all_settled: like gather but never short-circuits — collects {status: "ok"/"error"/"cancelled", value/error} for every task - Channels: asyncx.channel() returns [sender, receiver] for task-to-task communication via asyncx.send() and asyncx.recv() - Add regressions for both features

sb2bg · 2026-03-05T17:43:58Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 254a5c72eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-05T17:50:14Z

+                            TaskResolution::Ready(value) => {
+                                self.push(value);
+                                self.await_task_roots.pop();


Avoid pushing awaited result twice onto operand stack

When an awaited task is a runnable Pending function, run_pending_task_to_completion executes it and the callee Return path already pushes its return value onto the caller stack; this Await branch then pushes the same value again. That duplicates stack operands, so expressions can compute with the wrong values (for example 5 + await f() can consume two copies of f()), and repeated await statements leak extra stack entries in long-lived frames.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-05T17:50:14Z

+                if *self.user_channels[id].closed.borrow() {
+                    return Ok(false);
+                }


Make channel closed checks reachable

channel_send gates on user_channels[id].closed, but this flag is initialized to false and never set to true anywhere in the VM, so the closed path is effectively unreachable. As a result, asyncx.send cannot report a closed receiver and asyncx.recv waiters can remain pending forever once producers stop, leading to avoidable deadlocks in channel-based workflows.

Useful? React with 👍 / 👎.

sb2bg · 2026-03-06T07:58:19Z

@codex address the feedback

chatgpt-codex-connector · 2026-03-06T08:05:58Z

Summary

Fixed Opcode::Await so awaited task results are pushed exactly once by tracking whether the awaited task already placed its result on the operand stack; this removes duplicate stack values while preserving cleanup of non-awaited background task results. src/vm/mod.rsL3516-L3595
Added VM-level channel closing via channel_close, which marks a channel as closed so the existing send/recv closed-state logic is now reachable. src/vm/mod.rsL958-L972 src/vm/mod.rsL925-L927 src/vm/mod.rsL1273-L1278
Exposed channel close in stdlib as asyncx.close(endpoint) and wired it into native dispatch with a new NativeFunction::AsyncClose variant plus native registration/implementation. src/function.rsL138-L153 src/native_registry.rsL186-L200 src/native_registry.rsL1163-L1177

Testing

✅ cargo fmt
✅ cargo test -q

View task →

- Deleted execution mode files for various tests, simplifying the test suite. - Removed ExecMode enum and associated logic from language suite tests. - Updated test cases to run without specifying execution modes.

sb2bg · 2026-03-08T00:44:54Z

@codex review PR

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c32c4d154a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-08T00:53:14Z

+    if body_bytes.len() > parsed_head.content_length {
+        body_bytes.truncate(parsed_head.content_length);


Preserve unread bytes when parsing HTTP request bodies

The request reader drops any bytes already read beyond the current body's Content-Length: it copies the post-header tail into body_bytes and then truncates it. When a keep-alive client sends requests back-to-back, read() can return data that includes the next request, and this truncation silently discards that next request data, causing corrupted/lost follow-up requests on the same connection.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-08T00:53:14Z

+    }
+
+    pub(crate) fn run_queued_tasks(&mut self, span: Span) -> WalrusResult<()> {
+        while let Some(task_key) = self.next_runnable_task()? {


Refresh waiters even when no runnable task exists

run_queued_tasks only refreshes waiter state after executing a runnable task, so if the queue is empty it exits immediately without polling suspended dependencies. This misses cases where a task became ready due side effects outside the runnable queue (for example asyncx.send from currently running main code making a UserRecv task ready), leaving waiting tasks suspended and making asyncx.yield() fail to advance them.

Useful? React with 👍 / 👎.

Add async/await, non-blocking I/O, and networking stdlib

sb2bg and others added 15 commits March 4, 2026 18:00

Refactor parser implementation to use JsonParser struct

9c388e3

Implement VM async tasks with event-loop await scheduling

0c34f5b

Capture VM task failures and add async rejection regressions

0bb77ec

Add std/async spawn/sleep/timeout/gather on VM tasks

921f9b0

Add VM task cancellation primitives and regressions

23e9c1b

Add async closures/lambdas with regression tests

cda0de6

Support `async : args { body }` syntax for anonymous async functions, mirroring the existing lambda syntax with an `async` prefix.

Add std/net and std/http stdlib modules with TCP and HTTP primitives

4c73adf

Add async race() and status() primitives with regressions

81ea851

race(tasks) resolves to the first completed task result, skipping failures unless all tasks fail. status(task) returns the task state as a string: pending, ready, failed, or cancelled.

Replace hand-rolled JSON with serde_json

3669454

sb2bg requested a review from Copilot March 5, 2026 17:38

Repository owner deleted a comment from chatgpt-codex-connector Bot Mar 5, 2026

sb2bg review requested due to automatic review settings March 5, 2026 17:44

chatgpt-codex-connector Bot reviewed Mar 5, 2026

View reviewed changes

Repository owner deleted a comment from chatgpt-codex-connector Bot Mar 5, 2026

Add HTML page at '/'

72104ec

Repository owner deleted a comment from chatgpt-codex-connector Bot Mar 6, 2026

sb2bg added 5 commits March 6, 2026 02:11

Fix await stack duplication and channel close handling

53e8e9d

Refactor VM execution context management

82fff1f

Add async support to middleware and route handlers

bb354d7

Add HTTP server benchmark

bb7f3e2

Add TCP and HTTP utility functions to README and benchmark results

e8a239d

sb2bg added 4 commits March 6, 2026 03:11

Quality of life HTTP/net stdlib changes

a484ce9

Move stdlib into separate module

86184a5

Break up stdlib

66a4280

Remove interpreter, related tests, and dead code

c32c4d1

- Deleted execution mode files for various tests, simplifying the test suite. - Removed ExecMode enum and associated logic from language suite tests. - Updated test cases to run without specifying execution modes.

chatgpt-codex-connector Bot reviewed Mar 8, 2026

View reviewed changes

sb2bg added 5 commits March 7, 2026 19:14

Split VM

53c7963

Split VM

60899d4

Update benchmarks

be405a1

Update benchmark script

a3b7388

Patch http and scheduling bug

7cb3394

sb2bg merged commit 73ea8eb into master Mar 8, 2026
4 checks passed

sb2bg deleted the async-await-vm branch March 8, 2026 01:46

sb2bg added a commit that referenced this pull request Mar 8, 2026

Merge pull request #3 from sb2bg/async-await-vm

a4753e9

Add async/await, non-blocking I/O, and networking stdlib

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add async/await, non-blocking I/O, and networking stdlib#3

Add async/await, non-blocking I/O, and networking stdlib#3
sb2bg merged 30 commits into
masterfrom
async-await-vm

sb2bg commented Mar 5, 2026 •

edited

Loading

Uh oh!

sb2bg commented Mar 5, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Uh oh!

sb2bg commented Mar 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Mar 6, 2026

Uh oh!

sb2bg commented Mar 8, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Mar 8, 2026

Uh oh!

chatgpt-codex-connector Bot Mar 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		if body_bytes.len() > parsed_head.content_length {
		body_bytes.truncate(parsed_head.content_length);

Conversation

sb2bg commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Async Runtime

Non-blocking I/O

Networking & HTTP (std/net, std/http)

Standard Library Additions

Language Improvements

HTTP Server Example (examples/http_router_sim/)

Uh oh!

sb2bg commented Mar 5, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

sb2bg commented Mar 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Mar 6, 2026

Summary

Uh oh!

sb2bg commented Mar 8, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sb2bg commented Mar 5, 2026 •

edited

Loading

Networking & HTTP (`std/net`, `std/http`)

HTTP Server Example (`examples/http_router_sim/`)