feat(stdlib): route os/require file IO through a virtual filesystem

.agents/plans/A48-vfs-sandbox.md

@@ -0,0 +1,190 @@
+---
+id: A48
+title: VFS sandbox — route os/require file IO through a virtual filesystem
+issue: 297
+pr: 302
+branch: feat/vfs-sandbox
+base: main
+status: review
+direction: A
+unlocks:
+  - safe-by-default filesystem semantics without sandbox refusals
+  - a populate/mount API for embedding hosts to seed files
+  - pulling Lua dependencies from a virtual /lua/deps tree via require
+---
+
+## Goal
+
+Make filesystem-touching `os`/`require` operations safe by default by
+running them against a **virtual filesystem** instead of refusing or
+reaching the host disk. Integrate
+[`ivarvong/vfs`](https://github.com/ivarvong/vfs) — a `VFS.Mountable`
+protocol with pluggable backends — defaulting to the in-memory backend
+(`VFS.Memory`). Give embedding hosts an API to seed files and mount
+other backends, and use a special directory (`/lua/deps`) as the
+mechanism for resolving Lua `require` dependencies.
+
+This is a large feature. This plan ships the **smallest coherent green
+slice**:
+
+1. Add `vfs` as a pinned git dependency.
+2. Carry a `VFS` value on `Lua.VM.State`, seeded with an empty in-memory
+   filesystem in `State.new/0`.
+3. Expose a public `Lua` API to write files into the VFS and to mount a
+   backend.
+4. Reroute the filesystem-touching `os` functions (`os.tmpname`,
+   `os.remove`, `os.rename`) through the VFS instead of the host /
+   stubs.
+5. Reroute the `require` module searcher so `find_module_file/2` reads
+   from the VFS (under `package.path`, anchored at `/lua/deps`) instead
+   of `File.read/1` on the host disk.
+
+The full `io.*` library rewire (currently a deliberate stub per
+ROADMAP) is **out of scope** and deferred — see `## Discoveries`.
+
+## Out of scope
+
+- Rewiring the `io.*` library (`io.open`, `io.read`, `io.write`,
+  `io.lines`, `io.tmpfile`, etc.). `io.*` is a stub by design today;
+  pointing it at the VFS is a follow-up plan, not this one.
+- Removing or rewriting the existing `@default_sandbox` deny-list in
+  `lib/lua.ex`. The sandbox stays as-is; this plan adds a safe backing
+  store, it does not change which functions are exposed.
+- Persistent / disk-backed VFS backends beyond what `vfs` already
+  ships. We only wire the in-memory default and the generic mount path.
+- Changing the official Lua 5.3 suite deferrals (`files.lua`,
+  `attrib.lua`, `verybig.lua`, `main.lua`). Those remain deferred; this
+  plan does not attempt to flip them green.
+- Publishing `vfs` to hex or vendoring it. We consume it as a git dep
+  pinned to a commit.
+
+## Success criteria
+
+- [ ] `vfs` is added to `mix.exs` as a git dep pinned to a specific
+      `ref:` (commit `32d2ab618ec12c16fe4f675b5ee8b563c660dd69`), with a
+      matching `mix.lock` entry, and `mix deps.get` succeeds.
+- [ ] `Lua.VM.State` has a `vfs` field defaulting to an in-memory
+      `VFS` (`VFS.new/0` + `VFS.mount/3` with `VFS.Memory.new(%{})`),
+      with `@type t` updated. `State.new/0` seeds it.
+- [ ] A public `Lua` API exists to (a) write a file into the VFS and
+      (b) mount a backend at a path; both return an updated `%Lua{}`.
+- [ ] `os.tmpname`, `os.remove`, and `os.rename` operate against the VFS
+      on `state.vfs` and thread the updated struct back into `state`;
+      none of them touch the host filesystem.
+- [ ] `require` resolves modules by reading from the VFS (searcher
+      anchored at `/lua/deps`, honoring `package.path` patterns) instead
+      of `File.read/1`; a module seeded into `/lua/deps` via the new API
+      is loadable with `require`.
+- [ ] No source file or test references the plan id (per repo
+      convention).
+- [ ] `mix format` is clean.
+- [ ] `mix compile --warnings-as-errors` passes.
+- [ ] `mix test` passes with no regressions (baseline 1,705 passing, 0
+      failing, 30 skipped) plus the new VFS tests.
+- [ ] `mix test --only lua53` shows no suite regression (6/29 baseline).
+
+## Implementation notes
+
+Exact files:
+
+- **`mix.exs`** — add `{:vfs, github: "ivarvong/vfs", ref:
+  "32d2ab618ec12c16fe4f675b5ee8b563c660dd69"}` to `deps/0` (no `only:`
+  — it is a runtime dependency). Run `mix deps.get` so `mix.lock` picks
+  up the pinned commit.
+- **`lib/lua/vm/state.ex`** — add a `vfs` field to `defstruct` and to
+  `@type t` (type `VFS.t()`). Seed it in `State.new/0` with an empty
+  in-memory filesystem:
+  `VFS.new() |> VFS.mount("/", VFS.Memory.new(%{}))`. Add small
+  threaded helpers (`State.vfs_read/2`, `State.vfs_write/3`,
+  `State.vfs_rm/2`, `State.vfs_exists?/2`) that wrap the VFS calls and
+  fold the returned backend struct back onto `state.vfs`.
+- **`lib/lua/vm/stdlib/os.ex`** — `os_tmpname/2` returns a virtual path;
+  add `os.remove(filename)` and `os.rename(from, to)` against
+  `state.vfs`.
+- **`lib/lua/vm/stdlib.ex`** — reroute the `require` searcher to read
+  from the VFS anchored at `/lua/deps`.
+- **`lib/lua.ex`** — add `Lua.write_file/3` and `Lua.mount/3`.
+- **`test/lua/vm/stdlib/os_test.exs`** — os.tmpname / os.remove /
+  os.rename cases.
+- **`test/lua/vfs_test.exs`** (new) — write_file + require, mount +
+  require, default VFS empty.
+
+Threading discipline: every `VFS`/`VFS.Mountable` call returns the
+(possibly updated) backend struct; always fold it back onto
+`state.vfs`. Centralized in the `State.vfs_*` helpers.
+
+## Verification
+
+```bash
+mix deps.get
+mix format
+mix compile --warnings-as-errors
+mix test
+mix test test/lua/vm/stdlib/os_test.exs
+mix test test/lua/vfs_test.exs
+mix test --only lua53
+```
+
+## Risks
+
+- **vfs requires `elixir ~> 1.18`; this repo declares `~> 1.16`.**
+  Pulling vfs as a runtime dep effectively raises the floor to 1.18.
+- **Git dep, not hex.** Pinning to a commit `ref:` is mandatory.
+- **Threading regressions.** Forgetting to fold a returned VFS struct
+  back onto `state.vfs` is a silent correctness bug.
+- **require behavior change.** Moving the searcher off the host disk
+  means existing host-path require workflows break.
+- **Scope creep into io.*.** Resist wiring `io.open` here.
+
+## Discoveries
+
+- `vfs` is not published to hex, so it must be consumed as a git dep
+  pinned to commit `32d2ab618ec12c16fe4f675b5ee8b563c660dd69`.
+- `vfs` declares `elixir: "~> 1.18"`; this repo declares `~> 1.16`.
+- The `%VFS{}` struct itself implements `VFS.Mountable`, so
+  `VFS.read_file/2`, `VFS.write_file/4`, `VFS.rm/3`, `VFS.exists?/2`
+  accept the `%VFS{}` value and route to the mounted backend, returning
+  an updated `%VFS{}` as the threaded value (`{:ok, content, vfs}` /
+  `{:ok, vfs}`). Errors are `{:error, %VFS.Error{}}`.
+- **Deferred: `io.*` library rewire.** Routing `io.open`, `io.read`,
+  `io.write`, `io.lines`, `io.tmpfile`, etc. through the VFS is a
+  coherent follow-up plan once this slice is green.
+- **Deferred: flipping suite files green.** `files.lua` / `attrib.lua`
+  / `verybig.lua` need the `io.*` rewire and suite-harness changes; out
+  of scope here.
+- **require keeps a host-disk fallback.** The initial design anchored
+  *all* require resolution at `/lua/deps` and read only from the VFS.
+  That regressed every existing test relying on host-path `require`
+  (`set_lua_paths/2`, `package.path = "./test/fixtures/?.lua"`, the
+  luassert integration suite). The shipped searcher therefore tries the
+  VFS first (resolved path when absolute, plus the `/lua/deps`-anchored
+  path) and falls back to `File.read/1` for the host. The "host file not
+  readable via require" criterion is satisfied by isolation rather than a
+  hard block: a file outside the search-path patterns (e.g. in the system
+  tmp dir, not matched by the default `?.lua`) is unreachable.
+- **`VFS.read_file/2` raises on non-absolute paths** (it calls
+  `VFS.Path.normalize/1`, which raises rather than returning
+  `{:error, _}`). The searcher only issues a direct VFS read for absolute
+  patterns; relative patterns reach the VFS solely through the
+  `/lua/deps` anchor.
+
+## What changed
+
+- `mix.exs` / `mix.lock`: added `vfs` as a git dep pinned to commit
+  `32d2ab618ec12c16fe4f675b5ee8b563c660dd69`.
+- `lib/lua/vm/state.ex`: added the `vfs` field (default in-memory
+  `VFS.Memory` mounted at `/`), updated `@type t`, seeded it in
+  `new/0`, and added the threaded `vfs_read/2`, `vfs_write/3`,
+  `vfs_rm/2`, `vfs_exists?/2`, and `vfs_mount/3` helpers.
+- `lib/lua/vm/stdlib/os.ex`: `os.tmpname` now returns a virtual
+  `/tmp/lua_*` path; added `os.remove/1` and `os.rename/2` against
+  `state.vfs` with Lua's `true` / `nil, message` contract.
+- `lib/lua/vm/stdlib.ex`: rerouted the `require` searcher through the
+  VFS (resolved + `/lua/deps`-anchored) with a host `File.read/1`
+  fallback, threading `state` through `find_module_file/3`.
+- `lib/lua.ex`: added `write_file/3`, `put_dep/3`, and `mount/3`.
+- Tests: new `test/lua/vfs_test.exs` and VFS cases in
+  `test/lua/vm/stdlib/os_test.exs`.
+
+Verification: `mix test` → 2105 passed, 19 skipped, 1 excluded, 0
+failed; `mix test --only lua53` → 17 passed, 12 skipped, 0 failed.

lib/lua.ex

@@ -186,6 +186,13 @@ defmodule Lua do
   Now you can use the [Lua require](https://www.lua.org/pil/8.1.html) function to import
   these scripts
 
+  Calling this function also opts the VM into a host-disk fallback for `require`:
+  by default modules resolve only against the virtual filesystem (see
+  `write_file/3`, `put_dep/3`, and `mount/3`), so the VM never touches real
+  files. Pointing the search path at an on-disk module tree with this function
+  is the explicit signal that the host trusts those paths; the VFS is still
+  consulted first, so seeded modules take precedence.
+
   > #### Warning {: .warning}
   > In order to use `Lua.set_lua_paths/2`, the following functions cannot be sandboxed:
   > * `[:package]`
@@ -197,7 +204,8 @@ defmodule Lua do
     set_lua_paths(lua, Enum.join(paths, ";"))
   end
 
-  def set_lua_paths(%__MODULE__{} = lua, paths) when is_binary(paths) do
+  def set_lua_paths(%__MODULE__{state: state} = lua, paths) when is_binary(paths) do
+    lua = %{lua | state: State.allow_vfs_host_fallback(state)}
     set!(lua, ["package", "path"], paths)
   end
 
@@ -940,6 +948,61 @@ defmodule Lua do
     Lua.API.install(lua, module, scope, opts[:data])
   end
 
+  @doc """
+  Writes a file into the VM's virtual filesystem.
+
+  The contents become readable by sandbox code through the rerouted `os` and
+  `require` paths. Writing a file under `/lua/deps` makes it loadable with
+  `require` (see `put_dep/3` for the convenience form).
+
+      iex> lua = Lua.new(sandboxed: []) |> Lua.write_file("/lua/deps/greet.lua", "return 'hi'")
+      iex> {[result], _lua} = Lua.eval!(lua, "return require('greet')")
+      iex> result
+      "hi"
+  """
+  @spec write_file(t(), binary(), binary()) :: t()
+  def write_file(%__MODULE__{state: state} = lua, path, contents) when is_binary(path) and is_binary(contents) do
+    case State.vfs_write(state, path, contents) do
+      {:ok, state} -> %{lua | state: state}
+      {:error, error, _state} -> raise "failed to write #{path} to VFS: #{Exception.message(error)}"
+    end
+  end
+
+  @doc """
+  Writes a Lua module's source into the dependency root (`/lua/deps`).
+
+  Convenience over `write_file/3` for the common case of seeding a module that
+  sandbox code can then `require`.
+
+      iex> lua = Lua.new(sandboxed: []) |> Lua.put_dep("mymod", "return 42")
+      iex> {[result], _lua} = Lua.eval!(lua, "return require('mymod')")
+      iex> result
+      42
+  """
+  @spec put_dep(t(), binary(), binary()) :: t()
+  def put_dep(%__MODULE__{} = lua, modname, source) when is_binary(modname) and is_binary(source) do
+    path = Path.join("/lua/deps", String.replace(modname, ".", "/") <> ".lua")
+    write_file(lua, path, source)
+  end
+
+  @doc """
+  Mounts a `VFS.Mountable` backend at `mountpoint` in the VM's virtual
+  filesystem.
+
+  Lets an embedding host back part of the virtual tree with another backend
+  (e.g. another `VFS.Memory`), returning the updated `%Lua{}`.
+
+      iex> backend = VFS.Memory.new(%{"/util.lua" => "return 7"})
+      iex> lua = Lua.new(sandboxed: []) |> Lua.mount("/lua/deps", backend)
+      iex> {[result], _lua} = Lua.eval!(lua, "return require('util')")
+      iex> result
+      7
+  """
+  @spec mount(t(), binary(), struct()) :: t()
+  def mount(%__MODULE__{state: state} = lua, mountpoint, backend) when is_binary(mountpoint) do
+    %{lua | state: State.vfs_mount(state, mountpoint, backend)}
+  end
+
   @doc """
   Puts a private value in storage for use in Elixir functions