Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 141 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ emacs-backpack/
├── early-init.el # Emacs entry point; loads backpack.el, calls backpack-start
├── ensure.el # Batch-mode package sync (backpack ensure)
├── gc.el # Batch-mode orphan package cleanup (backpack gc)
├── bench.el # Batch-mode benchmark entry (backpack bench)
├── lisp/ # Core Backpack Emacs Lisp library
│ ├── backpack.el # Main module: bootstrap, startup optimisations, orchestration
Expand Down Expand Up @@ -120,6 +121,8 @@ emacs-backpack/
│ │ └── vterm.el
│ ├── ai/
│ │ └── anvil.el
│ └── os/
│ └── windows.el
│ └── editing/
│ ├── c.el
│ ├── cmake.el
Expand Down Expand Up @@ -149,13 +152,20 @@ emacs-backpack/
│ └── leaf-keywords.el/ # Extended leaf keywords
├── bin/
│ └── backpack # Shell CLI (backpack ensure, backpack gc)
│ ├── backpack # Shell CLI for Unix (ensure, gc, bench)
│ └── backpack.cmd # Batch CLI for Windows (ensure, gc, bench)
├── test/
│ ├── all-tests.el # Test runner
│ ├── startup-time.el # Startup time benchmark
│ └── pouch/
│ └── backpack-pouch.el # Unit tests for gear!/gearp! and backpack--extract-gear-form
│ ├── pouch/
│ │ └── backpack-pouch.el # Unit tests for gear!/gearp! and backpack--extract-gear-form
│ ├── treesit/
│ │ └── backpack-treesit.el # Unit tests for tree-sitter grammar handling
│ └── benchmark/ # A/B performance benchmark suite (backpack bench)
│ ├── backpack-bench-harness.el # Timing, stats, table & history output
│ ├── backpack-bench-defs.el # Windows-focused A/B benchmark pairs
│ └── run.el # Batch entry point for `backpack bench'
├── etc/scripts/
│ ├── prepare-and-run.sh # Test helper: copy config to tmpdir, run tests
Expand Down Expand Up @@ -236,7 +246,8 @@ activation are guaranteed to run.
### Batch sync mode (`backpack ensure`)

```
bin/backpack ensure
bin/backpack ensure # Unix
bin\backpack.cmd ensure # Windows
→ emacs --batch -l ensure.el
├─ Sets backpack-mode to 'sync
├─ Loads backpack.el, user init.el
Expand All @@ -250,13 +261,27 @@ bin/backpack ensure
### Garbage collection (`backpack gc`)

```
bin/backpack gc [--dry-run]
bin/backpack gc [--dry-run] # Unix
bin\backpack.cmd gc [--dry-run] # Windows
→ emacs --batch -l gc.el
├─ Collects declared packages from gear files
├─ Compares against installed packages
└─ Deletes orphaned packages (or reports in dry-run)
```

### Benchmarks (`backpack bench`)

```
bin/backpack bench # Unix
bin\backpack.cmd bench # Windows
→ emacs --batch -l bench.el
├─ Captures the real repo root before backpack-defaults rebinds user-emacs-directory
├─ Loads lisp/backpack.el (elpaca warnings suppressed -- bench runs without packages)
├─ Loads test/benchmark/run.el which registers specs and runs the suite
├─ Prints a human-readable A/B table to stdout
└─ Appends a timestamped .eld history file under .cache/etc/benchmarks/
```

## Gear File Conventions

### Structure of a gear file
Expand Down Expand Up @@ -429,6 +454,28 @@ restores it.
The side window routing uses `backpack--display-eldoc-side-window`, registered
in `display-buffer-alist` for the `*eldoc*` buffer.

#### windows (os)

The windows gear applies performance optimizations for GNU Emacs running natively
on Windows. It is opt-in — users add `(gear! :os windows)` to their configuration.

Settings applied (grouped by category):

- **Process/subprocess**: `w32-pipe-read-delay` → 0, `w32-pipe-buffer-size` → 64KB,
`w32-quote-process-args` → t, `process-connection-type` → nil (pipes, not ptys),
`read-process-output-max` → 64KB
- **File I/O**: `w32-get-true-file-attributes` → nil, `w32-get-true-file-link-count` → nil
- **Display**: `inhibit-compacting-font-caches` → t, `fast-but-imprecise-scrolling` → t,
`cursor-in-non-selected-windows` → nil, `highlight-nonselected-windows` → nil,
`auto-mode-case-fold` → nil
- **Bidi**: suppressed by default (`left-to-right`), opt out with the `bidi` flag
- **Native comp**: `native-comp-defer-time` → 1.0
- **Network**: `url-http-attempt-keepalives` → t
- **Shell**: auto-detects bash (Git for Windows / MSYS2) and sets `shell-file-name`

The `bidi` flag is **opt-in** (bidi is suppressed by default on Windows for performance).
Users who need RTL text support add `(gear! :os (windows bidi))`.

#### anvil (ai)

The anvil gear exposes Emacs as an MCP (Model Context Protocol) server so AI
Expand Down Expand Up @@ -556,6 +603,95 @@ The `devenv.nix` environment provides Emacs versions 29.1, 29.2, 29.3, 29.4,
30.1, and rolling. CI runs `devenv test` which executes
`etc/scripts/for-each-emacs.sh` to test against all versions.

## Benchmarks

The benchmark suite is separate from the ERT unit tests. It lives under
`test/benchmark/` and is invoked via the `bench` CLI subcommand. Unlike
`devenv test`, the benchmark suite is **not** run automatically in CI; it
is a developer tool for validating that performance tweaks (in particular
those in the `:os windows` gear) actually earn their line of code.

### Running

```sh
bin/backpack bench # Unix
bin\backpack.cmd bench # Windows
```

Useful environment variables (all optional):

| Variable | Effect |
|------------------------------|-----------------------------------------------------------|
| `BACKPACK_BENCH_FILTER` | Substring match; only run benchmarks whose name contains it |
| `BACKPACK_BENCH_ITERATIONS` | Override the iteration count of every registered bench |
| `BACKPACK_BENCH_NO_SAVE` | Non-empty string skips writing the `.eld` history file |

A full run takes a few minutes on Windows (the subprocess benches dominate).
For iterative work, use `BACKPACK_BENCH_FILTER=subprocess` (or similar) and a
small `BACKPACK_BENCH_ITERATIONS=3`.

### What is measured

Every benchmark is an **A/B pair**: variant A runs with stock Emacs defaults,
variant B with the values installed by Backpack's `:os windows` gear. The
harness reports the delta (both absolute milliseconds and percent) so you
can see at a glance whether a knob is helping, hurting, or merely noise.

The benches exercise the three dimensions the Windows gear targets:

| Benchmark | Primary knobs probed |
|---------------------------------|-------------------------------------------------------------------------------------|
| `bench-startup` | Cold startup cost of a child `emacs --batch -Q` loading early-init.el |
| `bench-file-attributes` | `w32-get-true-file-attributes`, `w32-get-true-file-link-count` |
| `bench-directory-scan` | Same, at recursive-walk scale |
| `bench-load-path-probe` | `file-name-handler-alist`, `auto-mode-case-fold` |
| `bench-auto-mode-match` | `auto-mode-case-fold` / `case-fold-search` during `auto-mode-alist` matching |
| `bench-subprocess-echo` | `process-connection-type`, `w32-pipe-read-delay`, `w32-pipe-buffer-size` |
| `bench-subprocess-large-output` | `w32-pipe-buffer-size`, `read-process-output-max`, `process-adaptive-read-buffering` |

On non-Windows hosts the W32-specific variables are declared special via
stub `defvar`s so `let`-binding them compiles clean; the underlying C code
simply ignores them. Those benches therefore report ~0 delta, which is a
cheap smoke check that the gear is not accidentally regressing other
platforms.

### Output

Two outputs are produced on every run:

1. A human-readable ASCII table printed to stdout.
2. A timestamped plist file under `.cache/etc/benchmarks/bench-YYYYMMDD-HHMMSS.eld`
containing the full per-variant stats (min, max, median, mean, stddev, n)
together with environment metadata (timestamp, system type, Emacs version,
native-comp availability, host name). These files are `read`-able Elisp
so they can be diffed, plotted, or aggregated externally.

### Adding a new benchmark

Register via `backpack-bench-defbench` in `test/benchmark/backpack-bench-defs.el`:

```elisp
(backpack-bench-defbench bench-my-new-thing
:doc "One-line description of what is measured."
:iterations 15
:warmup 2
:available-p (lambda () (eq system-type 'windows-nt)) ; optional
:setup (lambda () ...) ; optional one-shot fixture
:run-a (lambda () ...) ; stock defaults variant
:run-b (lambda () ...)) ; Windows-optimized variant
```

Guidelines:

- Keep per-iteration work in the single-digit-ms to low-hundred-ms range so
noise does not dwarf the signal. Very short bodies should loop internally.
- `let`-bind the variables the benchmark is measuring *inside* `:run-a` /
`:run-b`, not in `:setup` -- this keeps the two variants symmetric and
makes the bench source easy to read.
- Use `backpack-emacs-dir` (not `user-emacs-directory`) for any path that
must resolve to the repo root; `backpack-defaults.el` rebinds
`user-emacs-directory` to live inside `.cache/` early in startup.

## Key Conventions and Rules

### General
Expand Down
49 changes: 49 additions & 0 deletions bench.el
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
;;; bench.el --- Backpack benchmarking entry point -*- lexical-binding: t; -*-
;;
;; Usage: emacs --batch --eval "(setq user-emacs-directory \"/path/to/backpack/\")" -l bench.el
;;
;; Invoked by `bin/backpack bench' (POSIX) and `bin\backpack.cmd bench' (Windows).
;; The benchmark suite does *not* need elpaca or any user gears to be loaded:
;; it performs its own minimal setup so results are reproducible regardless of
;; what is pinned in the user's init.el.
;;
;; The actual benchmarks live under `test/benchmark/' so they participate in the
;; same code-review and history as the unit tests.

;;; Code:

;; Capture the original user-emacs-directory *before* loading backpack.el.
;; backpack-defaults.el deliberately rebinds `user-emacs-directory' to the
;; .cache/ subdir to keep $HOME tidy, which would break path resolution here
;; if we relied on it after the load.
(defvar backpack-bench--repo-root
(file-name-as-directory (expand-file-name user-emacs-directory))
"Absolute path to the Backpack source tree, captured before backpack.el loads.")

;; Mirror the path dance ensure.el / gc.el do, so base-packages libraries are
;; on the load-path for anything the benchmark harness might pull in.
(add-to-list 'load-path (expand-file-name "base-packages/leaf.el" backpack-bench--repo-root))
(add-to-list 'load-path (expand-file-name "base-packages/leaf-keywords.el" backpack-bench--repo-root))
(add-to-list 'load-path (expand-file-name "lisp" backpack-bench--repo-root))
(add-to-list 'load-path (expand-file-name "test/benchmark" backpack-bench--repo-root))

;; Suppress the "Elpaca is not installed" display-warning that backpack.el
;; emits in normal mode when the user hasn't run `backpack ensure' yet.
;; The benchmark suite doesn't need elpaca: its microbenches exercise
;; variables and Lisp primitives, not packages. Requiring `warnings'
;; before mutating `warning-suppress-log-types' avoids the "defining as
;; dynamic an already lexical var" complaint under lexical-binding.
(require 'warnings)
(add-to-list 'warning-suppress-log-types '(backpack))
(let ((backpack-file (expand-file-name "lisp/backpack.el"
backpack-bench--repo-root)))
(load backpack-file nil nil nil t))

;; Kick off the suite. run.el executes at load time and writes its own
;; output; we just forward its exit status.
(load (expand-file-name "test/benchmark/run.el" backpack-bench--repo-root)
nil nil nil t)

(kill-emacs 0)

;;; bench.el ends here
16 changes: 15 additions & 1 deletion bin/backpack
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ if [ -z "$ACTION" ]; then
echo "Available actions:" >&2
echo " ensure Install and build all packages (without activation)" >&2
echo " gc [--dry-run] Remove orphaned packages no longer needed by configuration" >&2
echo " bench Run the Backpack benchmark suite (A/B vs :os windows)" >&2
exit 1
fi

Expand Down Expand Up @@ -55,8 +56,21 @@ elif [ "$ACTION" = "gc" ]; then
-l "$BACKPACK_DIR/gc.el"
exit $?

elif [ "$ACTION" = "bench" ]; then
echo "Backpack: Running benchmark suite..."
echo "Backpack directory: $BACKPACK_DIR"
echo ""
echo "Tip: set BACKPACK_BENCH_FILTER=substring to run a subset"
echo " set BACKPACK_BENCH_ITERATIONS=N to override the default sample count"
echo ""
# Run Emacs in batch mode with bench.el
emacs --batch \
--eval "(setq user-emacs-directory \"$BACKPACK_DIR/\")" \
-l "$BACKPACK_DIR/bench.el"
exit $?

else
echo "Error: unknown action '$ACTION'" >&2
echo "Available actions: ensure, gc" >&2
echo "Available actions: ensure, gc, bench" >&2
exit 1
fi
82 changes: 82 additions & 0 deletions bin/backpack.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
@echo off
setlocal enabledelayedexpansion

if "%~1"=="" goto :usage
set "ACTION=%~1"
set "FLAG=%~2"

REM --- Resolve Backpack root from script location ---
set "BACKPACK_DIR=%~dp0.."
for %%I in ("%BACKPACK_DIR%") do set "BACKPACK_DIR=%%~fI"

REM --- Verify installation ---
if not exist "%BACKPACK_DIR%\ensure.el" (
echo Error: This script must be run from the Backpack installation directory>&2
echo Expected to find ensure.el in: %BACKPACK_DIR%>&2
exit /b 1
)
if not exist "%BACKPACK_DIR%\base-packages" (
echo Error: This script must be run from the Backpack installation directory>&2
echo Expected to find base-packages\ in: %BACKPACK_DIR%>&2
exit /b 1
)

REM --- Convert backslashes to forward slashes for Emacs.
REM NOTE: do not name this BACKPACK_EMACS_DIR or anything that case-
REM insensitively matches "emacs_dir" -- on Windows that would shadow
REM Emacs's own reserved emacs_dir variable (see src/emacs.c
REM decode_env_path) and break data-directory / exec-directory
REM resolution, causing fatal "charsets: No such file or directory". ---
set "BACKPACK_DIR_FWD=%BACKPACK_DIR:\=/%"

REM --- Defensive: clear any inherited emacs_dir so a parent shell that
REM happens to export it cannot poison Emacs's path resolution. ---
set "emacs_dir="

if "%ACTION%"=="ensure" goto :ensure
if "%ACTION%"=="gc" goto :gc
if "%ACTION%"=="bench" goto :bench
echo Error: unknown action '%ACTION%'>&2
echo Available actions: ensure, gc, bench>&2
exit /b 1

:ensure
echo Backpack: Synchronizing packages...
echo Backpack directory: %BACKPACK_DIR%
echo.
emacs --batch --eval "(setq user-emacs-directory \"%BACKPACK_DIR_FWD%/\")" -l "%BACKPACK_DIR_FWD%/ensure.el"
exit /b %errorlevel%

:gc
set "DRY_RUN=nil"
if "%FLAG%"=="--dry-run" set "DRY_RUN=t"
if "%FLAG%"=="-n" set "DRY_RUN=t"
if "!DRY_RUN!"=="t" (
echo Backpack: Garbage collection ^(dry run^)...
) else (
echo Backpack: Garbage collection...
)
echo Backpack directory: %BACKPACK_DIR%
echo.
emacs --batch --eval "(setq user-emacs-directory \"%BACKPACK_DIR_FWD%/\")" --eval "(setq backpack-gc-dry-run !DRY_RUN!)" -l "%BACKPACK_DIR_FWD%/gc.el"
exit /b %errorlevel%

:bench
echo Backpack: Running benchmark suite...
echo Backpack directory: %BACKPACK_DIR%
echo.
echo Tip: set BACKPACK_BENCH_FILTER=substring to run a subset
echo set BACKPACK_BENCH_ITERATIONS=N to override the default sample count
echo.
emacs --batch --eval "(setq user-emacs-directory \"%BACKPACK_DIR_FWD%/\")" -l "%BACKPACK_DIR_FWD%/bench.el"
exit /b %errorlevel%

:usage
echo Error: missing argument>&2
echo Usage: backpack ^<action^> [flags]>&2
echo.>&2
echo Available actions:>&2
echo ensure Install and build all packages ^(without activation^)>&2
echo gc [--dry-run] Remove orphaned packages no longer needed>&2
echo bench Run the Backpack benchmark suite ^(A/B vs :os windows^)>&2
exit /b 1
10 changes: 9 additions & 1 deletion early-init.el
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@
(load backpack nil nil nil t))
(setq user-init-file (expand-file-name "early-init" user-emacs-directory))
(setq load-prefer-newer t)
(setq gc-cons-threshold (* 16 1024 1024))))
(if backpack--system-windows-p
;; ─── Garbage collection ───
;; Windows pipe/IPC buffers are large (256 KB + 1 MiB read-max);
;; the default 16 MiB post-init threshold triggers GC too often
;; during LSP indexing bursts. 64 MiB is a good balance.
;; See: https://emacs-lsp.github.io/lsp-mode/page/performance/
;; https://emacsredux.com/blog/2025/03/28/speed-up-emacs-startup-by-tweaking-the-gc-settings/
(setq gc-cons-threshold (* 64 1024 1024))
(setq gc-cons-threshold (* 16 1024 1024)))))

(backpack-start (not noninteractive))
Loading
Loading