Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 104 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
name: CI

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
package:
name: Julia ${{ matrix.julia-version }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 75
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, macos-14, windows-2022]
julia-version: ['1.10.11', '1']
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: julia-actions/setup-julia@fa02766e078afaaf09b14210362cee14137e6a32 # v3
with:
version: ${{ matrix.julia-version }}
- uses: julia-actions/cache@a45e8fa8be21c18a06b7177052533149e61e9b38 # v3
with:
delete-old-caches: false
- name: Instantiate the pinned environment
run: julia --project=. -e 'using Pkg; Pkg.instantiate(); Pkg.precompile()'
- uses: julia-actions/julia-runtest@6e050c8013b833b1195105ff2fce9cd802f53271 # v1
with:
coverage: false

compat-floor:
name: Manifest-free resolution / Julia 1.10.11
runs-on: ubuntu-22.04
timeout-minutes: 45
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: julia-actions/setup-julia@fa02766e078afaaf09b14210362cee14137e6a32 # v3
with:
version: '1.10.11'
- uses: julia-actions/cache@a45e8fa8be21c18a06b7177052533149e61e9b38 # v3
with:
delete-old-caches: false
- name: Resolve and test without the checked-in manifest
run: julia --project=. -e 'using Pkg; rm("Manifest.toml"; force=true); Pkg.instantiate(); Pkg.test()'

tool-smoke:
name: Tool and template smoke tests
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
BRAINLESSLAB_AUTOTHREADS: '0'
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: julia-actions/setup-julia@fa02766e078afaaf09b14210362cee14137e6a32 # v3
with:
version: '1.10.11'
- uses: julia-actions/cache@a45e8fa8be21c18a06b7177052533149e61e9b38 # v3
with:
delete-old-caches: false
- name: Instantiate benchmark environment
run: julia --project=bench -e 'using Pkg; Pkg.instantiate()'
- name: Benchmark statistics and execution smoke
run: |
julia --project=bench bench/test_stats.jl
julia --project=bench -e 'include("bench/Benchmark.jl"); using .Benchmark; cfg = Benchmark.read_bench_config("bench/configs/smoke.toml"); out = Benchmark.run_benchmark(cfg; out_root=mktempdir()); @assert isfile(joinpath(out.dir, "summary.csv"))'
- name: Instantiate profile environment
run: julia --project=profile -e 'using Pkg; Pkg.instantiate()'
- name: Profile execution smoke
run: julia --project=profile -e 'include("profile/Profile.jl"); using .NodeProfile; out = NodeProfile.node_profile(:falandays; tasks=(:tracking,), n_seeds=1, canonical_N=Dict(:tracking => 12), gifs=false, out_root=mktempdir()); @assert isfile(out.metrics)'
- name: Sweep execution smoke
run: julia --project=. -e 'using BrainlessLab; out = run_sweep("configs/ci_sweep.toml"; root=mktempdir()); @assert isfile(out.results)'
- name: Unified plan and record smoke
run: |
julia --project=. bin/brainlesslab.jl check plans/examples/benchmark_core.toml
julia --project=. bin/brainlesslab.jl run plans/examples/profile_tracking.toml --root "${{ runner.temp }}/brainlesslab-records"
- name: Instantiate and execute project template
run: |
julia --project=examples/templates/new_project -e 'using Pkg; Pkg.develop(path=pwd()); Pkg.instantiate()'
julia --project=examples/templates/new_project examples/templates/new_project/run.jl --ticks 20 --n-nodes 12 --out "${{ runner.temp }}/brainlesslab-template-smoke"
julia --project=examples/templates/new_project examples/templates/new_project/run_plan.jl examples/templates/new_project/config.toml "${{ runner.temp }}/brainlesslab-template-records"

documentation:
name: Locked documentation build
runs-on: ubuntu-22.04
timeout-minutes: 20
defaults:
run:
working-directory: site
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.11
- run: bun install --frozen-lockfile
- run: bun run build
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ examples/output/
/experiments/runs/
/experiments/results/homeostatic_needs_v2/
/bench/genomes/
/bench/Manifest.toml
/bench/output/
/examples/templates/new_project/output/
/examples/templates/new_project/Manifest.toml
Expand Down
38 changes: 23 additions & 15 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,19 @@ human-readable guide; update it when the public contract changes.

## Preserve the scientific boundary

- `:falandays` is the canonical fixture-validated baseline; `:falandays_base` remains a
compatibility alias. Do not change their shared behavior, fixtures, or fidelity language
to make another change pass.
- Everything outside that baseline is experimental unless evidence says otherwise.
- `:falandays` is the canonical node validated on declared reference trajectories. Do not
change its behaviour, fixtures, or validation language to make another change pass.
- Each other node, task, analysis, and component has its own declared stability and
readiness. Do not inherit the Falandays validation boundary.
- A task score operationalizes performance on that task. It is not, by itself, evidence of
cognition, general capability, biological fidelity, or external validity.
- Use `task_outcome(sim)` for the task-declared outcome. Report its key, raw value, and
normalized value together; `nothing` means the task declares no scalar objective. Treat
normalised value together; `nothing` means the task declares no scalar objective. Treat
other metric fields as diagnostics unless the task contract says otherwise.
- Never present development seeds, tuned cells, representative runs, or exploratory plots
as sealed evidence.
- Do not inspect a sealed evaluation set to answer a planning or debugging question.
- Use the independent randomized block or trial as the inferential unit. Agents and ticks
- Use the independent randomised block or trial as the inferential unit. Agents and ticks
within one world do not create additional independent samples.

## Work safely
Expand All @@ -41,23 +41,24 @@ human-readable guide; update it when the public contract changes.
6. Use stable component and entity IDs; do not infer identity from tuple or vector position.
7. Use `apply_patch` for hand edits. Avoid destructive Git commands.
8. Run the narrowest relevant tests first, then the full package and site gates when the
public surface changes.
public interface changes.
9. Report what is verified, what is inferred, and what remains experimental.

## Keep software readiness separate from study evidence

- The canonical handbook lives under `site/src/content/docs/core/`; it documents stable
composition contracts.
- The public guide lives under `site/src/content/docs/`. Core pages document the main
composition and research interfaces.
- Experimental capabilities are listed under `site/src/content/docs/experimental/` with
repository-backed source, example, and test metadata.
- `available` and `integrated` describe software readiness. They do not validate a
biological interpretation, promote a study, or increase its evidence status.
- Experiment evidence (`exploratory`, `tuned`, `frozen`, `confirmed`, `promoted`, or
`retired`) belongs to the study record and is independent of component readiness.
- Experiment evidence (`planned`, `exploratory`, `tuned`, `frozen`, `confirmed`,
`promoted`, or `retired`) belongs to the versioned `ExperimentSpec` and its records. It
is independent of component readiness.

## Choose the narrowest extension

| Intent | Preferred seam |
| Intent | Preferred approach |
| --- | --- |
| Change parameters of an existing reservoir | config or registered preset |
| Add a genuinely different neural substrate | `Reservoir` methods plus node registration |
Expand All @@ -81,12 +82,16 @@ Use the evidence ladder in `site/src/content/docs/core/design-study.mdx`:
conformance → calibration → exploration → tuning/training → variance pilot → frozen
protocol → sealed confirmation → robustness → promoted evidence.

For causal comparisons, share randomized worlds within a paired block while giving distinct
For causal comparisons, share randomised worlds within a paired block while giving distinct
mechanisms their declared streams. Keep training, selection, variance-pilot, confirmation,
and robustness seeds disjoint. The null must match the claim: random action, blind input,
shifted or sham input, an ablation, a baseline model, and an oracle answer different
questions.

Use an `ExperimentSpec` when a study needs a stable question, version, named conditions,
limitations, and one or more operations. Keep these bundles under `experiments/`. Do not
revive the archived bespoke experiment runner or add another operation-specific schema.

## Verification

For Julia changes:
Expand All @@ -96,7 +101,7 @@ julia --project=. -e 'using Pkg; Pkg.test()'
```

Also run focused contract tests while iterating. For hot paths, warm the call before checking
inference or allocations. Fixture parity is a separate gate from behavioral performance.
inference or allocations. Fixture parity is a separate gate from behavioural performance.

For site changes:

Expand All @@ -111,7 +116,7 @@ For public changes, also check:
- `Test.detect_ambiguities(BrainlessLab; recursive=true)` has no new ambiguity;
- examples referenced by docs execute;
- no absolute local paths or retired study references remain;
- README, site, examples, and both skills use the same vocabulary.
- README, site, examples, and both skills use the same vocabulary;
- canonical Core routes and Experimental feature metadata resolve.

## Documentation standard
Expand All @@ -129,3 +134,6 @@ guide should say:

Do not describe planned automation as implemented. Keep current limits visible and link to
`site/src/content/docs/platform-limits.mdx`.

Follow `docs/WRITING.md`: use British English, stable technical terms, soft-STE sentence
control, and minimal decorative emphasis.
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Changelog

## 0.2.0 — 2026-07-22

BrainlessLab 0.2.0 is an experimental research preview and the first typed research-platform release.

- Add typed node, task, composition, evaluation, operation, and experiment contracts.
- Add one version-one TOML plan schema for profile, sweep, ablate, evolve, and benchmark.
- Add portable research records with raw CSV data, seed ledgers, checksums, statistics,
resolved provenance, and generated HTML reports.
- Add four experimental Plank CartPole profiles while keeping Tracking and Pong as the
core qualification benchmark.
- Add checked-in operation plans, a unified CLI, and an external-project template.
- Align package and citation metadata on version 0.2.0.
- Add reproducible package, compatibility-floor, tool-smoke, and documentation CI.
- Add package-quality checks without weakening numerical conformance or calibration tests.
- Clarify repository installation and the pre-1.0 stability boundary.

## 0.1.0 — 2026-07-04

The public tag named `v0.1.0` was created from code whose `Project.toml` still reported
version `0.0.1`. That historical tag remains immutable; 0.2.0 is the first release in
which the package and citation versions are aligned.
13 changes: 7 additions & 6 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
cff-version: 1.2.0
message: >-
If you use BrainlessLab, please cite this software, and also cite the
original Falandays et al. work that the :falandays_base model reimplements.
original Falandays et al. work on which the :falandays model is based.
title: BrainlessLab.jl
abstract: >-
An extensible Julia lab for "brainless" cognition — behaviour that emerges
from collectives of simple neuron-like nodes — built around an authors-faithful
reimplementation of the Falandays et al. homeostatic spiking reservoir.
An experimental Julia research platform for behaviour that emerges from
collectives of simple neuron-like nodes, built around a Falandays et al.
homeostatic spiking reservoir that is validated on declared reference
trajectories.
type: software
authors:
- given-names: Polyphony
Expand All @@ -17,8 +18,8 @@ authors:
family-names: Jackson
- given-names: William
family-names: O'Hearn
version: 0.0.1
date-released: "2026-07-04"
version: 0.2.0
date-released: "2026-07-22"
license: MIT
repository-code: "https://github.com/btgaskin/brainless-lab"
url: "https://brainless-lab.pages.dev"
Expand Down
21 changes: 11 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ You do not need to know Julia before contributing. You can begin by running an e
simulation, improving a task description, checking an example, or working with a coding
agent. The repository contains guidance for both humans and agents.

Start with the online [Getting started](https://brainless-lab.pages.dev/getting-started/)
Start with the online [Getting started](https://brainless-lab.pages.dev/core/getting-started/)
guide. If you are using an agent, point it at `AGENTS.md`; that file tells it which
repository skills and safeguards to follow.

Expand All @@ -20,12 +20,13 @@ repository skills and safeguards to follow.
- **Add a body component:** extend the narrow physical interface and add strict config
materialization, an example, and conformance evidence.
- **Add a metric or analysis:** declare its input channels, unit of analysis, diagnostics,
valid null, failure behavior, and scientific limitations.
- **Add an experiment:** use only public BrainlessLab APIs and follow the evidence ladder in
the [Research workflow](https://brainless-lab.pages.dev/research-workflow/).
valid null, failure behaviour, and scientific limitations.
- **Add an experiment:** compose named conditions and typed operations in an
`ExperimentSpec`, then follow the evidence ladder in
[Design a study](https://brainless-lab.pages.dev/core/design-study/).

The [Extending](https://brainless-lab.pages.dev/extending/) page maps every public family to
its contract, example, and required tests.
[Extend the lab](https://brainless-lab.pages.dev/core/extend/) maps each public interface to
its example and required tests.

## Local setup

Expand All @@ -49,12 +50,12 @@ The first run compiles the package and can be slower than later runs.
## Change discipline

1. Create a branch or isolated worktree.
2. Reproduce the current behavior before editing.
2. Reproduce the current behaviour before editing.
3. Add or update the narrow contract test.
4. Implement through public dispatch boundaries.
5. Run focused tests, then the full applicable gates.
6. Update the canonical site page and executable example.
7. Keep exploratory output out of the evidence surface.
7. Keep exploratory output out of promoted research records.

Do not modify fidelity fixtures, committed evidence, or a sealed protocol merely to make a
new implementation agree with an expectation. If a scientific expectation changes, explain
Expand All @@ -75,7 +76,7 @@ bun run build
```

The relevant change should also have a focused test. New hot-loop code should be checked
after warmup for inference and avoidable allocations. New stochastic behavior needs reset,
after warm-up for inference and avoidable allocations. New stochastic behaviour needs reset,
replay, stream-ownership, and iteration-order tests.

## Pull-request handoff
Expand All @@ -89,4 +90,4 @@ State:
- the evidence status of scientific outputs;
- current limitations and follow-up work.

Keep implementation conformance, behavioral observations, and scientific claims separate.
Keep implementation conformance, behavioural observations, and scientific claims separate.
Loading