NeuralCompose

A privacy-first, fully on-device macOS prototype for EEG-driven communication and sleep-cycle research. A Muse headband streams brain signals through BrainFlow, a Core ML classifier on the Apple Neural Engine detects intent (jaw clench / blink / rest / select), and a local MLX LLM suggests the next word. No cloud APIs. No telemetry. No network at runtime.

Live signal

Left: EEGScalpPlotterView, the 2D depth-stacked plotter, replaying the project's first golden recording — TP9/AF7/AF8/TP10 live from a Muse S. Right: NeuralWorkspaceView, the same session in 3D — node brightness tracks broadband RMS, elevation tracks theta-band power, edge tint and pulse track the live intent classifier's output.

Channel	Contact	Clipping	RMS	Overall
TP9	Excellent	0.65%	162.5 µV	Good
AF7	Excellent	0.85%	176.6 µV	Good
AF8	Excellent	0.94%	175.7 µV	Good
TP10	Excellent	0.34%	146.4 µV	Good

98 blink-like transients and 19 EMG bursts detected across the narrated protocol (eyes open/closed, blinks, jaw clenches, and a deliberate electrode-lift on each channel in turn). Full report — PSD, spectrogram, rolling band power, RMS timeline — and the raw recording's provenance: Recordings/golden/README.md.

This recording also backs Tests/BCIEEGTests/GoldenRecordingRegressionTests.swift: every test run replays it deterministically (see Playback & synchronization below) through the real windowing → feature-extraction → classifier → channel-health pipeline and checks the output against a committed reference.

On AF7: an earlier validation session (validate-muse-physiology.py) found AF7 saturated (~900 µV RMS) across 4 consecutive runs and read that as a hardware defect. It wasn't — headband tautness was the actual cause; once corrected, AF7 recorded as cleanly as the other three channels. Worth remembering before writing off a "bad" channel as broken hardware.

Project status (July 2026)

Status	Component
✅	Native BrainFlow integration (BLE + BCIBridge)
✅	Live Muse S acquisition (256 Hz, 4 channels)
✅	Communication mode (intent → carousel → MLX LLM)
✅	Phase B Sleep Validation Toolkit — 2D plotter + 3D live topography
✅	Deterministic playback (PlaybackEEGStream.normalized) + CI regression against a golden recording
✅	3D workspace driven entirely by live classifier output (no manual controls)
🚧	Sleep-stage classifier (4-class: Wake / N1 / N2_N3 / Uncertain_REM)
🚧	Dream-session controller + session FSM
🚧	LLM primer generation + dream-report analogy extraction
🧪	Cognitive-incubation experiments (pre-registration pending)

Architecture

                    ┌──────────────────────────────────────────────────────┐
                    │                NeuralComposeApp                       │
                    │  (SwiftUI: comms window, Phase B debug, menu-bar UI)  │
                    └─────┬──────────────────────────┬──────────────────────┘
                          │                          │ Cmd+Shift+D
                          ▼                          ▼
                ┌──────────────────┐        ┌─────────────────────────┐
                │ TextComposition  │        │  SleepValidationView    │
                │ Controller       │        │  (2D plotter, 3D scene) │
                └────────┬─────────┘        └─────────────────────────┘
                         ▼
                ┌──────────────────┐    ┌──────────────────┐
                │ IntentSmoother   │    │ EEGWindowing     │  (2s comms / 30s sleep)
                │ (BCICore actor)  │    │ (BCICore actor)  │
                └────────┬─────────┘    └────────┬─────────┘
                         ▼                       │
                ┌──────────────────┐             │
                │ Core ML on ANE   │◄────────────┘
                │ (BCIClassifier)  │
                └────────┬─────────┘
                         ▼
                ┌──────────────────┐
                │ EEGStreaming     │ ← BrainFlow / synthetic / playback
                │  (BCIEEG)        │
                └──────────────────┘

Module boundaries (MLX isolation is load-bearing):

• BCICore — pure-Swift models, protocols, FSMs, buffers. No third-party deps.
• BCIBridge — Obj-C++ shim for BrainFlow (stub by default, gated by BCI_BRAINFLOW_AVAILABLE).
• BCIEEG — BrainFlowService, SyntheticEEGStream, PlaybackEEGStream, EEGScalpPlotterView, NeuralWorkspaceView.
• BCIClassifier — Core ML wrapper + deterministic mock (also the CI classifier).
• BCILLM — MLX-Swift linked only here. Adapter + stub + tokenizer.
• NeuralComposeApp — SwiftUI views, Phase B debug window, menu-bar UI.

The app talks to BCILLM through NextWordPredicting, so there's exactly one MLX runtime copy in the linked binary.

Four-layer model

The codebase is organized into four layers, named for role (not current contents) so they remain meaningful as the platform grows:

┌─────────────────────────────────────────────────────┐
│                      Interface                      │
│  SwiftUI · SceneKit · Plotters · Channel-health UI  │
└────────────────────────▲────────────────────────────┘
                         │ depends on
┌────────────────────────┴────────────────────────────┐
│                     Intelligence                    │
│  DSP · Features · Classifier · Embeddings · Project │
└────────────────────────▲────────────────────────────┘
                         │ depends on
┌────────────────────────┴────────────────────────────┐
│                      Runtime                        │
│  EEGStreaming · AsyncMulticastChannel · Supervisors │
│  Recording · Diagnostics                            │
└────────────────────────▲────────────────────────────┘
                         │ depends on
┌────────────────────────┴────────────────────────────┐
│                  External Systems                   │
│   Muse · BrainFlow · OSC · Playback · Synthetic     │
└─────────────────────────────────────────────────────┘

  ▼ data flows downward

External Systems is whatever produces samples — the Muse over BrainFlow, a remote Muse over OSC, a recorded file in playback, a deterministic synthetic stream. The layer is named "External" rather than "Hardware" because playback and synthetic are not hardware; the shared property is "outside the process boundary of the analysis pipeline."

Runtime owns the streaming substrate: the single-owner EEGStreaming (see ADR-001), the AsyncMulticastChannel that distributes samples to multiple consumers, the supervisors that handle stalls and reconnects, the recording subsystem, and the transport diagnostics.

Intelligence is the analysis layer: feature extraction, the intent classifier, the (future) sentence embedder, the projection that turns a high-dimensional embedding into a 3D point. It does not know what produced the samples or what will render the output.

Interface is everything the user sees: SwiftUI windows, the SceneKit 3D workspace, the 2D plotter, the privacy indicator, the channel-health badge. It consumes Intelligence outputs and never imports Core ML or MLX directly.

The dependency direction is strictly downward: Interface depends on Intelligence, Intelligence depends on Runtime, Runtime depends on External Systems. A component that needs to know about a non-adjacent layer is a sign that either the data flow should be redesigned, or the missing protocol should be added at the layer boundary where the knowledge should live.

See docs/architecture/PRINCIPLES.md for the engineering values these layers implement, and docs/architecture/decision-log/ for the specific architectural decisions recorded under those principles.

Playback & synchronization math

Live BLE acquisition is a noisy clock — inter-sample gaps jitter with radio conditions. PlaybackEEGStream.normalized resamples a recording onto an exact uniform grid before replay, via linear interpolation between the two nearest recorded samples $(t_a, x_a)$, $(t_b, x_b)$:

$$x(t) = x_a + (x_b - x_a)\cdot\frac{t - t_a}{t_b - t_a}, \qquad t_a \le t \le t_b$$

Two replays of the same file at the same target rate then produce byte-identical sample sequences, independent of the original jitter — the property the CI regression test depends on.

Classifier confidence driving the 3D workspace's edge pulse is EMA-smoothed so an async prediction arrival doesn't visibly pop:

$$\hat c_n = \hat c_{n-1} + \alpha,(c_n - \hat c_{n-1}), \qquad \alpha = 0.15$$

and node brightness is broadband RMS under a log compression so small changes stay visible without large ones saturating:

$$I = \mathrm{clamp}\big(\log(1 + 0.05\cdot\mathrm{RMS}),\ 0,\ 1\big)$$

Predictions and samples arrive on independent streams; if a prediction goes stale (no update for classifierStaleThreshold while samples keep flowing), intent-driven color/pulse dim rather than keep showing a confidently-colored but outdated classification.

Measured performance (Apple Silicon, debug build): replaying the golden recording (77,966 samples / 305s) through the full windowing → features → classifier → channel-health → 3D-scene-checkpoint pipeline takes ~6.9s wall-clock — about 44× faster than real time, consistent with .instant pacing bypassing per-sample sleeps entirely. NeuralWorkspaceView.recompute() (the per-frame node/edge material update) costs ~0.42ms/call — at the view's 30Hz target refresh, that's ~1.3% of the frame budget, leaving headroom for a future embedding-projection node without a redesign.

Scientific motivation

This is a platform, not a clinical or productivity tool. The aim is to build the on-device infrastructure that lets a small research team:

1. Validate consumer-grade EEG against physiological expectations (alpha rise on eyes-closed, blink transients, jaw-clench EMG contamination).
2. Estimate sleep stage from 4 frontal channels (Muse S: TP9, AF7, AF8, TP10 — no chin EMG, no EOG). A 4-class output is the honest upper bound.
3. Test whether TMR cues during N2/SWS paired with LLM-generated dream analysis improve creative problem solving — pre-registration required before claiming any effect.
4. Ship the platform regardless of (3): the validation toolkit and codebase are useful contributions on their own.

Established neuroscience (alpha dropout, AASM staging, TMR for declarative memory) is treated as established. Novel claims (LLM analogy extraction, insight improvement) are treated as unproven. Every claim in SLEEP_CYCLE_DESIGN.md carries a confidence rating.

Core signal-processing definitions (full derivations in docs/Math.md):

$$X(t) \in \mathbb{R}^{4 \times N}, \qquad P_b = \sum_{f \in \text{band}_b} |\mathcal{F}{x}_f|^2 \cdot \frac{1}{N_{\text{bin}}}, \qquad r_\alpha(t) = \frac{P_\alpha^{\text{baseline}}}{P_\alpha(t)}$$

$P_b$ is Welch-style band power; $r_\alpha > 1$ means current alpha is lower than baseline — the canonical N1-onset signature.

Repository layout

NeuralCompose/
├── Sources/
│   ├── BCIBridge/        Obj-C++ shim for BrainFlow (stub by default)
│   ├── BCICore/          pure-Swift models, protocols, FSMs, buffers
│   ├── BCIEEG/           EEG streams, 2D plotter, 3D workspace (Phase B)
│   ├── BCIClassifier/    Core ML wrapper + deterministic mock
│   ├── BCILLM/           MLX adapter + stub + tokenizer  ← only MLX target
│   └── NeuralComposeApp/ SwiftUI views, Phase B debug window
├── Tests/                unit + golden-recording regression tests
├── Scripts/
│   ├── build.sh / run-synthetic.sh / run-muse-s.sh
│   ├── record-golden.sh              # capture a new golden reference recording
│   ├── analyze-eeg-session.py        # PSD/band-power/spectrogram/quality report for any recording
│   └── validate-muse-physiology.py   # live 5-condition acquisition sanity check
├── Recordings/           per-session EEG (gitignored) + golden/ (committed reference + report)
├── docs/                 long-form documentation
├── SLEEP_CYCLE_DESIGN.md full D1–D8 sleep architecture spec
└── HARDWARE_SETUP.md / MODEL_SETUP.md / CALIBRATION.md / TROUBLESHOOTING.md

Quick start

Synthetic mode — no hardware, no models:

git clone https://github.com/aurascoper/NeuralCompose.git
cd NeuralCompose
./Scripts/build.sh
./Scripts/run-synthetic.sh

Live Muse S (after BrainFlow is installed at ~/Developer/brainflow/):

./Scripts/build.sh --with-brainflow
./Scripts/run-muse-s.sh

Phase B debug window (Cmd+Shift+D in the running app) — live EEGScalpPlotterView (2D) and NeuralWorkspaceView (3D) tabs.

Replay the golden recording:

python3 Scripts/analyze-eeg-session.py Recordings/golden/golden_20260710-141352.eeg.csv
swift test --filter GoldenRecordingRegressionTests

Experimental status & limitations

Claim	Status
Live Muse S EEG acquisition through BrainFlow is reproducible on macOS	Established
Per-channel RMS, alpha power, and blink detection are observable on consumer Muse hardware	Established
Deterministic playback + CI regression against real hardware data	Established
4-class sleep staging from Muse S is achievable at research accuracy	Plausible — domain shift from PSG is the largest expected error source
TMR cues + LLM dream analysis improves engineering insight	Unproven — D8 crossover, pre-registration pending
5-class AASM sleep staging on Muse S	Hardware-limited — no chin EMG, atonia is the defining REM criterion

The platform ships regardless of the unproven claims — the validation toolkit, architectural spec, and codebase are useful on their own.

Documentation

• Recordings/golden/README.md — golden recording provenance, full report, plots.
• HARDWARE_SETUP.md · MODEL_SETUP.md · CALIBRATION.md · TROUBLESHOOTING.md
• SLEEP_CYCLE_DESIGN.md — full D1–D8 sleep architecture spec.
• docs/Architecture.md · docs/Math.md · docs/Validation.md · docs/Research.md

Citation

A paper draft is in paper/. Suggested citation when published:

Kinder, H. (2026). An open-source, privacy-preserving platform for EEG-guided cognitive incubation and dream-report analysis using consumer-grade hardware. In preparation.

License

Research prototype code. Do not use NeuralCompose to make clinical or safety-critical decisions. License terms: see LICENSE (MIT).

Acknowledgements

• BrainFlow for the unified biosensor acquisition API.
• MLX-Swift for the local on-device LLM runtime.
• Apple Neural Engine for low-power Core ML inference.
• The Muse headband community for open BLE protocol documentation.
• The sleep-staging research community for the AASM standard.