Add topology-aware lighting with Rynk control - #1031
Open
colonelpanic8 wants to merge 77 commits into
Open
Conversation
colonelpanic8
force-pushed
the
glove80-rmk/lighting-v2
branch
from
August 5, 2026 08:03
7a5dbe7 to
dfdafb1
Compare
Size Report
|
Add an opaque, bounded application payload that firmware can exchange between the split central and peripheral alongside the normal split traffic, without ever taking priority over key events. * `split_app` module: `SplitAppData` (a small, `MaxSize`, postcard length-prefixed payload) plus four statics — `SPLIT_APP_TX` (central -> peripheral), `SPLIT_APP_PERIPH_TX` (peripheral -> central), the symmetric `SPLIT_APP_RX` inbox, and the `SPLIT_APP_LINK` watch that reports split-link state to the application. * Producers use `try_send` only (bounded, drop-on-full) so the split read/write loops never block or get starved by application traffic; key events are always polled first. * The split driver and peripheral drain the application queues as the lowest-priority arm of their outgoing selects and forward received `SplitMessage::Application` payloads into the inbox. * `SPLIT_APP_LINK` is state-based (a `Watch`), so a late-subscribing application still observes the current link state; the `false -> true` edge is a resync trigger. Link-down edges are emitted from a drop guard so they survive async cancellation of the split session. * On the peripheral the link is raised on the FIRST inbound message from the central rather than on bare connection: over BLE, notifications to a central that has not yet subscribed are silently dropped, so the connection alone is not proof the application channel is usable. Developed for a split keyboard port.
Add the protocol types for runtime-configurable per-layer scenes: LightingSceneCell (layer + stable LED id + effect), LightingLayerPolicy, a revision-pinned scenes page, single-cell set/unset requests, an atomic Begin/Put/Commit/Abort replacement transaction, and SetLightingLayerPolicy. Discovery deliberately leaves LightingCapabilities and LightingState byte-identical: postcard is positional, so appending fields would break new-host/old-firmware decode. Scenes are advertised through a new LAYER_SCENES bit in the existing LightingFeatureFlags plus a dedicated GetLightingSceneStatus endpoint carrying capacity, occupancy, and policy. New LightingError variants (UnknownLayer, SceneFull) are appended, so existing encodings are unchanged. Protocol version bumps to v0.3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Give the standard engine an owned, fixed-capacity SceneTable (new SCENE_CAP const generic, default 0) holding per-layer, per-slot effects with the same EffectiveOnly/ActiveStack composition semantics as the static LayerScenes. The table composes in the layer band immediately after the board's static scenes — a runtime cell overrides a static default for the same slot — while the TTL overlay stays highest-priority. Scene mutations are revision-checked commands; whole-table replacement stages inside the engine via bounded chunks (Begin/Put/Commit/Abort) with inactivity expiry and an idempotent commit. Staging engine-side instead of reusing the host-side overlay staging keeps kilobyte-sized scene batches off the bounded mailbox channels, whose payloads are copied by value. Scene reads and transaction reservation need non-state readback, so the engine's reply type becomes the StandardReply enum and StandardState gains scene_len/scene_policy. The Rynk service exposes the table through nine endpoints for status, revision-pinned paged reads, single-cell set/unset, layer policy, and the chunked replacement transaction. It validates effects, layer bounds (against the live keymap), and stable LED ids (against the topology) before anything reaches the engine; boards opt in by advertising a capacity via RynkLightingController::with_scene_capacity, and scene endpoints reject with Unsupported when none is wired. Scene writes honor the same unlock gate as the other lighting writes. Scene configuration is durable config like the keymap: after every successful mutation the adapter reads the table back out of the engine and persists it as one header record (len + policy) plus chunk-sized shard records in wire (stable LED id) form. The storage task compares before writing so unchanged shards cost no flash traffic; a storage reset clears them. Boards load persisted scenes at boot with Storage::read_lighting_scenes + install_lighting_scenes, which skips cells whose LED id no longer resolves. Lighting featuresets join the CI test/clippy matrix. Composed with the split renderer replicas underneath: the runtime scene table travels inside StandardReplicaState, so replica renderers draw runtime scenes exactly like the authority without ever seeing the incremental scene mutation commands. StandardReplicaState/-Slot and StandardCommand gain a trailing SCENE_CAP const generic (default 0), ExportReplica snapshots the table and ApplyReplica installs it, and the replica round-trip test now covers a runtime scene cell. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ndings Typed client methods for every scene endpoint, plus two alloc conveniences: read_all_lighting_scenes pages the whole table under one pinned revision and restarts on a concurrent-mutation conflict, and replace_all_lighting_scenes drives the begin/put/commit transaction with a best-effort abort on staging failure. The wasm client re-exports the same surface, so the generated TypeScript picks up the scene types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ProtocolVersion numbers belong to upstream (HaoboGu/rmk feat/rynk); a fork that bumps CURRENT will eventually collide with upstream reusing the same minor for a different protocol. Revert CURRENT to the upstream base's v0.1 and drop every fork-minted claim (v0.2 native lighting, v0.3 build info, v0.4 routed split bootloader entry, v0.5 layer scenes). Downstream features are discoverable without version numbers, using the protocol's existing negotiation machinery: - lighting endpoints: `DeviceCapabilities.lighting_enabled` (declared by upstream v0.1), then `GetLightingCapabilities.features` - layer scenes: `LightingFeatureFlags::LAYER_SCENES` (set iff the board advertises a scene capacity) plus `GetLightingSceneStatus.capacity` - `GetBuildInfo` / `PeripheralBootloaderJump`: per-command probing — firmware without them answers `UnknownCmd`, which hosts already receive as `Rejected(UnknownCmd)` No code ever gated on the minted minors (the handshake only rejects `major` mismatches), so firmware already in the field reporting v0.2–v0.5 stays compatible with reverted hosts and vice versa. Regenerate the wire snapshots and the generated protocol reference, and state the rule in the reference's Compatibility section and the changelog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ndalone config macro Boards whose hardware cannot use the generated main (custom matrix drivers, nonstandard USB bring-up) previously had no way to consume `[lighting]` from keyboard.toml, and animated extension sources could not own the RGB mode/value/speed keys: apply_light_action routed every RGB action to the uniform background. - LightingSource::handle_light_action (default: decline) lets the standard engine offer each LightAction to its extension source before the built-in background handling. - rmk_lighting_config! emits the same flash-resident physical-layout and lighting statics as #[rmk_keyboard], resolved standalone so no [matrix]/[split] section is required; [keymap].layers alone provides the layer count for scene validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exercise layout_standalone/lighting_standalone plus the geometry and
topology expansion on an exact copy of the rmk-zsa-voyager keyboard.toml:
52 emitters over two outputs, with { all = true } layer scenes expanding
to every slot.
The test caught that expand_standalone_lighting_config loaded the toml
through new_from_toml_path, which panics without [keyboard].chip — a
section hand-written-main firmware intentionally omits. Load with event
defaults only instead, as rmk-types' build.rs already does; the skipped
chip-default layer never touches [layout], [lighting], or [keymap].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The capability was set only when a board bound a key to cycling the mode, so replacing that binding with a real keycode withdrew the whole capability -- hosts then stopped reading or writing the mode, and a config asking for powered-only silently read back as always-on. The standard engine always owns a policy, so the flag reflects that rather than the binding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream rebased feat/rynk before merging it, so this topic replays onto a main whose rynk internals differ from the ones it was written against: - `Serve::<E, _>::serve(x, msg)` became the free `serve::<E, _>(x, msg)`, and the lock endpoints take the session's `HostLock` rather than the session. - `RynkSession` lost its `locker` and `topics` fields, which are now locals in `run_session`. It keeps only the lighting overlay transaction, which has to outlive a single dispatch because it spans Begin/Put/Commit. - `RMK_VERSION` moved into `handlers::system`; it lives here again so it can be public alongside `RMK_VERSION_STRING` and the build-label helpers. - `rynk::endpoint` folded into `rynk::command`, and `TopicEvent::cmd()` is gone. - The new simulator codegen predates `bootloader_requires_unlock`, so the `run_tests!` scenarios could not build a `LockConfig`. The lighting handler tests derive their gate from the service under test, the way `run_session` does, now that the session no longer carries one.
Upstream replaced the hand-written loopback suite with a TOML-scenario simulator, which these cases cannot use: the topology a lighting board advertises is a compile-time fixture rather than a keymap, and the stateful endpoints answer from an engine that has to be driven concurrently with the request. Both are things a timeline has no vocabulary for, so they live beside the other cases a scenario cannot express and play the host over the same in-memory duplex `run_session` gets from a real transport. Restores discovery with revision-pinned paging, the extension endpoints across the full mailbox/adapter/engine path, and the lighting readback topic. The Unsupported case is split out, since it is the one that needs no engine. The host half is reimplemented rather than pulled from `rynk`: as a dev-dependency it would unify `rmk-types/host` into every rmk test build, so a lean feature row would compile `Cmd` variants whose dispatch arms are cfg'd out. Both ends share `rmk-types` and decode with the same strictness instead.
Scene lookup scanned the whole table for every cell it resolved, so composition cost grew with the total number of cells rather than with the cells the active layers actually contribute. Keep cells grouped by layer with a `layer_offsets` index instead, so a layer's run is a slice and `included_len` is arithmetic over the active layers rather than a filtered count. Insertion and removal shift within the array to hold the grouping; layers past `LayerState::CAPACITY` fall back to a partition point. Also stop publishing a LightingChangedEvent when a board signals `snapshot_changed`. That signal means "re-render from fresh context", which every board raises on ordinary battery and sensor movement; the event is a host readback invalidation and does not belong on that path. Tests: 649 pass on the split,rynk,storage,async_matrix,_ble,lighting row under nextest, including a new differential test that checks the grouped table resolves identically to a naive scan. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reject extended conditional-scene cells whose connection profile is past NUM_BLE_PROFILE, empty the legacy conditional table whenever the V2 table is written so a firmware downgrade cannot resurrect stale rules, and document that legacy read-modify-write cycles drop connection predicates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…able Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An extension is enabled while its ExtensionState value is non-zero, which is exactly what RgbTog flips. Nothing could observe that, so a key bound to the toggle had no way to show its own state: its cell painted the same colour whether effects were rendering or blanked. Add an `effects` predicate to ConditionSet, answered from the extension source the engine already holds. Compiled rules pass None and so stay unsatisfiable, on the same grounds as output_mode: a source that cannot observe the state must not let either polarity fire. The predicate rides the extended conditional cell, which changes that cell's encoding rather than merely adding a field to it. Firmware advertising only RUNTIME_CONNECTION_CONDITIONS speaks the earlier cell, so RUNTIME_EFFECTS_CONDITIONS describes the new encoding and a host that cannot see it must fall back to the legacy endpoints. The added predicate also costs a cell one byte, so an extended page now holds five. Persisted V2 tables written by the earlier extended cell no longer parse -- the shrunk chunk alone would have done that -- and fetch_data reports a failed record as absent, so such a table boots empty and is restored by the host's next apply rather than misparsed into stale rules.
Maintain a bitmap alongside the profile manager's bonded-device list so subscribers outside the BLE task can see every slot's paired-versus-empty state at once, not just the active slot's. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conditional rules gain connection.bonded = { slot, bonded }, matched
against the bond bitmap independently of the active profile, so one rule
per slot key can light paired and empty slots differently. The extended
wire cell grows accordingly (pre-release surface iterated in lockstep
with the in-tree hosts); pages now carry five cells. Stale stored V2
tables from the previous cell layout decode as absent and are restored
by the next host apply.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rules can gate on USB being plugged and routable independently of the active transport — the difference between a USB indicator shown ready (plugged, typing elsewhere) and active (carrying typing). Wire cell grows by the trailing option; five-cell pages still fit the payload bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The extended conditional exemplars still built `LightingConnectionCondition` without `bonded` and `usb_connected`, so the `host` feature build of rmk-types failed to compile. Populate both and repin the frame snapshot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only the lighting overlay handlers take the session, so `rynk` without `lighting` tripped -D unused-variables in example builds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The engine could report what it holds but never what the LEDs show. On a split renderer replica those are different questions: the replica renders from a snapshot that can go stale, and nothing observable distinguished a stale replica from a correct one. Record each render's revision and lighting context, promote it when the output acknowledges the write, and page the committed frame back through `ReadFrame`. The committed frame is post-`OutputTransform`, so what comes back is what the driver was handed rather than the pre-brightness composition.
Two new lighting endpoints for diagnosing a split renderer replica from the host: `GetLightingFrame` pages back what one node last presented, and `GetLightingReplicaStatus` returns both sides of the replication handshake in one read. `LightingFeatureFlags` is out of bits, so these are discovered by probing — firmware without them answers `UnknownCmd`, which is the documented downstream path anyway. Existing layouts and the upstream protocol version are untouched. A remote frame page has to be reassembled from split application packets, whose queues are shallow and lossy, so the page holds 24 cells rather than the ~80 the payload budget would allow. Availability is encoded explicitly: `Option` for never-observed values, and two appended `LightingError` variants separating "no such node" from "that node could not answer".
Wires GetLightingFrame and GetLightingReplicaStatus through the host service: the local half answers from its own engine, the peripheral through the board-provided remote-frame and replication-status ports, and the loopback integration tests drive both endpoints end to end. Native rmk-types and rynk_lighting suites pass; the cross-feature matrix and no_std target checks have not run yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
colonelpanic8
force-pushed
the
glove80-rmk/lighting-v2
branch
from
August 7, 2026 19:58
dfdafb1 to
f5202f3
Compare
Expose split lighting frame and replica observability
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Why
RMK currently has no general model for per-key lighting that can span irregular and split layouts while keeping firmware, Rynk, and Vial state consistent. This implementation makes one lighting service authoritative, so changes made by either host protocol are reflected by later state queries and rendered output.
Relationship to earlier PRs
This replaces #987, which was opened against
feat/rynkbefore that branch was merged intomain(9448f7a) and so could no longer be reviewed against a live base. This branch is the same work rebased ontomainand carried forward, and it merges cleanly.It is self-contained: it includes the bounded split application-message side channel from #984, since split keyboards use that channel to propagate lighting state. It also supersedes the smaller abstraction draft in #982.
Checks