diff --git a/AGENTS.md b/AGENTS.md index 9ec6d08d..dd2d537f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ ## Purpose -Python control plane for DotBots. Serial / cloud / edge adapters talk to a DotBot gateway (often via Mari → marilib); a FastAPI REST + WebSocket server exposes state; a React web UI provides joystick/map/lighthouse-position visualization. Also ships CLI tools (`dotbot-controller`, `dotbot-edge-gateway`, `dotbot-keyboard`, `dotbot-joystick`, `dotbot-qrkey`) and DotBot/SailBot simulators. +Python control plane for DotBots. Serial / cloud / edge adapters talk to a DotBot gateway (often via Mari → marilib); a FastAPI REST + WebSocket server exposes state; a React web UI provides joystick/map/lighthouse-position visualization. Ships a unified `dotbot` CLI whose top level is four object-namespaces: `fw` (firmware artifacts: build/fetch/list/make), `device` (one cabled device: flash/info), `swarm` (the fleet over the air), and `run` (host-side processes you launch — `dotbot run controller`, `run gateway`, `run simulator`, `run lh2-calibration`, `run demo`, `run keyboard`, `run joystick`), plus DotBot/SailBot simulators. The `dotbot` dispatcher is the only console script — there are no per-command `dotbot-*` binaries. This is the most active repo in the ecosystem (187 commits in last 90 days as of 2026-05-05). @@ -15,7 +15,8 @@ This is the most active repo in the ecosystem (187 commits in last 90 days as of ## Entry points -- `dotbot/controller_app.py` — main CLI (`dotbot-controller`); wires adapters and settings +- `dotbot/cli/main.py` — unified `dotbot` Click group (lazy subcommand loader) +- `dotbot/controller_app.py` — `dotbot run controller` subcommand backend; wires adapters and settings - `dotbot/controller.py:1` — 737-line `Controller` class; central object - `dotbot/frontend/src/App.tsx` — React UI root @@ -23,8 +24,14 @@ This is the most active repo in the ecosystem (187 commits in last 90 days as of ```bash pip install pydotbot # or `pip install -e .` -dotbot-controller --help -# Other entry points: dotbot-edge-gateway, dotbot-keyboard, dotbot-joystick, dotbot-qrkey +dotbot --help # unified dispatcher: fw / device / swarm / run +dotbot fw --help # firmware artifacts: build / fetch / list / make +dotbot device --help # one cabled device: flash an app/role, read info +dotbot swarm --help # the fleet over the air (swarmit; in the base install) +dotbot run --help # host-side processes (controller, gateway, simulator, ...) +dotbot run controller --help # start the controller +dotbot run lh2-calibration --help # LH2 calibration (optional: pip install pydotbot[calibrate]) +dotbot run demo --list # built-in research demos # Tests / lint / build tox # envs: tests, check, cli, web=npm run lint, doc @@ -45,7 +52,14 @@ CI: `.github/workflows/continuous-integration.yml` — `tox` on Linux/macOS/Wind - **`PyDotBot-utils`** — `pyproject.toml:49`; used by `utils/hooks/sdist.py:build_frontend` - **`DotBot-libs`** — checked out in CI to build `utils/control_loop` C library - **`DotBot-firmware`** — referenced only in README (flashing instructions); no code dep -- No references to: `swarmit`, `dotbot-lh2-calibration`, `dotbot-provision` +- **`swarmit`** — sibling package, a core dependency (`pyproject.toml`); + imported lazily inside `dotbot/cli/swarm.py`, which bridges the unified + config's `conn`/`swarm_id` into swarmit's flags at the mount boundary. +- **`dotbot-provision`** — vendored into `dotbot/provision/` (Phase 2, + 2026-05). Standalone PyPI package scheduled for deprecation. +- **`dotbot-lh2-calibration` (Python)** — vendored into + `dotbot/calibration/` (Phase 2, 2026-05). The C firmware stays in + its own repo. ## State of repo (snapshot 2026-05-05) diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..182fc968 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,100 @@ +# Changelog + +All notable changes to PyDotBot are recorded here. + +Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added + +- Unified `dotbot` CLI dispatcher that mounts every workflow (controller, + simulator, testbed ops, calibration, demos, keyboard/joystick) under one + command. Subcommand modules are loaded lazily so `dotbot --help` stays + cheap. +- `dotbot run demo` discoverable launcher; `dotbot run demo qr` runs the + qrkey phone-bridge demo. +- `dotbot fw` mock surface (scaffold/build/flash subcommands; placeholder + for the firmware-developer workflow). +- **Vendored `dotbot-provision`** into `dotbot/provision/`. All five + subcommands available as `dotbot testbed provision `. +- **Vendored `dotbot-lh2-calibration` (Python side)** into + `dotbot/calibration/`. Surfaced as `dotbot run lh2-calibration` with + two subcommands: + - `collect` — runs the Textual TUI (default — bare + `dotbot run lh2-calibration` invokes this for muscle memory) + - `apply ` — write the saved calibration as a C header to + `` (replaces the previous `dotbot-calibration-exporter`; + today the only consumer is the swarmit secure bootloader which + `#include`s the file at compile time) + The C firmware in the `dotbot-lh2-calibration` repo is unchanged. + Future OTA / swarm-wide counterparts (`collect` over MQTT, + `apply` as OTA push) will live under `dotbot swarm + calibrate-lh2`. +- Calibration records are now saved as timestamped, schema-versioned + TOML files (`~/.dotbot/calibration-.toml`) carrying + metadata (number of LH stations, calibration distance, creation + time) alongside the homography bytes (hex-encoded under + `[calibration].data_hex`). The legacy `~/.dotbot/calibration.out` + binary is still written as a back-compat byproduct so external + consumers (swarmit OTA, `dotbot testbed provision flash`) keep + working unchanged; once they learn to read TOML the legacy write + will be dropped. `load_calibration()` prefers the newest TOML and + falls back to `calibration.out` if no TOML files exist. +- `dotbot testbed provision flash --calibration ` accepts a + `.toml` calibration file in addition to the legacy binary format + (the file extension drives the parsing path). +- Optional dependency groups (revised): + - `pip install dotbot[testbed]` adds `swarmit` (still external) + - `pip install dotbot[provision]` adds `intelhex` (provision runtime) + - `pip install dotbot[calibrate]` adds `opencv-python` + `textual` + - `pip install dotbot[all]` pulls all three + +### Changed + +- **Breaking — CLI reorganized into four object-namespaces.** The top + level is now exactly `fw` (firmware artifacts), `device` (one cabled + device), `swarm` (the fleet), and `run` (host-side processes). The flat + process verbs moved under `run`: `dotbot controller` → `dotbot run + controller`, and likewise `gateway` / `simulator` / `demo` / `keyboard` / + `joystick`; `dotbot calibrate-lh2` → `dotbot run lh2-calibration`. The + Makefile escape hatch moved from `dotbot make` to `dotbot fw make`. + `run` subcommands are still loaded lazily, so `dotbot run --help` stays + cheap. +- The qrkey integration moved from `dotbot/qrkey.py` to + `dotbot/examples/qrkey_demo/`. The demo is now a separate process that + consumes the controller's REST API — the controller stays agnostic to + qrkey. +- `dotbot/examples/qrkey_demo/` is a thin client of the upstream `qrkey` + package (now pinned `>= 0.12.2`); none of its code is vendored. +- Frontend polls qrkey count every 1 s for faster Show QR button + feedback. + +### Removed + +- `dotbot-qrkey` console script — use `python -m dotbot.examples.qrkey_demo` + or `dotbot run demo qr` instead. +- `dotbot-edge-gateway` console script — the referenced module + `dotbot.edge_gateway_app` never existed; the entry was silently broken. +- `pin_code` tox env — referenced `dotbot/pin_code_ui/` which never + existed. +- `dotbot-provision` and `dotbot-lh2-calibration` PyPI dependencies + (folded into the `dotbot` package). The standalone PyPI packages are + scheduled for deprecation releases that point users at `pip install + dotbot[provision]` / `pip install dotbot[calibrate]`. +- `dotbot-controller`, `dotbot-keyboard`, and `dotbot-joystick` console + scripts — removed outright (no longer aliased). Use `dotbot run + controller` / `dotbot run keyboard` / `dotbot run joystick`. + +### Deprecated + +- The standalone `dotbot-provision` and `dotbot-lh2-calibration` PyPI + packages will issue `DeprecationWarning` on their next release and + point users at `pip install dotbot[provision]` / + `pip install dotbot[calibrate]`. Their console scripts + (`dotbot-provision`, `dotbot-calibration`, + `dotbot-calibration-exporter`) are not re-exported by `dotbot` + because they never shipped from this package; use the unified + subcommands instead. diff --git a/README.md b/README.md index a1c5616a..35f7558b 100644 --- a/README.md +++ b/README.md @@ -6,102 +6,196 @@ # PyDotBot -This package contains a complete environment for controlling and visualizing -[DotBots](http://www.dotbots.org). +The control plane for the [DotBot](http://www.dotbots.org) - a small wireless +wheeled robot built to operate in large swarms, for research and education. -The DotBots hardware design can be found [here (PCB)][dotbot-pcb-repo]. -The firmware running on the DotBots can be found [here][dotbot-firmware-repo]. +PyDotBot allows you to flash a robot and control a whole fleet over the air, +from one bot to a thousand. -This package can also be used to control devices running the SailBot firmware -application. +[▶️ Click to see a DotBot swarm in action](https://www.youtube.com/watch?v=pXGTLqafReU) -![DotBots controller overview][pydotbot-overview] +```text +┌───────────┐ ┌────────────┐ ┌─────────┐ +│ web UI / │ │ │ │ │ +│ CLI / │──REST/WS─▶│ controller │──serial/MQTT─▶│ gateway │──radio─▶ 🤖🤖🤖 DotBot swarm +│ your code │ │ │ │ │ +└───────────┘ └────────────┘ └─────────┘ + ╰─────────── PyDotBot ───────────╯ +``` + +**What you can do** + +- 🕹️ Drive one bot or a whole fleet from a **web UI** (live map + joystick) or your own **Python** code +- 📡 Flash the swarm **over the air** - one command, hundreds of bots at once +- 🛰️ Get real-world **(x, y) positions** with Lighthouse 2 localization +- 🧪 Try it all with **zero hardware** using the built-in simulator +- 🛠️ One `dotbot` CLI takes you from build → flash → run + +## Install + +PyDotBot is available on [PyPi](https://pypi.org/project/pydotbot/), install it with: + +```bash +pip install --pre pydotbot +``` + +Then, check your installation with `dotbot --version` and learn what's possible with `dotbot --help`. + +Every command and flag is documented in the [CLI reference][cli-doc]. + +## Try the simulator + +See the whole thing run with nothing but Python! + +The command below will run a simulated swarm, which you can observe in a web UI at http://localhost:8000/PyDotBot/ : + +```bash +dotbot run simulator -w +``` + +Drive the simulated bots from the UI, or run a bundled demo in a +second terminal: + +```bash +dotbot run demo circle # drive one bot in a circle (the simplest demo) +``` + +Learn how to script the swarm from your own code, run the richer examples, and more - all with +no hardware - in the [simulator guide][simulator-doc]. + +## Deploy a real swarm + +The DotBot is made to operate as a swarm, here is how you can deploy it on real robots. + +### Prerequisites + +Minimal hardware setup: +- DotBot v3, as well as a USB-C cable and a barrel-jack charger (2.5 mm, 6–18 V, 5/10 A) +- nRF5340-DK to use as gateway, as well as a micro-USB cable + +Software to install (as needed): +- Python ≥ 3.11 - ensure you also have [pip](https://pip.pypa.io/en/stable/) available in your PATH +- [nRF Command Line Tools](https://www.nordicsemi.com/Products/Development-tools/nRF-Command-Line-Tools) (`nrfjprog`), for commands such as `dotbot device flash` -## Installation +### Setup -Run `pip install pydotbot` +To operate as a swarm, set your swarm connection config: -## Setup +```bash +dotbot config init --conn mqtts://argus.paris.inria.fr:8883 --swarm-id 1234 +``` + +> `argus.paris.inria.fr` is our Inria Paris broker and `1234` our swarm - pass +> your own `--conn` and `--swarm-id` (your testbed admin provides these). This +> writes `./dotbot.toml`; commands run from this directory pick it up, so you +> don't repeat the flags. Full schema: the [configuration reference][config-doc]. + +The swarm mode also requires a special "sandbox" firmware in each dotbot. +We also need a more powerful gateway firmware. Let's flash both - the network +id comes from your config: + +```bash +dotbot fw fetch # pull the pinned pre-compiled firmwares (swarmit + dotbot-firmware) +dotbot device flash-mari-gateway -s 10 # flash the gateway +dotbot device flash-swarmit-sandbox -s 77 # the sandbox firmware - do this on each dotbot +``` + +(`device flash-mari-gateway` / `flash-swarmit-sandbox` auto-fetch +the firmware into `~/.dotbot/artifacts/` if it isn't already there.) + +Now, run the gateway (the broker comes from your config): + +```bash +dotbot run gateway -p /dev/cu.usbmodem0010500324491 +``` + +### Deploy and control -Flash the required firmwares on the DotBots and gateway board (use an -nRF52833DK/nRF52840DK/nrf5340DK board as gateway), as explained in -[the DotBots firmware repository][dotbot-firmware-repo]. +You can flash as many dotbots as you want, all at once! First, how about making them spinnnn 🔄 🔄 -## Usage +```bash +dotbot swarm flash ~/.dotbot/artifacts/dotbot-firmware-1.22.0rc1/spin-sandbox-dotbot-v3.bin -ys # flash the whole fleet with a simple spinning app +``` + +(`dotbot swarm` reads the same `dotbot.toml` as the rest - pass `--conn` / +`--swarm-id` to override it for one run. That path is the `dotbot-firmware` +release `dotbot fw fetch` cached - run `dotbot fw list` to see the exact paths +and versions on your machine.) + +Then, flash another experiment: +```bash +dotbot swarm stop # ensure all robots are in bootloader +dotbot swarm flash ~/.dotbot/artifacts/dotbot-firmware-1.22.0rc1/dotbot-sandbox-dotbot-v3.bin -ys # this firmware allows bots to be remote-controlled ``` -dotbot-controller --help -Usage: dotbot-controller [OPTIONS] - - DotBotController, universal SailBot and DotBot controller. - -Options: - -a, --adapter [serial|edge|cloud|dotbot-simulator|sailbot-simulator] - Controller interface adapter. Defaults to - serial - -p, --port TEXT Serial port used by 'serial' and 'edge' - adapters. Defaults to '/dev/ttyACM0' - -b, --baudrate INTEGER Serial baudrate used by 'serial' and 'edge' - adapters. Defaults to 1000000 - -H, --mqtt-host TEXT MQTT host used by cloud adapter. Default: - localhost. - -P, --mqtt-port INTEGER MQTT port used by cloud adapter. Default: - 1883. - -T, --mqtt-use_tls / --no-mqtt-use_tls - Use TLS with MQTT (for cloud adapter). - -g, --gw-address TEXT Gateway address in hex. Defaults to - 0000000000000000 - -s, --network-id TEXT Network ID in hex. Defaults to 0000 - -c, --controller-http-port INTEGER - Controller HTTP port of the REST API. Defaults - to '8000' - -w, --webbrowser / --no-webbrowser - Open a web browser automatically - -v, --verbose Run in verbose mode (all payloads received are - printed in terminal) - --log-level [debug|info|warning|error] - Logging level. Defaults to info - --log-output PATH Filename where logs are redirected - --config-path FILE Path to a .toml configuration file. - -m, --map-size TEXT Map size in mm. Defaults to '2000x2000' - --help Show this message and exit. + +Observe and control your swarm from a web interface: + +```bash +dotbot run controller -w # will open a webpage at http://localhost:8000/PyDotBot/ ``` -By default, the controller expects the serial port to be `/dev/ttyACM0`, as on -Linux, use the `--port` option to specify another one if it's different. For -example, on Windows, you'll need to check which COM port is connected to the -gateway and add `--port COM3` if it's COM3. +Full walkthrough of fleet operations - status, OTA flash, start/stop, monitor - +is in the [`swarm` reference][swarm-doc]. -Using the `--webbrowser` option, a tab will automatically open at -[http://localhost:8000/PyDotBot](http://localhost:8000/PyDotBot). The page maintains -a list of available DotBots, allows to set which one is selected and controllable -and provide a virtual joystick to control it or change the color of the on-board -RGB LED. +### Calibrate positions (optional) -Use `--config-path` to specify the file: +Give the bots real-world `(x, y)` with Lighthouse 2 - capture once from any bot +over the air, then push the result to the whole fleet (needs the `[calibrate]` +extra, below): ```bash -# Use settings from the config file -dotbot-controller --config-path config_sample.toml -# Use config file but override port and adapter (simulator example) -dotbot-controller --config-path config_sample.toml -a dotbot-simulator +dotbot swarm stop # bots must be idle to capture +dotbot swarm lh2-calibration collect --device -d 500 # capture + solve + save +dotbot swarm lh2-calibration push ~/.dotbot/calibration-.toml # apply to every bot ``` -Values defined in the config file behave exactly like CLI options. -If both are provided, CLI flags override config values. +`-d` is your reference square's side, in mm (one bot's capture calibrates the +whole arena). Full walkthrough - arena sizing and the cabled alternative - is in +the [LH2 calibration guide][lh2-doc]. + +## Going further + +- **Drive a single bot** end to end - build, flash, and control one DotBot: + the [one-bot guide][one-bot-doc]. +- **Position tracking with Lighthouse 2** - give the fleet real-world `(x, y)`, + calibrated over the air: the [LH2 calibration guide][lh2-doc] (a cabled + alternative is covered there too). +- **The controller + web UI** - drive and visualize a swarm from the browser: + the [controller guide][controller-doc]. +- **Build firmware from source** instead of `dotbot fw fetch` - needs + [SEGGER Embedded Studio](https://www.segger.com/products/development-tools/embedded-studio/) + and a DotBot-firmware checkout: + ```bash + git clone --recurse-submodules --branch develop https://github.com/DotBots/DotBot-firmware.git + export DOTBOT_FIRMWARE_REPO=$(pwd)/DotBot-firmware + ``` + then `dotbot fw build` / `dotbot fw artifacts` (see [`fw`][fw-doc]). +- **Everything else** - the full `dotbot` CLI (`fw` / `device` / `swarm` / `run` + + `config`), the REST/WS and MQTT surfaces, and hardware notes: the + [documentation][doc-link]. + +Most of `dotbot` is in the base install; only LH2 calibration needs an extra: + +```bash +pip install --pre 'pydotbot[calibrate]' # opencv (the LH2 homography solve) +``` -**Firefox users:** -If the webapp is not working, press `Ctrl + L`, type `about:config`, -and set `network.http.http2.websockets` to `false`. +Hitting a snag (e.g. the web UI not loading in Firefox)? See +[Troubleshooting][troubleshooting-doc]. ## Tests -To run the tests, install [tox](https://pypi.org/project/tox/) and use it: +To run the tests, run [tox](https://pypi.org/project/tox/): ``` tox ``` +## License + +See `LICENSE` in each component repository. + [ci-badge]: https://github.com/DotBots/PyDotBot/workflows/CI/badge.svg [ci-link]: https://github.com/DotBots/PyDotBot/actions?query=workflow%3ACI+branch%3Amain [pypi-badge]: https://badge.fury.io/py/pydotbot.svg @@ -112,6 +206,12 @@ tox [license-link]: https://github.com/DotBots/pydotbot/blob/main/LICENSE.txt [codecov-badge]: https://codecov.io/gh/DotBots/PyDotBot/branch/main/graph/badge.svg [codecov-link]: https://codecov.io/gh/DotBots/PyDotBot -[pydotbot-overview]: https://github.com/DotBots/PyDotBot/blob/main/dotbots.png?raw=True -[dotbot-firmware-repo]: https://github.com/DotBots/DotBot-firmware -[dotbot-pcb-repo]: https://github.com/DotBots/DotBot-hardware +[cli-doc]: https://pydotbot.readthedocs.io/en/latest/cli/index.html +[fw-doc]: https://pydotbot.readthedocs.io/en/latest/cli/fw.html +[swarm-doc]: https://pydotbot.readthedocs.io/en/latest/cli/swarm.html +[config-doc]: https://pydotbot.readthedocs.io/en/latest/reference/configuration.html +[simulator-doc]: https://pydotbot.readthedocs.io/en/latest/guides/simulator.html +[controller-doc]: https://pydotbot.readthedocs.io/en/latest/guides/controller.html +[one-bot-doc]: https://pydotbot.readthedocs.io/en/latest/guides/one-bot.html +[lh2-doc]: https://pydotbot.readthedocs.io/en/latest/guides/lh2-calibration.html +[troubleshooting-doc]: https://pydotbot.readthedocs.io/en/latest/reference/troubleshooting.html diff --git a/codecov.yml b/codecov.yml new file mode 100644 index 00000000..020ac4f0 --- /dev/null +++ b/codecov.yml @@ -0,0 +1,25 @@ +# Codecov config — split project-level (long-term) from patch-level +# (per-PR) policy. Project-level keeps the long-term floor honest; +# patch-level is informational because vendoring / refactor PRs can +# legitimately ship diffs with low instantaneous coverage even when +# the project total stays healthy. + +coverage: + status: + project: + default: + # Compare against main's current coverage. Allow tiny dips + # (rounding / one-off branches) without flapping CI. + target: auto + threshold: 1% + patch: + default: + # Report patch coverage but don't fail CI on it. Reviewers can + # still see the number in the PR comment; the gate just lives + # in human judgment instead of a hard threshold. + informational: true + +comment: + layout: "reach, diff, flags, files" + behavior: default + require_changes: false diff --git a/doc/cli/config.md b/doc/cli/config.md new file mode 100644 index 00000000..7bd0e54a --- /dev/null +++ b/doc/cli/config.md @@ -0,0 +1,54 @@ +# `dotbot config` - inspect and scaffold the config + +`dotbot` reads a `dotbot.toml` so commands don't repeat shared settings - your +gateway connection, swarm id, firmware paths. `config` scaffolds that file and +shows you what the CLI actually resolved. For the full file format - every key, +deployments, the precedence rules - see the +[configuration reference](../reference/configuration.md). + +## Which command do I want? + +| Goal | Command | +|---|---| +| Write a starter `dotbot.toml` you can edit | `dotbot config init` | +| See the merged, effective config + which file it came from | `dotbot config show` | +| Print just the resolved config-file path | `dotbot config path` | + +## `init` + +Writes a minimal `./dotbot.toml` - a one-line pointer to the docs, plus any keys +you pre-fill. `--global` writes your per-machine `~/.dotbot/config.toml` instead; +`-f/--force` overwrites an existing file. + +```bash +dotbot config init # commented starter in ./dotbot.toml +dotbot config init --conn mqtts://broker:8883 --swarm-id 1234 # pre-fill the two common keys +dotbot config init --global # ~/.dotbot/config.toml +``` + +> MQTT credentials are never file keys - set `DOTBOT_MQTT_USER` / +> `DOTBOT_MQTT_PASS` in the environment. + +## `show` / `path` + +`show` prints the source file, the selected deployment, and the resolved config +as TOML - only the keys actually set, not the full schema. `path` prints just +the file path (or notes that built-in defaults are in use). Both are read-only; +there is no per-key `set` - edit the file, it's yours. + +```bash +dotbot config show +dotbot config path +``` + +## Where the config comes from + +`dotbot` uses the first of: a `-c/--config FILE` flag (or `DOTBOT_CONFIG`), a +`dotbot.toml` in the current directory, then `~/.dotbot/config.toml`. A flag or a +`DOTBOT_*` env var overrides the file for a single run. The full precedence chain +is in the [configuration reference](../reference/configuration.md#precedence). + +## See also + +- [Configuration reference](../reference/configuration.md) - the file format, every key, deployments, precedence. +- [`dotbot fw`](fw.md) - reads its `[fw]` keys (`segger_dir`, `firmware_repo`) from this same config. diff --git a/doc/cli/device.md b/doc/cli/device.md new file mode 100644 index 00000000..ca1f94ab --- /dev/null +++ b/doc/cli/device.md @@ -0,0 +1,139 @@ +# `dotbot device` - flash one cabled board + +`dotbot device` programs **one board on your desk**, connected over a cable. It +talks to the board's on-board programmer over the SWD/J-Link interface - no +external probe needed for normal flashing. On the **DotBot v3** the programmer +(a J-Link-OB / DAPLink behind an SWD mux) is reached over **USB-C**; on an +nRF5340-DK over its micro-USB port. A separate J-Link is only required for +[`flash-programmer`](#flash-programmer). + +To put firmware on the **whole fleet over the air**, use [`swarm`](swarm.md) +instead. To build the `.hex` first, see [`fw`](fw.md). + +```{tip} +**`device flash-mari-gateway` flashes _firmware onto a board_.** The host-side +UART↔MQTT bridge process is a different thing - that's [`run gateway`](run.md). +``` + +```{note} +`dotbot device` drives **`nrfjprog`** (Nordic's nRF Command Line Tools), not +`nrfutil` - install it and put its `bin/` on your `PATH` before flashing. +``` + +## Commands + +| Command | What it does | +|---|---| +| `flash ` | Whole-chip program one app (or a `.hex`/`.bin`) onto the board | +| `flash-mari-gateway` | Turn an nRF5340-DK into the swarm gateway (both cores + network id) | +| `flash-swarmit-sandbox` | Turn a DotBot v3 into a swarm sandbox host (bootloader + netcore + id) | +| `flash-programmer` | Re-flash the board's on-board debug chip (J-Link OB / DAPLink) - needs a J-Link | +| `info` | Read a board's provisioning state (chip id + network id) | + +## Flash an app + +`flash` resolves `` to a cached `-.hex` (under +`~/.dotbot/artifacts/`), **building it if the file isn't there**; an explicit +`.hex`/`.bin` path is flashed as-is. + +```bash +export DOTBOT_FIRMWARE_REPO=$(pwd)/repos/DotBot-firmware + +# Build, then flash the bare DotBot app onto a DotBot v3 (board defaults to dotbot-v3) +dotbot fw artifacts --app dotbot +dotbot device flash dotbot -s 77 +``` + +`-b/--board` selects the **chip family and core** to program. The nrfjprog family +and coprocessor are derived from it: nRF52 boards → `-f NRF52`, no coprocessor; +nRF5340 → `-f NRF53` with `CP_APPLICATION`, or `CP_NETWORK` for a `*-net` board. + +`-b` only sets the family/core nrfjprog is *told* to program; the CLI doesn't +read it back from the attached chip. Make sure the cabled board matches `-b` - +e.g. don't flash an nRF53 image onto a connected nRF52 (or vice versa). + +```bash +# Gateway onto an nRF52840-DK (device flash picks -f NRF52 from the board) +dotbot fw artifacts --app dotbot_gateway -t nrf52840dk +dotbot device flash dotbot_gateway -b nrf52840dk -s 10 +``` + +### nRF5340 = two cores + +The nRF5340's radio lives on the **net core**, so an app-core app also needs a +net-core image. Build and flash each for its own target - the app image is +`dotbot_gateway`, the net image is **`nrf5340_net`** (not `dotbot_gateway`): + +```bash +# App core +dotbot fw artifacts --app dotbot_gateway -t nrf5340dk-app +dotbot device flash dotbot_gateway -b nrf5340dk-app -s 10 + +# Net core (-b *-net routes to CP_NETWORK) +dotbot fw artifacts --app nrf5340_net -t nrf5340dk-net +dotbot device flash nrf5340_net -b nrf5340dk-net -s 10 +``` + +**`flash` flags** (see `dotbot device flash --help` for the full list): + +| Flag | Meaning | +|---|---| +| `-b, --board` | Target board → chip family + core (default `dotbot-v3`) | +| `-s, --sn-starting-digits` | J-Link serial **prefix**, e.g. `77` (v3) or `10` (DK) | +| `--sandbox` | Resolve the sandbox-app flavor (`.bin`) | +| `--build-config` | `Debug` \| `Release` (default `Release`) | + +## Flash a role + +`flash-mari-gateway` and `flash-swarmit-sandbox` flash a **complete system firmware** +(both cores) and write the **network identity** in one shot. They auto-fetch the +named release into `~/.dotbot/artifacts/` if it isn't cached. + +```bash +# nRF5340-DK → swarm gateway +dotbot device flash-mari-gateway --swarm-id 0100 -f 0.8.0rc1 -s 10 + +# DotBot v3 → swarm sandbox host (the firmware that runs OTA apps) +dotbot device flash-swarmit-sandbox --swarm-id 0100 -f 0.8.0rc1 -s 77 +``` + +| Flag | `flash-mari-gateway` | `flash-swarmit-sandbox` | +|---|---|---| +| `--swarm-id` | 16-bit hex swarm id (or from config) | 16-bit hex swarm id (or from config) | +| `-f, --fw-version` | release to flash (default: the pinned swarmit version) | release to flash (default: the pinned swarmit version) | +| `-s, --sn-starting-digits` | J-Link serial prefix | J-Link serial prefix | +| `-l, --calibration` | - | optional LH2 calibration file to bake in | + +A board flashed with `flash-swarmit-sandbox` is what [`swarm flash`](swarm.md) +targets to run sandboxed apps over the air. + +## Inspect a board + +```bash +dotbot device info -s 77 +``` + +Reports the chip id and network identity. It never fails on a blank board - it +says *not provisioned* and how to fix it. + +## flash-programmer + +Re-flashes the on-board debug chip's own firmware (J-Link OB or DAPLink). This is +obscure, one-time-per-board bring-up and **requires an external J-Link**. + +```bash +dotbot device flash-programmer -p daplink -d ./programmer-firmware/ +``` + +| Flag | Meaning | +|---|---| +| `-p, --programmer-firmware` | `jlink` \| `daplink` (required) | +| `-d, --files-dir` | directory with the programmer firmware files (required) | +| `--probe-uid` | pyOCD probe UID, when multiple probes are attached | + +```{note} +**Never run `nrfjprog` (or these commands) under `sudo`.** One sudo run leaves +`/tmp/boost_interprocess/` owned by root and every later call fails with +*Operation not permitted*. Recover with +`sudo rm -rf /tmp/boost_interprocess`. +``` diff --git a/doc/cli/fw.md b/doc/cli/fw.md new file mode 100644 index 00000000..d43fb314 --- /dev/null +++ b/doc/cli/fw.md @@ -0,0 +1,179 @@ +# `dotbot fw` - firmware artifacts + +Build, fetch, and inventory firmware **without touching hardware**. Flashing +lives elsewhere: one cabled board → [`dotbot device`](device.md), the fleet +over the air → [`dotbot swarm`](swarm.md). + +You get **bare apps** by default. Add `--sandbox` for the TrustZone +non-secure (NS) flavor that runs inside the swarm sandbox host. + +## Setup + +`dotbot fw` builds from a local `DotBot-firmware` checkout via SEGGER Embedded +Studio (SES), so it needs to find both. The checkout defaults to +`./DotBot-firmware/`; SES is auto-detected only on macOS (a standard +`/Applications/SEGGER/` install), so on Linux/Windows builds fail with +"SEGGER Embedded Studio ... wasn't found" until you point at it. + +Your firmware checkout is per-project, so set it in the project's `./dotbot.toml` +(or export `DOTBOT_FIRMWARE_REPO`): + +```toml +# ./dotbot.toml (per project) +[fw] +firmware_repo = "/path/to/DotBot-firmware" +``` + +Your SES install rarely changes, so set it once per machine in +`~/.dotbot/config.toml`: + +```toml +# ~/.dotbot/config.toml (once per machine) +[fw] +segger_dir = "/path/to/SEGGER Embedded Studio X.YY" +``` + +> **First SES build needs the nRF + CMSIS_5 packages.** A fresh SES install has +> no chip headers, so the build fails with `fatal error: 'nrf.h' file not found`. +> In SES, open **Tools → Package Manager** and install the **nRF** and +> **CMSIS_5** packages (`nrf.h` ships in the nRF one). One-time per SES install. + +## Which command do I want? + +| Goal | Command | +|---|---| +| Compile an app, leave it in the SES `Output/` tree (path echoed) | `dotbot fw build` | +| Compile **and** collect a flat `-.hex` into the cache | `dotbot fw artifacts` | +| Download the pinned release(s) into `~/.dotbot/artifacts/-/` | `dotbot fw fetch [-S -f ]` | +| List targets you can build | `dotbot fw targets [--sandbox]` | +| List what's cached locally | `dotbot fw list` | +| A Makefile knob the CLI doesn't model | `dotbot fw make ` | + +**`build` vs `artifacts`**: both compile via SES. `build` stops once SES is +done (output stays buried in the per-target `Output/` tree). `artifacts` goes +one step further and copies a flat, predictably-named `-.hex` into +the cache (`~/.dotbot/artifacts/dotbot-firmware-local/`) - which is exactly +where `dotbot device flash ` and the swarm tools look. Reach for +`artifacts` when you intend to flash; `build` when you only want to know it +compiles. + +## `build` / `artifacts` flags + +Both share the same build options: + +| Flag | Meaning | +|---|---| +| `-a, --app ` | Build one app (default: every app for the target) | +| `-t, --target ` | Board/target (default: `dotbot-v3`) | +| `--build-config Debug\|Release` | Build configuration (default: `Release`) | +| `--sandbox` | TrustZone NS flavor → `sandbox-`, emits `.bin` | +| `--rebuild` | Force a full rebuild (default: incremental) | +| `-v, --verbose` | Full SES output | + +`artifacts` adds `--out ` (default `~/.dotbot/artifacts/dotbot-firmware-local/`) +and `--print-path` (report where the artifact would land without building). See +`dotbot fw --help` for the full list. + +> **Flag mismatch to remember:** `fw` selects a board with `--target/-t`, but +> [`device flash`](device.md) uses `--board/-b`. + +## Targets × apps + +What builds where (verified `2026-05-30`). Run `dotbot fw targets` to list +bare targets, `dotbot fw targets --sandbox` for the sandbox set. + +**Bare targets:** + +| Target | Chip | Apps available | +|---|---|---| +| `dotbot-v1` / `v2` / `v3` | DotBot board (v3 = nRF5340) | `dotbot`, `lh2_calibration`, `log_dump` | +| `nrf52833dk`, `nrf52840dk` | nRF52833 / nRF52840 DK | `dotbot`, `dotbot_gateway`, `dotbot_gateway_lr`, `lh2_calibration`, `lh2_mini_mote_app`, `lh2_mini_mote_test`, `log_dump`, `sailbot` | +| `nrf5340dk-app` | nRF5340 **app core** | `dotbot`, `dotbot_gateway`, `dotbot_gateway_lr`, `lh2_calibration`, `log_dump`, `sailbot`, `lh2_mini_mote_*` | +| `nrf5340dk-net` | nRF5340 **net core** | `dotbot_gateway`, `dotbot_gateway_lr`, `log_dump`, `nrf5340_net` | +| `sailbot-v1` | SailBot | `lh2_calibration`, `log_dump`, `sailbot` | +| `freebot-v1.0` | FreeBot | `freebot` | +| `lh2-mini-mote` | LH2 mini-mote | `lh2_calibration`, `lh2_mini_mote_*`, `log_dump` | +| `xgo-v1` / `v2` | XGO | `xgo` | + +**Sandbox targets** (`--sandbox` → `sandbox-dotbot-v2`, `sandbox-dotbot-v3`, +`sandbox-nrf5340dk`): apps `dotbot`, `dotbot-simple`, `motors`, `move`, +`rgbled`, `spin`, `timer`. These run over the air via +[`dotbot swarm`](swarm.md). + +Notes: +- The gateway (`dotbot_gateway`) builds for the **DK** targets, not the DotBot + boards - it runs on a DK plugged into your computer. +- The nRF5340 radio lives on the **net core**, so a gateway needs two images: + `dotbot_gateway` on `nrf5340dk-app` **and** `nrf5340_net` on `nrf5340dk-net`. + +## `fetch` - pinned firmware + +`dotbot fw fetch` downloads prebuilt firmware from two release sources - +**swarmit** (the swarm system images) and **DotBot-firmware** (the bare and +sandbox apps) - into `~/.dotbot/artifacts/-/`, each with a +`manifest.json` recording where it came from. + +With no flags it fetches the **exact versions this `dotbot` is pinned to**, so a +given `dotbot` always pulls a known-good, reproducible set: + +- **swarmit** is also a Python dependency, so its firmware version is read from + the installed `swarmit` package. +- **DotBot-firmware** is not a Python package, so the version `dotbot` is built + and tested against is declared in `dotbot` and bumped deliberately. + +Override per source when you need something else - the two version +independently, so `-f` requires `-S`: + +```bash +dotbot fw fetch # pinned versions, both sources +dotbot fw fetch -S dotbot-firmware -f latest # newest dotbot-firmware release +dotbot fw fetch -S swarmit -f 0.8.0rc2 # a specific swarmit version +``` + +The cache is user-level and shared across projects (override the location with +`$DOTBOT_ARTIFACTS_DIR`); `dotbot device flash` and the swarm tools resolve +their input from it. + +## Examples + +```bash +export DOTBOT_FIRMWARE_REPO=/path/to/DotBot-firmware + +# Bare DotBot app for a DotBot v3 → ~/.dotbot/artifacts/dotbot-firmware-local/dotbot-dotbot-v3.hex +dotbot fw artifacts --app dotbot + +# Just confirm an app compiles (no collection) +dotbot fw build --app sailbot -t nrf52840dk + +# Gateway for an nRF5340-DK - both cores +dotbot fw artifacts --app dotbot_gateway -t nrf5340dk-app +dotbot fw artifacts --app nrf5340_net -t nrf5340dk-net + +# Sandbox (NS) "spin" app for a DotBot v3 → .bin, for OTA via swarm +dotbot fw artifacts --app spin -t dotbot-v3 --sandbox + +# Download released firmware instead of building (see "fetch" above) +dotbot fw fetch # the versions this dotbot is pinned to (both sources) + +# See what's cached +dotbot fw list +``` + +## `make` - the escape hatch + +`dotbot fw make` runs `make` inside your `DotBot-firmware` checkout with the +workspace-resolved `SEGGER_DIR`, forwarding every argument verbatim. Use it +only when `build`/`artifacts` don't model the Makefile knob you need. + +```bash +dotbot fw make list-projects +``` + +Do **not** run `make docker` - that's the CI path and crawls under emulation on +this machine. + +## See also + +- [`dotbot device`](device.md) - flash an artifact onto one cabled board. +- [`dotbot swarm`](swarm.md) - push a sandbox app to the fleet over the air. +- [LH2 calibration (cabled)](../guides/lh2-calibration-cabled.md) - the `lh2_calibration` app workflow. diff --git a/doc/cli/index.md b/doc/cli/index.md new file mode 100644 index 00000000..c763f1ac --- /dev/null +++ b/doc/cli/index.md @@ -0,0 +1,70 @@ +# The `dotbot` CLI + +```{toctree} +:hidden: +fw +device +swarm +run +config +``` + +One CLI for the whole DotBot workflow: build firmware, flash one board, control a +whole swarm, and launch the host-side processes that tie it together - +from one bot to a thousand. + +```bash +dotbot --help +``` + +## The four commands + +`dotbot` has four top-level commands - pick by what you're doing right now: + +| Command | What it does | Reach for it when… | +|---|---|---| +| [`fw`](fw.md) | Build, fetch, and list firmware files. No hardware needed. | You want a `.hex`/`.bin` to flash later, or to see what builds. | +| [`device`](device.md) | Flash one cabled board and read its info. | A DotBot or DK is plugged into your USB port right now. | +| [`swarm`](swarm.md) | Drive the whole fleet over the air - status, OTA flash, start/stop, monitor. | You're operating many provisioned bots through a gateway. | +| [`run`](run.md) | Start host processes on your computer - controller, gateway bridge, simulator, demos, teleop. | You need the web UI, a gateway bridge, the simulator, or a demo. | + +Beyond the four namespaces, [`config`](config.md) scaffolds and inspects the +shared `dotbot.toml` the other commands read their defaults from. + +## Which one do I want? + +```text +Do I have hardware? +├── No ─────────────────────────► fw (build/fetch artifacts, simulator under run) +└── Yes + ├── One board on a cable ─────► device (flash app/role, read info) + └── A fleet over the air ─────► swarm (status, OTA flash, start/stop) + +Need a process running on my computer (UI, gateway bridge, demo)? ─► run +``` + +A few signposts so the namespaces don't blur together: + +- **`fw` never touches hardware.** It only produces or lists artifacts in + `~/.dotbot/artifacts/`. Flashing always happens under `device` (cabled) or + `swarm` (OTA). +- **Bare vs. sandbox artifacts.** `fw` builds bare apps (`.hex`) by default; + `fw artifacts --sandbox` builds TrustZone apps (`.bin`) - the payload `swarm` + flashes over the air. +- **Same word, different object.** `dotbot device flash-mari-gateway` flashes + *firmware onto a board*; `dotbot run gateway` starts the *host bridge + process*. They are not the same thing. +- **A DotBot v3 has an on-board programmer.** Normal flashing over USB-C needs + no external probe - a separate J-Link is only for + `dotbot device flash-programmer`. + +## Next + +- [`fw`](fw.md) - build, fetch, and list firmware artifacts. +- [`device`](device.md) - flash and inspect one cabled board. +- [`swarm`](swarm.md) - run experiments across the fleet. +- [`run`](run.md) - launch the controller, gateway bridge, simulator, and demos. +- [`config`](config.md) - scaffold and inspect the shared `dotbot.toml`. + +Two end-to-end walkthroughs put these together: [build and flash one +board](device.md), and [operate a swarm over the air](swarm.md). diff --git a/doc/cli/run.md b/doc/cli/run.md new file mode 100644 index 00000000..02db38cd --- /dev/null +++ b/doc/cli/run.md @@ -0,0 +1,101 @@ +# `dotbot run` - host-side processes + +`dotbot run` launches the things that run **on your computer**: the control +plane, the gateway bridge, a simulator, calibration, demos, and teleop +drivers. (`fw` / [`device`](device.md) / [`swarm`](swarm.md) are the things you +*manage*; `run` is the long-lived processes that talk to them.) + +```bash +dotbot run --help # the full list +``` + +| Subcommand | Launches | +|---|---| +| `controller` | Control plane: REST/WS API + web dashboard. The hub everything else talks to. | +| `gateway` | Host bridge: gateway firmware UART ↔ MQTT broker. | +| `simulator` | Standalone simulator (no hardware). | +| `lh2-calibration` | LH2 calibration on one cabled board (capture / apply); deployed bots use `swarm lh2-calibration`. | +| `demo` | Built-in research demos (qrkey phone bridge, …). | +| `keyboard` | Drive a DotBot from the keyboard. | +| `joystick` | Drive a DotBot from a joystick. | + +## `controller` - the control plane + web UI + +Connect to a swarm and serve the dashboard at `http://localhost:8000/PyDotBot/`. +`--conn` is one discriminated string: `mqtts://host:port`, a serial path, or +`simulator`. + +```bash +dotbot run controller --conn mqtts://argus.paris.inria.fr:8883 --swarm-id 1234 -w +dotbot run controller --conn /dev/ttyACM0 -w +``` + +| Flag | Meaning | +|---|---| +| `-n/--conn` | `mqtts://host:port`, serial path, or `simulator` | +| `-s/--swarm-id` | hex swarm id - **required for MQTT**, ignored for serial/simulator | +| `-w/--webbrowser` | open the dashboard automatically | +| `--csv-data-output` | record robot data to a CSV file | + +Full options and the dashboard tour live in +[the controller guide](../guides/controller.md). See `dotbot run controller --help`. + +## `gateway` - UART ↔ MQTT bridge + +Runs wherever the gateway firmware is plugged in. With `--mqtt-url` it bridges +serial frames to the broker; without it, it just prints what it receives. + +```bash +dotbot run gateway -m mqtts://argus.paris.inria.fr:8883 -p /dev/cu.usbmodem1234 +dotbot run gateway # autodetect port, print-only (no broker) +``` + +> **`run gateway` ≠ `device flash-mari-gateway`.** This is the *host process* that +> bridges a gateway board to MQTT. [`device flash-mari-gateway`](device.md) is the +> *firmware* you flash onto that board, once. Same word, different objects. + +## `simulator` - standalone simulator + +No hardware, no gateway. Exactly equivalent to `run controller --conn simulator`, +so it shares the controller's flags and serves the same dashboard. + +```bash +dotbot run simulator -w +``` + +## `lh2-calibration` - capture & apply (cabled) + +Lighthouse v2 calibration against a single serial-attached board. `collect` +opens a TUI to capture LH2 counts; `apply` writes the saved calibration out as +a C header. This is the cabled, bench path - for deployed bots, capture over the +air with [`swarm lh2-calibration`](swarm.md) instead. + +```bash +dotbot run lh2-calibration collect +dotbot run lh2-calibration apply ./lh2_calibration.h +``` + +See [the cabled LH2 calibration guide](../guides/lh2-calibration-cabled.md). To +capture without a cable, or to push a saved calibration to the fleet over the +air, use [`swarm lh2-calibration`](swarm.md). + +## `demo` - built-in demos + +```bash +dotbot run demo --list # what's available +dotbot run demo qr # qrkey phone bridge +``` + +## `keyboard` / `joystick` - teleop + +Drive a DotBot live through a running controller (start one with +`run controller` first). Both default to `localhost:8000`; pass `-d` to target a +specific robot by hex address. + +```bash +dotbot run keyboard +dotbot run joystick -j 0 -d 1234567890abcdef +``` + +See `dotbot run keyboard --help` / `dotbot run joystick --help` for the host, +port, and application (`dotbot`/`sailbot`) flags. diff --git a/doc/cli/swarm.md b/doc/cli/swarm.md new file mode 100644 index 00000000..badaa5e4 --- /dev/null +++ b/doc/cli/swarm.md @@ -0,0 +1,134 @@ +# `dotbot swarm` - operate the fleet over the air + +Run experiments across many robots at once. `dotbot swarm` drives the +[SwarmIT](https://github.com/DotBots/swarmit) orchestration backend: it +OTA-flashes a sandbox app to every bot, starts/stops it, and watches status - +all wirelessly through a gateway. + +For one cabled board, use [`device`](device.md). To build the apps you flash, +see [`fw`](fw.md). The host bridge and dashboard come from [`run`](run.md). + +## The flow + +```text +1. provision (once) device flash-mari-gateway + device flash-swarmit-sandbox +2. host bridge run gateway (UART <-> MQTT) +3. build the payload fw artifacts --sandbox (or fw fetch) +4. operate swarm flash | start | stop | status | monitor +``` + +## 1. Provision once + +Each robot needs the SwarmIT sandbox-host firmware; the gateway is an +nRF5340-DK running the Mari gateway firmware. Both are cabled flashes over +USB-C (the DotBot v3 has an on-board programmer - no separate J-Link needed). +Details and chip caveats live in [`device`](device.md). + +```bash +dotbot device flash-mari-gateway --swarm-id 1234 -s 10 -f 0.8.0rc1 # a DK -> gateway, net id 0x1234 +dotbot device flash-swarmit-sandbox --swarm-id 1234 -s 77 -f 0.8.0rc1 # each bot -> sandbox host +``` + +## 2. Start the host bridge + +The gateway board needs a host process bridging its UART to MQTT: + +```bash +dotbot run gateway -m mqtts://argus.paris.inria.fr:8883 -p /dev/cu.usbmodem... +``` + +`run gateway` is the host *process*; `device flash-mari-gateway` flashed the +*firmware* - same word, different objects. + +## 3. Build the OTA payload + +The OTA payload is a **sandbox** app - a TrustZone non-secure `.bin`. Build it, +or fetch a pre-compiled release: + +```bash +dotbot fw artifacts --sandbox # builds -> ~/.dotbot/artifacts/dotbot-firmware-local/-sandbox-.bin +dotbot fw fetch # or pull the pinned releases into ~/.dotbot/artifacts/-/ +``` + +Sandbox apps include `dotbot`, `move`, `rgbled`, `spin`, `timer`. Artifact +names look like `spin-sandbox-dotbot-v3.bin`. (Bare `.hex` apps are *not* OTA +payloads - those are cabled via [`device flash`](device.md).) + +## 4. Connect + +The connection is given as global options *before* the subcommand, or in a +`.toml` via `-c`: + +| Option | Meaning | +|---|---| +| `-n`, `--conn`, `--connection` | one string: `mqtts://host:port` (broker) or `/dev/ttyACM0` (serial gateway) | +| `-s`, `--swarm-id` | hex swarm id - **required for MQTT**, ignored for serial | +| `-c`, `--config-path` | a `.toml` carrying the same fields | +| `-b`, `--baudrate` | serial baudrate (default `1000000`) | +| `-d`, `--devices` | restrict to a comma-separated subset of addresses | + +See `dotbot swarm --help` for the full list. + +```bash +dotbot config init --conn mqtts://argus.paris.inria.fr:8883 --swarm-id 1234 +``` + +This writes `./dotbot.toml`; `dotbot swarm` discovers it from the current +directory like the other `dotbot` commands (pass `--conn` / `--swarm-id` / `-c` +to override). If the broker needs auth, set `DOTBOT_MQTT_USER` / +`DOTBOT_MQTT_PASS`. + +## 5. Operate the fleet + +```bash +dotbot swarm status # who's out there + their state +dotbot swarm status -w # keep watching +dotbot swarm flash ~/.dotbot/artifacts/dotbot-firmware-local/spin-sandbox-dotbot-v3.bin -ys +dotbot swarm stop # back to bootloader (before re-flashing) +dotbot swarm start # (re)start the loaded app +dotbot swarm monitor # tail SWARMIT_EVENT_LOG from bots +dotbot swarm message "hello" # custom text to the bots +``` + +To replace a running experiment: `stop`, then `flash ... -ys`. + +### `swarm flash` flags + +| Flag | Meaning | +|---|---| +| `-y`, `--yes` | flash without the confirmation prompt | +| `-s`, `--start` | start the app once flashed | +| `-t`, `--ota-timeout` | seconds per OTA ACK (default `0.7`) | +| `-r`, `--ota-max-retries` | retries per OTA message (default `10`) | + +## 6. LH2 calibration over the air + +Capture and push a Lighthouse-2 calibration for one DotBot without a cable, +driving it over the swarm. The arena geometry and `-d` sizing live in the +[LH2 calibration guide](../guides/lh2-calibration.md). + +```bash +dotbot swarm stop # capture only runs in READY +dotbot swarm lh2-calibration collect --device BC3D... -d 500 # capture from one bot -> solve -> save +dotbot swarm lh2-calibration push ~/.dotbot/calibration-.toml # apply to every ready bot +``` + +`collect` walks one bot through the four arena corners over the air, solves the +homography, and saves it under `~/.dotbot/`. `push` (no `--device`) then sends +that calibration to **every ready bot** - the arena shares one transform. +(`collect --push` is a single-bot shortcut: it sends only to the captured bot.) +`push` takes a `calibration-*.toml` or the legacy raw payload - the format is +picked by file extension. Get the `--device` address from `dotbot swarm status`. + +## Two web servers - don't mix them up + +| Command | What it serves | Default port | +|---|---|---| +| `dotbot run controller -w` | drive/visualize Web UI + REST/WS | `8000` | +| `dotbot swarm serve` | SwarmIT FastAPI orchestration backend | `8001` | + +`dotbot swarm` auto-discovers a running `serve` daemon; pass `--no-server` to +skip the probe and run an in-process controller for that one invocation. Use +`serve --local` for a zero-config local backend. + +See `dotbot swarm --help` for every flag. diff --git a/doc/conf.py b/doc/conf.py index 52eb0e2f..35b78e48 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -58,6 +58,8 @@ html_static_path = ["_static"] myst_enable_extensions = ["html_image"] +# Generate slugged anchors for headings so `[text](#heading-slug)` links resolve. +myst_heading_anchors = 3 # Define the json_url for our version switcher. json_url = "https://pydotbot.readthedocs.io/en/latest/_static/switcher.json" @@ -117,7 +119,26 @@ # -- Options for linkcheck --------------------------------------------- -linkcheck_ignore = [r"http://localhost:\d+/"] +linkcheck_ignore = [ + r"http://localhost:\d+/", + # nordicsemi.com's WAF returns 403 to the linkcheck bot; the link is valid + # for humans (the nRF Command Line Tools download linked from the README). + r"https://www\.nordicsemi\.com/", + # The README deep-links into this same docs site; those pages exist only + # once this build is published, so linkcheck can't reach them yet. + r"https://pydotbot\.readthedocs\.io/", + # YouTube (demo video + its thumbnail) bot-blocks the linkcheck crawler. + r"https://www\.youtube\.com/", + r"https://img\.youtube\.com/", + # Badge services (shields.io, badge.fury.io) are decorative and + # intermittently time out or rate-limit the linkcheck bot. + r"https://img\.shields\.io/", + r"https://badge\.fury\.io/", + # dotbots.org and segger.com return 403 to the linkcheck bot (WAF / + # user-agent block); both links are valid for humans. + r"https?://www\.dotbots\.org", + r"https://www\.segger\.com/", +] # -- Options for autosummary/autodoc output ----------------------------------- autosummary_generate = True diff --git a/doc/getting_started.md b/doc/getting_started.md deleted file mode 100644 index 3959d825..00000000 --- a/doc/getting_started.md +++ /dev/null @@ -1,104 +0,0 @@ -# Getting started - -This document will help guide you through the setup of PyDotBot connected to -a DotBot gateway and a DotBot robot. - -## Prerequisites - -1. Make sure that you have access to the internet since the controller has to -connect to [https://broker.hivemq.com](https://broker.hivemq.com) to communicate -with the web client. - -2. Make sure you have access to an nRF DK board (nRF52833DK, nRF52840DK or -nRF5340DK) and to a DotBot (v1 or v2). - -3. Follow the instructions in the -[DotBot firmware getting started page][dotbot-firmware-getting-started]. - -## Install PyDotBot - -Use pip to install the latest version of PyDotBot from [pypi][pydotbot-pypi]: - -``` -pip install pydotbot -U -``` - -## Setup the gateway - -The gateway is an nRF DK used to bridge the UART communication between PyDotBot -running on a computer and the BLE radio used to communicate wirelessly with the -DotBot(s). - -1. Connect the nRF DK gateway to your computer - -2. Identify the TTY port it is connected to. On Linux, it should be `/dev/ttyACM0`. - On Windows, check the device manager, it should be `COM1`, `COM2`, `COM3`, etc. - If using an nRF5340DK, you might see 2 TTY port, use the one with the lowest - id. - -3. From a terminal window (or powershell on Windows), run `dotbot-controller` - with the TTY port you identified above and the `--webbrowser` flag to - automatically open the web client: - -``` -dotbot-controller --port --webbrowser -``` - -At this point, if the DotBot is powered on with fully charged batteries, you -should see an output in the logs that looks something like: - -``` -Welcome to the DotBots controller (version: 0.xx). -2023-11-29T07:55:11.725907Z [info ] Lighthouse initialized [pydotbot] context=dotbot.lighthouse2 -2023-11-29T07:55:11.726746Z [info ] Starting web server [pydotbot] context=dotbot.server -2023-11-29T07:55:11.739085Z [info ] Serial port thread started [pydotbot] context=dotbot.serial_interface -2023-11-29T07:55:12.197714Z [info ] New dotbot [pydotbot] application=DotBot context=dotbot.controller msg_id=90350129 payload_type=ADVERTISEMENT source=9903ef26257feb31 -``` - -## Control your DotBot - -1. In the web client opened in the browser, you should have one item - corresponding to your DotBot. - -2. Select it by clicking on the DotBot item: - -```{image} _static/images/pydotbot-ui-activate.png -:alt: Single DotBot item not selected -:class: bg-primary -:width: 400px -:align: center -``` - -3. The item should now be expanded: a joystick and a color picker widgets are - visible: - -```{image} _static/images/pydotbot-ui-active.png -:alt: Single selected DotBot item, with widgets -:class: bg-primary -:width: 400px -:align: center -``` - -4. Check that you can control the DotBot: - - by clicking on the joystick and dragging it in the direction that you want - the DotBot to move - - by using the color selector in the UI - -5. In a separate command window, launch `dotbot-keyboard`: -``` -Welcome to the DotBots keyboard interface (version: 0.16). -2023-12-08T10:07:32.597536Z [info ] Controller initialized [pydotbot] context=dotbot.keyboard -``` - -6. Check that you can control the DotBot using your keyboard: - - control it using the arrow keys - - change the RGB LED color by pressing "r", "g", "b", "y", "w", "n" keys -```{admonition} Note -:class: info -You might have to set the mouse focus on a separate application to have the keyboard -key events correctly taken into account. This is a limitation of the `pynput` -library used to track the keyboard events. -``` - -[dotbot-firmware-getting-started]: https://dotbot-firmware.readthedocs.io/en/latest/getting_started.html -[pydotbot-pypi]: https://pypi.org/project/pydotbot/ diff --git a/doc/guides/controller.md b/doc/guides/controller.md new file mode 100644 index 00000000..ee009066 --- /dev/null +++ b/doc/guides/controller.md @@ -0,0 +1,81 @@ +# Run the controller + web UI + +The controller is the host-side control plane: it talks to your gateway (or a +simulator), exposes a REST + WebSocket API, and serves a web UI to drive your +robots. + +## Start it + +Point the controller at a connection and open the web UI: + +```bash +# serial gateway plugged into your computer (no swarm-id needed) +dotbot run controller --conn /dev/ttyACM0 -w + +# a swarm over MQTT (swarm-id required - the broker carries many swarms) +dotbot run controller --conn mqtts://argus.paris.inria.fr:8883 --swarm-id 1234 -w + +# no hardware at all - pure software simulator +dotbot run controller --conn simulator -w +``` + +`--conn` takes one string: a serial device path (`/dev/ttyACM0`, `COM3` on +Windows), an MQTT broker (`mqtts://host:port`), or `simulator`. + +`-w` / `--webbrowser` opens a tab automatically. Otherwise browse to + yourself. + +| Flag | What it does | +|---|---| +| `-n, --conn` | Connection: serial path, `mqtts://host:port`, or `simulator` | +| `-s, --swarm-id` | Swarm id in hex (required for MQTT, ignored otherwise) | +| `-w, --webbrowser` | Open the web UI automatically | +| `--controller-http-port` | HTTP/REST port (default `8000`) | +| `--config-path` | Path to a `.toml` config file | +| `--dotbot / --sailbot` | With `--conn simulator`: which robot to simulate | + +See `dotbot run controller --help` for the full list (logging, CSV export, map +size, background map, simulator init state). + +`dotbot run simulator` is shorthand for `dotbot run controller --conn simulator` - try +the UI with no robot or gateway. + +## Use a config file + +Save your connection once instead of repeating flags: + +```bash +# save where to connect (writes ./dotbot.toml) +dotbot config init --conn mqtts://broker:8883 --swarm-id 1234 + +# the controller picks it up automatically when run from here +dotbot run controller + +# override the saved connection for one run (a simulator instead) +dotbot run controller --conn simulator +``` + +CLI flags override config-file values when both are given. See the +[configuration reference](../reference/configuration.md) for how the file is +discovered and the full schema. + +## The web UI + +At the page lists every DotBot the controller +sees. Select one to control it: + +- **Joystick** - a virtual joystick drives the selected bot. +- **RGB LED** - pick a color and the bot's LED follows. +- If you flashed Lighthouse 2 localization, bots report their `(x, y)` position + on the map (see [LH2 calibration](lh2-calibration.md)). + +## Firefox websockets note + +If the web UI does not connect under Firefox, the WebSocket stream is likely +being blocked. Open `about:config` (Ctrl + L, then type it), find +`network.http.http2.websockets`, and set it to `false`. + +## Next steps + +- Flash robots and a gateway first - see [device flashing](../cli/device.md). +- Operate the whole fleet over the air - see [swarm](../cli/swarm.md). diff --git a/doc/guides/index.md b/doc/guides/index.md new file mode 100644 index 00000000..60fe7321 --- /dev/null +++ b/doc/guides/index.md @@ -0,0 +1,23 @@ +# Guides + +Task-oriented walkthroughs that span several commands. + +```{toctree} +:hidden: +simulator +one-bot +controller +lh2-calibration +lh2-calibration-cabled +``` + +- [Try it in the simulator](simulator.md) - run the full UI and script bots with + no hardware. +- [Drive a single DotBot](one-bot.md) - build, flash, and control one bot end to + end. +- [Run the controller + web UI](controller.md) - drive and visualize a swarm + from the browser. +- [Lighthouse 2 localization](lh2-calibration.md) - give your bots real-world + `(x, y)` positions, calibrated over the air. +- [LH2 calibration over a cable](lh2-calibration-cabled.md) - the bench + alternative, for a single USB-connected bot. diff --git a/doc/guides/lh2-calibration-cabled.md b/doc/guides/lh2-calibration-cabled.md new file mode 100644 index 00000000..9f20122c --- /dev/null +++ b/doc/guides/lh2-calibration-cabled.md @@ -0,0 +1,87 @@ +# LH2 calibration over a serial cable + +The bench alternative to the [over-the-air flow](lh2-calibration.md): calibrate a +single DotBot connected over USB, with no swarm provisioned. Reach for this when +you're working with one bot on the bench, or before the fleet is set up. For +already-deployed bots, prefer the over-the-air flow - no cable, no firmware swap. + +What LH2 calibration is, and the arena geometry (the `-d` square sizing), are +covered in the [main guide](lh2-calibration.md); this page is just the cabled +capture path. + +## Prerequisites + +- A DotBot v3 you can cable to your machine over USB-C (no external probe - the + v3 flashes over its on-board programmer). +- Two LH2 base stations facing the arena, and a square marked on the floor (see + [Sizing `-d`](lh2-calibration.md) in the main guide). +- The `[calibrate]` extra: + +```bash +pip install --pre 'pydotbot[calibrate]' +``` + +## 1. Flash the capture firmware + +The `lh2_calibration` app streams raw LH2 counts over serial. Flash it to the +cabled bot (see [device](../cli/device.md) for serial-prefix selection): + +```bash +dotbot device flash lh2_calibration -s 77 # board defaults to dotbot-v3 +``` + +## 2. Capture the four reference points + +Place the bot on the floor square and run the TUI. `-d` is the side length of +the square, in millimeters: + +```bash +dotbot run lh2-calibration collect -p /dev/cu.usbmodem... -d 500 +``` + +Move the bot to each corner - Top left -> Top right -> Bottom left -> Bottom +right - pressing the matching button in the TUI at each. When all four are +captured, save. The calibration is written under `~/.dotbot/` (a +`calibration-.toml`), the same place the over-the-air flow uses. + +| Flag | Default | Meaning | +|---|---|---| +| `-p`, `--port` | auto-detect | Serial port of the calibration firmware. | +| `-d`, `--distance` | calibration default | Square side length, **in mm**. | +| `-n`, `--extra-lh-num` | `0` | Extra base stations beyond the first (0–5). | +| `--input-data` | - | Re-process a saved capture instead of capturing live. | + +See `dotbot run lh2-calibration collect --help` for the full list. + +## 3. Use the calibration + +Send the saved calibration to the fleet over the air (the same command the +over-the-air flow uses - stop any running app first): + +```bash +dotbot swarm stop +dotbot swarm lh2-calibration push ~/.dotbot/calibration-.toml +``` + +### Bake it into the bootloader (header path) + +For a fresh board whose bootloader bakes the calibration in at compile time +(rather than receiving it over the air), export the saved calibration as a C +header instead: + +```bash +dotbot run lh2-calibration apply ./lh2_calibration.h +``` + +The swarmit secure bootloader `#include`s this file; rebuild and reflash the +bootloader for it to take effect. For already-running bots, prefer the +over-the-air push above - no reflash needed. + +## Troubleshooting + +- **No counts in the TUI** - wrong `-p` port, or the bot can't see both base + stations. Confirm line-of-sight and that the base-station LEDs are steady. +- **Positions look skewed or mirrored** - the corners were captured out of + order. Re-run `collect` and follow TL -> TR -> BL -> BR exactly. +- **Positions are scaled wrong** - `-d` didn't match the real square. It's in + millimeters, not centimeters. diff --git a/doc/guides/lh2-calibration.md b/doc/guides/lh2-calibration.md new file mode 100644 index 00000000..f39eb36d --- /dev/null +++ b/doc/guides/lh2-calibration.md @@ -0,0 +1,117 @@ +# Lighthouse 2 (LH2) calibration + +Lighthouse 2 gives every DotBot a real-world **(x, y) position** on your arena +floor. Two SteamVR base stations sweep the room with IR; each bot's LH2 sensor +times the sweeps. Calibration is the one-time step that maps those raw sweep +counts to metric coordinates: you place one bot on four known corners of a +square, capture, and the resulting transform is pushed to the whole fleet. + +You do this once per physical setup (move a base station -> recalibrate). + +**The default flow is over the air** - drive one already-deployed bot through the +corners over the swarm, no cable and no firmware swap. If you'd rather calibrate +a single bot on the bench over USB, see +[LH2 calibration over a cable](lh2-calibration-cabled.md). + +## Prerequisites + +- A provisioned swarm: a gateway plus sandbox-host bots, reachable from your + config (see [swarm](../cli/swarm.md)). +- Two LH2 base stations mounted ~2 m up, facing the arena. +- A square marked on the floor with a known side length. +- The `[calibrate]` extra (the homography solve uses opencv): + +```bash +pip install --pre 'pydotbot[calibrate]' +``` + +## Capture and push + +The calibration is a property of the **arena** (the base-station layout), not the +individual bot, so you capture once from any one bot and apply the result to the +whole fleet. Two steps: + +```bash +dotbot swarm stop # put the bots in READY +dotbot swarm lh2-calibration collect \ + --device BC3D3C8A2A6F8E68 -d 500 # capture from one bot -> solve -> save +dotbot swarm lh2-calibration push \ + ~/.dotbot/calibration-.toml # apply to every ready bot +``` + +`collect` walks one bot through the four corners - **top-left -> top-right -> +bottom-left -> bottom-right** - (capture only runs while the bot is in READY, so +`swarm stop` first). Each prompt triggers a raw-count capture over the air; it +then solves the homography and saves a `calibration-.toml` under `~/.dotbot/` +(the path is printed at the end). Find the `--device` address with +`dotbot swarm status`. + +`push` with **no `--device`** sends that calibration to **every ready bot** - the +whole arena shares one transform. It accepts a `calibration-*.toml` or the legacy +raw `calibration.out` payload; the format is picked by file extension. + +```{note} +`collect --push` is a **single-bot shortcut**: it sends the result to *only* the +`--device` bot you captured from (handy to spot-check that one bot, or for a +single-bot setup). To calibrate the fleet, run the standalone `push` above - it +targets all ready bots. +``` + +Once pushed, the bots report positions, which show up live in the +[controller](../cli/run.md) Web UI. + +### `collect` flags + +| Flag | Default | Meaning | +|---|---|---| +| `--device` | (required) | DotBot link-layer address in hex (from `dotbot swarm status`). | +| `-d`, `--distance` | calibration default | Square side length, **in mm** (see sizing below). | +| `-n`, `--conn` / `-s`, `--swarm-id` | from config | Swarm connection, like the other `dotbot swarm` commands. | +| `--timeout` | `5` s | Seconds to wait for each capture before re-triggering. | +| `--retries` | `3` | Re-trigger this many times per corner before giving up. | +| `--tag` | - | Arena/setup label (e.g. `office-2x2m`) added to the filename + metadata. | +| `--push` | off | After solving, send to the captured `--device` bot **only** (use the standalone `push` for the whole fleet). | + +See `dotbot swarm lh2-calibration collect --help` for the full list. + +## Sizing `-d` + +`-d` is the **side of your reference square, in millimeters**. The usable arena +is **5× the square side**, with the square centered (a `2·d` margin on every +side): + +``` + ←─────────────── 5·d ────────────→ +┌──────────────────────────────────┐ ↑ +│ │ │ +│ │ │ +│ ←─── d ───→ │ │ +│ TL ●─────────● TR │ │ +│ │ │ │ 5·d +│ │ │ │ │ +│ BL ●─────────● BR │ │ +│ │ │ +│←── 2·d ──→ │ │ +└──────────────────────────────────┘ ↓ + + ⌖ LH2 base station (mounted ~2 m up, facing the arena) +``` + +`TL/TR/BL/BR` are the four reference points you place the bot on; `d` is the +square side (`--distance`, in mm), `5·d` the resulting arena. + +| `-d` | Square | Usable arena | +|---|---|---| +| `400` | 40 cm | 2.0 m × 2.0 m | +| `500` | 50 cm | 2.5 m × 2.5 m | +| `800` | 80 cm | 4.0 m × 4.0 m (used for the 725-bot Limerick run) | + +## Troubleshooting + +- **Capture times out** - the bot isn't in READY (run `dotbot swarm stop` + first), or it can't see both base stations. The address passed to `--device` + must match one from `dotbot swarm status`. +- **Positions look skewed or mirrored** - the corners were captured out of + order. Re-run `collect` and follow TL -> TR -> BL -> BR exactly. +- **Positions are scaled wrong** - `-d` didn't match the real square. It's in + millimeters, not centimeters. diff --git a/doc/guides/one-bot.md b/doc/guides/one-bot.md new file mode 100644 index 00000000..7b059608 --- /dev/null +++ b/doc/guides/one-bot.md @@ -0,0 +1,53 @@ +# Drive a single DotBot + +Build and flash one DotBot and a gateway, cable them to your computer, and drive +the bot from the web UI. This is the smallest real-hardware setup. For the +no-hardware path, use the [simulator](simulator.md) instead. + +Building firmware needs SEGGER Embedded Studio and `nrfjprog` (see the README +prerequisites). To skip building, fetch a pre-built release with `dotbot fw +fetch -f ` and flash those artifacts instead. + +## 1. Build and flash the DotBot + +The DotBot v3 is an nRF5340, which has two cores - the application core (your +app) and the network core (the radio) - so you build and flash two images: + +```bash +# build the bare dotbot apps into the cache (needs SEGGER Embedded Studio) +dotbot fw artifacts --app dotbot +dotbot fw artifacts --app nrf5340_net --target nrf5340dk-net +# cable-flash to the bot whose J-Link serial starts with 77 +dotbot device flash dotbot -s 77 # app core +dotbot device flash nrf5340_net -b nrf5340dk-net -s 77 # network core +``` + +## 2. Build and flash the gateway + +The gateway is a dev board (e.g. an nRF52840-DK) plugged into your computer; it +bridges the robot's radio to USB serial. + +```bash +dotbot fw artifacts --app dotbot_gateway --target nrf52840dk +# cable-flash to the DK whose J-Link serial starts with 10 +dotbot device flash dotbot_gateway -b nrf52840dk -s 10 +``` + +## 3. Drive it from the web UI + +With the gateway plugged in, point the controller at its serial port and open +the web UI: + +```bash +dotbot run controller --conn /dev/ttyACM0 -w # serial gateway; no swarm-id needed +``` + +Select the bot in the browser and steer it with the joystick. See the +[controller + web UI guide](controller.md) for the full UI tour, and +[`fw`](../cli/fw.md) / [`device`](../cli/device.md) for the firmware commands. + +## Next + +- Operate many bots over the air - the [`swarm`](../cli/swarm.md) reference. +- Add real-world positions - [LH2 calibration over a cable](lh2-calibration-cabled.md) + (or [over the air](lh2-calibration.md) once you've provisioned a swarm). diff --git a/doc/guides/simulator.md b/doc/guides/simulator.md new file mode 100644 index 00000000..75b1e717 --- /dev/null +++ b/doc/guides/simulator.md @@ -0,0 +1,66 @@ +# Try it in the simulator + +The simulator runs the **full controller and web UI with no hardware** - no bot, +no gateway, no radio. It's the fastest way to see DotBot work, and because it +exposes the exact same REST/WebSocket API as the real controller, code you write +against the simulator runs unchanged against real robots. + +## Start it + +```bash +dotbot run simulator -w +``` + +This opens the web UI at driving a simulated +swarm. `dotbot run simulator` is shorthand for +`dotbot run controller --conn simulator`, so everything in the +[controller + web UI guide](controller.md) applies. Drive the bots from the +joystick and watch them on the map. + +## Drive one bot in a circle + +The simplest demo - it grabs the first bot the controller sees and drives it in a +circle. With the simulator running, in a second terminal: + +```bash +dotbot run demo circle +``` + +## Drive it from your own code + +That demo is ~15 lines talking to the controller's REST API over +[`requests`](https://pypi.org/project/requests/) (bundled with pydotbot) - here +is the whole thing, a template for your own scripts: + +```python +import requests, time + +BASE = "http://localhost:8000" +bot = requests.get(f"{BASE}/controller/dotbots").json()[0]["address"] + +# roll in a circle for ~5 s - left_y and right_y are the two wheel speeds +for _ in range(50): + requests.put(f"{BASE}/controller/dotbots/{bot}/0/move_raw", + json={"left_x": 0, "left_y": 60, "right_x": 0, "right_y": 30}) + time.sleep(0.1) +requests.put(f"{BASE}/controller/dotbots/{bot}/0/move_raw", + json={"left_x": 0, "left_y": 0, "right_x": 0, "right_y": 0}) +``` + +The full surface - every endpoint, the live WebSocket stream, and CSV data +logging - is in the [REST / WebSocket reference](../reference/rest.md) (or the +[MQTT bridge](../reference/mqtt.md)). A higher-level Python SDK is planned; today +you talk to the controller over REST/WebSocket/MQTT. + +## More examples + +```bash +dotbot run demo --list # what's available +dotbot run demo qr # phone-as-joystick over QrKey +``` + +Richer multi-bot scenarios - work-and-charge, charging-station, labyrinth, the +naming game, motion shapes - live in `dotbot/examples/`, each with its own +README and a simulator init state. They drive the controller over the same +REST/WebSocket API shown above, so they run against the simulator or real +hardware unchanged. diff --git a/doc/hardware/index.md b/doc/hardware/index.md new file mode 100644 index 00000000..983a431a --- /dev/null +++ b/doc/hardware/index.md @@ -0,0 +1,89 @@ +# Know your DotBot v3 + +A quick tour of the hardware you'll plug things into. This is orientation only - +for the PCB, schematics, and CAD see the +[DotBot-hardware repo](https://github.com/DotBots/DotBot-hardware). + +Three pieces make up a working setup: + +- **The DotBot v3** - the robot. An nRF5340-based wheeled bot. +- **The gateway** - an nRF5340-DK that bridges your computer to the swarm over the air. +- **A Lighthouse 2 base station** - for indoor localization (optional, per-experiment). + +## Cables and connectors + +What to have on hand (the two USB cables are the ones you'll reach for most): + +| Cable / connector | For | +|---|---| +| **USB-C to USB-A (or USB-C)** | Flash and power the DotBot v3 (its USB-C port, J2). | +| **micro-USB to USB-A (or USB-C)** | The nRF5340-DK gateway's on-board J-Link. | +| **Barrel-jack charger** (2.5 mm, 6-18 V) | Charges the DotBot v3 supercap (J4); free-roaming only. | + +## DotBot v3 - the robot + +The robot has two connectors you'll use: + +| Connector | What it's for | +|---|---| +| **USB-C (J2)** | Flash and program the bot. Also powers it while plugged in. | +| **Barrel jack (J4)** | Charges the on-board supercapacitor (the bot's "battery"). | + +**USB-C (J2) - flashing.** The DotBot v3 has an **on-board programmer** behind +the USB-C port: a J-Link-OB / DAPLink debug chip plus an SWD mux that routes the +debug lines to the nRF5340. **You do not need a separate J-Link** for normal +flashing - just a USB-C cable. Plug it in and flash: + +```bash +# cabled flash of one bot (board defaults to dotbot-v3) +dotbot device flash dotbot -s 77 +``` + +A standalone J-Link is only needed to re-flash the on-board programmer's *own* +firmware (`dotbot device flash-programmer`) - a rare, one-time bring-up step. +See [device](../cli/device.md) for the full flashing workflow. + +**Barrel jack (J4) - charging.** The barrel jack feeds the BQ24640 charger, +which tops up the on-board supercapacitor (a ~240 F stack at 3.0 V max). The +supercap is what runs the bot when it's untethered; expect short, fast charges +rather than a slow battery cycle. + +```{note} +The bot is powered whenever USB-C is connected, so you can flash and bench-test +without charging first. For free-roaming, charge via the barrel jack. +``` + +## Gateway - nRF5340-DK + +The gateway is a stock **Nordic nRF5340-DK** with its own on-board J-Link (over +the DK's micro-USB port). It runs the Mari gateway firmware and bridges your +host to the swarm radio. + +```bash +# flash the gateway role onto a DK (writes the network id + both cores) +dotbot device flash-mari-gateway --swarm-id 0100 -f 0.8.0rc1 -s 10 + +# then run the host-side UART<->MQTT bridge +dotbot run gateway +``` + +Geovane's serial-prefix convention: DotBot v3 boards start `77`, nRF5340-DKs +start `10` (the `-s` prefix selects which probe to talk to). See +[swarm](../cli/swarm.md) for driving the fleet once the gateway is up. + +## Lighthouse 2 base station + +For position tracking, the testbed uses **Valve Lighthouse 2** base stations. +Each DotBot v3 carries an LH2 sensor shield (a TS4231 light-to-digital receiver +with a photodiode) that decodes the base station's sweeping IR beams into a +position. One base station illuminates the arena; the bots compute where they +are from what they see. + +Once the optical setup is in place, calibrate it before relying on the +coordinates - see [LH2 calibration](../guides/lh2-calibration.md). + +## Next steps + +- [device](../cli/device.md) - flash an app or role onto one cabled board. +- [swarm](../cli/swarm.md) - control the whole fleet over the air. +- [DotBot-hardware](https://github.com/DotBots/DotBot-hardware) - schematics, BOM, and CAD. diff --git a/doc/index.md b/doc/index.md index 1388949e..fdcce713 100644 --- a/doc/index.md +++ b/doc/index.md @@ -1,9 +1,37 @@ ```{toctree} :hidden: -getting_started -rest -mqtt -api +:maxdepth: 2 +CLI +Python SDK +Hardware +Guides +Reference +``` + +```{admonition} Where do you want to start? +:class: tip + +New here? DotBots are small wheeled robots you drive from your browser or your +own code - one bot, or a swarm of hundreds. Pick a starting point: + +- **Try it with no hardware** - the simulator runs the full web UI with no bot + or gateway needed: `dotbot run simulator -w`. See the + [simulator guide](guides/simulator.md). +- **Get one bot moving** - build and cable-flash a single DotBot and gateway, + then drive it from the browser. See the [one-bot guide](guides/one-bot.md). +- **Run a swarm experiment** - provision and command many bots over the air. + The swarm quickstart below is the main path; then see [`swarm`](cli/swarm.md) + and [LH2 localization](guides/lh2-calibration.md). +- **Script it / collect data** - drive the swarm from your own code today over + [REST / WebSocket](reference/rest.md) or [MQTT](reference/mqtt.md), and log + runs with `dotbot run controller --csv-data-output`. (A higher-level + [Python SDK](sdk/index.md) is planned.) +- **Extend the platform** - every command and flag is in the + [CLI reference](cli/index.md); the firmware flows live under [`fw`](cli/fw.md) + and [`device`](cli/device.md). +- **Before flashing hardware** - the simulator and driving an existing swarm + need only Python; building and cable-flashing firmware also need SES and the + nRF Command Line Tools (`nrfjprog`). See **Prerequisites** below before you start. ``` ```{include} ../README.md diff --git a/doc/mqtt.md b/doc/mqtt.md deleted file mode 100644 index c28ad654..00000000 --- a/doc/mqtt.md +++ /dev/null @@ -1,211 +0,0 @@ -# MQTT - -For a brief introduction to MQTT, have a look at -[HiveMQ MQTT Essentials](https://www.hivemq.com/mqtt/). - -At startup the controller automatically connects by default to -[https://broker.hivemq.com](https://broker.hivemq.com), a fully open MQTT broker. -If you want to use a different broker, see `.env.example` for the list of -possible MQTT options. - -Then it subscribes to commands messages published to the -`/dotbots/2SzQsZWfOV8OXrWQtEEdIA==/0000/+/+/move_raw` and -`/dotbots/2SzQsZWfOV8OXrWQtEEdIA==/0000/+/+/rgb_led` topics, among others. These -topics are used to control the motors and on-board RGB LED. - -They can be described as follows: -`/dotbots/////` -where: -- `secret topic` is a base64 encoded topic, derived from a random 8 digits - pin code using [HKDF](https://en.wikipedia.org/wiki/HKDF), -- `swarm-id` is a 4 hexadecimal string (2B long) identifier corresponding to a swarm, - typically all DotBots behind a single gateway -- `dotbot-address` is a 18 hexadecimal string (8B long) unique identifier of a DotBot, -- `application` is the type of application (0: DotBot, 1: SailBot) -- `command` is the type of command (`move_raw` or `rgb_led`) - -Since all messages are exchanged unauthentified via a public broker, all payloads -exchanged between MQTT clients and the controller are encrypted using -the standard [JSON Web Encryption protocol](https://datatracker.ietf.org/doc/html/rfc7516). -The symmetric keys used to encrypt the payload are also derived from a random 8 digits -pin code using [HKDF](https://en.wikipedia.org/wiki/HKDF). -All topics used by one controller and its PyDotBot clients use the same -`/dotbots/` base topic to make sure multiple controller running -at the same time won't interfere. - -One last thing about the 8 digit pin code: it rotates every 15 minutes (with a -grace period of 2 minutes) to ensure it cannot reused later and to make brut -force attacks harder. This means that every 15 minutes, the encryption key and -base topic changes for a given controller. All clients are notified of this -change and recomputes (or rederive) their key/topic accordingly. - -## Prerequisites - -Make sure you already followed the [getting started](getting_started) page and -have a functional setup with `dotbot-controller` running and connected to a -nRF DK gateway. - -To interact with the MQTT broker, you will use a Python script that require -several packages: -- [paho-mqtt](https://pypi.org/project/paho-mqtt) to connect and publish - messages to the MQTT broker, -- [requests](https://pypi.org/project/requests/) to directly fetch dotbots and - the pin code from the controller REST api, -- [cryptography](https://pypi.org/project/cryptography/) to derive the secret - topic and encryption key using HKDF, -- [joserfc](https://pypi.org/project/joserfc/) to encrypt the payload using JSON Web Encryption standard. - -Install all the Python dependencies using pip: -``` -pip install cryptography joserfc paho-mqtt requests -``` - -## The basics - -Running the controller is as easy as running the following command: - -``` -dotbot-controller -``` - -The logs should contain information about the MQTT broker connection and the -topic subscriptions: - -``` -Welcome to the DotBots controller (version: 0.17). -2024-01-11T13:42:02.738414Z [info ] Lighthouse initialized [pydotbot] context=dotbot.lighthouse2 -2024-01-11T13:42:02.740025Z [info ] Starting web server [pydotbot] context=dotbot.controller -2024-01-11T13:42:02.752914Z [info ] Serial port thread started [pydotbot] context=dotbot.serial_interface -2024-01-11T13:42:02.949352Z [info ] Connected [pydotbot] context=dotbot.mqtt flags=0 rc=0 receive_maximum=[10] topic_alias_maximum=[5] -2024-01-11T13:42:03.128297Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/command/0000/+/+/move_raw [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.128606Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/command/0000/+/+/rgb_led [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.128790Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/command/0000/+/+/waypoints [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.128940Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/command/0000/+/+/clear_position_history [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.129056Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/lh2/add [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.129159Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/lh2/start [pydotbot] context=dotbot.mqtt qos=(0,) -2024-01-11T13:42:03.129280Z [info ] Subscribed to /dotbots/2SzQsZWfOV8OXrWQtEEdIA==/request [pydotbot] context=dotbot.mqtt qos=(0,) -``` - -In the output above you can see that the _secret topic_ is `2SzQsZWfOV8OXrWQtEEdIA==`. - -Let's start by fetching available dotbots and the pin code using our own Python script: - -```py -import requests - -dotbots = requests.get('http://localhost:8000/controller/dotbots').json() - -if not dotbots: - print("No DotBot found!, exiting") - sys.exit(0) - -dotbot = dotbots[0] - -if dotbot["status"] != 0: - print("DotBot is not active!, exiting") - sys.exit(0) - -dotbot_addr = dotbot["address"] -print(f"DotBot address: {dotbot_addr}") - -pin_data = requests.get('http://localhost:8080/pin_code').json() -pin = str(pin_data["pin"]).encode() -print(f"Pin code: {pin.decode()}") -``` - -If you have a running DotBot, at this point you should have an output like this (with different address/pin values): -``` -DotBot address: 9903ef26257feb31 -Pin code: 30206157 -``` - -Know let's derive the secret topic and symmetric key using HKDF (extend the -previous script with the following content): - -```py -from cryptography.hazmat.primitives import hashes -from cryptography.hazmat.primitives.kdf.hkdf import HKDF - -from dotbot_utils.protocol import PROTOCOL_VERSION - -version = PROTOCOL_VERSION - -# derive topic and key -kdf_topic = HKDF( - algorithm=hashes.SHA256(), - length=16, - salt=b"", - info=f"secret_topic_{version}".encode() -) -topic = base64.urlsafe_b64encode(kdf_topic.derive(pin)).decode() -print(f"Secret topic: {topic}") - -kdf_key = HKDF( - algorithm=hashes.SHA256(), - length=32, - salt=b"", - info=f"secret_key_{version}".encode() -) -key = kdf_key.derive(pin) -print(f"Encryption AES key: {key.hex()}") -``` - -To ensure consistent values on both ends the salt parameter is left empty and -the info field contains a string built from the PyDotBot protocol version. This -ensures different PyDotBot protocol versions cannot be used together. - -At this point, when you run the script, you should have an output like: -``` -DotBot address: 9903ef26257feb31 -Pin code: 30206157 -Secret topic: 2RIP5S_xgDvu6wGJVZH6tw== -Encryption AES key: ecddf00497b30b57d965310a46b0502e06ebe89374e4167f15fc06a44e9a06bf -``` - -We are now ready to add the MQTT client code to our script which is based on paho-mqtt: - -```py -import paho.mqtt.client as mqtt - -# Connect to the MQTT broker -client = mqtt.Client(protocol=mqtt.MQTTv5) -client.tls_set_context(context=None) -client.connect("broker.hivemq.com", 8883, 60) -``` - -## Change the color of the RGB LED - -Let's change the RGB LED color of the DotBot by sending an `rgb_led` command. -This command takes a payload parameter containing a json with the red, green and blue -values to apply. -But first the payload has to be encrypted using JWE. This can be done by -extenting our script as follows: - -```py -import json -from joserfc import jwe - -# Encryption using AESGCM -rgb_led = json.dumps({"red": 255, "green": 0, "blue": 0}) -protected = {'alg': 'dir', 'enc': 'A256GCM'} -rgb_led_payload = jwe.encrypt_compact(protected, rgb_led, key) -print(f"RGB LED Payload: {rgb_led_payload}") - -client.publish(f"/dotbots/{topic}/command/0000/{dotbot_addr}/0/rgb_led", rgb_led_payload) -``` - -And the RGB LED should turn red. - -## Move one DotBot - -Let's now try to make the DotBot move forward briefly using the `move_raw` -command: - -```py -move = json.dumps({"left_x": 0, "left_y": 80, "right_x": 0, "right_y": 80}) -move_payload = jwe.encrypt_compact(protected, move, key) -print(f"Move Payload: {move_payload}") -client.publish(f"/dotbots/{topic}/command/0000/{dotbot_addr}/0/move_raw", move_payload, qos=1) -``` - -And the DotBot should move forward during 200ms! diff --git a/doc/reference/configuration.md b/doc/reference/configuration.md new file mode 100644 index 00000000..ead8a3ea --- /dev/null +++ b/doc/reference/configuration.md @@ -0,0 +1,269 @@ +# Configuration + +`dotbot` reads one optional config file so you don't retype the same flags on +every command. A value can come from a flag, an environment variable, the file, +or a built-in default - the resolver merges them through a single precedence +chain, so the file is just a place to park the defaults you'd otherwise pass by +hand. + +You never need a config file: every setting also has a flag and an env var. The +file just makes a repeated setup (a broker URL, a board name, a swarm id) the +default. + +Create one with `dotbot config init` (it writes a minimal `./dotbot.toml`); +pass `--conn` / `--swarm-id` to pre-fill the two most common keys: + +```bash +dotbot config init --conn mqtts://broker:8883 --swarm-id 1234 +``` + +This page is the file-format reference. For the `config` command itself +(`init` / `show` / `path`), see [`dotbot config`](../cli/config.md). + +## Where the file comes from + +`dotbot` looks in this order and uses the first hit: + +| Order | Source | How | +|---|---|---| +| 1 | `-c PATH` / `--config PATH` | An explicit path on the command line. | +| 2 | `DOTBOT_CONFIG` | An explicit path in the environment. | +| 3 | `./dotbot.toml` | A `dotbot.toml` in the current directory (the cwd only - parent directories are not searched). | +| 4 | `~/.dotbot/config.toml` | Your user-level file. | +| 5 | (none) | Built-in defaults only. | + +A `dotbot.toml` in your working directory (3) takes precedence over your +personal file (4), so a per-experiment config wins while you work in that +directory. Discovery looks only at the cwd - it does not walk up to parent +directories, so the active config is always unambiguous. + +`~/.dotbot/config.toml` (4) is the per-machine fallback for settings you set +once and want everywhere - typically `[fw].segger_dir`, since the SES install +path rarely changes. Per-project settings like `[fw].firmware_repo` belong in +the project's `./dotbot.toml` instead. Every command, including `dotbot fw`, +reads through this same resolver. + +## Precedence + +For any single setting, the highest-priority source that has a value wins: + +```text +CLI flag > env DOTBOT_
_ (then shared DOTBOT_) + > file: section value > selected deployment > top-level + > built-in default +``` + +Inside the file, a key set in its own section table beats the same key on the +selected deployment, which beats a shared top-level key. + +**Worked example** - resolving the controller's broker URL (`conn`): + +| Source | Value | Wins? | +|---|---|---| +| `--conn mqtts://cli:8883` flag | `mqtts://cli:8883` | yes, flag is highest | +| `DOTBOT_RUN_CONN` env | `mqtts://env:8883` | only if no flag | +| `[run] conn` in the file | `mqtts://run:8883` | only if no flag/env | +| `[deployment.inria] conn` (selected) | `mqtts://inria:8883` | only if `[run]` has no `conn` | +| top-level `conn` | `mqtts://shared:8883` | only if nothing above is set | +| built-in default | - | last resort | + +Env-var names are mechanical: a section key becomes `DOTBOT_
_` +(e.g. `DOTBOT_FW_BOARD`, `DOTBOT_RUN_CONN`), and a shared top-level key becomes +`DOTBOT_` (e.g. `DOTBOT_CONN`, `DOTBOT_SWARM_ID`). A sectioned key also +accepts the shared `DOTBOT_` form as a fallback. + +## Top-level (shared) keys + +Set once at the top of the file; any section or deployment can override them. + +| Key | Meaning | +|---|---| +| `conn` | Default connection string (`mqtts://host:port`, a serial path, or `simulator`). | +| `swarm_id` | Swarm id selecting the MQTT topic namespace. | +| `log_level` | Logging verbosity. | +| `default_deployment` | Name of the deployment to select when neither `--deployment` nor `DOTBOT_DEPLOYMENT` is given. | + +## Section tables + +The four tables mirror the four CLI namespaces (`fw` / `device` / `swarm` / +`run`); a section key is the per-namespace default for the matching flag. + +`[fw]` - firmware-artifact builds (`dotbot fw`): + +| Key | Meaning | +|---|---| +| `board` | Target board, e.g. `dotbot-v3`. | +| `sandbox` | Build TrustZone sandbox apps (`.bin`) instead of bare apps. | +| `build_config` | `Debug` or `Release`. | +| `segger_dir` | SEGGER Embedded Studio install path. | +| `firmware_repo` | Path to your `DotBot-firmware` clone (so `fw build`/`artifacts` find it without `cd` or `DOTBOT_FIRMWARE_REPO`). | + +`[device]` - one cabled device (`dotbot device`): + +| Key | Meaning | +|---|---| +| `board` | Target board for flashing. | +| `sn_starting_digits` | J-Link serial-number prefix selecting which probe. | +| `build_config` | `Debug` or `Release`. | + +`[swarm]` - the fleet over the air (`dotbot swarm`): + +| Key | Meaning | +|---|---| +| `conn` | Connection string for the fleet link. | +| `swarm_id` | Swarm id (topic namespace). | +| `devices` | Device selection for fleet operations. | + +`[run]` plus `[run.controller]` and `[run.gateway]` - host processes +(`dotbot run`): + +| Key | Meaning | +|---|---| +| `conn` | Connection string for `dotbot run`. | +| `swarm_id` | Swarm id (topic namespace). | +| `[run.controller] http_port` | REST/WebSocket port (default 8000). | +| `[run.controller] map_size` | Controller map size. | +| `[run.controller] background_map` | Background map image. | +| `[run.controller] log_output` | Log output path. | +| `[run.controller] csv_data_output` | CSV data output path. | +| `[run.controller] webbrowser` | Open the web UI on start. | +| `[run.controller] gw_address` | Gateway address. | +| `[run.controller] simulator_init_state` | Initial simulator state. | +| `[run.gateway] serial_port` | Gateway serial port. | +| `[run.gateway] mqtt` | Gateway MQTT connection string. | + +Unknown keys are rejected: a typo in a section or key name fails loud rather +than being silently ignored. + +## What a deployment is + +A **deployment** here means one physical deployment - one set of real DotBots +behind one broker, in one place (e.g. the ~100-bot setup at Inria Paris, or a +1000-bot campaign). You define each one as a `[deployment.]` table and +**select** it; you do not edit the file to switch between them. + +Select the active deployment with, in precedence order, `--deployment NAME`, the +`DOTBOT_DEPLOYMENT` env var, or the top-level `default_deployment`. The selected +deployment's keys slot into the file layer (above top-level, below sections), so an +explicit flag or env var still overrides it. Selecting a name with no matching +`[deployment.]` table is an error that lists the defined deployments. + +A deployment is **not** the simulator. To drive simulated bots, set the connection +to `simulator` (`--conn simulator`, or `conn = "simulator"`); that is a +connection kind, not a deployment. + +A `[deployment.]` table holds the deployment-binding keys plus descriptive +metadata: + +| Key | Meaning | +|---|---| +| `conn` | Broker / link for this deployment. | +| `swarm_id` | Swarm id for this deployment. | +| `serial_port` | Default serial port for this deployment. | +| `location` | Descriptive label (shown by `dotbot deployment list`). | +| `bots` | Descriptive bot count. | + +## Managing deployments + +The `dotbot deployment` group inspects, switches, and fetches deployments: + +| Command | Does | +|---|---| +| `dotbot deployment list` | List defined deployments; mark the active one. | +| `dotbot deployment show NAME` | Print one deployment's fields. | +| `dotbot deployment use NAME` | Set NAME as `default_deployment`, written into your config file (comments preserved). | +| `dotbot deployment fetch [SOURCE]` | Fetch published deployments and merge them into your config. | + +`fetch` takes a URL or a local file holding `[deployment.*]` tables; with no +SOURCE it uses the built-in DotBots registry. It **merges**: a same-named +deployment is replaced (you are asked first), and everything else in the file +(other deployments, sections, comments) is left intact. Like `dotbot fw fetch`, +it only acquires the deployment - select it afterwards with `dotbot deployment +use` or `--deployment`. Useful flags: `--into project` (write the nearest +`dotbot.toml` instead of `~/.dotbot/config.toml`), `--dry-run`, and `--yes`. + +Because MQTT credentials are env-only (below), a published deployment file is not +secret - it carries only the broker URL, swarm id, and descriptive labels. + +## MQTT credentials are env-only + +MQTT username and password are read **only** from the environment: + +```bash +export DOTBOT_MQTT_USER=alice +export DOTBOT_MQTT_PASS=… +``` + +They are never file keys - don't put them in `dotbot.toml`, and don't commit +them. Keep the broker URL in the file and the credentials in your environment +(or a secret manager). + +## Inspecting the resolved config + +Two helpers show what `dotbot` actually resolved, so you don't have to trace the +precedence chain by hand: + +| Command | Shows | +|---|---| +| `dotbot config show` | The merged, effective config and which file (if any) it came from. | +| `dotbot deployment list` | The defined deployments, their metadata, and which one is selected. | + +## Full example + +An annotated `dotbot.toml` exercising every layer: + +```toml +# Top-level shared keys: every section and deployment inherits these unless it +# sets its own value. +default_deployment = "inria" # used when --deployment / DOTBOT_DEPLOYMENT unset +conn = "mqtts://broker.local:8883" +swarm_id = "0001" +log_level = "info" + +# A physical deployment. Select it with `--deployment inria`, DOTBOT_DEPLOYMENT, or +# default_deployment above - don't edit this table to switch deployments. +[deployment.inria] +conn = "mqtts://broker.inria.fr:8883" +swarm_id = "0001" +serial_port = "/dev/ttyACM0" +location = "Inria Paris" # descriptive, for `dotbot deployment list` +bots = 100 # descriptive + +[deployment.limerick] +conn = "mqtts://broker.limerick:8883" +swarm_id = "0002" +location = "Limerick campaign" +bots = 725 + +# Firmware-artifact builds (dotbot fw). +[fw] +board = "dotbot-v3" +sandbox = false +build_config = "Release" +# segger_dir = "/Applications/SEGGER/SEGGER Embedded Studio 8.22a" + +# One cabled device (dotbot device). +[device] +board = "dotbot-v3" +sn_starting_digits = "77" # J-Link serial prefix +build_config = "Release" + +# The fleet over the air (dotbot swarm). +[swarm] +swarm_id = "0001" + +# Host-side processes (dotbot run). +[run] +conn = "mqtts://broker.local:8883" + +[run.controller] +http_port = 8000 +webbrowser = true +# background_map = "./map.png" + +[run.gateway] +serial_port = "/dev/ttyACM0" + +# Note: MQTT credentials are env-only - DOTBOT_MQTT_USER / DOTBOT_MQTT_PASS. +# Never a file key. +``` diff --git a/doc/reference/index.md b/doc/reference/index.md new file mode 100644 index 00000000..503b8abe --- /dev/null +++ b/doc/reference/index.md @@ -0,0 +1,21 @@ +# Reference + +Language-neutral surfaces for talking to the controller and the swarm, plus +fixes for common snags. + +```{toctree} +:hidden: +rest +mqtt +configuration +troubleshooting +/api +``` + +- [REST / WebSocket API](rest.md) - the controller's HTTP + WebSocket surface. +- [MQTT](mqtt.md) - topic vocabulary for non-Python integrations. +- [Configuration](configuration.md) - the single `dotbot` config file: + discovery, precedence, sections, and testbeds. +- [Troubleshooting](troubleshooting.md) - fixes for the rough edges (e.g. the + Firefox web-UI workaround). +- The autogenerated **Python API** reference is in the sidebar. diff --git a/doc/reference/mqtt.md b/doc/reference/mqtt.md new file mode 100644 index 00000000..1e644505 --- /dev/null +++ b/doc/reference/mqtt.md @@ -0,0 +1,97 @@ +# MQTT + +Talk to the swarm from any language that speaks MQTT - no Python, no SDK. This +is the low-magic integration path: subscribe to bot state, publish commands, on +standard topics. For Python, the [REST API](rest.md) is usually simpler. + +## How it works + +The MQTT surface is provided by the **qrkey bridge**, a small process that runs +next to the [controller](../guides/controller.md) and mirrors its state onto an +MQTT broker: + +```bash +dotbot run controller # one terminal - drives the gateway +dotbot run demo qr -w # another terminal - the qrkey MQTT bridge +``` + +The bridge connects to a broker (a public HiveMQ instance by default) and: + +- **publishes** notifications (state changes, position updates) for consumers to read; +- **subscribes** to command topics, forwarding what it receives to the controller. + +So an external consumer subscribes to notifications and publishes commands - it +never talks to the controller directly. + +## Topic vocabulary + +Every topic is rooted at `/pydotbot/`, where `` is a +base64 string derived from the current PIN code (see [Secured brokers](#secured-brokers)). + +| Topic (under `/pydotbot/`) | Direction | Purpose | +|---|---|---| +| `/command//
//` | you publish | drive a bot (`move_raw`, `rgb_led`, `waypoints`, `clear_position_history`) | +| `/notify` | you subscribe | controller state changes + position updates | +| `/request` / `/reply/` | request/reply | one-shot queries (e.g. list of bots, map size) | + +Command-topic fields: + +- `` - 4-hex swarm identifier (bots behind one gateway), e.g. `0000`. +- `
` - 16-hex DotBot address, e.g. `9903ef26257feb31`. +- `` - application type: `0` = DotBot, `1` = SailBot. +- `` - the command name (last segment). + +Get a bot's address and the swarm id from the controller's +[REST API](rest.md) (`GET /controller/dotbots`). + +## Send commands + +Payloads are JSON. Drive a bot forward and turn its LED red: + +```bash +# move_raw - left_y / right_y drive the wheels, values in [-100, 100] +mosquitto_pub -h \ + -t '/pydotbot//command/0000/9903ef26257feb31/0/move_raw' \ + -m '{"left_x": 0, "left_y": 80, "right_x": 0, "right_y": 80}' + +# rgb_led - 0..255 per channel +mosquitto_pub -h \ + -t '/pydotbot//command/0000/9903ef26257feb31/0/rgb_led' \ + -m '{"red": 255, "green": 0, "blue": 0}' +``` + +## Read state + +```bash +mosquitto_sub -h -t '/pydotbot//notify' | jq +``` + +Notifications carry a `cmd` field: `RELOAD` (refetch all bots), `UPDATE` +(per-bot state delta, incl. LH2 position), `PIN_CODE_UPDATE` (the secret topic +and key are about to rotate - see below). + +## Secured brokers + +Topics and payloads are not in the clear. The secret topic and a symmetric +AES-GCM key are both derived from a rotating 8-digit PIN code; the PIN refreshes +periodically (with a grace window), so the topic and key change over time, and +all payloads are encrypted. A consumer therefore needs to derive the topic/key +from the current PIN and decrypt - the bare `mosquitto_pub/sub` calls above are +the shape of the integration, not a drop-in for a live secured broker. + +The PIN and the full key-derivation + encryption scheme are +[qrkey](https://github.com/DotBots/qrkey)'s job. Use it (or a port of its +derivation) rather than reimplementing the crypto. A complete working consumer - +deriving the topic/key, encrypting commands, decrypting notifications, and +rotating on `PIN_CODE_UPDATE` - ships as the `qrkey_demo` example +(`dotbot run demo qr`); read its source as the reference implementation. + +For a fully language-neutral bridge that publishes plain dotbot-semantic topics +(`pydotbot//position`, `.../cmd/move_raw`) with no per-message crypto, see +the [Python SDK](../sdk/index.md) roadmap - that bridge is planned, not yet shipped. + +## See also + +- [REST API](rest.md) - the controller surface the bridge mirrors. +- [`dotbot run`](../cli/run.md) - `run controller` and `run demo qr`. +- [Controller guide](../guides/controller.md) - what the controller does. diff --git a/doc/reference/rest.md b/doc/reference/rest.md new file mode 100644 index 00000000..4fb51026 --- /dev/null +++ b/doc/reference/rest.md @@ -0,0 +1,93 @@ +# REST API + +`dotbot run controller` exposes a FastAPI REST server for reading DotBot state +and sending commands. The React web UI and the [MQTT](mqtt.md) bridge use the +same controller; REST is the simplest way to script the swarm from your own +code. + +## Where it lives + +Start a controller (see [`dotbot run`](../cli/run.md)): + +```bash +dotbot run controller --conn /dev/ttyACM0 # serial gateway +dotbot run controller --conn mqtts://broker:8883 --swarm-id 1234 # over MQTT +``` + +The server listens on **port 8000** by default (`--controller-http-port` to +change it). Interactive OpenAPI docs - schemas, payloads, and a "try it out" +button - are served by the running app at: + +``` +http://localhost:8000/api +``` + +```{image} ../_static/images/pydotbot-ui-openapi.png +:alt: OpenAPI UI +:width: 700px +:align: center +``` + +That page is the authoritative, version-matched reference. The table below is a +quick map; treat `/api` as the source of truth. + +## Endpoints + +All paths are under `http://localhost:8000`. `{address}` is the 8-byte hex +DotBot id; `{application}` is `0` (DotBot) or `1` (SailBot). + +| Method | Path | Purpose | +|---|---|---| +| `GET` | `/controller/dotbots` | List connected DotBots | +| `GET` | `/controller/dotbots/{address}` | One DotBot's state | +| `GET` | `/controller/map_size` | Controller map size | +| `GET` | `/controller/background_map` | Background map image (base64) | +| `PUT` | `/controller/dotbots/{address}/{application}/move_raw` | Drive the motors | +| `PUT` | `/controller/dotbots/{address}/{application}/rgb_led` | Set the RGB LED | +| `PUT` | `/controller/dotbots/{address}/{application}/waypoints` | Set navigation waypoints | +| `DELETE` | `/controller/dotbots/{address}/positions` | Clear position history | + +Two WebSocket endpoints push live updates: `/controller/ws/status` (state +stream) and `/controller/ws/dotbots` (send `move_raw` / `rgb_led` / `waypoints` +as JSON). + +## Quick examples + +Install [requests](https://pypi.org/project/requests/): `pip install requests`. + +**List DotBots** - `address` identifies a bot; `status` is `0` active, `1` +inactive, `2` lost. + +```py +import requests +print(requests.get("http://localhost:8000/controller/dotbots").json()) +``` + +**Set the RGB LED** (`red`/`green`/`blue`, 0–255): + +```py +import requests +addr = "9903ef26257feb31" # from the list above +requests.put( + f"http://localhost:8000/controller/dotbots/{addr}/0/rgb_led", + json={"red": 255, "green": 0, "blue": 0}, +) +``` + +**Drive the motors** - only `left_y` / `right_y` are used; values in `[-100, +100]`, and absolute values below ~50 won't overcome friction. + +```py +import requests +addr = "9903ef26257feb31" +requests.put( + f"http://localhost:8000/controller/dotbots/{addr}/0/move_raw", + json={"left_x": 0, "left_y": 60, "right_x": 0, "right_y": 60}, +) +``` + +```{admonition} Motors stop after 200 ms +:class: info +The firmware halts the motors if no `move_raw` arrives within 200 ms. To keep a +DotBot moving, send commands in a loop with a delay under 200 ms. +``` diff --git a/doc/reference/troubleshooting.md b/doc/reference/troubleshooting.md new file mode 100644 index 00000000..7f8be403 --- /dev/null +++ b/doc/reference/troubleshooting.md @@ -0,0 +1,33 @@ +# Troubleshooting + +Fixes for the rough edges you're most likely to hit. + +## Web UI won't load in Firefox + +Firefox's HTTP/2 handling can break the WebSocket stream the web UI uses to talk +to the controller, so the map and joystick never come alive. Turn it off: + +1. Press `Ctrl + L`, type `about:config`, and accept the warning. +2. Find `network.http.http2.websockets` and set it to `false`. +3. Reload the web UI. + +Chromium-based browsers (Chrome, Edge, Brave) are unaffected. + +## Web UI is missing (frontend build not found) + +Starting the controller or simulator logs: + +``` +Frontend build not found at .../dotbot/frontend/build; the web UI will be unavailable. +``` + +The React web UI ships inside the published wheel but is **not** built by a +plain source checkout, so a git clone (or a wheel-less install) has no +`dotbot/frontend/build/`. The controller and its REST/WebSocket API still run - +only the browser UI is unavailable. Fixes: + +- **From PyPI** - install the wheel, which bundles the UI: `pip install --pre pydotbot`. +- **From a git checkout** - build the UI once: `cd dotbot/frontend && npm install && npm run build`. + +On a version without this check, the same cause surfaces as a startup crash, +`RuntimeError: Directory '.../dotbot/frontend/build' does not exist`. diff --git a/doc/rest.md b/doc/rest.md deleted file mode 100644 index f90554a7..00000000 --- a/doc/rest.md +++ /dev/null @@ -1,176 +0,0 @@ -# REST - -While connected to a DotBot gateway, the `dotbot-controller` -application provides a REST server to send commands to and receive information -from connected DotBots. - -The REST API is documented in the running `dotbot-controller` application itself -at [http://localhost:8000/api](http://localhost:8000/api). This page also allows -you to play with the API directly from the browser. - -```{image} _static/images/pydotbot-ui-openapi.png -:alt: Open API UI -:class: bg-primary -:width: 700px -:align: center -``` - -## Prerequisites - -Make sure you already followed the [getting started](getting_started) page and -have a functional setup with `dotbot-controller` running and connected to a -nRF DK gateway. - -To interact with the REST API, you will use the Python -[requests](https://pypi.org/project/requests/) package. You can install it on -your computer using pip: - -``` -pip install -U requests -``` - -## The basics - -First, let's start by fetching the information about available DotBots using -the following script: - -```py -import json -import requests - -get_endpoint = "controller/dotbots" - -print( - json.dumps( - requests.get( - f"http://localhost:8000/{get_endpoint}" - ).json() - ) -) -``` - -If a DotBot is connected, this script should give an output similar to: -```json -[ - { - "address": "9903ef26257feb31", - "application": 0, - "swarm": "0000", - "status": 2, - "mode": 0, - "last_seen": 1701244665.8099585, - "waypoints": [], - "waypoints_threshold": 50, - "position_history": [] - } -] -``` - -This is a list of all DotBots connected to the `dotbot-controller`. In the -example above, there is only one DotBot connected. -The 8-byte `address` uniquely identifies a DotBot in the controller. The -`status` indicates whether the DotBot is `Active` (value=0, the DotBot has been -seen within the last 5 seconds), `Inactive` (value=1, the DotBot hasn't been seen -within the last 5 sec) or `Lost` (value=2, the DotBot hasn't been seen for more -than 60 sec). - -If the DotBot `address` is already known by the controller, e.g. it identifies -one of the DotBots returned a the previous request, use the -`controller/dotbots/
` to fetch information about that particular -DotBot (for example `controller/dotbots/9903ef26257feb31`). - -## Change the color of the RGB LED - -Use the `controller/dotbots/{address}/{application}/rgb_led` endpoint to change -the RGB LED color on the DotBot. The `address` parameter in the URL can be -retrieved from the list of available DotBots that we got in the previous -section. The `application` parameter is 0 (DotBot) in our case. - -It's important to note that this request, according to the API is a PUT request -and requires a payload: - -``` -{ - "red": 0, - "green": 0, - "blue": 0 -} -``` - -Here is an example Python script to send a "RGB LED" request to one DotBot: - -```py -import requests - -ADDRESS = "DOTBOT_ADDRESS_HERE" # edit this line with the DotBot address you want to control -RGB_LED_VALUE = { - "red": 255, - "green": 0, - "blue": 0, -} - -requests.put( - f"http://localhost:8000/controller/dotbots/{ADDRESS}/0/rgb_led", - json=RGB_LED_VALUE, -) -``` - -Play with the red/green/blue values to change the DotBot RGB LED. - -## Move one DotBot - -Use the `controller/dotbots/{address}/{application}/move_raw` endpoint to move a -DotBot. - -This request, according to the API is also a PUT request and requires a payload: - -``` -{ - "left_x": 0, - "left_y": 0, - "right_x": 0, - "right_y": 0 -} -``` - -To control the DotBot motors, only `left_y` and `right_y` values are useful, -`left_x` and `right_x` being ignored by the firmware running on the DotBots. - -```{admonition} Note 1 -:class: info -left_{x,y} and right_{x,y} values must be within the range **[-100, 100]** -and it's important to know that absolute values below 50 won't move the motors -(because of limited power in electronic circuit and internal friction of the motors). -``` - -```{admonition} Note 2 -:class: info -The firmware running on the DotBot stops automatically the motors if -no move command is received after 200ms. To move the DotBot continuously, -several commands must be sent with a delay below 200ms between them. -``` - -Here is an example Python script to send a "move raw" request to one DotBot: - -```py -import requests - -ADDRESS = "DOTBOT_ADDRESS_HERE" # edit this line with the DotBot address you want to control -MOVE_RAW_VALUE = { - "left_x": 0, - "left_y": 60, - "right_x": 0, - "right_y": 60 -} - -requests.put( - f"http://localhost:8000/controller/dotbots/{ADDRESS}/0/move_raw", - json=MOVE_RAW_VALUE, -) -``` - -Adapt the script above to: - -- move a DotBot forward during 10 seconds (use the sleep function from the - Python `time` module for example) -- rotate a DotBot during 20 seconds diff --git a/doc/sdk/index.md b/doc/sdk/index.md new file mode 100644 index 00000000..e6e2315a --- /dev/null +++ b/doc/sdk/index.md @@ -0,0 +1,70 @@ +# Python SDK (preview) + +```{admonition} Planned - not yet available +:class: warning + +The Python **Swarm SDK** described on this page is a **design preview**, not +shipped code. None of the snippets below run today - they show the API we +intend to build. The imports (`from dotbot import Swarm`) and every method +(`Swarm.connect`, `Swarm.run`, `bot.move_to`, ...) are **aspirational**. + +To script the swarm **today**, use the CLI: start a controller with +[`dotbot run controller`](../cli/run.md), then drive bots over its +[REST](../reference/rest.md) / [WebSocket](../reference/rest.md) surface +(or the [MQTT bridge](../reference/mqtt.md)). +``` + +## What it will be + +The SDK will be a thin Python wrapper over a running controller's REST/WS +surface, so you write swarm logic in Python instead of hand-rolling HTTP and +asyncio. You start a controller once (`dotbot run controller`), then a script +connects to it and commands bots. The same script targets real hardware, the +simulator, or a remote testbed - the backend is chosen at run time, not in the +code. + +## Intended API + +All three snippets are **aspirational** - they will not run until the SDK ships. + +**Connect and drive one bot** - connect to a local controller, grab a bot, set +its color, move it: + +```python +from dotbot import Swarm + +async with Swarm.connect() as swarm: # defaults to http://localhost:8000 + bot = next(iter(swarm)) + bot.set_color(red=255) + await bot.move_to(500, 500) +``` + +**Run an algorithm** - `Swarm.run()` handles argv parsing and `asyncio.run()`, +so a student writes only the algorithm body: + +```python +from dotbot import Swarm + +async def algorithm(swarm): + for bot in swarm: + bot.set_color(red=255) + +if __name__ == "__main__": + Swarm.run(algorithm) +``` + +**Switch backends without editing code** - the same script runs against the +local default, the simulator, or a remote class testbed, picked by a CLI flag: + +```bash +python my_assignment.py # local controller +python my_assignment.py --sim # simulator +python my_assignment.py --swarm-url http://classroom:8000 # shared testbed +``` + +## Until then + +- Launch the control plane: [`dotbot run`](../cli/run.md). +- Talk to it directly: [REST / WebSocket reference](../reference/rest.md) and + the [MQTT bridge](../reference/mqtt.md). +- New to the platform? Start at the [getting-started quickstarts](../index.md). diff --git a/dotbot/adapter.py b/dotbot/adapter.py index 959a3c24..3742a269 100644 --- a/dotbot/adapter.py +++ b/dotbot/adapter.py @@ -15,6 +15,7 @@ from marilib.communication_adapter import MQTTAdapter as MarilibMQTTAdapter from marilib.communication_adapter import SerialAdapter as MarilibSerialAdapter from marilib.mari_protocol import Frame as MariFrame +from marilib.mari_protocol import NextProto from marilib.marilib_cloud import MarilibCloud from marilib.marilib_edge import MarilibEdge from marilib.model import EdgeEvent, MariNode @@ -42,7 +43,14 @@ def send_payload(self, destination: int, payload: Payload): class SerialAdapter(GatewayAdapterBase): - """Class used to interface with the serial port.""" + """Raw (non-Mari) serial gateway interface. + + Deprecated: the `--conn` CLI no longer selects this — a device-path + connection maps to the Mari `edge` adapter, since bare DotBot apps now + emit Mari-shaped frames and the gateway speaks Mari-shaped UART + packets (so the edge adapter handles both sandbox and bare). Kept for + now as no CLI path constructs it; likely removed in a future cleanup. + """ def __init__( self, @@ -115,6 +123,8 @@ def _on_mari_event(event: EdgeEvent, event_data: MariNode | MariFrame): elif event == EdgeEvent.NODE_LEFT: LOGGER.debug(f"Node left: {event_data.address:016x}") elif event == EdgeEvent.NODE_DATA: + if event_data.header.next_proto != NextProto.DOTBOT_APP: + return try: packet = Packet.from_bytes(event_data.payload) except (ValueError, ProtocolPayloadParserException) as exc: @@ -143,6 +153,7 @@ def send_payload(self, destination: int, payload: Payload): self.mari.send_frame( dst=destination, payload=Packet.from_payload(payload).to_bytes(), + next_proto=NextProto.DOTBOT_APP, ) @@ -155,11 +166,15 @@ def __init__( port: int, use_tls: bool, network_id: int, + username: str | None = None, + password: str | None = None, ): self.host = host self.port = port self.use_tls = use_tls self.network_id = network_id + self.username = username + self.password = password async def start(self, on_frame_received: callable): self.on_frame_received = on_frame_received @@ -172,6 +187,8 @@ def _on_mari_event(event: EdgeEvent, event_data: MariNode | MariFrame): elif event == EdgeEvent.NODE_LEFT: LOGGER.debug(f"Node left: {event_data.address:016x}") elif event == EdgeEvent.NODE_DATA: + if event_data.header.next_proto != NextProto.DOTBOT_APP: + return try: packet = Packet.from_bytes(event_data.payload) except (ValueError, ProtocolPayloadParserException) as exc: @@ -183,10 +200,24 @@ def _on_mari_event(event: EdgeEvent, event_data: MariNode | MariFrame): queue.put_nowait, Frame(header=event_data.header, packet=packet) ) + # Broker credentials (from DOTBOT_MQTT_USER / DOTBOT_MQTT_PASS, + # threaded down by controller_app) are passed only when set. + # NOTE: requires the marilib companion that adds username/password + # to MarilibMQTTAdapter; until that lands, set credentials are a + # no-op (anonymous connect), which matches today's behaviour. + mqtt_kwargs = {} + if self.username is not None: + mqtt_kwargs["username"] = self.username + if self.password is not None: + mqtt_kwargs["password"] = self.password self.mari = MarilibCloud( _on_mari_event, MarilibMQTTAdapter( - self.host, self.port, use_tls=self.use_tls, is_edge=False + self.host, + self.port, + use_tls=self.use_tls, + is_edge=False, + **mqtt_kwargs, ), self.network_id, ) @@ -204,12 +235,17 @@ def send_payload(self, destination: int, payload: Payload): self.mari.send_frame( dst=destination, payload=Packet.from_payload(payload).to_bytes(), + next_proto=NextProto.DOTBOT_APP, ) class SimulatorAdapterBase(GatewayAdapterBase): """Base class used to interface with the simulator.""" + # Assigned in start(); stays None if start() failed before the + # simulator was constructed, so close() can no-op instead of raising. + simulator = None + @abstractmethod def create_simulator(self, _byte_received: callable): """Create the simulator instance.""" @@ -232,6 +268,8 @@ def _frame_received(frame): self.on_frame_received(frame) def close(self): + if self.simulator is None: + return LOGGER.info("Disconnect from simulator...") self.simulator.stop() diff --git a/dotbot/calibration/__init__.py b/dotbot/calibration/__init__.py new file mode 100644 index 00000000..a67d7dc7 --- /dev/null +++ b/dotbot/calibration/__init__.py @@ -0,0 +1,5 @@ +"""Lighthouse v2 calibration tooling. + +Vendored from the standalone `dotbot-lh2-calibration` package as part +of the unified-dx consolidation. +""" diff --git a/dotbot/calibration/app.py b/dotbot/calibration/app.py new file mode 100644 index 00000000..000cb7cd --- /dev/null +++ b/dotbot/calibration/app.py @@ -0,0 +1,517 @@ +import asyncio +import csv +import dataclasses +import logging +import traceback + +import serial +from dotbot_utils.hdlc import HDLCHandler, HDLCState +from textual.app import App, ComposeResult +from textual.containers import Container, Horizontal +from textual.widgets import ( + Button, + Header, + Label, + RichLog, + Select, + TabbedContent, + TabPane, +) + +from dotbot.calibration.lighthouse2 import ( + CALIBRATION_DIR, + LH2CalibrationSample, + LH2Counts, + LighthouseManager, +) + +# Tracebacks from inside the Textual TUI don't make it to the terminal, +# so we tee everything we'd want to see to a file under CALIBRATION_DIR +# (~/.dotbot/), the same directory that already holds the calibration +# output. Predictable location for pip-installed users (no dependency on +# the cwd they ran the command from). +_CALIB_LOG_PATH = CALIBRATION_DIR / "calibration.log" +CALIBRATION_DIR.mkdir(parents=True, exist_ok=True) +_CALIB_LOGGER = logging.getLogger("dotbot.calibration") +if not _CALIB_LOGGER.handlers: + _h = logging.FileHandler(_CALIB_LOG_PATH, mode="a") + _h.setFormatter( + logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s") + ) + _CALIB_LOGGER.addHandler(_h) + _CALIB_LOGGER.setLevel(logging.DEBUG) + _CALIB_LOGGER.propagate = False # avoid feedback through structlog + + +def _log_exception(target_log, message: str, exc: Exception) -> None: + # Must be called from inside an except: block — reads the active traceback. + target_log.write(f"[red]{message}: {exc}[/]") + for line in traceback.format_exc().rstrip().splitlines(): + target_log.write(f"[red]{line}[/]") + _CALIB_LOGGER.exception(message) + + +@dataclasses.dataclass +class CalibrationButton: + """Calibration button dataclass.""" + + button: Button + value: int = -1 + data_set: bool = False + + +BUTTONS = { + "top_left": CalibrationButton( + button=Button("Top left", id="top_left", classes="point-btn"), value=0 + ), + "top_right": CalibrationButton( + button=Button("Top right", id="top_right", classes="point-btn"), + value=1, + ), + "bottom_left": CalibrationButton( + button=Button("Bottom left", id="bottom_left", classes="point-btn"), + value=2, + ), + "bottom_right": CalibrationButton( + button=Button("Bottom right", id="bottom_right", classes="point-btn"), + value=3, + ), +} + +EXTRA_LH_BUTTONS = { + "lh1": CalibrationButton( + button=Button("Add point", id="lh1", classes="lh-btn", disabled=True), + value=1, + ), + "lh2": CalibrationButton( + button=Button("Add point", id="lh2", classes="lh-btn", disabled=True), + value=2, + ), + "lh3": CalibrationButton( + button=Button("Add point", id="lh3", classes="lh-btn", disabled=True), + value=3, + ), + "lh4": CalibrationButton( + button=Button("Add point", id="lh3", classes="lh-btn", disabled=True), + value=3, + ), + "lh5": CalibrationButton( + button=Button("Add point", id="lh3", classes="lh-btn", disabled=True), + value=3, + ), +} + + +def read_calibration_data_from_csv( + file_path: str, +) -> list[LH2CalibrationSample]: + """Read calibration data from CSV file.""" + calibration_samples: list[LH2CalibrationSample] = [] + with open(file_path) as input_file: + reader = csv.DictReader( + input_file, + quoting=csv.QUOTE_STRINGS, + fieldnames=[ + "lh_index", + "count1", + "count2", + "ref_lh_index", + "ref_count1", + "ref_count2", + ], + ) + for row in reader: + calibration_samples.append(LH2CalibrationSample(**row)) + return calibration_samples + + +class CalibrationApp(App): + """Calibration application.""" + + CSS_PATH = "app.tcss" + + def __init__( + self, + port, + baudrate, + distance, + extra_lh_num, + output_data=None, + input_data=None, + ): + super().__init__() + self.port = port + self.baudrate = baudrate + self.extra_lh_num = extra_lh_num + self.output_data = output_data + self.input_data = input_data + self.calibration_samples: list[LH2CalibrationSample] = [None] * 4 + if self.input_data is not None: + self.calibration_samples = read_calibration_data_from_csv(self.input_data) + else: + self.serial = serial.Serial(self.port, self.baudrate, timeout=0.1) + self.serial.flushInput() + self.csv_writer = None + if self.output_data is not None: + output_data_file = open(self.output_data, "w", newline="") + self.csv_writer = csv.writer(output_data_file) + + self.hdlc_handler = HDLCHandler() + self.lh2_manager = LighthouseManager( + calibration_distance=distance, extra_lh_num=self.extra_lh_num + ) + self.data_log = None + self.app_log = None + self.save_calibration_button = None + self.last_counts: list[LH2Counts | None] = [None, None, None, None] + self.extra_lh_samples_num: list[int] = [0] * self.extra_lh_num + self.extra_lh_index_references: list[int] = [0] * self.extra_lh_num + self.extra_lh_logs = [] + + def compose(self) -> ComposeResult: + """Compose the UI.""" + yield Header(show_clock=True) + self.main_container = Container(id="main-container") + with self.main_container: + with Container(classes="calibration-controls"): + with Container(classes="calibration-point-controls"): + with Horizontal(classes="calibration-label"): + yield Label("Reference calibration points (LH0):") + with Horizontal(classes="calibration-point-button-group"): + yield BUTTONS["top_left"].button + yield BUTTONS["top_right"].button + with Horizontal(classes="calibration-point-button-group"): + yield BUTTONS["bottom_left"].button + yield BUTTONS["bottom_right"].button + with Container(id="data-logs"): + self.data_log = RichLog(id="log", highlight=True, markup=True) + yield self.data_log + if self.extra_lh_num > 0: + with TabbedContent(id="extra-lh-tabs", initial="tab-lh1"): + for lh in range(self.extra_lh_num): + with TabPane(f"LH{lh + 1} calibration", id=f"tab-lh{lh+1}"): + with Container(classes="extra-lh-calibration-section"): + with Container( + classes="calibration-extra-lh-container" + ): + with Horizontal( + classes="calibration-extra-lh-point-controls" + ): + yield Select( + classes="lh-reference-select", + id=f"lh{lh + 1}_reference", + options=[ + ( + f"Reference: LH{index}", + index, + ) + for index in range( + 0, self.extra_lh_num + 1 + ) + if index < lh + 1 + ], + value=0, + ) + yield EXTRA_LH_BUTTONS[f"lh{lh+1}"].button + with Container(classes="calibration-state-info"): + log = RichLog( + id=f"extra_lh_logs_{lh + 1}", + highlight=True, + markup=True, + ) + self.extra_lh_logs.append(log) + yield log + with Container(id="app-logs"): + self.app_log = RichLog(id="app_log", highlight=True, markup=True) + yield self.app_log + with Horizontal(): + self.save_calibration_button = Button( + "Save calibration", id="save-btn", variant="primary" + ) + yield self.save_calibration_button + yield Button("Reset calibration", id="reset-btn", variant="warning") + yield Button("Exit", id="exit-btn", variant="error") + + async def on_button_pressed(self, event: Button.Pressed): + """Handle button presses.""" + btn_id = event.button.id + if btn_id == "save-btn": + self.save_calibration() + return + + if btn_id == "reset-btn": + self.reset_calibration() + return + + if btn_id == "exit-btn": + await self.action_quit() + return + + if btn_id in BUTTONS: + self.add_initial_calibration_point(btn_id) + + if btn_id in EXTRA_LH_BUTTONS: + self.add_extra_lh_point(btn_id) + + async def on_select_changed(self, event: Select.Changed): + """Handle select changes.""" + select_id = event.select.id + lh_index = int(select_id[2:3]) + self.extra_lh_index_references[lh_index - 1] = event.value + + async def on_mount(self): + """Initialize the serial connection.""" + self.save_calibration_button.disabled = True + if self.input_data is None: + self.data_log.write( + f"[green]Connected to {self.port} @ {self.baudrate} baud[/]" + ) + self.read_task = asyncio.create_task(self.read_serial()) + + def handle_received_payload(self, payload: bytes): + """Handle a received frame.""" + if len(payload) != 9: + self.data_log.write(f"[red]Invalid payload received '{payload.hex()}'[/]") + return + + counts: LH2Counts = LH2Counts( + lh_index=int.from_bytes(payload[0:1], byteorder="little", signed=False), + count1=int.from_bytes(payload[1:5], byteorder="little", signed=False), + count2=int.from_bytes(payload[5:9], byteorder="little", signed=False), + ) + + # The firmware reports counts for every LH it sees, including ones + # outside the configured calibration range (other base stations in + # the room). Drop those — they would crash the fixed-size + # last_counts list and they have no use here anyway. + if counts.lh_index > self.extra_lh_num or counts.lh_index >= len( + self.last_counts + ): + self.data_log.write( + f"[dim]Ignoring counts for LH{counts.lh_index} " + f"(outside configured range 0..{self.extra_lh_num})[/]" + ) + return + + message = f"[cyan]Counts received: {counts}" + if self.lh2_manager.has_calibration(counts.lh_index): + coords = self.lh2_manager.ground_coordinate_from_counts(counts) + message += f" -> coords: ({coords[0]:.2f}, {coords[1]:.2f})" + message += "[/]" + self.data_log.write(message) + self.last_counts[counts.lh_index] = counts + + def on_byte_received(self, byte: bytes): + """Handle a received byte from serial.""" + self.hdlc_handler.handle_byte(byte) + if self.hdlc_handler.state == HDLCState.READY: + try: + data = self.hdlc_handler.payload + except Exception: + _CALIB_LOGGER.exception("HDLC payload extraction failed") + return + self.handle_received_payload(data) + + async def read_serial(self): + """Read bytes from serial port.""" + while self.serial and self.serial.is_open: + try: + byte = await asyncio.to_thread(self.serial.read, 1) + if byte: + self.on_byte_received(byte) + except Exception as e: + _log_exception(self.data_log, "Error reading serial port", e) + break + await self.action_quit() + + def add_initial_calibration_point(self, point_id: str): + """Add a calibration point.""" + + if self.input_data is not None: + calibration_sample = self.calibration_samples[BUTTONS[point_id].value] + self.last_counts[0] = LH2Counts( + lh_index=calibration_sample.lh_index, + count1=calibration_sample.count1, + count2=calibration_sample.count2, + ) + + if self.last_counts[0] is None: + self.app_log.write( + "[red]Error: No LH2 counts available, cannot add calibration point[/]" + ) + return + + counts = self.last_counts[0] + self.last_counts[0] = None + + if self.input_data is None: + self.calibration_samples[BUTTONS[point_id].value] = LH2CalibrationSample( + lh_index=counts.lh_index, + count1=counts.count1, + count2=counts.count2, + ) + + if self.csv_writer is not None: + self.csv_writer.writerow( + [ + counts.lh_index, + counts.count1, + counts.count2, + None, + None, + None, + ] + ) + + BUTTONS[point_id].button.variant = "success" + BUTTONS[point_id].data_set = True + self.app_log.write( + f"[cyan]Calibration point {BUTTONS[point_id].value} added for LH0 ({counts.count1}, {counts.count2}).[/]" + ) + if all(button.data_set for button in BUTTONS.values()): + if self.extra_lh_num > 0: + self.app_log.write( + "[yellow]All initial calibration points set, " + f"proceed to the {self.extra_lh_num} other lighthouses calibration[/]" + ) + EXTRA_LH_BUTTONS["lh1"].button.disabled = False + else: + self.app_log.write( + "[green]All calibration points set, " + "ready to save calibration.[/]" + ) + self.save_calibration_button.disabled = False + + def add_extra_lh_point(self, lh_id: str): + """Add a shared calibration point.""" + + lh_index = EXTRA_LH_BUTTONS[lh_id].value + ref_index = self.extra_lh_index_references[lh_index - 1] + + if self.input_data is not None: + samples = [s for s in self.calibration_samples if s.lh_index == lh_index] + if self.extra_lh_samples_num[lh_index - 1] >= len(samples): + self.app_log.write( + f"[red]Error: No more calibration samples available for LH{lh_index}[/]" + ) + return + sample = samples[self.extra_lh_samples_num[lh_index - 1]] + self.last_counts[lh_index] = LH2Counts( + lh_index=sample.lh_index, + count1=sample.count1, + count2=sample.count2, + ) + self.last_counts[ref_index] = LH2Counts( + lh_index=sample.ref_lh_index, + count1=sample.ref_count1, + count2=sample.ref_count2, + ) + + # Get reference counts from LH0 + ref_counts = self.last_counts[ref_index] + self.last_counts[ref_index] = None + if ref_counts is None: + self.app_log.write( + "[red]Error: No reference LH counts available, cannot add calibration sample[/]" + ) + return + + # Get counts from extra lighthouse + new_counts = self.last_counts[lh_index] + self.last_counts[lh_index] = None + if new_counts is None: + self.app_log.write( + f"[red]Error: No new LH{lh_index} counts available, cannot add calibration sample[/]" + ) + return + + # Create counts are mathching expected lighthouse + if lh_index != new_counts.lh_index: + self.app_log.write( + f"[red]Error: Received counts polynomial index {new_counts.lh_index} does not match expected LH{lh_index}.[/]" + ) + return + + sample = LH2CalibrationSample( + lh_index=lh_index, + count1=new_counts.count1, + count2=new_counts.count2, + ref_lh_index=ref_index, + ref_count1=ref_counts.count1, + ref_count2=ref_counts.count2, + ) + + if self.input_data is None: + self.calibration_samples.append(sample) + + if self.csv_writer is not None: + self.csv_writer.writerow( + [ + sample.lh_index, + sample.count1, + sample.count2, + sample.ref_lh_index, + sample.ref_count1, + sample.ref_count2, + ] + ) + + self.extra_lh_samples_num[lh_index - 1] += 1 + self.app_log.write( + f"[cyan]Calibration point {self.extra_lh_samples_num[lh_index - 1]} " + f"added for LH{lh_index} ({new_counts.count1}, {new_counts.count2})[/]" + ) + if self.extra_lh_samples_num[lh_index - 1] >= 4: + EXTRA_LH_BUTTONS[lh_id].button.variant = "success" + EXTRA_LH_BUTTONS[lh_id].data_set = True + next_lh_index = lh_index + 1 + if next_lh_index <= self.extra_lh_num: + next_btn_id = f"lh{next_lh_index}" + EXTRA_LH_BUTTONS[next_btn_id].button.disabled = False + self.app_log.write( + f"[green]LH{lh_index} calibration ready, proceed to LH{next_lh_index}[/]" + ) + if all( + button.data_set + for button in list(EXTRA_LH_BUTTONS.values())[: self.extra_lh_num] + ): + self.app_log.write( + "[green]All additional calibration points set, ready to save calibration[/]" + ) + self.save_calibration_button.disabled = False + + def reset_calibration(self): + """Reset calibration data.""" + for button in BUTTONS.values(): + button.button.variant = "default" + button.data_set = False + for button in EXTRA_LH_BUTTONS.values(): + button.data_set = False + button.button.variant = "default" + button.button.disabled = True + self.calibration_samples = [None] * 4 + self.extra_lh_samples_num = [0] * self.extra_lh_num + self.extra_lh_index_references = [0] * self.extra_lh_num + self.save_calibration_button.disabled = True + self.app_log.write("[green]Calibration data reset[/]") + + def save_calibration(self): + """Save calibration data to file.""" + try: + self.lh2_manager.compute_calibration(self.calibration_samples) + except Exception as e: + _log_exception(self.app_log, "Error computing calibration", e) + return + try: + saved_path = self.lh2_manager.save_calibration() + except Exception as e: + _log_exception(self.app_log, "Error saving calibration", e) + return + + self.app_log.write(f"[green]Calibration data saved to {saved_path}[/]") + + async def on_unmount(self): + """Cleanup on app exit.""" + if self.input_data is None and self.serial and self.serial.is_open: + await asyncio.to_thread(self.serial.close) + self.serial = None diff --git a/dotbot/calibration/app.tcss b/dotbot/calibration/app.tcss new file mode 100644 index 00000000..75df2ec7 --- /dev/null +++ b/dotbot/calibration/app.tcss @@ -0,0 +1,132 @@ +Screen { + layout: vertical; + align: center middle; + padding: 0; +} + +#main-container { + layout: vertical; + align: center middle; + height: 35; + width: 100%; + padding: 0; +} + +.calibration-controls { + layout: horizontal; + align: center middle; + height: 12; + width: 100%; + margin-bottom: 0; +} + +.extra-lh-calibration-section { + layout: horizontal; + align: center middle; + height: 6; + width: 100%; +} + +.calibration-point-controls { + layout: vertical; + align: center middle; + height: 100%; + width: 40%; + border: round cyan; +} + +.calibration-extra-lh-point-controls { + layout: horizontal; + align: center middle; + height: 100%; + width: 100%; +} + +.calibration-extra-lh-container { + layout: vertical; + align: center middle; + height: 100%; + width: 40%; + border: round cyan; +} + +.calibration-state-info { + layout: vertical; + align: center middle; + height: 100%; + width: 60%; + border: round green; +} + +#data-logs { + layout: vertical; + align: center middle; + height: 100%; + width: 60%; + border: round green; +} + +#app-logs { + layout: vertical; + align: center middle; + height: 10; + width: 100%; + border: round green; +} + +.extra-lh-label { + max-height: 1; + margin: 0; +} + +.calibration-label { + max-height: 1; +} + +.lh-reference-select{ + align: center middle; + max-width: 25; + margin-top: 1; +} + +.calibration-point-button-group { + align: center middle; +} + +Button { + min-width: 20; + min-height: 5; + max-height: 3; +} + +#exit-btn { + min-height: 3; +} + +#save-btn { + margin-left: 2; + margin-right: 2; + min-height: 3; +} + +#reset-btn { + margin-left: 2; + margin-right: 2; + min-height: 3; +} + +.point-btn { + margin: 1; + width: 20%; +} + +.lh-btn { + margin: 1; + max-width: 30%; +} + +RichLog { + height: 100%; + width: 100%; + margin-left: 1; +} diff --git a/dotbot/calibration/cli.py b/dotbot/calibration/cli.py new file mode 100644 index 00000000..b4987c6f --- /dev/null +++ b/dotbot/calibration/cli.py @@ -0,0 +1,117 @@ +"""CLI for DotBot LH2 calibration tools.""" + +# SPDX-FileCopyrightText: 2022-present Inria +# SPDX-FileCopyrightText: 2022-present Alexandre Abadie +# +# SPDX-License-Identifier: BSD-3-Clause + +#!/usr/bin/env python3 + +import logging +import sys +import traceback + +import click +import serial +import structlog +from serial.tools import list_ports + +from dotbot.calibration.app import CalibrationApp +from dotbot.calibration.lighthouse2 import CALIBRATION_DISTANCE_DEFAULT + + +def get_default_port(): + """Return default serial port. Called lazily by Click on subcommand + invocation — `import` of this module no longer enumerates serial + ports (that side effect was inherited from the pre-fold layout).""" + ports = list(list_ports.comports()) + if sys.platform != "win32": + ports = sorted(ports) + if not ports: + return "/dev/ttyACM0" + return ports[0].device + + +SERIAL_BAUDRATE_DEFAULT = 115200 +LH_NUM_DEFAULT = 0 + + +@click.command() +@click.option( + "-p", + "--port", + type=str, + default=get_default_port, + show_default="auto-detected serial port", + help="Serial port used to read LH2 counts from the calibration firmware.", +) +@click.option( + "-b", + "--baudrate", + type=int, + default=SERIAL_BAUDRATE_DEFAULT, + help=f"Serial baudrate used by 'serial' and 'edge' adapters. Defaults to {SERIAL_BAUDRATE_DEFAULT}", +) +@click.option( + "-d", + "--distance", + default=CALIBRATION_DISTANCE_DEFAULT, + type=int, + help="Distance between reference calibration points in millimeters.", +) +@click.option( + "-n", + "--extra-lh-num", + default=LH_NUM_DEFAULT, + type=click.IntRange(min=0, max=5), + help="Extra lighthouse number to calibrate.", +) +@click.option( + "--output-data", + type=click.Path(file_okay=True, dir_okay=False, writable=True), + required=False, + help="Path to save calibration data.", +) +@click.option( + "--input-data", + type=click.Path(exists=True, readable=True), + required=False, + help="Path to load calibration data.", +) +def main( + port, baudrate, distance, extra_lh_num, output_data, input_data +): # pylint: disable=redefined-builtin + """Lighthouse calibration application.""" + + # Configure structlog to suppress logs below CRITICAL level + structlog.configure( + wrapper_class=structlog.make_filtering_bound_logger(logging.CRITICAL), + ) + + app = CalibrationApp( + port, baudrate, distance, extra_lh_num, output_data, input_data + ) + try: + app.run() + except serial.serialutil.SerialException as exc: + sys.exit(exc) + except (SystemExit, KeyboardInterrupt): + pass + except Exception: + # Textual swallows exceptions from its event loop; tee to stderr + # (visible after teardown) and to the calibration log file. + traceback.print_exc() + logging.getLogger("dotbot.calibration").exception("CalibrationApp crashed") + sys.exit(1) + + # The TUI's "saved" log line is invisible after teardown — echo the + # path to stdout so the user knows where the calibration landed. + saved_path = getattr( + getattr(app, "lh2_manager", None), "last_saved_toml_path", None + ) + if saved_path is not None: + click.echo(f"Calibration saved to {saved_path}") + + +if __name__ == "__main__": + main() # pragma: nocover, pylint: disable=no-value-for-parameter diff --git a/dotbot/calibration/exporter.py b/dotbot/calibration/exporter.py new file mode 100644 index 00000000..ae295bcf --- /dev/null +++ b/dotbot/calibration/exporter.py @@ -0,0 +1,88 @@ +# SPDX-FileCopyrightText: 2025-present Inria +# SPDX-FileCopyrightText: 2025-present Alexandre Abadie +# +# SPDX-License-Identifier: BSD-3-Clause + +import logging +import os +import sys +from pathlib import Path + +import click +import structlog + +from dotbot.calibration.lighthouse2 import LighthouseManager + +CALIBRATION_HEADER_FILENAME = Path("lh2_calibration.h") +CALIBRATION_HEADER_HEADER = """// Auto-generated file, do not edit! +#ifndef __LH2_CALIBRATION_H +#define __LH2_CALIBRATION_H + +#include "localization.h" + +#define LH2_CALIBRATION_IS_VALID (1) +""" + +CALIBRATION_HEADER_FOOTER = """}; + +#endif // __LH2_CALIBRATION_H +""" + + +def export_calibration(calibrations: list[bytes]) -> str: + """Export the calibration file to a user-defined location.""" + # Store homography matrix as C header to use in SwarmIT bootloader + output = CALIBRATION_HEADER_HEADER + output += f"#define LH2_CALIBRATION_COUNT ({len(calibrations)})\n\n" + output += "static int32_t swrmt_homographies[LH2_CALIBRATION_COUNT][3][3] = {\n" + + for calibration in calibrations: + output += " {\n" + matrix_int = [ + int.from_bytes(calibration[i : i + 4], "little", signed=True) + for i in range(0, 36, 4) + ] + matrix = [matrix_int[i : i + 3] for i in range(0, 9, 3)] + for row in matrix: + output += " {" + ", ".join(str(v) for v in row) + "},\n" + output += " },\n" + output += CALIBRATION_HEADER_FOOTER + return output + + +@click.command() +@click.argument("output_path", nargs=1) +def main(output_path): + """Export DotBot calibration data to a file.""" + # Disable logging + structlog.configure( + wrapper_class=structlog.make_filtering_bound_logger(logging.CRITICAL), + ) + + if not os.path.exists(output_path): + print(f"Error: '{output_path}' doesn't exist", file=sys.stderr) + sys.exit(1) + + lh2_manager = LighthouseManager() + if not os.path.exists(lh2_manager.calibration_output_path): + print("Error: Lighthouse is not calibrated", file=sys.stderr) + sys.exit(1) + + calibrations = lh2_manager.load_calibration() + if not calibrations: + print("Error: No calibration data found", file=sys.stderr) + sys.exit(1) + try: + output = export_calibration(calibrations) + header_path = Path(output_path) / CALIBRATION_HEADER_FILENAME + with open(header_path, "w") as header_file: + header_file.write(output) + print(output) + print(f"Calibration data exported to '{header_path}'") + except Exception as e: + print(f"Error exporting calibration data: {e}", file=sys.stderr) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/dotbot/calibration/lighthouse2.py b/dotbot/calibration/lighthouse2.py new file mode 100644 index 00000000..f181c5d2 --- /dev/null +++ b/dotbot/calibration/lighthouse2.py @@ -0,0 +1,446 @@ +# SPDX-FileCopyrightText: 2022-present Inria +# SPDX-FileCopyrightText: 2022-present Filip Maksimovic +# SPDX-FileCopyrightText: 2022-present Alexandre Abadie +# +# SPDX-License-Identifier: BSD-3-Clause + +"""Module containing the API to convert LH2 raw data to relative positions.""" + +# pylint: disable=invalid-name,unspecified-encoding,no-member + +import dataclasses +import datetime +import math +import os +import re +import tomllib +from dataclasses import dataclass +from pathlib import Path +from typing import Optional + +import numpy as np + +# cv2 is imported lazily inside `compute_homography_matrix` (the only +# function that uses it). This keeps `dotbot calibrate export` usable +# without opencv-python installed — the exporter only reads / writes +# bytes and does no homography math itself. + +CALIBRATION_DIR = Path.home() / ".dotbot" +CALIBRATION_DISTANCE_DEFAULT = 500 # in millimeters +CALIBRATION_SCHEMA_VERSION = 1 +# Legacy binary file. Kept as a back-compat byproduct of save_calibration() +# so external consumers (swarmit OTA `calibrate-lh2 `, +# dotbot-provision `flash --calibration `) keep working until they +# learn to read the new TOML format. Once they do, drop the .out write. +CALIBRATION_LEGACY_OUT = "calibration.out" +CALIBRATION_TOML_GLOB = "calibration-*.toml" +REFERENCE_POINTS_DEFAULT = [ + [0.4, 0.4], # Top-left + [0.6, 0.4], # Top-right + [0.4, 0.6], # Bottom-left + [0.6, 0.6], # Bottom-right +] +LH_PERIODS = [ + 959000, # mode 1 + 957000, # mode 2 + 953000, # mode 3 + 949000, # mode 4 + 947000, # mode 5 + 943000, # mode 6 + 941000, # mode 7 + 939000, # mode 8 + 937000, # mode 9 + 929000, # mode 10 + 919000, # mode 11 + 911000, # mode 12 + 907000, # mode 13 + 901900, # mode 14 + 893000, # mode 15 + 887000, # mode 16 +] + + +@dataclass +class LH2Homography: + """Dataclass that holds computed LH2 homography for a basestation indicated by index.""" + + matrix: np.ndarray = dataclasses.field( + default_factory=lambda: np.zeros((3, 3), dtype=np.float64) + ) + + +@dataclass +class LH2Counts: + """Class that stores LH2 counts.""" + + lh_index: int + count1: int + count2: int + + def __repr__(self): + return f"{dataclasses.asdict(self)}" + + +@dataclass +class LH2CalibrationSample: + """Class that stores LH2 calibration data.""" + + lh_index: int + count1: int + count2: int + ref_lh_index: Optional[int] = None + ref_count1: Optional[int] = None + ref_count2: Optional[int] = None + + def __post_init__(self): + self.lh_index = int(self.lh_index) + self.count1 = int(self.count1) + self.count2 = int(self.count2) + if self.ref_lh_index is not None: + self.ref_lh_index = int(self.ref_lh_index) + if self.ref_count1 is not None: + self.ref_count1 = int(self.ref_count1) + if self.ref_count2 is not None: + self.ref_count2 = int(self.ref_count2) + + +def calculate_camera_point(counts: LH2Counts) -> np.ndarray: + """Calculate camera points from counts.""" + period = LH_PERIODS[counts.lh_index] + + a1 = (counts.count1 * 8 / period) * 2 * math.pi + a2 = (counts.count2 * 8 / period) * 2 * math.pi + + cam_x = -math.tan(0.5 * (a1 + a2)) + if counts.count1 < counts.count2: + cam_y = -math.sin(a2 / 2 - a1 / 2 - 60 * math.pi / 180) / math.tan(math.pi / 6) + else: + cam_y = -math.sin(a1 / 2 - a2 / 2 - 60 * math.pi / 180) / math.tan(math.pi / 6) + + return np.asarray([cam_x, cam_y], dtype=np.float64) + + +def camera_points_from_counts(counts: list[LH2Counts]) -> np.ndarray: + """Convert counts to camera points.""" + camera_points = np.zeros((len(counts), 2), dtype=np.float64) + for index, count in enumerate(counts): + camera_points[index] = calculate_camera_point(count) + return camera_points + + +def compute_homography_matrix( + camera_points: np.ndarray, + reference_points: np.ndarray, +) -> np.ndarray: + """Compute homography matrix from camera points to reference points.""" + import cv2 # lazy: opencv-python is only required for the capture path + + M, _ = cv2.findHomography( + camera_points, + reference_points, + method=cv2.RANSAC, + ransacReprojThreshold=0.001, + ) + + if M is None: + raise ValueError("Cannot find a valid homography matrix.") + + return M + + +def apply_homography( + homography: np.ndarray, camera_view_points: np.ndarray +) -> np.ndarray: + """Apply homography to camera points.""" + ground_plane_coordinates = np.zeros((0, 2), dtype=np.float64) + for row in camera_view_points: + projected = np.dot(homography, np.array([row[0], row[1], 1.0])) + projected /= projected[2] + ground_plane_coordinates = np.vstack((ground_plane_coordinates, projected[:2])) + + return ground_plane_coordinates + + +def homography_as_bytes(matrix: np.ndarray) -> bytes: + """Convert homography matrix to bytes.""" + matrix_bytes = bytearray() + try: + for bytes_block in [ + int(n * 1e3).to_bytes(4, "little", signed=True) for n in matrix.ravel() + ]: + matrix_bytes += bytes_block + except Exception: # noqa: BLE001 - defensive fallback for overflow + matrix_bytes = bytearray(36) + return matrix_bytes + + +def _build_calibration_payload( + homographies: list[LH2Homography], extra_lh_num: int +) -> bytes: + """Pack homographies as 1-byte count + N × 36-byte matrices. + + Same wire shape the legacy `calibration.out` carried; the TOML + payload also stores this byte-for-byte (hex-encoded) so external + consumers can decode it without ambiguity. + """ + payload = bytearray() + payload.append(1 + extra_lh_num) + for homography in homographies: + payload += homography_as_bytes(homography.matrix) + return bytes(payload) + + +def _slug_tag(tag: str) -> str: + """Filename-safe slug for a free-form calibration tag. + + Keeps ASCII letters, digits, dot, dash and underscore; collapses any + other run of characters to a single dash and trims dashes and dots off + the ends (so a tag like "../x" can't smuggle in a leading ".."). Returns + "" when nothing usable remains, so callers can treat the tag as absent. + The slug is safe to drop into both a filename and a TOML string. + """ + return re.sub(r"[^A-Za-z0-9._-]+", "-", tag).strip("-.") + + +def _parse_calibration_payload(payload: bytes) -> list[bytes]: + """Inverse of `_build_calibration_payload`: yields the per-LH 36-byte + matrix chunks. Used when loading from either TOML or legacy .out.""" + if not payload: + return [] + count = payload[0] + matrices = [] + for i in range(count): + start = 1 + i * 36 + matrices.append(payload[start : start + 36]) + return matrices + + +def _read_toml_payload(path: Path) -> bytes: + """Read a calibration-*.toml file and return the raw byte payload. + + Validates `schema_version` so future writers can break compatibility + explicitly instead of silently corrupting reads. + """ + with open(path, "rb") as f: + data = tomllib.load(f) + schema = data.get("schema_version", 0) + if schema != CALIBRATION_SCHEMA_VERSION: + raise ValueError( + f"{path}: unsupported calibration schema_version {schema} " + f"(this build supports {CALIBRATION_SCHEMA_VERSION})" + ) + hex_data = data["calibration"]["data_hex"] + return bytes.fromhex(hex_data) + + +class LighthouseManager: + """Class to manage the LightHouse positionning state and workflow.""" + + def __init__( + self, + calibration_distance: float = CALIBRATION_DISTANCE_DEFAULT, + extra_lh_num: int = 0, + ): + Path.mkdir(CALIBRATION_DIR, exist_ok=True) + # Legacy path, kept for back-compat with external consumers. + # The primary record is now timestamped TOML files in CALIBRATION_DIR. + self.calibration_output_path = CALIBRATION_DIR / CALIBRATION_LEGACY_OUT + self.calibration_distance = calibration_distance + self.extra_lh_num = extra_lh_num + self.homographies: list[LH2Homography] = [LH2Homography()] * ( + 1 + self.extra_lh_num + ) + self.last_saved_toml_path: Optional[Path] = None + + def _compute_reference_homography( + self, calibration_counts: list[LH2Counts] + ) -> LH2Homography: + """Compute the reference calibration values and matrices.""" + # Convert reference counts to camera view points + camera_points = camera_points_from_counts(calibration_counts) + + reference_points = np.array(REFERENCE_POINTS_DEFAULT, dtype=np.float64) + # Scale reference points according to calibration distance + reference_points *= self.calibration_distance * 5 + + # Compute homography from camera points to ground plane coordinates + homography = compute_homography_matrix( + camera_points, + reference_points, + ) + + print(f"reference homography: {homography}") + + # Project camera points using computed homography for verification + ref_coordinates = apply_homography(homography, camera_points) + + # compare with reference points + for i, ref_point in enumerate(reference_points): + if not np.allclose(ref_coordinates[i], ref_point, atol=1e-3): + raise ValueError( + f"Projected point {ref_coordinates[i]} does not match reference point {ref_point}" + ) + + return LH2Homography(matrix=homography) + + def _compute_extra_calibration( + self, samples: list[LH2CalibrationSample] + ) -> LH2Homography: + """Compute the extra lighthouse calibration values and matrices.""" + + print(f"ref: {samples[0].ref_lh_index}, homographies: {self.homographies}") + + # Convert reference counts to camera points + ref_camera_points = camera_points_from_counts( + [LH2Counts(s.ref_lh_index, s.ref_count1, s.ref_count2) for s in samples] + ) + + print(f"ref_camera_points: {ref_camera_points}") + + # Convert reference camera points to ground plane coordinates using reference homography + ref_coordinates = apply_homography( + self.homographies[samples[0].ref_lh_index].matrix, + ref_camera_points, + ) + + print(f"ref_coordinates: {ref_coordinates}") + + # Convert new LH counts to new camera points + new_camera_points = camera_points_from_counts( + [LH2Counts(s.lh_index, s.count1, s.count2) for s in samples] + ) + + print(f"new_camera_points: {new_camera_points}") + + # Compute homography from new camera points to ground plane coordinates + homography = compute_homography_matrix( + new_camera_points, + ref_coordinates, + ) + + # Project camera points using computed homography for verification + ref_coordinates = apply_homography(homography, new_camera_points) + + # compare with reference points + for i, ref_point in enumerate(ref_coordinates): + if not np.allclose(ref_coordinates[i], ref_point, atol=1e-3): + raise ValueError( + f"Projected point {ref_coordinates[i]} does not match reference point {ref_point}" + ) + + print(f"Computed homography: {homography}") + + return LH2Homography(matrix=homography) + + def compute_calibration( + self, + calibration_samples: list[LH2CalibrationSample], + ) -> list[LH2Homography]: + """Compute the calibration values and matrices.""" + reference_counts = [ + LH2Counts(s.lh_index, s.count1, s.count2) + for s in calibration_samples + if s.lh_index == 0 + ] + self.homographies[0] = self._compute_reference_homography(reference_counts) + + print(f"Computing {self.extra_lh_num} extra lighthouse calibrations...") + if self.extra_lh_num > 0: + for lh_index in range(self.extra_lh_num): + print(f"Computing calibration for LH{lh_index + 1}") + samples = [s for s in calibration_samples if s.lh_index == lh_index + 1] + self.homographies[lh_index + 1] = self._compute_extra_calibration( + samples + ) + + def has_calibration(self, lh_index) -> bool: + """Check if calibration is available for a given lighthouse index.""" + return len(self.homographies) > lh_index and not np.all( + self.homographies[lh_index].matrix == 0 + ) + + def load_calibration(self) -> list[bytes]: + """Load the most recent calibration as a flat list of matrix bytes. + + Prefers the newest timestamped `calibration-*.toml`; falls back + to the legacy binary `calibration.out` if no TOML files exist + (so setups predating the format change keep working). + """ + toml_files = sorted( + CALIBRATION_DIR.glob(CALIBRATION_TOML_GLOB), + key=lambda p: p.stat().st_mtime, + reverse=True, + ) + if toml_files: + return _parse_calibration_payload(_read_toml_payload(toml_files[0])) + if os.path.exists(self.calibration_output_path): + return _parse_calibration_payload(self.calibration_output_path.read_bytes()) + return [] + + def save_calibration(self, tag: Optional[str] = None) -> Path: + """Save the calibration as a timestamped TOML file (+ legacy .out). + + The TOML file is the new primary record: versioned, metadata- + bearing, human-inspectable. The legacy `.out` file is also + written so external consumers (swarmit OTA, dotbot-provision) + keep working until they learn to read TOML. + + `tag`, when given, is a free-form arena/setup label (e.g. + "office-2x2m"); a filename-safe slug of it is inserted into the + filename and recorded under `[metadata]` so the calibration stays + self-describing even after a rename. + + Returns the path of the TOML file just written, and also stores + it on `self.last_saved_toml_path` so a caller that lost the + return value (e.g. the TUI handler) can still surface it after + the fact. + """ + payload = _build_calibration_payload(self.homographies, self.extra_lh_num) + + now = datetime.datetime.now(datetime.timezone.utc) + # Filename-safe variant of ISO 8601: `:` is rejected on Windows + # and a footgun on some Unix tools. + ts_for_filename = now.strftime("%Y-%m-%dT%H-%M-%SZ") + slug = _slug_tag(tag) if tag else "" + stem = ( + f"calibration-{slug}-{ts_for_filename}" + if slug + else f"calibration-{ts_for_filename}" + ) + toml_path = CALIBRATION_DIR / f"{stem}.toml" + tag_line = f'tag = "{slug}"\n' if slug else "" + # Explicit UTF-8 — TOML is spec'd as UTF-8, and Path.write_text + # defaults to the platform encoding (cp1252 on Windows), which + # mangles any non-ASCII byte and breaks the tomllib reader. + toml_path.write_text( + f"schema_version = {CALIBRATION_SCHEMA_VERSION}\n" + "\n" + "[metadata]\n" + f'created_at = "{now.strftime("%Y-%m-%dT%H:%M:%SZ")}"\n' + f"calibration_distance_mm = {int(self.calibration_distance)}\n" + f"num_lh_stations = {1 + self.extra_lh_num}\n" + f"{tag_line}" + "\n" + "[calibration]\n" + "# 1-byte homography count + N x 36-byte int32 LE matrices,\n" + "# hex-encoded. Same bytes as the legacy calibration.out.\n" + f'data_hex = "{payload.hex()}"\n', + encoding="utf-8", + ) + + # Legacy back-compat write — drop once swarmit OTA + provision + # read TOML. + self.calibration_output_path.write_bytes(payload) + self.last_saved_toml_path = toml_path + return toml_path + + def ground_coordinate_from_counts(self, counts: LH2Counts) -> np.ndarray: + """Convert counts to ground plane coordinates using homography.""" + # Convert counts to camera points + camera_points = np.zeros((1, 2), dtype=np.float64) + camera_points[0] = calculate_camera_point(counts) + + # Apply homography to get ground plane coordinates + return apply_homography( + self.homographies[counts.lh_index].matrix, camera_points + )[0] diff --git a/dotbot/calibration/ota.py b/dotbot/calibration/ota.py new file mode 100644 index 00000000..d11e1686 --- /dev/null +++ b/dotbot/calibration/ota.py @@ -0,0 +1,134 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Over-the-air LH2 calibration collection (swarmit transport). + +Variant A: a single DotBot, no serial cable. The bot's secure bootloader +samples its own raw LH2 counts on request (READY mode only) and ships them +back inside a SWARMIT_EVENT_LOG. This module triggers one capture per arena +corner and decodes the samples; the homography solve and save live in +`lighthouse2.LighthouseManager`, exactly as in the serial flow. +""" + +from __future__ import annotations + +import queue +import threading +import time +from collections.abc import Callable + +from dotbot.calibration.lighthouse2 import LH2CalibrationSample + +# The four reference corners, in the order LighthouseManager expects them: +# it zips the collected counts against REFERENCE_POINTS_DEFAULT positionally +# (top-left, top-right, bottom-left, bottom-right), so the collection order +# is load-bearing, not cosmetic. +CORNERS = ("top-left", "top-right", "bottom-left", "bottom-right") + +CAPTURE_TIMEOUT_DEFAULT = 5.0 +CAPTURE_RETRIES_DEFAULT = 3 + +# Each raw sample inside the LOG payload is [lh_index:1][count1:4 LE][count2:4 LE]. +_SAMPLE_SIZE = 9 + + +def parse_capture_payload(data: bytes, tag: int) -> list[LH2CalibrationSample]: + """Decode a SWARMIT_EVENT_LOG payload of raw LH2 samples. + + Layout (mirrors the swarmit bootloader): a 1-byte `tag`, then N + fixed-size records. Returns [] for any payload that is not a capture + (regular text log lines do not carry `tag` as their first byte). + """ + if len(data) < 1 or data[0] != tag: + return [] + body = data[1:] + samples: list[LH2CalibrationSample] = [] + for off in range(0, len(body) - _SAMPLE_SIZE + 1, _SAMPLE_SIZE): + lh_index = body[off] + count1 = int.from_bytes(body[off + 1 : off + 5], "little") + count2 = int.from_bytes(body[off + 5 : off + 9], "little") + samples.append(LH2CalibrationSample(lh_index, count1, count2)) + return samples + + +class CaptureSession: + """One shared log-event stream for a whole collect session. + + The bot only emits raw counts in reply to a trigger, so nothing arrives + unsolicited - a single `watch_log_events()` stream serves every corner. + A background reader thread decodes samples addressed to `device` into a + queue; `capture()` triggers and waits, re-triggering on timeout because + the trigger send is best-effort (no transport-level ack). + """ + + def __init__(self, client, device: str, tag: int): + self._client = client + self._device = device.upper() + self._tag = tag + self._queue: queue.Queue = queue.Queue() + self._stop = threading.Event() + self._thread = threading.Thread(target=self._reader, daemon=True) + + def __enter__(self) -> CaptureSession: + self._thread.start() + return self + + def __exit__(self, *exc) -> None: + self._stop.set() + + def _reader(self) -> None: + try: + for event in self._client.watch_log_events(): + if self._stop.is_set(): + break + if str(event.get("addr", "")).upper() != self._device: + continue + data = bytes.fromhex(event.get("data_hex", "")) + for sample in parse_capture_payload(data, self._tag): + self._queue.put(sample) + except Exception as exc: # surfaced on the next capture() get() + self._queue.put(exc) + + def capture( + self, + lh_index: int, + timeout: float, + retries: int, + on_attempt: Callable[[int, int], None] | None = None, + ) -> LH2CalibrationSample: + """Trigger a capture and return the first sample for `lh_index`. + + Retries the trigger up to `retries` times; raises TimeoutError if + no matching sample arrives. `on_attempt(n, total)` runs just before + each trigger so callers can show progress during the otherwise silent + wait. + """ + # Discard anything left over from the previous corner. + while not self._queue.empty(): + self._queue.get_nowait() + + attempts = retries + 1 + for attempt in range(attempts): + if on_attempt is not None: + on_attempt(attempt + 1, attempts) + self._client.request_lh2_capture(self._device) + deadline = time.monotonic() + timeout + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + break + try: + item = self._queue.get(timeout=remaining) + except queue.Empty: + break + if isinstance(item, Exception): + raise item + if item.lh_index == lh_index: + return item + # A sample for a different lighthouse: ignore, keep waiting. + + raise TimeoutError( + f"no LH{lh_index} sample from {self._device} after " + f"{retries + 1} attempt(s); is the bot in READY (app stopped) " + f"and in view of the lighthouse?" + ) diff --git a/dotbot/cli/__init__.py b/dotbot/cli/__init__.py new file mode 100644 index 00000000..7b60d0da --- /dev/null +++ b/dotbot/cli/__init__.py @@ -0,0 +1,11 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Unified `dotbot` CLI dispatcher. + +Mounts existing Click commands from this package and sibling packages +(swarmit) as subcommands so users see one tool instead of seven +console_scripts. +""" + +from dotbot.cli.main import cli # noqa: F401 diff --git a/dotbot/cli/__main__.py b/dotbot/cli/__main__.py new file mode 100644 index 00000000..9b965645 --- /dev/null +++ b/dotbot/cli/__main__.py @@ -0,0 +1,9 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Allow `python -m dotbot.cli` (and `python -m dotbot`).""" + +from dotbot.cli.main import cli + +if __name__ == "__main__": + cli() # pragma: no cover, pylint: disable=no-value-for-parameter diff --git a/dotbot/cli/_artifacts.py b/dotbot/cli/_artifacts.py new file mode 100644 index 00000000..b5516a78 --- /dev/null +++ b/dotbot/cli/_artifacts.py @@ -0,0 +1,162 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Shared artifact-resolution + friendly-error helpers for `fw` / `device`. + +Owns the user-level ``~/.dotbot/artifacts/`` cache convention, the +absolute-path echo on every cache read/write (so you always see where files +landed, regardless of the directory you ran from), the auto-resolve decision +tree used by ``dotbot device flash `` (present → build → error), and +the two centralized tool-missing messages (SES for builds, nrfjprog for +device ops). + +Leaf module: it imports `_fw_helpers` / `dotbot.firmware` lazily *inside* +functions, so importing it (e.g. for `device info`, which needs neither +SES nor a firmware repo) stays cheap and side-effect-free. +""" + +import os +from pathlib import Path + +import click + +# Single source of truth for the default cache location, so changing it is a +# one-line edit. `artifacts_dir()` is the behavioral path (env-overridable, ~ +# expanded, resolved); `DEFAULT_ARTIFACTS_DISPLAY` is the human-readable form +# for help text. (Docstrings and the Markdown docs can't interpolate a +# variable, so they spell the path out literally - keep them in sync by hand.) +_DEFAULT_ARTIFACTS_PARTS = (".dotbot", "artifacts") +DEFAULT_ARTIFACTS_DISPLAY = "~/" + "/".join(_DEFAULT_ARTIFACTS_PARTS) + + +def artifacts_dir() -> Path: + """The firmware cache: ``~/.dotbot/artifacts/`` by default. + + User-level and shared across working directories (like other tools cache + downloaded firmware), so you don't re-download a release per directory and + the launch dir stays clean. Override with ``$DOTBOT_ARTIFACTS_DIR``. + """ + override = os.environ.get("DOTBOT_ARTIFACTS_DIR") + base = ( + Path(override) if override else Path.home().joinpath(*_DEFAULT_ARTIFACTS_PARTS) + ) + return base.expanduser().resolve() + + +def echo_artifact_path(path: Path, *, action: str = "using") -> None: + """Announce the resolved absolute artifact path on a cache read/write. + + ``action`` is a short verb ("writing", "reading", "using", ...). The + point is that a user who ran the command from an unexpected directory + immediately sees where files actually landed. + """ + click.echo(f"[artifacts] {action}: {Path(path).resolve()}", err=True) + + +def friendly_nrfjprog_error() -> click.ClickException: + """Message for a missing `nrfjprog`. It's an external binary (no pip).""" + return click.ClickException( + "`nrfjprog` (Nordic command-line tools) was not found on PATH.\n" + "Device commands flash over the J-Link cable via nrfjprog.\n" + " • Install nRF Command Line Tools from " + "https://www.nordicsemi.com/Products/Development-tools/nRF-Command-Line-Tools\n" + " • Or, on a fresh shell, make sure its `bin/` is on PATH." + ) + + +def ensure_nrfjprog() -> None: + """Raise the friendly nrfjprog message if the tool isn't installed.""" + from dotbot.firmware.nrf import nrfjprog_available + + if not nrfjprog_available(): + raise friendly_nrfjprog_error() + + +def _find_in_cache(name: str) -> Path | None: + """Find a firmware file across the source-qualified cache dirs. + + The cache holds ``-/`` (fetched releases) and + ``-local[-link]/`` (built/linked) subdirs. Prefer a local build + (you're iterating on it), else the newest-versioned release dir. + """ + root = artifacts_dir() + if not root.is_dir(): + return None + local_hits: list[Path] = [] + released_hits: list[Path] = [] + for sub in sorted(root.iterdir()): + if not sub.is_dir(): + continue + candidate = sub / name + if candidate.is_file(): + if sub.name.endswith(("-local", "-local-link")): + local_hits.append(candidate) + else: + released_hits.append(candidate) + if local_hits: + return local_hits[0] + if released_hits: + # newest by dir name (a coarse version sort; good enough for picking + # the latest release of a given app) + return sorted(released_hits, key=lambda p: p.parent.name, reverse=True)[0] + return None + + +def resolve_app_artifact( + app: str, + *, + board: str = "dotbot-v3", + config: str = "Release", + sandbox: bool = False, +) -> Path: + """Auto-resolve a single app's firmware artifact for cable-flashing. + + Decision tree (npm-style): present in ``~/.dotbot/artifacts/`` → build + from source → clear error pointing at build/fetch. An *explicit file path* + is handled by the caller before this is reached. + + - Flat ``-.hex`` (bare) or ``-sandbox-.bin`` + (sandbox), as produced by `dotbot fw artifacts`, found across the + cache's ``-/`` and ``-local/`` dirs. + - Else, if a DotBot-firmware repo is locatable, build it (needs SES) + and use the SES output path. + - Else, a friendly error telling the user to build or fetch first. + """ + name = f"{app}-sandbox-{board}.bin" if sandbox else f"{app}-{board}.hex" + cached = _find_in_cache(name) + if cached is not None: + echo_artifact_path(cached, action="using") + return cached + + # Not cached → try to build from source. + from dotbot.cli import _fw_helpers + + try: + repo = _fw_helpers.resolve_firmware_repo() + except click.ClickException: + repo = None + if repo is not None: + target = f"sandbox-{board}" if sandbox else board + click.echo( + f"[artifacts] {name} not cached; building {app} for {target}...", + err=True, + ) + # run_make → resolve_segger_dir already raises the friendly + # "no SES, use fetch + device flash" message when SES is absent. + _fw_helpers.run_make(target, config, app, rebuild=False, quiet=True) + built = _fw_helpers.artifact_path(target, app, config) + if built.is_file(): + echo_artifact_path(built, action="built") + return built + raise click.ClickException( + f"Build finished but {built} was not produced; check the app name." + ) + + raise click.ClickException( + f"No artifact for {app!r} ({board}) in {artifacts_dir()} and no " + "DotBot-firmware source to build from.\n" + " • `dotbot fw build " + f"-a {app} -t {board}{' --sandbox' if sandbox else ''}` to build, or\n" + " • `dotbot fw fetch` to download the latest releases, then retry, or\n" + " • pass an explicit path: `dotbot device flash `." + ) diff --git a/dotbot/cli/_cfg.py b/dotbot/cli/_cfg.py new file mode 100644 index 00000000..a88d0c46 --- /dev/null +++ b/dotbot/cli/_cfg.py @@ -0,0 +1,42 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Bridge between a Click command's options and the unified config resolver. + +Phase 3 wiring: `fw` / `device` options read their defaults from the loaded +config (stashed on `ctx.obj` by the root group), while an explicit flag on +the command line still wins. The trick is Click's parameter-source check: an +option whose value came from `COMMANDLINE` is a real user choice and beats +the file; an option still sitting at its built-in default yields to the +config/env layers. + +Keeping this in one helper means every command resolves identically, and the +no-config common case stays byte-for-byte the same as before (the option's +own default flows straight through `resolve(..., default=value)`). +""" + +import click + +from dotbot.config import resolve + + +def from_config(ctx: click.Context, param_name: str, key: str, section: str): + """CLI flag if given on the command line, else config > env > the option's default. + + `param_name` is the Click parameter name (what `ctx.params` keys on); + `key` / `section` address the value in the config resolver. When the + option was set on the command line we return it verbatim; otherwise we let + the resolver fall through config (section > deployment > top-level) and env, + using the option's current value as the built-in default. + """ + value = ctx.params.get(param_name) + if ctx.get_parameter_source(param_name) is click.core.ParameterSource.COMMANDLINE: + return value + obj = ctx.obj or {} + return resolve( + key, + section=section, + config=obj.get("config"), + deployment=obj.get("deployment"), + default=value, + ) diff --git a/dotbot/cli/_conn.py b/dotbot/cli/_conn.py new file mode 100644 index 00000000..66cecf6d --- /dev/null +++ b/dotbot/cli/_conn.py @@ -0,0 +1,92 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Parse a `--conn` connection string into a typed result. + +`dotbot run controller --conn CONNECTION` takes one discriminated +connection string whose *form* selects the kind of connection - the +`git remote` / `docker -H` / MAVSDK `add_any_connection()` pattern: + +| value | kind | +|----------------------------------------|-------------| +| `mqtts://host:port` / `mqtt://host:port` | `mqtt` | +| `/dev/ttyACM0`, `/dev/tty.usbmodem…`, `COM3` | `serial` | +| `simulator` / `sim` | `simulator` | + +Keeping this a pure function (no Click, no I/O) so it's exhaustively +unit-testable without hardware. +""" + +from dataclasses import dataclass +from typing import Optional + +from marilib.communication_adapter import parse_mqtt_url + + +@dataclass(frozen=True) +class ConnectionSpec: + """Result of parsing a `--conn` value. + + `kind` is one of "mqtt" | "serial" | "simulator". The other fields + are populated per kind: + - mqtt: host, port, use_tls + - serial: serial_port + - simulator: (no extra fields; robot type is a separate flag) + """ + + kind: str + host: Optional[str] = None + port: Optional[int] = None + use_tls: bool = False + serial_port: Optional[str] = None + + +class ConnError(ValueError): + """Raised when a `--conn` value can't be parsed.""" + + +# Accepted spellings for the simulator (documented: `simulator`). +_SIMULATOR_VALUES = {"simulator", "sim"} + + +def parse_connection(value: str) -> ConnectionSpec: + """Parse a `--conn` string. Raises ConnError on a malformed value.""" + if not value: + raise ConnError("empty connection string") + + lowered = value.strip().lower() + + # Simulator — a bare keyword. + if lowered in _SIMULATOR_VALUES: + return ConnectionSpec(kind="simulator") + + # MQTT — discriminated by the mqtt:// / mqtts:// scheme. Delegate the + # url → parts mapping to marilib's parse_mqtt_url, the single source of + # truth shared with swarmit (so the default-port logic can't drift). + if lowered.startswith(("mqtt://", "mqtts://")): + host, port, use_tls, _user, _pass = parse_mqtt_url(value) + if not host: + raise ConnError(f"no host in MQTT connection string: {value!r}") + return ConnectionSpec(kind="mqtt", host=host, port=port, use_tls=use_tls) + + # Anything else that has a `scheme://` we don't recognize is an error, + # rather than being silently treated as a device path. + if "://" in value: + raise ConnError( + f"unrecognized connection scheme in {value!r} " + "(expected mqtt:// or mqtts://, a device path, or 'simulator')" + ) + + # Otherwise: treat as a serial device path (`/dev/ttyACM0`, `COM3`, …). + # Plain path (no `serial://` scheme) so shell tab-completion works. + return ConnectionSpec(kind="serial", serial_port=value) + + +def needs_swarm_id(parsed: ConnectionSpec) -> bool: + """True if this connection requires `--swarm-id`. + + Only MQTT does: the broker carries traffic for many swarms, and the + swarm id selects the topic namespace. A serial gateway already + belongs to one swarm; a simulator has none. + """ + return parsed.kind == "mqtt" diff --git a/dotbot/cli/_fw_helpers.py b/dotbot/cli/_fw_helpers.py new file mode 100644 index 00000000..a40f3bf2 --- /dev/null +++ b/dotbot/cli/_fw_helpers.py @@ -0,0 +1,321 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Shared helpers for the `dotbot fw` build commands (bare + --sandbox). + +`dotbot fw build/clean/targets/artifacts` shell out to the same +`DotBot-firmware` Makefile, which discriminates bare vs sandbox by +`BUILD_TARGET` prefix (`sandbox-*` routes to `apps-sandbox/`, everything +else to `apps/`). Sandbox is the `--sandbox` flavor flag on those +commands (not a separate namespace). The helpers here keep target +validation, SEGGER_DIR resolution, and the make invocation contract in +one place. + +## Configuration + +`SEGGER_DIR` can be persisted in `~/.dotbot/config.toml` so it doesn't +have to ride in every shell: + +```toml +[fw] +segger_dir = "/Applications/SEGGER/SEGGER Embedded Studio 8.30" +``` + +Resolution order (first match wins): +- SEGGER: `SEGGER_DIR` env var → `[fw].segger_dir` in config → glob + `/Applications/SEGGER/SEGGER Embedded Studio*` on macOS. +- firmware repo: `DOTBOT_FIRMWARE_REPO` env var → `[fw].firmware_repo` in + config → `/DotBot-firmware/`. No parent walk-up or `repos/` heuristics: + set the env var, persist the path in config, or `cd` to where your clone is. +""" + +import difflib +import glob +import os +import subprocess +import sys +import time +from pathlib import Path +from typing import Iterable, Optional + +import click + +from dotbot.firmware.boards import BOARDS + +# Glob used to discover SES installs on macOS. Picks the lexicographically +# largest match (e.g. "Studio 8.30" beats "Studio 8.22a"), which is good +# enough as a fallback when the user hasn't set SEGGER_DIR or written +# `[fw].segger_dir` in their dotbot config. +_SEGGER_MACOS_GLOB = "/Applications/SEGGER/SEGGER Embedded Studio*" + +# BUILD_TARGET / flashable board names. Single source of truth is the board +# table in `dotbot.firmware.boards` (which also carries each board's nrfjprog +# family + core) — so a valid build target and a flashable board can't drift +# apart. An unrecognized target falls through to the Makefile's catch-all +# `find apps/` rule (opaque SES errors), so we validate up-front. +BARE_TARGETS = frozenset(BOARDS) + +# BUILD_TARGET = "sandbox-" + BOARD for the sandbox path. Boards +# supported by the SES `.emProject` files at the DotBot-firmware root. +SANDBOX_BOARDS = frozenset({"dotbot-v2", "dotbot-v3", "nrf5340dk"}) + +# Valid `BUILD_CONFIG` values. +CONFIGS = ("Debug", "Release") +DEFAULT_CONFIG = "Release" +DEFAULT_BARE_TARGET = "dotbot-v3" +DEFAULT_SANDBOX_BOARD = "dotbot-v3" + + +def _config_fw_value(key: str) -> Optional[str]: + """Read `[fw].` from the resolved unified config, or None. + + Uses the config the root `dotbot` group already resolved onto the Click + context when one is active (so `-c`, the cwd `dotbot.toml`, the + `~/.dotbot/config.toml` fallback, and flag precedence all apply); for + direct (non-CLI) calls it discovers and loads the config fresh. + """ + ctx = click.get_current_context(silent=True) + cfg = ( + ctx.obj.get("config") + if (ctx is not None and isinstance(ctx.obj, dict)) + else None + ) + if cfg is None: + from dotbot import config as _config + + try: + cfg, _ = _config.load_discovered() + except _config.ConfigError as exc: + raise click.ClickException(str(exc)) from exc + val = getattr(cfg.fw, key, None) + return str(val) if val else None + + +def _glob_macos_segger() -> Optional[Path]: + """Pick the lexicographically-latest SES install matching the glob. + + Returns None if no match has a usable `bin/emBuild`. The sort order + favours newer versions (e.g. `8.30` > `8.22a`) for typical SES + version strings. + """ + if sys.platform != "darwin": + return None + matches = sorted(glob.glob(_SEGGER_MACOS_GLOB)) + for match in reversed(matches): + candidate = Path(match) + if (candidate / "bin" / "emBuild").is_file(): + return candidate + return None + + +def resolve_segger_dir() -> Path: + """SEGGER_DIR env → config → macOS glob → error.""" + env = os.environ.get("SEGGER_DIR") + if env: + return Path(env) + cfg = _config_fw_value("segger_dir") + if cfg: + return Path(cfg) + macos = _glob_macos_segger() + if macos: + return macos + raise click.ClickException( + "Building firmware from source needs SEGGER Embedded Studio (SES), " + "which wasn't found.\n" + " • Export SEGGER_DIR, or add to ~/.dotbot/config.toml:\n" + " [fw]\n" + ' segger_dir = "/path/to/SEGGER Embedded Studio X.YY"\n' + " • You do NOT need SES to run firmware: `dotbot fw fetch -f ` " + "downloads pre-built release binaries and `dotbot device flash` flashes " + "them.\n" + "(A license-free CMake/GCC build path is planned; until then, building " + "from source needs SES.)" + ) + + +def resolve_firmware_repo() -> Path: + """DOTBOT_FIRMWARE_REPO env → `[fw].firmware_repo` config → ./DotBot-firmware/ → error. + + Mirrors `resolve_segger_dir`: an env var wins, else the persisted + `[fw].firmware_repo` in `~/.dotbot/config.toml`, else the user `cd`'d to a + directory containing a sibling `DotBot-firmware/` clone. No parent walk-up + or `repos/` heuristics. + """ + env = os.environ.get("DOTBOT_FIRMWARE_REPO") + if env: + candidate = Path(env) + if (candidate / "Makefile").is_file(): + return candidate + raise click.ClickException( + f"DOTBOT_FIRMWARE_REPO={env!r} does not contain a Makefile." + ) + cfg = _config_fw_value("firmware_repo") + if cfg: + candidate = Path(cfg) + if (candidate / "Makefile").is_file(): + return candidate + raise click.ClickException( + f"[fw].firmware_repo={cfg!r} does not contain a Makefile." + ) + candidate = Path.cwd() / "DotBot-firmware" + if (candidate / "Makefile").is_file(): + return candidate + raise click.ClickException( + "Could not locate DotBot-firmware. Either:\n" + " - `cd` to the directory containing your DotBot-firmware clone,\n" + " - export DOTBOT_FIRMWARE_REPO=/path/to/DotBot-firmware, or\n" + ' - add to ~/.dotbot/config.toml: [fw]\\n firmware_repo = "/path/to/DotBot-firmware"' + ) + + +def suggest_close_match(name: str, candidates: Iterable[str]) -> str: + """One-shot 'did you mean X?' suggestion, or empty string if none close.""" + close = difflib.get_close_matches(name, list(candidates), n=1, cutoff=0.6) + return f" Did you mean {close[0]!r}?" if close else "" + + +def validate_bare_target(target: str) -> None: + if target.startswith("sandbox-"): + raise click.ClickException( + f"{target!r} is a sandbox target. Use " + f"`dotbot fw build -t {target[len('sandbox-'):]} --sandbox` instead." + ) + if target not in BARE_TARGETS: + hint = suggest_close_match(target, BARE_TARGETS) + raise click.ClickException( + f"Unknown bare target {target!r}.{hint}\n" + f"Run `dotbot fw targets` to list valid bare targets." + ) + + +def validate_sandbox_board(board: str) -> None: + if board.startswith("sandbox-"): + raise click.ClickException( + f"Drop the `sandbox-` prefix — pass just the board name: " + f"{board[len('sandbox-'):]!r}." + ) + if board not in SANDBOX_BOARDS: + hint = suggest_close_match(board, SANDBOX_BOARDS) + raise click.ClickException( + f"Unknown sandbox board {board!r}.{hint}\n" + f"Run `dotbot fw targets --sandbox` to list valid sandbox boards." + ) + + +def _make_env(segger_dir: Path) -> dict: + env = dict(os.environ) + env["SEGGER_DIR"] = str(segger_dir) + return env + + +def list_projects(target: str) -> list[str]: + """Return the post-filter project list for `target` via `make list-projects`.""" + repo = resolve_firmware_repo() + segger = resolve_segger_dir() + result = subprocess.run( + ["make", "-s", "list-projects", f"BUILD_TARGET={target}"], + cwd=repo, + env=_make_env(segger), + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + raise click.ClickException( + f"`make list-projects BUILD_TARGET={target}` failed:\n{result.stderr}" + ) + # The Makefile recipe prints an ANSI-styled header line we want to skip; + # take only lines that look like bare project identifiers. + return [ + line.strip() + for line in result.stdout.splitlines() + if line.strip() + and not line.strip().startswith(("\x1b", "\\e[")) + and "Available projects" not in line + ] + + +def run_make( + target: str, + config: str, + project: Optional[str] = None, + *, + rebuild: bool = False, + quiet: bool = True, + make_targets: Optional[list[str]] = None, +) -> float: + """Invoke `make BUILD_TARGET=... BUILD_CONFIG=... [project|make_target]`. + + rebuild=False passes an empty `BUILD_MODE=` to make so the + `emBuild` recipe runs with no action flag — emBuild defaults to + incremental builds in that case. rebuild=True passes + `BUILD_MODE=-rebuild` to force full rebuilds. Requires the + `BUILD_MODE` knob added in DotBot-firmware Makefile (commit + "makefile: parameterize emBuild -rebuild via BUILD_MODE knob"). + + quiet=True passes `QUIET=1` so the Makefile suppresses SES's + `-verbose -echo` flood; the per-project "Building project X" / + "Done" banners still come through. quiet=False also echoes the full + make command line to stderr so the user has a copy-pasteable line + to reproduce outside the CLI. + + If `make_targets` is given, those are the make-level targets passed + on the command line (e.g. `["clean"]`, `["artifacts"]`). Otherwise + `project` is appended (or nothing, which means default `all` → + every project for the BUILD_TARGET). + + Returns elapsed wall-clock seconds. Raises `ClickException` on + non-zero exit so callers can short-circuit. + """ + repo = resolve_firmware_repo() + segger = resolve_segger_dir() + embuild = segger / "bin" / "emBuild" + if not embuild.is_file(): + raise click.ClickException( + f"emBuild not found at {embuild}. Check that SEGGER_DIR points " + f"at a real SES install." + ) + cmd = ["make", f"BUILD_TARGET={target}", f"BUILD_CONFIG={config}"] + if quiet: + cmd.append("QUIET=1") + cmd.append(f"BUILD_MODE={'-rebuild' if rebuild else ''}") + if make_targets: + cmd.extend(make_targets) + elif project: + cmd.append(project) + if not quiet: + # Verbose mode: print the make command so the user can copy/paste + # it to reproduce outside the CLI. + click.echo(f"$ {' '.join(cmd)}", err=True) + t0 = time.perf_counter() + rc = subprocess.call(cmd, cwd=repo, env=_make_env(segger)) + elapsed = time.perf_counter() - t0 + if rc != 0: + raise click.ClickException(f"`make` exited {rc} after {elapsed:.1f}s.") + return elapsed + + +def artifact_path(target: str, project: str, config: str) -> Path: + """Return where SES writes the artifact for (target, project, config). + + SES uses its internal `$(BuildTarget)` macro for the Output directory + and the suffix on the file name. The `.emProject` solution files set + that macro to match the make-level `BUILD_TARGET` exactly (bare + `dotbot-v3` → `BuildTarget=dotbot-v3`, sandbox `sandbox-dotbot-v3` → + `BuildTarget=sandbox-dotbot-v3`), so the on-disk path mirrors what + the Makefile's `ARTIFACT_BASE` formula expects. + """ + is_sandbox = target.startswith("sandbox-") + apps_dir = "apps-sandbox" if is_sandbox else "apps" + ext = "bin" if is_sandbox else "hex" + repo = resolve_firmware_repo() + return ( + repo + / apps_dir + / project + / "Output" + / target + / config + / "Exe" + / f"{project}-{target}.{ext}" + ) diff --git a/dotbot/cli/_lazy.py b/dotbot/cli/_lazy.py new file mode 100644 index 00000000..dd1870ce --- /dev/null +++ b/dotbot/cli/_lazy.py @@ -0,0 +1,75 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Helper for mounting subcommands that live in optional sibling packages. + +Each subcommand sits behind a `pip install dotbot[]` boundary so +the core install stays lean. When the extra is missing we still want +`dotbot --help` to list the subcommand (so users see what exists) and +running it should print an actionable install hint instead of a +traceback. +""" + +import sys +from typing import Callable, Optional + +import click + + +def lazy_subcommand( + *, + name: str, + extra: str, + package: str, + help: str, + loader: Callable[[], click.Command], + transform: Optional[Callable[[click.Command], click.Command]] = None, +) -> click.Command: + """Return a Click command that defers import until invocation. + + If `loader()` raises ImportError, we expose a stub group/command + that prints a clean install hint and exits 1. The stub keeps the + name visible in `dotbot --help` so missing extras are discoverable. + + `transform`, when given, wraps the successfully loaded command — used to + inject behavior at the mount boundary (e.g. config-driven flag defaults). + It is not applied to the missing-extra stub. + """ + try: + cmd = loader() + except ImportError as exc: + return _missing_extra_stub( + name=name, extra=extra, package=package, help=help, error=str(exc) + ) + + if transform is not None: + return transform(cmd) + + # Don't mutate cmd.name — the source package has its own tests that + # assert on the original name. Click uses the lookup-key name from + # the parent's `commands` dict for usage display, so the dispatcher + # still shows e.g. `Usage: dotbot deployment ...` correctly. + return cmd + + +def _missing_extra_stub( + *, name: str, extra: str, package: str, help: str, error: Optional[str] +) -> click.Command: + @click.command( + name=name, + help=f"{help} [install: pip install dotbot[{extra}]]", + context_settings=dict(ignore_unknown_options=True, allow_extra_args=True), + ) + @click.pass_context + def _stub(ctx): # pylint: disable=unused-argument + click.echo( + f"`dotbot {name}` needs the `{package}` package " + f"(not installed in this environment).", + err=True, + ) + click.echo(f"Install with: pip install dotbot[{extra}]", err=True) + if error: + click.echo(f"(import error was: {error})", err=True) + sys.exit(1) + + return _stub diff --git a/dotbot/cli/_lazygroup.py b/dotbot/cli/_lazygroup.py new file mode 100644 index 00000000..c6dcc396 --- /dev/null +++ b/dotbot/cli/_lazygroup.py @@ -0,0 +1,68 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""A Click group that lists subcommands eagerly but imports them on demand. + +Subcommands are declared as static `(cli-name, module path, short help)` +triples. `--help` renders from the triples alone — cheap, no imports — +and a subcommand's module is only imported when that subcommand is +actually invoked. Each module must expose a `cmd` attribute (the Click +command/group to mount). + +Why lazy: importing e.g. `dotbot.controller_app` pulls in `dotbot.server`, +which mounts FastAPI StaticFiles at module load. That's fine for the +`controller` subcommand but `dotbot run --help` shouldn't pay the cost (or +fail when the frontend bundle isn't built). The root group and the `run` +group both use this so the laziness holds at every level of the tree. +""" + +import importlib +from typing import Optional, Tuple + +import click + +# (cli-name, dotted module path, short help shown in the parent's --help) +Subcommand = Tuple[str, str, str] + + +class LazyGroup(click.Group): + """Click group resolving subcommands by importing their module on demand. + + Pass the static subcommand table via the `subcommands=` keyword (it is + captured here and never forwarded to the base `click.Group`, which + would reject the unknown kwarg). + """ + + def __init__(self, *args, subcommands: Tuple[Subcommand, ...] = (), **kwargs): + super().__init__(*args, **kwargs) + self.lazy_subcommands: Tuple[Subcommand, ...] = tuple(subcommands) + self._help_index = {name: short for name, _, short in self.lazy_subcommands} + self._module_index = {name: mod for name, mod, _ in self.lazy_subcommands} + + def list_commands(self, ctx): + return [name for name, _, _ in self.lazy_subcommands] + + def get_command(self, ctx, cmd_name) -> Optional[click.Command]: + module_path = self._module_index.get(cmd_name) + if module_path is None: + return None + module = importlib.import_module(module_path) + command = getattr(module, "cmd", None) + if command is None: + raise RuntimeError( + f"{module_path} is registered as the `{cmd_name}` subcommand " + "but does not expose a `cmd` attribute." + ) + return command + + def format_commands(self, ctx, formatter): + """Render the command list from the static table. + + Overriding this is what keeps `--help` from importing every + subcommand module just to read its one-line help — that would + defeat the lazy load. + """ + rows = [(name, self._help_index[name]) for name, _, _ in self.lazy_subcommands] + if rows: + with formatter.section("Commands"): + formatter.write_dl(rows) diff --git a/dotbot/cli/_swarm_inject.py b/dotbot/cli/_swarm_inject.py new file mode 100644 index 00000000..9bb09f05 --- /dev/null +++ b/dotbot/cli/_swarm_inject.py @@ -0,0 +1,57 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Config -> swarmit argument injection for `dotbot swarm`. + +`dotbot swarm` wraps swarmit's own CLI, which does not read the unified dotbot +config. This translates the resolved `conn` / `swarm_id` into swarmit's flags at +the mount boundary, so a saved `dotbot.toml` drives the fleet like every other +command - while an explicit swarmit flag still wins (swarmit's own precedence is +CLI flag > config file). + +Kept separate from `swarm.py` so it imports without pulling in swarmit, whose +protocol registry collides with PyDotBot's inside a shared test process. +""" + +from typing import Optional, Sequence + +# swarmit's connection flags. Any of these means the user is steering the +# connection explicitly, so we leave their args untouched. +_CONN_FLAGS = ("-n", "--conn", "--connection") +_SWARM_ID_FLAGS = ("-s", "--swarm-id") +_CONFIG_FLAGS = ("-c", "--config-path") +_HELP_FLAGS = ("-h", "--help") + + +def _has_flag(args: Sequence[str], flags: Sequence[str]) -> bool: + """True if any of `flags` (or its `--flag=value` form) appears in `args`.""" + eqs = tuple(f + "=" for f in flags if f.startswith("--")) + return any(arg in flags or (eqs and arg.startswith(eqs)) for arg in args) + + +def inject_config(args: Sequence[str], obj: Optional[dict]) -> list: + """Prepend `--conn` / `--swarm-id` from the resolved config to `args`. + + No-op when the user already passes a `--conn` / `--swarm-id` / `-c` flag or + `--help` (those win), or when no config supplies the value. Injected flags + go first so swarmit parses them as group options ahead of the subcommand. + """ + args = list(args) + if _has_flag(args, _HELP_FLAGS) or _has_flag(args, _CONFIG_FLAGS): + return args + + from dotbot.config import resolve + + obj = obj or {} + config = obj.get("config") + deployment = obj.get("deployment") + injected: list = [] + conn = resolve("conn", section="swarm", config=config, deployment=deployment) + swarm_id = resolve( + "swarm_id", section="swarm", config=config, deployment=deployment + ) + if conn and not _has_flag(args, _CONN_FLAGS): + injected += ["--conn", str(conn)] + if swarm_id and not _has_flag(args, _SWARM_ID_FLAGS): + injected += ["--swarm-id", str(swarm_id)] + return injected + args diff --git a/dotbot/cli/calibrate.py b/dotbot/cli/calibrate.py new file mode 100644 index 00000000..4d4dfa8c --- /dev/null +++ b/dotbot/cli/calibrate.py @@ -0,0 +1,121 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run lh2-calibration` - LH2 calibration. + +Native subgroup mounting the vendored `dotbot.calibration` package, for +single-device calibration over either transport. + +Subcommands: + +- `collect` — capture LH2 counts via the Textual TUI from a single + serial-attached nRF DK; writes ~/.dotbot/calibration.out. +- `apply ` — write the saved calibration as a C header to + . Today the only consumer is the swarmit secure + bootloader (it #includes the file at compile time). + +Cable-free, over-the-air calibration of a DotBot in the arena lives under +`dotbot swarm lh2-calibration` (it drives the fleet transport, not a serial +DK). + +Calibration runtime deps (`opencv-python`, `textual`) live behind the +`[calibrate]` extra; ImportError at subcommand invocation prints an +install hint instead of a traceback. +""" + +import sys + +import click + + +def _run_tui(ctx: click.Context) -> None: + """Lazy-load the TUI Click command and hand off this process's argv tail.""" + try: + from dotbot.calibration.cli import main as _tui_main + except ImportError as exc: + click.echo( + "`dotbot run lh2-calibration collect` needs the calibration " + "runtime deps (opencv-python, textual).\n" + "Install with: pip install dotbot[calibrate]", + err=True, + ) + click.echo(f"(import error was: {exc})", err=True) + sys.exit(1) + # Forward this process's argv tail (anything after `collect`) to the + # TUI Click command. Click's parent group already consumed the + # subcommand name itself, so ctx.args/ctx.parent.args don't carry + # the right tail — let the TUI re-parse from a clean state. + _tui_main.main(args=list(ctx.args), standalone_mode=True) + + +@click.group( + name="lh2-calibration", + help="LH2 calibration for one serial-attached device: capture, apply.", + invoke_without_command=True, +) +@click.pass_context +def cmd(ctx: click.Context) -> None: + if ctx.invoked_subcommand is not None: + return + # Bare `dotbot run lh2-calibration` with no subcommand defaults to + # collect — the most common action — so it works without recalling + # the subcommand name. + _run_tui(ctx) + + +@cmd.command( + name="collect", + context_settings=dict( + ignore_unknown_options=True, + allow_extra_args=True, + help_option_names=[], + ), + add_help_option=False, + help="Capture LH2 counts via the Textual TUI (serial-attached DK).", +) +@click.pass_context +def _collect(ctx: click.Context) -> None: + _run_tui(ctx) + + +@cmd.command( + name="apply", + help=( + "Write the saved calibration as a C header to PATH. Today the " + "consumer is the swarmit secure bootloader (#includes the file " + "at compile time). The over-the-air / runtime equivalent is " + "`dotbot swarm lh2-calibration push`." + ), +) +@click.argument( + "path", + type=click.Path(dir_okay=False, writable=True), +) +def _apply(path: str) -> None: + try: + from dotbot.calibration.exporter import export_calibration + from dotbot.calibration.lighthouse2 import LighthouseManager + except ImportError as exc: + click.echo( + "`dotbot run lh2-calibration apply` needs the calibration " + "runtime deps.\nInstall with: pip install dotbot[calibrate]", + err=True, + ) + click.echo(f"(import error was: {exc})", err=True) + sys.exit(1) + + lh2_manager = LighthouseManager() + calibrations = lh2_manager.load_calibration() + if not calibrations: + click.echo( + "No saved calibration found at " + f"{lh2_manager.calibration_output_path}.\n" + "Run `dotbot run lh2-calibration collect` first.", + err=True, + ) + sys.exit(1) + + output = export_calibration(calibrations) + with open(path, "w") as f: + f.write(output) + click.echo(f"Wrote calibration ({len(calibrations)} matrices) to {path}") diff --git a/dotbot/cli/config_cmd.py b/dotbot/cli/config_cmd.py new file mode 100644 index 00000000..de8713bf --- /dev/null +++ b/dotbot/cli/config_cmd.py @@ -0,0 +1,153 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot config` - scaffold and inspect the dotbot configuration. + +A management group (like `git config` / `kubectl config`): `init` writes a +starter config file (optionally pre-filling `conn` / `swarm_id`); `path` and +`show` are read-only inspectors over what the root group resolved onto the +Click context (`ctx.obj`): the loaded `DotbotConfig`, its source path, and the +selected deployment. There is no per-key `set` - edit the file, it is yours. +""" + +from pathlib import Path +from typing import Any + +import click +import tomlkit + +from dotbot.config import USER_CONFIG_PATH + +_CONFIG_DOCS_URL = ( + "https://pydotbot.readthedocs.io/en/latest/reference/configuration.html" +) + + +# `dotbot config init` writes a *minimal* file: just the keys you pass, plus a +# one-line pointer to the full reference. No wall of commented options - the +# schema lives in the docs, not in everyone's config file. +def _starter_template(conn: str | None = None, swarm_id: str | None = None) -> str: + header = ( + f"# dotbot config. Options + examples: {_CONFIG_DOCS_URL}\n" + "# (MQTT credentials are env-only: DOTBOT_MQTT_USER / DOTBOT_MQTT_PASS.)\n" + ) + keys = [] + if conn: + keys.append(f'conn = "{conn}"') + if swarm_id: + keys.append(f'swarm_id = "{swarm_id}"') + if keys: + return header + "\n" + "\n".join(keys) + "\n" + return header + + +@click.group( + name="config", + help="Show the resolved config + where it came from; scaffold one with init.", +) +def cmd(): + pass + + +@cmd.command() +@click.option( + "--global", + "global_", + is_flag=True, + help="Write the user-level ~/.dotbot/config.toml instead of ./dotbot.toml.", +) +@click.option("--force", "-f", is_flag=True, help="Overwrite an existing file.") +@click.option( + "--conn", + help="Pre-fill the shared connection (broker URL, serial path, or 'simulator').", +) +@click.option("--swarm-id", help="Pre-fill the shared swarm id.") +def init(global_, force, conn, swarm_id): + """Write a minimal starter config file you can edit. + + Defaults to ./dotbot.toml in the current directory; --global writes your + user-level ~/.dotbot/config.toml. Refuses to overwrite unless --force. + `--conn` / `--swarm-id` pre-fill those top-level keys; the file otherwise + holds just a one-line pointer to the full reference (no wall of options). + """ + if conn is not None: + from dotbot.cli._conn import ConnError, parse_connection + + try: + parse_connection(conn) + except ConnError as exc: + raise click.ClickException(f"invalid --conn: {exc}") from exc + + target = USER_CONFIG_PATH if global_ else Path.cwd() / "dotbot.toml" + if target.exists() and not force: + raise click.ClickException( + f"{target} already exists. Pass --force to overwrite it." + ) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(_starter_template(conn, swarm_id)) + click.echo(f"Wrote {target}") + if conn or swarm_id: + filled = " and ".join( + label for label, val in (("conn", conn), ("swarm_id", swarm_id)) if val + ) + click.echo(f"Set {filled}; review it, then run `dotbot config show`.") + else: + click.echo( + "Add your settings (see the link inside), then `dotbot config show`." + ) + + +@cmd.command() +@click.pass_context +def path(ctx): + """Print the resolved config file path (or note the built-in defaults).""" + config_path = (ctx.obj or {}).get("config_path") + if config_path is None: + click.echo("(none; using built-in defaults)") + click.echo("Create one with: dotbot config init", err=True) + else: + click.echo(str(config_path)) + + +def _prune(value: Any) -> Any: + """Recursively drop None values and empty tables so only set keys remain.""" + if isinstance(value, dict): + pruned = {k: _prune(v) for k, v in value.items() if v is not None} + return {k: v for k, v in pruned.items() if v != {}} + return value + + +@cmd.command() +@click.pass_context +def show(ctx): + """Print the source path, the active deployment, and the loaded config. + + None-valued fields are skipped so only what is actually set shows up. + """ + obj = ctx.obj or {} + config = obj.get("config") + config_path = obj.get("config_path") + deployment_name = obj.get("deployment_name") + + source = ( + str(config_path) if config_path is not None else "(none; built-in defaults)" + ) + click.echo(f"source: {source}") + click.echo(f"deployment: {deployment_name or '(none)'}") + click.echo("") + + if config is None: + click.echo("(no config loaded)") + return + + # Prune unset Optionals so the dump shows only what the file explicitly set + # (matches the resolver's "unset vs default" model), then render via tomlkit + # so the output is real, round-trippable TOML. + data = _prune(config.model_dump()) + if not data: + if config_path is None: + click.echo("No config file found. Create one with: dotbot config init") + else: + click.echo("(the file sets nothing yet; all built-in defaults)") + return + click.echo(tomlkit.dumps(data).rstrip()) diff --git a/dotbot/cli/controller.py b/dotbot/cli/controller.py new file mode 100644 index 00000000..6e1c4866 --- /dev/null +++ b/dotbot/cli/controller.py @@ -0,0 +1,16 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run controller` — start the controller + REST/WS + dashboard. + +Today this re-mounts the existing `dotbot.controller_app:main` Click +command verbatim. A future refactor will extract the engine from the +controller monolith. +""" + +from dotbot.controller_app import main as _controller_main + +# Re-export the existing command without mutation. The `run` group +# registers it under name="controller"; Click's usage formatter uses +# that lookup name, not cmd.name, so we don't need to rewrite it. +cmd = _controller_main diff --git a/dotbot/cli/demo.py b/dotbot/cli/demo.py new file mode 100644 index 00000000..ff78543c --- /dev/null +++ b/dotbot/cli/demo.py @@ -0,0 +1,48 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run demo` — discoverable launcher for built-in research demos. + +Demos live in `dotbot/examples/`. Each demo consumes the controller's +REST/WS API and runs as a separate process — the controller stays +agnostic to whichever demo (if any) is talking to it. Adding a new +demo means dropping a Click command somewhere under `dotbot/examples/` +and registering it below. +""" + +import click + +from dotbot.examples.circle.circle import main as _circle_main +from dotbot.examples.qrkey_demo.cli import main as _qrkey_main + + +@click.group( + name="demo", + help="Built-in research demos (qrkey phone bridge, formations, ...).", + invoke_without_command=True, +) +@click.option( + "--list", + "list_demos", + is_flag=True, + help="List available demos.", +) +@click.pass_context +def cmd(ctx, list_demos): + if list_demos or ctx.invoked_subcommand is None: + click.echo("Available demos:") + for name, sub in cmd.commands.items(): + short = (sub.help or "").splitlines()[0] if sub.help else "" + click.echo(f" {name:<12} {short}") + click.echo("") + click.echo("Run one with: dotbot run demo [OPTIONS]") + ctx.exit(0) + + +# Register demos. New entries go here. Keep the alias short (the +# subcommand IS the discovery surface — long names defeat the point). +# We pass `name=...` rather than mutating `_qrkey_main.name` so the +# demo's own test suite (which imports the same Click command) stays +# unaffected. +cmd.add_command(_circle_main, name="circle") +cmd.add_command(_qrkey_main, name="qr") diff --git a/dotbot/cli/deployment_cmd.py b/dotbot/cli/deployment_cmd.py new file mode 100644 index 00000000..f418d4fc --- /dev/null +++ b/dotbot/cli/deployment_cmd.py @@ -0,0 +1,277 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot deployment` - list / show / switch the configured deployments. + +A deployment is one named physical deployment (Inria/100, La Poste/1000, ...) +defined by a `[deployment.]` table in the config file. You *select* one +(`--deployment` / `DOTBOT_DEPLOYMENT` / `default_deployment`) per invocation; +`deployment use` writes the `default_deployment` for you, so switching is one +command rather than a hand edit. `list` / `show` are read-only inspectors. +""" + +import tomllib +from pathlib import Path + +import click +import httpx +import tomlkit + +from dotbot.config import ( + PROJECT_CONFIG_NAME, + USER_CONFIG_PATH, + ConfigError, + discover_config_path, + load_config_text, +) + +# Where `deployment fetch` (no SOURCE) looks for the published registry. The +# `/releases/latest/download/` path 302-redirects to the newest release asset, +# so this URL is stable across republishes. Not published yet - Geovane owns it. +_DEFAULT_REGISTRY_URL = ( + "https://github.com/DotBots/deployments/releases/latest/download/deployments.toml" +) + + +@click.group( + name="deployment", + help="List / show deployments; switch the default with `use`, fetch published ones.", +) +def cmd(): + pass + + +# The descriptive fields worth showing inline, in display order. +_FIELDS = ("conn", "swarm_id", "serial_port", "location", "bots") + + +def _deployment_fields(deployment) -> list[tuple[str, object]]: + """The (name, value) pairs that are actually set on a deployment.""" + return [ + (field, getattr(deployment, field)) + for field in _FIELDS + if getattr(deployment, field) is not None + ] + + +@cmd.command(name="list") +@click.pass_context +def list_deployments(ctx): + """List configured deployment names, marking the active one (*).""" + obj = ctx.obj or {} + config = obj.get("config") + active = obj.get("deployment_name") + + deployments = config.deployment if config is not None else {} + if not deployments: + click.echo("(no deployments configured)") + return + + for name in sorted(deployments): + marker = "* " if name == active else " " + click.echo(f"{marker}{name}") + for field, value in _deployment_fields(deployments[name]): + click.echo(f" {field}: {value}") + + +@cmd.command() +@click.argument("name") +@click.pass_context +def show(ctx, name): + """Print one deployment's fields. Errors if NAME isn't defined.""" + obj = ctx.obj or {} + config = obj.get("config") + + deployments = config.deployment if config is not None else {} + if name not in deployments: + known = ", ".join(sorted(deployments)) or "(none defined)" + raise click.ClickException( + f"unknown deployment {name!r}; defined deployments: {known}" + ) + + active = obj.get("deployment_name") + suffix = " (active)" if name == active else "" + click.echo(f"{name}{suffix}") + fields = _deployment_fields(deployments[name]) + if not fields: + click.echo(" (no fields set)") + return + for field, value in fields: + click.echo(f" {field}: {value}") + + +def _set_default_deployment(path: Path, name: str) -> None: + """Write `default_deployment = ""` into `path`, preserving the rest. + + tomlkit round-trips the document, so existing comments and structure stay + intact; it auto-places a new top-level key before the first `[table]` so the + result is valid TOML whether or not the key already existed. + """ + doc = tomlkit.parse(path.read_text()) + doc["default_deployment"] = name + path.write_text(tomlkit.dumps(doc)) + + +@cmd.command() +@click.argument("name") +@click.pass_context +def use(ctx, name): + """Set NAME as the default deployment, writing it to the active config file. + + Updates `default_deployment` in the file `dotbot` is currently using (the + one `dotbot config path` reports), keeping the rest of the file - comments + included - intact. NAME must be a defined `[deployment.]`. + """ + obj = ctx.obj or {} + config = obj.get("config") + config_path = obj.get("config_path") + + if config_path is None: + raise click.ClickException( + "no config file in use to write to; create one with " + "`dotbot config init` (or point at one with `dotbot -c PATH`)." + ) + deployments = config.deployment if config is not None else {} + if name not in deployments: + known = ", ".join(sorted(deployments)) or "(none defined)" + raise click.ClickException( + f"unknown deployment {name!r}; defined deployments: {known}" + ) + + _set_default_deployment(Path(config_path), name) + click.echo(f"Set default deployment to {name!r} in {config_path}") + + +def _read_source(source: str) -> str: + """Return the text of SOURCE - a local file path, or an http(s) URL.""" + if Path(source).is_file(): + return Path(source).read_text() + if source.startswith(("http://", "https://")): + try: + response = httpx.get(source, follow_redirects=True, timeout=30.0) + response.raise_for_status() + except httpx.HTTPError as exc: + raise click.ClickException(f"could not fetch {source}: {exc}") from exc + return response.text + raise click.ClickException(f"not a URL or an existing file: {source!r}") + + +def _merge_target(into: str) -> Path: + """The file `fetch` writes into: the user config, or the project dotbot.toml.""" + if into == "project": + found = discover_config_path(include_user_file=False) + return found if found is not None else Path.cwd() / PROJECT_CONFIG_NAME + return USER_CONFIG_PATH + + +def _diff_deployments(target: Path, fetched: dict) -> list[tuple[str, str]]: + """Per-name status of fetched vs target: 'added' / 'changed' / 'same'.""" + existing = {} + if target.is_file(): + existing = tomllib.loads(target.read_text()).get("deployment", {}) + changes = [] + for name in sorted(fetched): + new_fields = fetched[name].model_dump(exclude_none=True) + old = existing.get(name) + if old is None: + changes.append((name, "added")) + elif old == new_fields: + changes.append((name, "same")) + else: + changes.append((name, "changed")) + return changes + + +def _write_deployments(target: Path, fetched: dict, changes: list) -> None: + """Upsert the added/changed `[deployment.*]` tables, preserving everything else. + + Uses tomlkit so a hand-edited target keeps its comments, other deployments, + and `[fw]`/`[device]`/`[swarm]`/`[run]` sections; only the named tables that + actually changed are replaced. + """ + if target.is_file(): + doc = tomlkit.parse(target.read_text()) + else: + doc = tomlkit.document() + doc.add( + tomlkit.comment( + " DotBot deployments - managed by `dotbot deployment fetch`." + ) + ) + deployments = doc.get("deployment") + if deployments is None: + deployments = tomlkit.table(is_super_table=True) + doc["deployment"] = deployments + + status_by_name = dict(changes) + for name in sorted(fetched): + if status_by_name.get(name) == "same": + continue + table = tomlkit.table() + for key, value in fetched[name].model_dump(exclude_none=True).items(): + table[key] = value + deployments[name] = table + + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(tomlkit.dumps(doc)) + + +@cmd.command() +@click.argument("source", required=False) +@click.option( + "--into", + type=click.Choice(["user", "project"]), + default="user", + help="Which file to write into (default: your ~/.dotbot/config.toml).", +) +@click.option("--dry-run", is_flag=True, help="Show what would change; write nothing.") +@click.option( + "--yes", + "-y", + is_flag=True, + help="Don't prompt before replacing an existing deployment.", +) +def fetch(source, into, dry_run, yes): + """Fetch published deployments and merge them into your config. + + SOURCE is a URL or a local file holding `[deployment.*]` tables; with no + SOURCE the built-in DotBots registry is used. Existing deployments of the + same name are replaced (you are asked first); everything else in the file - + other deployments, sections, comments - is left intact. Like `fw fetch`, + this only acquires: select one with `dotbot deployment use` / `--deployment`. + """ + source = source or _DEFAULT_REGISTRY_URL + text = _read_source(source) + try: + config = load_config_text(text, source=source) + except ConfigError as exc: + raise click.ClickException(str(exc)) from exc + fetched = config.deployment + if not fetched: + raise click.ClickException(f"no [deployment.*] tables found in {source}") + + target = _merge_target(into) + changes = _diff_deployments(target, fetched) + + symbol = {"added": "+", "changed": "~", "same": "="} + for name, status in changes: + click.echo(f" {symbol[status]} {name}") + + if all(status == "same" for _, status in changes): + click.echo(f"Already up to date; {target} unchanged.") + return + if dry_run: + click.echo(f"(dry run; {target} unchanged)") + return + + changed = [name for name, status in changes if status == "changed"] + if changed and not yes: + click.confirm( + f"This replaces {len(changed)} existing deployment(s) " + f"({', '.join(changed)}) in {target}. Continue?", + abort=True, + ) + + _write_deployments(target, fetched, changes) + written = sum(1 for _, status in changes if status != "same") + click.echo(f"Wrote {written} deployment(s) to {target}") diff --git a/dotbot/cli/device.py b/dotbot/cli/device.py new file mode 100644 index 00000000..f0b92323 --- /dev/null +++ b/dotbot/cli/device.py @@ -0,0 +1,257 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot device` — operate on ONE connected device over the J-Link cable. + +Single-device, cabled (nrfjprog / J-Link) operations: flash a user app, +flash the sandbox-host or gateway role (2-image bundle + shared config +page + network identity), flash the on-board programmer chip, and read +provisioning state. The fleet/OTA equivalents live under `dotbot swarm`; +firmware ARTIFACT build/fetch/list live under `dotbot fw`. + +NOTE: `dotbot device flash-mari-gateway` FLASHES gateway firmware onto a board +over the cable. `dotbot run gateway` is something else entirely — the +host-side UART<->MQTT bridge process. Different verbs, different objects. +""" + +from pathlib import Path + +import click + +from dotbot.cli._artifacts import ( + DEFAULT_ARTIFACTS_DISPLAY, + artifacts_dir, + ensure_nrfjprog, + resolve_app_artifact, +) +from dotbot.cli._cfg import from_config + + +@click.group( + name="device", + help="One connected device (J-Link cable): flash an app/role, read info.", +) +def cmd(): + pass + + +def _looks_like_path(value: str) -> bool: + """True if `value` is a firmware file rather than an app name.""" + return ( + value.endswith((".hex", ".bin")) + or "/" in value + or "\\" in value + or Path(value).is_file() + ) + + +@cmd.command() +@click.argument("app") +@click.option("--sn-starting-digits", "-s", help="J-Link serial prefix, e.g. 77.") +@click.option( + "--board", + "-b", + default="dotbot-v3", + show_default=True, + help=( + "Target board: selects the chip family + core to flash (nRF52 vs " + f"nRF5340 app/net) and resolves - in {DEFAULT_ARTIFACTS_DISPLAY}/." + ), +) +@click.option("--sandbox", is_flag=True, help="Resolve the sandbox-app flavor (.bin).") +@click.option( + "--build-config", + "config", + type=click.Choice(("Debug", "Release")), + default="Release", + show_default=True, + help="Build configuration (for auto-resolving the artifact).", +) +@click.pass_context +def flash(ctx, app, sn_starting_digits, board, sandbox, config): + """Flash a firmware image to one cabled device (whole-chip program). + + APP is an app name (resolved against ~/.dotbot/artifacts/, building from source + if needed) or an explicit `.hex`/`.bin` file path. `--board` selects the + chip family + core to program (see `dotbot fw targets`); no sandbox host + is required. + """ + from dotbot.firmware.flash import flash_app_image + + board = from_config(ctx, "board", "board", "device") + sn_starting_digits = from_config( + ctx, "sn_starting_digits", "sn_starting_digits", "device" + ) + config = from_config(ctx, "config", "build_config", "device") + ensure_nrfjprog() + if _looks_like_path(app): + image = Path(app) + if not image.is_file(): + raise click.ClickException(f"Firmware image not found: {image}") + else: + image = resolve_app_artifact(app, board=board, config=config, sandbox=sandbox) + flash_app_image(image, board=board, sn_starting_digits=sn_starting_digits) + + +def _fw_version_option(f): + return click.option( + "--fw-version", + "-f", + default=None, + help=( + "Release version to flash, e.g. 0.8.0rc2 (default: the swarmit " + f"version pydotbot pins). Binaries are fetched into {DEFAULT_ARTIFACTS_DISPLAY}/ if not cached." + ), + )(f) + + +def _sn_option(f): + return click.option( + "--sn-starting-digits", "-s", help="J-Link serial prefix, e.g. 77." + )(f) + + +@cmd.command(name="flash-swarmit-sandbox") +@click.option( + "--swarm-id", + default=None, + help="16-bit hex swarm id (e.g. 0100); defaults to your config's swarm_id.", +) +@click.option( + "--calibration", + "-l", + "calibration_path", + type=click.Path(path_type=Path, dir_okay=False, exists=True), + help="Optional LH2 calibration file to bake into the config page.", +) +@_fw_version_option +@_sn_option +@click.pass_context +def flash_swarmit_sandbox( + ctx, swarm_id, calibration_path, fw_version, sn_starting_digits +): + """Turn a DotBot v3 into a swarm sandbox host (was `provision -d dotbot-v3`). + + Flashes the SwarmIT bootloader (app core) + netcore + writes the + network identity. Auto-fetches the release if not already in + ~/.dotbot/artifacts/swarmit-/. + """ + from dotbot.firmware.fetch import pinned_version + from dotbot.firmware.flash import flash_role, normalize_network_id + + swarm_id = from_config(ctx, "swarm_id", "swarm_id", None) + if swarm_id is None: + raise click.ClickException( + "no swarm id: pass --swarm-id, or set swarm_id (or a " + "deployment) in your config." + ) + if fw_version is None: + fw_version = pinned_version("swarmit") + click.echo( + f"No version specified, using the pinned swarmit version: {fw_version}" + ) + ensure_nrfjprog() + net_id = normalize_network_id(swarm_id) + flash_role( + "dotbot-v3", + net_id=net_id, + fw_version=fw_version, + calibration_path=calibration_path, + bin_dir=artifacts_dir(), + sn_starting_digits=sn_starting_digits, + ) + + +@cmd.command(name="flash-mari-gateway") +@click.option( + "--swarm-id", + default=None, + help="16-bit hex swarm id (e.g. 0100); defaults to your config's swarm_id.", +) +@_fw_version_option +@_sn_option +@click.pass_context +def flash_mari_gateway(ctx, swarm_id, fw_version, sn_starting_digits): + """Turn an nRF5340-DK into the swarm gateway (was `provision -d gateway`). + + Flashes the Mari gateway firmware (both cores) + writes the network + identity. Auto-fetches the release if absent. (To run the host-side + UART<->MQTT bridge instead, use `dotbot run gateway`.) + """ + from dotbot.firmware.fetch import pinned_version + from dotbot.firmware.flash import flash_role, normalize_network_id + + swarm_id = from_config(ctx, "swarm_id", "swarm_id", None) + if swarm_id is None: + raise click.ClickException( + "no swarm id: pass --swarm-id, or set swarm_id (or a " + "deployment) in your config." + ) + if fw_version is None: + fw_version = pinned_version("swarmit") + click.echo( + f"No version specified, using the pinned swarmit version: {fw_version}" + ) + ensure_nrfjprog() + net_id = normalize_network_id(swarm_id) + flash_role( + "gateway", + net_id=net_id, + fw_version=fw_version, + bin_dir=artifacts_dir(), + sn_starting_digits=sn_starting_digits, + ) + + +@cmd.command(name="flash-programmer") +@click.option( + "--programmer-firmware", + "-p", + type=click.Choice(("jlink", "daplink")), + required=True, +) +@click.option( + "--files-dir", + "-d", + type=click.Path(path_type=Path, file_okay=False, dir_okay=True), + required=True, +) +@click.option("--probe-uid", help="pyOCD probe UID (when multiple probes attached).") +def flash_programmer(programmer_firmware, files_dir, probe_uid): + """Flash J-Link OB / DAPLink firmware to the on-board debug chip. + + Obscure, one-time-per-board bring-up (was `provision flash-bringup`). + """ + from dotbot.firmware.flash import flash_programmer as _flash_programmer + + _flash_programmer(programmer_firmware, files_dir, probe_uid) + + +@cmd.command() +@_sn_option +def info(sn_starting_digits): + """Read a device's provisioning state (chip id + network identity). + + Never fails on a blank/unprovisioned board — reports 'not + provisioned' and how to fix it. + """ + from dotbot.firmware.flash import read_config_report + + ensure_nrfjprog() + try: + net_id, device_id = read_config_report(sn_starting_digits) + except RuntimeError as exc: + raise click.ClickException(f"Could not read the device: {exc}") from exc + + last6 = device_id[-6:] + last6_spaced = " ".join(last6[i : i + 2] for i in range(0, len(last6), 2)) + click.echo(f"device-id: {device_id} (last 6: {last6_spaced})") + if net_id == "unprovisioned": + click.echo("config: not provisioned (no swarm config on this device)") + click.echo( + " → run `dotbot device flash-swarmit-sandbox` (robot) or " + "`flash-mari-gateway` (gateway) first." + ) + else: + click.echo("config: provisioned") + click.echo(f" net-id: 0x{net_id}") diff --git a/dotbot/cli/fw.py b/dotbot/cli/fw.py new file mode 100644 index 00000000..a1d4966e --- /dev/null +++ b/dotbot/cli/fw.py @@ -0,0 +1,378 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot fw` — firmware artifacts: build, fetch, list, make. + +`fw` is the *artifacts* namespace — it produces or downloads firmware +files. It never touches hardware: flashing a device lives under `fw`'s +sibling `dotbot device`, and OTA-flashing the fleet under `dotbot swarm`. + +- `build` compiles from source via SES (`emBuild`) in `DotBot-firmware`, + leaving the result in the SES `Output/.../Exe/` tree and echoing that + path — it does *not* copy into the cache. Bare apps by default; + `--sandbox` builds the TrustZone NS flavor (`sandbox-`, `.bin`). +- `artifacts` builds *and* collects the result into the cache + (`~/.dotbot/artifacts/dotbot-firmware-local/`), with the flat + `-.hex` / `-sandbox-.bin` names. +- `fetch` downloads the pinned release (or a `-f `/`latest` + override) into `~/.dotbot/artifacts/-/`. +- `list` shows what's cached in `~/.dotbot/artifacts/`. +- `make` is the low-level escape hatch: it forwards arbitrary arguments + to `make` in the firmware repo (workspace-resolved SEGGER_DIR) for the + Makefile knobs `build` deliberately doesn't model. + +Only `artifacts` and `fetch` populate the cache. The device-flash +commands then auto-resolve their input, by *different* rules: `dotbot +device flash ` resolves an app image present in `~/.dotbot/artifacts/` → +build-from-source → error (it never fetches); `device flash-swarmit-sandbox` +/ `flash-mari-gateway` resolve a release's system firmware present in +`~/.dotbot/artifacts/` → fetch (they never build). +""" + +import sys +from pathlib import Path + +import click + +from dotbot.cli._artifacts import ( + DEFAULT_ARTIFACTS_DISPLAY, + artifacts_dir, + echo_artifact_path, +) +from dotbot.cli._cfg import from_config +from dotbot.cli._fw_helpers import ( + BARE_TARGETS, + CONFIGS, + DEFAULT_BARE_TARGET, + DEFAULT_CONFIG, + SANDBOX_BOARDS, + artifact_path, + list_projects, + run_make, + validate_bare_target, + validate_sandbox_board, +) + +_NOT_READY = ( + "`dotbot fw {sub}` is not implemented yet.\n" + "For now: use SEGGER Embedded Studio directly, or invoke the " + "Makefile in your DotBot-firmware checkout (set `DOTBOT_FIRMWARE_REPO`)." +) + + +@click.group( + name="fw", + help=( + "Firmware artifacts: build (from source via SES), fetch (a release), " + "list. Bare apps by default; `--sandbox` for TrustZone NS apps. " + "Flashing lives under `dotbot device` (one board) and `dotbot swarm` " + "(the fleet). Need a Makefile knob? `dotbot fw make` forwards to `make`." + ), +) +def cmd(): + pass + + +def _target_option(f): + """Reusable `--target/-t` option for build/clean/artifacts.""" + return click.option( + "--target", + "-t", + default=DEFAULT_BARE_TARGET, + show_default=True, + help=( + "Board/target (e.g. dotbot-v3, nrf5340dk-app). With --sandbox, " + "pass the board name without the `sandbox-` prefix. See " + "`dotbot fw targets [--sandbox]`." + ), + )(f) + + +def _project_option(f): + """Reusable `--app/-a NAME` option for build/clean/artifacts.""" + return click.option( + "--app", + "-a", + "project", + type=str, + default=None, + help=( + "Build a single app (e.g. `dotbot`, `spin`). " + "Default: build every app available for the target." + ), + )(f) + + +def _config_option(f): + """Reusable `--build-config` option for build/clean/artifacts.""" + return click.option( + "--build-config", + "config", + type=click.Choice(CONFIGS), + default=DEFAULT_CONFIG, + show_default=True, + help="Build configuration (Debug or Release).", + )(f) + + +def _sandbox_option(f): + """Reusable `--sandbox` flavor flag (TrustZone NS apps).""" + return click.option( + "--sandbox", + is_flag=True, + default=False, + help="Build/list the TrustZone sandbox (NS) flavor — `sandbox-`, emits .bin.", + )(f) + + +def _resolve_build_target(target: str, sandbox: bool) -> str: + """Validate and return the make BUILD_TARGET for (board, flavor).""" + if sandbox: + validate_sandbox_board(target) + return f"sandbox-{target}" + validate_bare_target(target) + return target + + +@cmd.command() +@_target_option +@_project_option +@_config_option +@_sandbox_option +@click.option( + "--rebuild", + is_flag=True, + default=False, + help="Force full rebuild (pass `-rebuild` to emBuild). Default: incremental.", +) +@click.option( + "-v", + "--verbose", + is_flag=True, + default=False, + help="Show full SES `-verbose -echo` output.", +) +@click.pass_context +def build(ctx, target, project, config, sandbox, rebuild, verbose): + """Build firmware from source (default target: dotbot-v3).""" + target = from_config(ctx, "target", "board", "fw") + config = from_config(ctx, "config", "build_config", "fw") + sandbox = from_config(ctx, "sandbox", "sandbox", "fw") + build_target = _resolve_build_target(target, sandbox) + flavor = "sandbox " if sandbox else "" + apps_to_build = [project] if project else list_projects(build_target) + if project and project not in list_projects(build_target): + raise click.ClickException( + f"App {project!r} is not available for target {target!r}.\n" + f"Available: {', '.join(list_projects(build_target))}" + ) + mode = "rebuild" if rebuild else "incremental" + what = project or f"all {flavor}apps" + click.echo(f"Building {what} for {target} ({config}, {mode})...", err=True) + elapsed = run_make( + build_target, config, project, rebuild=rebuild, quiet=not verbose + ) + click.echo(f"✓ Built {target} in {elapsed:.1f}s", err=True) + # Echo each produced artifact path on its own stdout line so pipelines + # like `dotbot fw build | xargs -n1 ...` work. + for app in apps_to_build: + out = artifact_path(build_target, app, config) + if out.is_file(): + click.echo(str(out)) + + +@cmd.command() +@_target_option +@_config_option +@_sandbox_option +@click.option("-v", "--verbose", is_flag=True, default=False) +@click.pass_context +def clean(ctx, target, config, sandbox, verbose): + """Clean SES build outputs (default target: dotbot-v3).""" + target = from_config(ctx, "target", "board", "fw") + config = from_config(ctx, "config", "build_config", "fw") + sandbox = from_config(ctx, "sandbox", "sandbox", "fw") + build_target = _resolve_build_target(target, sandbox) + click.echo(f"Cleaning {target} ({config})...", err=True) + elapsed = run_make(build_target, config, make_targets=["clean"], quiet=not verbose) + click.echo(f"✓ Cleaned in {elapsed:.1f}s", err=True) + + +@cmd.command(name="targets") +@_sandbox_option +def list_targets(sandbox): + """List valid targets for `dotbot fw build` (one per line).""" + boards = SANDBOX_BOARDS if sandbox else BARE_TARGETS + for t in sorted(boards): + click.echo(t) + + +@cmd.command() +@_target_option +@_project_option +@_config_option +@_sandbox_option +@click.option( + "--out", + "out_dir", + type=click.Path(file_okay=False, dir_okay=True), + default=None, + help=f"Where to collect artifacts. Default: {DEFAULT_ARTIFACTS_DISPLAY}/dotbot-firmware-local/.", +) +@click.option( + "--print-path", + is_flag=True, + default=False, + help="Print where the artifact lives without building.", +) +@click.option("-v", "--verbose", is_flag=True, default=False) +@click.pass_context +def artifacts(ctx, target, project, config, sandbox, out_dir, print_path, verbose): + """Build + collect artifacts into the local cache (default). + + Without --out, built apps land in the source-qualified + ``/dotbot-firmware-local/`` dir so `device flash ` finds them + alongside fetched releases. + """ + import shutil + + target = from_config(ctx, "target", "board", "fw") + config = from_config(ctx, "config", "build_config", "fw") + sandbox = from_config(ctx, "sandbox", "sandbox", "fw") + build_target = _resolve_build_target(target, sandbox) + if print_path: + if not project: + raise click.ClickException( + "`--print-path` requires `--app NAME` — there is no canonical " + "artifact path without a specific project." + ) + click.echo(str(artifact_path(build_target, project, config))) + return + out = ( + Path(out_dir).resolve() + if out_dir + else artifacts_dir() / "dotbot-firmware-local" + ) + click.echo( + f"Building + collecting artifacts for {target} ({config}) → {out}/...", + err=True, + ) + # Force a full rebuild: bare and sandbox share the SES Output dir per + # board (`$(BuildTarget)`), so incremental can pick up stale objects + # from the other flavor and link-error. + elapsed = run_make(build_target, config, project, rebuild=True, quiet=not verbose) + out.mkdir(parents=True, exist_ok=True) + apps_to_collect = [project] if project else list_projects(build_target) + copied = [] + for app in apps_to_collect: + src = artifact_path(build_target, app, config) + if src.is_file(): + dst = out / src.name + shutil.copy2(src, dst) + copied.append(dst) + echo_artifact_path(out, action="collected into") + click.echo(f"✓ Collected {len(copied)} artifact(s) in {elapsed:.1f}s", err=True) + for p in copied: + click.echo(str(p)) + + +@cmd.command() +@click.option( + "--source", + "-S", + type=click.Choice(list(("swarmit", "dotbot-firmware"))), + default=None, + help="Limit to one source (default: fetch the pinned version from all sources).", +) +@click.option( + "--fw-version", + "-f", + default=None, + help="Override the pinned version for --source: a release tag, 'latest', or 'local'.", +) +@click.option( + "--local-root", + type=click.Path(path_type=Path, file_okay=False, dir_okay=True), + help="Root of a local build tree (with --source --fw-version local).", +) +def fetch(source, fw_version, local_root): + """Download firmware into ~/.dotbot/artifacts/-/. + + With no flags, fetches the exact release this pydotbot is pinned to, from + every source: swarmit (swarm system images, version inferred from the + installed swarmit package) and DotBot-firmware (bare + sandbox apps, the + version pydotbot is tested against). The two version independently, so + overriding with -f requires a --source - pass `-f latest` for the newest + release or `-f ` for a specific one. + """ + from dotbot import pydotbot_version + from dotbot.firmware.fetch import ( + DEFAULT_FETCH_SOURCES, + _short_path, + fetch_assets, + pinned_version, + ) + + if fw_version is not None and source is None: + raise click.ClickException( + "Pass --source with -f/--fw-version: swarmit and dotbot-firmware " + "version independently." + ) + sources = [source] if source else list(DEFAULT_FETCH_SOURCES) + fetched: list[Path] = [] + for src in sources: + if fw_version is None: + version = pinned_version(src) + click.echo( + f"Fetching {src} {version} (pinned by pydotbot {pydotbot_version()})..." + ) + else: + version = fw_version + if version == "latest": + click.echo(f"Fetching the latest {src} release...") + fetched.append(fetch_assets(src, version, artifacts_dir(), local_root)) + click.echo("\nDone. Firmware fetched into:") + for path in fetched: + click.echo(f" {_short_path(path)}") + + +@cmd.command(name="list") +def list_artifacts(): + """List firmware artifacts cached in ~/.dotbot/artifacts/.""" + root = artifacts_dir() + echo_artifact_path(root, action="listing") + if not root.is_dir(): + click.echo( + "(nothing cached yet — run `dotbot fw fetch` or `dotbot fw artifacts`)" + ) + return + found = sorted( + p for p in root.rglob("*") if p.is_file() and p.suffix in (".hex", ".bin") + ) + if not found: + click.echo("(empty)") + return + for p in found: + click.echo(str(p.relative_to(root))) + + +@cmd.command() +@click.argument("name") +@click.option( + "--template", + type=click.Choice(["swarmit-app", "bare"]), + default="swarmit-app", + show_default=True, +) +def new(name, template): # pylint: disable=unused-argument + """Scaffold a new firmware project (NOT IMPLEMENTED).""" + click.echo(_NOT_READY.format(sub="new"), err=True) + sys.exit(2) + + +# The low-level Makefile escape hatch, mounted next to its high layer +# `fw build`. Importing `make` here is cheap (no SES/firmware import at +# module load), so it doesn't compromise the dispatcher's lazy loading. +from dotbot.cli.make import cmd as _make_cmd # noqa: E402 + +cmd.add_command(_make_cmd) diff --git a/dotbot/cli/gateway.py b/dotbot/cli/gateway.py new file mode 100644 index 00000000..8075067d --- /dev/null +++ b/dotbot/cli/gateway.py @@ -0,0 +1,123 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run gateway` — host-side Mari gateway bridge. + +Runs on whatever computer the gateway firmware is plugged into (a +laptop for a starter setup, a Pi for a permanent install). Bridges UART +HDLC frames to/from an MQTT broker, so a `dotbot run controller --conn +mqtts://…` can reach the swarm from anywhere. + +Thin re-mount of marilib's `mari-edge`: wraps a `MarilibEdge` with a +serial adapter and (optionally) an MQTT adapter. Without `--mqtt-url` +it just prints received frames (handy to sanity-check a freshly-flashed +gateway with zero MQTT infra); with `--mqtt-url` it bridges to the +broker. Frame printing is on by default in both modes (`--no-print` to +silence). Metrics probing is off (`metrics_probe_period=0`), so the +print-only mode is passive — it doesn't inject traffic onto the link. + +Phase 1 is a raw bridge (mari frames + raw mari topics). DotBot-semantic +MQTT topics are a later phase, tracked in the controller-CLI-redesign +plan. +""" + +import os +import time + +import click + +from dotbot.cli._cfg import from_config +from dotbot.cli._conn import parse_connection + + +def _run_gateway(port, mqtt_url, do_print): # pragma: no cover - needs a gateway + """Construct a MarilibEdge bridge and pump it until interrupted. + + Imports marilib lazily so `dotbot run gateway --help` is cheap and the + command is importable without a serial port present. + """ + from marilib.communication_adapter import MQTTAdapter, SerialAdapter + from marilib.marilib_edge import MarilibEdge + from marilib.model import EdgeEvent + from marilib.serial_uart import get_default_port + + port = port or get_default_port() + + def on_event(event, event_data): + if do_print and event == EdgeEvent.NODE_DATA: + click.echo( + f"<- {event_data.header.source:016x}: {event_data.payload.hex()}" + ) + + mqtt_interface = None + if mqtt_url is not None: + # Broker credentials come from the environment (DOTBOT_MQTT_USER / + # DOTBOT_MQTT_PASS); they override any user:pass in the URL. + mqtt_interface = MQTTAdapter.from_url( + mqtt_url, + is_edge=True, + username=os.environ.get("DOTBOT_MQTT_USER"), + password=os.environ.get("DOTBOT_MQTT_PASS"), + ) + + # metrics_probe_period=0 → MarilibEdge starts no metrics thread, so a + # print-only run stays passive (no probe traffic on the serial link). + mari = MarilibEdge( + on_event, + serial_interface=SerialAdapter(port), + mqtt_interface=mqtt_interface, + metrics_probe_period=0, + ) + where = mqtt_url if mqtt_url else "(no broker — print only)" + click.echo(f"dotbot run gateway: {port} <-> {where}", err=True) + try: + while True: + mari.update() + time.sleep(0.5) + except KeyboardInterrupt: + pass + finally: + mari.close() + + +@click.command( + name="gateway", + help=( + "Host-side Mari gateway bridge (UART <-> MQTT). Runs wherever the " + "gateway firmware is plugged in. With --mqtt-url it bridges to the " + "broker; without it, it just prints received frames. Printing is on " + "by default (--no-print to silence)." + ), +) +@click.option( + "-p", + "--port", + type=str, + default=None, + help="Serial port of the attached gateway firmware. Default: autodetect.", +) +@click.option( + "-m", + "--mqtt-url", + type=str, + default=None, + help="MQTT broker to bridge to (`mqtts://host:port`). Absent → print-only.", +) +@click.option( + "--print/--no-print", + "do_print", + default=True, + help="Print received frames to stdout (default: on).", +) +@click.pass_context +def cmd(ctx, port, mqtt_url, do_print): + """Run the gateway bridge.""" + if mqtt_url is None: + # No --mqtt-url on the command line: fall back to the selected + # deployment's (or config's) connection, but only when it names an + # MQTT broker - a serial/simulator conn is not a broker to bridge to, + # so we leave mqtt_url None and keep the print-only behavior. + conn = from_config(ctx, "mqtt_url", "conn", "run") + if conn and parse_connection(conn).kind == "mqtt": + mqtt_url = conn + _run_gateway(port, mqtt_url, do_print) diff --git a/dotbot/cli/joystick.py b/dotbot/cli/joystick.py new file mode 100644 index 00000000..e6526194 --- /dev/null +++ b/dotbot/cli/joystick.py @@ -0,0 +1,8 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run joystick` — drive a bot live from a USB joystick.""" + +from dotbot.joystick import main as _joystick_main + +cmd = _joystick_main diff --git a/dotbot/cli/keyboard.py b/dotbot/cli/keyboard.py new file mode 100644 index 00000000..3fa3e7fb --- /dev/null +++ b/dotbot/cli/keyboard.py @@ -0,0 +1,8 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run keyboard` — drive a bot live from the keyboard.""" + +from dotbot.keyboard import main as _keyboard_main + +cmd = _keyboard_main diff --git a/dotbot/cli/main.py b/dotbot/cli/main.py new file mode 100644 index 00000000..8fca3951 --- /dev/null +++ b/dotbot/cli/main.py @@ -0,0 +1,139 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Root `dotbot` Click group: four object-namespaces + management commands. + +The top level is the four object-namespaces, each one *kind of thing*: + + fw — firmware artifacts (cached in ~/.dotbot/artifacts/, no hardware) + device — one connected device (cable / probe) + swarm — the fleet (radio / OTA) + run — host-side processes (software you launch on your computer) + +Three are nouns (things you manage); `run` is the verb (the thing you do). +Alongside them sit the read-only management commands - `config` (what +config is in effect, and where it came from) and `deployment` (which +deployments are defined, and which is active). + +Each group lives in its own module under `dotbot.cli.` exposing a +`cmd` attribute. The root lists the groups eagerly (so `dotbot --help` is +cheap) but only imports a group's module when it's actually invoked — see +`dotbot.cli._lazygroup.LazyGroup`. + +Adding a new top-level group: + 1. Create `dotbot/cli/.py` exposing `cmd = click.Command(...)`. + 2. Add a `(cli-name, module path, short help)` entry to `_SUBCOMMANDS`. + 3. If the backend lives in an optional sibling package, wrap it with + `dotbot.cli._lazy.lazy_subcommand` inside that module. +""" + +import click + +from dotbot import pydotbot_version +from dotbot.cli._lazygroup import LazyGroup + +# (cli-name, dotted module path, short help shown by `dotbot --help`) +_SUBCOMMANDS = ( + ( + "fw", + "dotbot.cli.fw", + "Firmware artifacts (no hardware): build / fetch / list / make.", + ), + ( + "device", + "dotbot.cli.device", + "One connected device (cable/probe): flash an app/role, read info.", + ), + ( + "swarm", + "dotbot.cli.swarm", + "The fleet over the air: status, start/stop, OTA flash, monitor.", + ), + ( + "run", + "dotbot.cli.run", + "Host-side processes: controller, gateway, simulator, calibration, demos, teleop.", + ), + ( + "config", + "dotbot.cli.config_cmd", + "Show the resolved config + where it came from.", + ), + ( + "deployment", + "dotbot.cli.deployment_cmd", + "List / show configured deployments.", + ), +) + + +@click.group( + cls=LazyGroup, + subcommands=_SUBCOMMANDS, + help=( + "One CLI for the whole DotBot workflow: build and flash firmware, " + "program and control a single robot, and run experiments over the air " + "across a swarm - from one bot to a thousand." + ), +) +@click.option( + "-c", + "--config", + "config_path", + type=click.Path(dir_okay=False), + default=None, + help=( + "Config file to use (default: a dotbot.toml in the current directory, " + "else ~/.dotbot/config.toml)." + ), +) +@click.option( + "--deployment", + "deployment_name", + default=None, + metavar="NAME", + help="Which configured deployment to target; overrides default_deployment.", +) +@click.version_option( + version=pydotbot_version(), + prog_name="dotbot", + message="%(prog)s %(version)s", +) +@click.pass_context +def cli(ctx, config_path, deployment_name): + """Load the unified config + select the deployment, then dispatch. + + The resolved config and the selected deployment are stashed on the Click + context (`ctx.obj`) so each subcommand can read its defaults from them; + flags and env vars still override the file (see `dotbot.config`). + + Discovery order: `-c` / `DOTBOT_CONFIG` > a `dotbot.toml` in the cwd > + `~/.dotbot/config.toml` (the per-machine fallback). `fw` reads its `[fw]` + keys (`segger_dir`, `firmware_repo`, ...) through this same resolver. + """ + from dotbot.config import ( + ConfigError, + discover_config_path, + load_config, + select_deployment, + ) + + ctx.ensure_object(dict) + try: + path = discover_config_path(config_path) + config = load_config(path) + deployment, deployment_resolved = select_deployment( + config, cli_name=deployment_name + ) + except ConfigError as exc: + raise click.ClickException(str(exc)) from exc + + if path is not None: + click.echo(f"using config file at {path}", err=True) + else: + click.echo("no config file found; using built-in defaults", err=True) + + ctx.obj["config"] = config + ctx.obj["config_path"] = path + ctx.obj["deployment"] = deployment + ctx.obj["deployment_name"] = deployment_resolved diff --git a/dotbot/cli/make.py b/dotbot/cli/make.py new file mode 100644 index 00000000..186f7c75 --- /dev/null +++ b/dotbot/cli/make.py @@ -0,0 +1,61 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot fw make` — escape hatch to `make` in your DotBot-firmware checkout. + +`dotbot fw build` deliberately models only +the flags that matter for the daily edit/build loop: TARGET, `--app`, +`--config`, `--rebuild`, `-v`. Anything else (PACKAGES_DIR_OPT, DOCKER +overrides, `make doc`, custom CLANG_FORMAT_TYPE, …) is intentionally +not modelled — the Makefile is fully featured and the flag matrix +shouldn't grow to chase it. + +This subcommand is the honest answer: it forwards arbitrary arguments +to `make` in the firmware repo, with two affordances that bare `cd +DotBot-firmware && make ...` doesn't give you: + +1. SEGGER_DIR is auto-resolved (env → macOS default → clear error). +2. The firmware repo is auto-located (`DOTBOT_FIRMWARE_REPO` env → + `./DotBot-firmware/`). + +Everything else is plain make. +""" + +import os +import subprocess +import sys + +import click + +from dotbot.cli._fw_helpers import resolve_firmware_repo, resolve_segger_dir + + +@click.command( + name="make", + context_settings={ + "ignore_unknown_options": True, + "allow_extra_args": True, + "help_option_names": ["-h", "--help"], + }, + help=( + "Escape hatch: run `make` in your DotBot-firmware checkout with " + "workspace-resolved SEGGER_DIR. Forwards all args verbatim. " + "Use this when `dotbot fw build` doesn't model the Makefile knob " + "you need." + ), +) +@click.pass_context +def cmd(ctx): + """Run `make` in the firmware repo. Examples: + + \b + dotbot fw make help + dotbot fw make list-targets + dotbot fw make BUILD_TARGET=dotbot-v3 BUILD_CONFIG=Debug + dotbot fw make BUILD_TARGET=dotbot-v3 PACKAGES_DIR_OPT="-packagesdir /opt/pkgs" + dotbot fw make docker BUILD_TARGET=sandbox-dotbot-v3 + """ + repo = resolve_firmware_repo() + segger = resolve_segger_dir() + env = {**os.environ, "SEGGER_DIR": str(segger)} + sys.exit(subprocess.call(["make", *ctx.args], cwd=repo, env=env)) diff --git a/dotbot/cli/run.py b/dotbot/cli/run.py new file mode 100644 index 00000000..7533ba9f --- /dev/null +++ b/dotbot/cli/run.py @@ -0,0 +1,72 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run` — launch host-side processes (software on your computer). + +The fourth top-level namespace. `fw` / `device` / `swarm` are *nouns* — +things you manage (artifacts, one board, the fleet). `run` is the *verb*: +the long-running software you start on your own machine — the control +plane, the gateway bridge, the simulator, the calibration workflow, the +demos, the teleop drivers. Every reference CLI keeps "launch a process" +as a top-level `run` (docker, cargo, ros2), so the asymmetry is correct, +not an inconsistency. + +Note the two "gateway"s the namespaces disambiguate: +`dotbot device flash-mari-gateway` flashes gateway firmware onto a board; +`dotbot run gateway` runs the host-side UART<->MQTT bridge process that +talks to that board. Different objects, named by their namespace. + +Each subcommand is loaded lazily (see `_lazygroup`) so `dotbot run --help` +lists everything without importing `dotbot.controller_app` (FastAPI + +StaticFiles) or the teleop drivers' pygame/pynput. +""" + +import click + +from dotbot.cli._lazygroup import LazyGroup + +# (cli-name, dotted module path exposing `cmd`, short help under `run --help`) +_RUN_SUBCOMMANDS = ( + ( + "controller", + "dotbot.cli.controller", + "Control plane + REST/WS + dashboard.", + ), + ( + "gateway", + "dotbot.cli.gateway", + "Host-side Mari gateway bridge (UART <-> MQTT).", + ), + ( + "simulator", + "dotbot.cli.simulator", + "Standalone simulator (≡ run controller --conn simulator).", + ), + ( + "lh2-calibration", + "dotbot.cli.calibrate", + "LH2 calibration: capture, apply, export (serial-side / single device).", + ), + ( + "demo", + "dotbot.cli.demo", + "Built-in research demos (qrkey phone bridge, ...).", + ), + ("keyboard", "dotbot.cli.keyboard", "Drive a DotBot from the keyboard (live)."), + ("joystick", "dotbot.cli.joystick", "Drive a DotBot from a joystick (live)."), +) + + +@click.group( + cls=LazyGroup, + name="run", + subcommands=_RUN_SUBCOMMANDS, + help=( + "Launch host-side processes: the controller (+ REST/WS + UI), the " + "gateway bridge, a simulator, LH2 calibration, demos, and teleop " + "drivers. These run on your computer; `fw` / `device` / `swarm` are " + "the things you manage." + ), +) +def cmd(): + pass diff --git a/dotbot/cli/simulator.py b/dotbot/cli/simulator.py new file mode 100644 index 00000000..e589846a --- /dev/null +++ b/dotbot/cli/simulator.py @@ -0,0 +1,39 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot run simulator` — standalone simulator (no hardware). + +Equivalent to `dotbot run controller --conn simulator`. The name +advertises the no-hardware case so students can discover the offline path +from `dotbot run --help` without reading connection docs. + +Implementation: prepend `--conn simulator` to argv and delegate to the +controller's Click command. `dotbot run simulator --sailbot` forwards through to +the controller's robot-type selector. A future refactor may turn this +into a first-class entry (and possibly a separate sim process). +""" + +import click + +from dotbot.controller_app import main as _controller_main + + +@click.command( + name="simulator", + context_settings=dict( + ignore_unknown_options=True, + allow_extra_args=True, + help_option_names=[], # forward --help to the controller + ), + add_help_option=False, +) +@click.pass_context +def cmd(ctx): + """Run a standalone simulator (no hardware required). + + `dotbot run simulator` runs a dotbot simulator; `dotbot run simulator --sailbot` + runs a sailbot one. Other controller flags are forwarded as-is. Try + `dotbot run simulator --help` for the full option list. + """ + args = ["--conn", "simulator", *ctx.args] + _controller_main.main(args=args, standalone_mode=True) diff --git a/dotbot/cli/swarm.py b/dotbot/cli/swarm.py new file mode 100644 index 00000000..dead6f17 --- /dev/null +++ b/dotbot/cli/swarm.py @@ -0,0 +1,89 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot swarm` — fleet operations over the air (status/start/stop/flash/...). + +Mounts the upstream `swarmit` Click group as the `dotbot swarm` parent: +operators get `status|start|stop|flash|monitor|reset|message|calibrate-lh2| +serve` with their existing flags. `swarm` is strictly the *many-devices, +over-the-radio* namespace. + +Single-device, cabled operations moved out: firmware-artifact build/fetch/ +list live under `dotbot fw`, and per-device flashing/inspection (including +what used to be `swarm provision …`) lives under `dotbot device`. + +swarmit has its own config loader, so the unified `dotbot.toml` is bridged in +at the mount boundary: `conn` / `swarm_id` resolved by the root group are +translated into swarmit's flags (see `_swarm_inject`), so `dotbot swarm status` +inherits a saved deployment like every other command. An explicit swarmit +`--conn` / `--swarm-id` / `-c` still wins. +""" + +import click + +from dotbot.cli._lazy import lazy_subcommand +from dotbot.cli._swarm_inject import inject_config + +_HELP = ( + "Fleet ops over the air: status, start/stop, OTA-flash, monitor, " + "reset, calibrate-lh2. Wraps swarmit." +) + + +def _load_swarmit_group(): + from swarmit.cli.main import main as swarmit_group + + return swarmit_group + + +def _run_swarmit( + swarmit_group, args +): # pragma: no cover - delegates to swarmit (needs MQTT/serial) + swarmit_group.main(args=args, prog_name="dotbot swarm", standalone_mode=True) + + +def _with_config_injection(swarmit_group): + """Wrap the swarmit group so `dotbot swarm` injects config-driven conn/swarm_id. + + A passthrough command that captures every token, prepends the resolved + connection (unless the user gave it explicitly), and re-invokes swarmit. + `--help` and subcommand help flow straight through. + """ + + @click.command( + name="swarm", + help=_HELP, + context_settings=dict(ignore_unknown_options=True, allow_extra_args=True), + add_help_option=False, + ) + @click.argument("args", nargs=-1, type=click.UNPROCESSED) + @click.pass_context + def cmd(ctx, args): + args = list(args) + # `lh2-calibration` is PyDotBot-native (the homography solve lives + # here, not in swarmit), so intercept it before the passthrough and + # hand off to our own group, carrying the resolved config along. + if args and args[0] == "lh2-calibration": + from dotbot.cli.swarm_lh2 import cmd as lh2_group + + lh2_group.main( + args=args[1:], + prog_name="dotbot swarm lh2-calibration", + standalone_mode=True, + obj=ctx.obj, + ) + return + final = inject_config(args, ctx.obj) if args else args + _run_swarmit(swarmit_group, final) + + return cmd + + +cmd = lazy_subcommand( + name="swarm", + extra="swarm", + package="swarmit", + help=_HELP, + loader=_load_swarmit_group, + transform=_with_config_injection, +) diff --git a/dotbot/cli/swarm_lh2.py b/dotbot/cli/swarm_lh2.py new file mode 100644 index 00000000..544020b8 --- /dev/null +++ b/dotbot/cli/swarm_lh2.py @@ -0,0 +1,266 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot swarm lh2-calibration` - over-the-air LH2 calibration. + +The fleet-side home for LH2 calibration: capture and send a calibration +without a serial cable, driving a single DotBot through the swarmit +transport. Two subcommands: + +- `collect` - walk one DotBot through the 4 arena corners, trigger a + raw-count capture per corner over the air, solve the + homography, and save the calibration under ~/.dotbot/. +- `push ` - send a saved calibration to the bot over the air. A thin + forward to swarmit's `calibrate-lh2`, which picks the payload + format (legacy `.out` or `calibration-*.toml`) by extension. + +The homography solve lives in PyDotBot (`dotbot.calibration.lighthouse2`); the +transport lives in swarmit. `collect` therefore runs natively here, while +`push` is pure transport and reuses swarmit's own command. + +Serial-cable (single DK) calibration and the C-header `apply` export stay +under `dotbot run lh2-calibration`. + +Calibration runtime deps (`opencv-python`) live behind the `[calibrate]` +extra; ImportError at invocation prints an install hint instead of a +traceback. +""" + +import sys +import time + +import click + + +def _build_swarmit_client(ctx, conn, swarm_id, device): + """Build a swarmit client targeting a single `device`. + + Reuses swarmit's own conn-string translation so the two CLIs can't + drift, and falls back to the unified dotbot config's `conn` / `swarm_id` + (like `dotbot swarm`) when the flags are omitted. Imported lazily: the + swarmit protocol registry must not load during PyDotBot test collection. + + Transport selection is swarmit's call: `build_client` probes for a running + swarmit server and falls back to an in-process controller on its own, so + there is no flag to choose here. + """ + from swarmit.cli.main import DEFAULTS, _conn_to_config + from swarmit.client import build_client + from swarmit.testbed.controller import ControllerSettings + + if conn is None or swarm_id is None: + from dotbot.config import resolve + + obj = ctx.obj or {} + config = obj.get("config") + deployment = obj.get("deployment") + if conn is None: + conn = resolve("conn", config=config, deployment=deployment) + if swarm_id is None: + swarm_id = resolve("swarm_id", config=config, deployment=deployment) + + final = {**DEFAULTS, **_conn_to_config(conn, swarm_id)} + settings = ControllerSettings( + serial_port=final["serial_port"], + serial_baudrate=final["baudrate"], + mqtt_host=final["mqtt_host"], + mqtt_port=final["mqtt_port"], + mqtt_use_tls=final["mqtt_use_tls"], + mqtt_username=final.get("mqtt_username"), + mqtt_password=final.get("mqtt_password"), + network_id=int(final["swarmit_network_id"], 16), + adapter=final["adapter"], + devices=[device.upper()], + verbose=False, + ) + return build_client(settings) + + +@click.group( + name="lh2-calibration", + help="Over-the-air LH2 calibration for one DotBot: collect, push.", +) +def cmd() -> None: + pass + + +@cmd.command( + name="collect", + help=( + "Collect LH2 calibration from one DotBot over the air (no serial " + "cable). Walks you through the 4 arena corners, triggers a capture " + "per corner via swarmit, solves the homography, and saves the " + "calibration." + ), +) +@click.option( + "--device", + required=True, + help="DotBot link-layer address in hex (e.g. BC3D3C8A2A6F8E68).", +) +@click.option( + "-n", + "--conn", + "--connection", + "conn", + default=None, + help=( + "Swarm connection string: an MQTT broker `mqtts://host:port` or a " + "serial gateway `/dev/ttyACM0`. Falls back to the dotbot config." + ), +) +@click.option( + "-s", + "--swarm-id", + "swarm_id", + default=None, + help="Swarm id in hex (required for an MQTT broker connection).", +) +@click.option( + "-d", + "--distance", + default=None, + type=int, + help=( + "Distance between reference corners in millimeters " + "(default: the calibration package default)." + ), +) +@click.option( + "--timeout", + default=None, + type=float, + help="Seconds to wait for each capture before re-triggering.", +) +@click.option( + "--retries", + default=None, + type=int, + help="Re-trigger this many times per corner before giving up.", +) +@click.option( + "--tag", + default=None, + help=( + 'Optional arena/setup label (e.g. "office-2x2m") added to the saved ' + "filename and metadata, so calibrations stay self-describing." + ), +) +@click.option( + "--push", + is_flag=True, + help="Send the computed calibration back to the bot over the air.", +) +@click.pass_context +def _collect(ctx, device, conn, swarm_id, distance, timeout, retries, tag, push): + try: + from swarmit.testbed.protocol import LH2_CALIB_TAG + + from dotbot.calibration.lighthouse2 import ( + CALIBRATION_DISTANCE_DEFAULT, + LighthouseManager, + ) + from dotbot.calibration.ota import ( + CAPTURE_RETRIES_DEFAULT, + CAPTURE_TIMEOUT_DEFAULT, + CORNERS, + CaptureSession, + ) + except ImportError as exc: + click.echo( + "`dotbot swarm lh2-calibration collect` needs the calibration " + "runtime deps (opencv-python).\n" + "Install with: pip install dotbot[calibrate]", + err=True, + ) + click.echo(f"(import error was: {exc})", err=True) + sys.exit(1) + + distance = distance if distance is not None else CALIBRATION_DISTANCE_DEFAULT + timeout = timeout if timeout is not None else CAPTURE_TIMEOUT_DEFAULT + retries = retries if retries is not None else CAPTURE_RETRIES_DEFAULT + + try: + client = _build_swarmit_client(ctx, conn, swarm_id, device) + except click.ClickException: + raise + except Exception as exc: + click.echo(f"Could not reach the swarm: {exc}", err=True) + sys.exit(1) + + samples = [] + with client: + with CaptureSession(client, device, LH2_CALIB_TAG) as session: + # Give the transport's own connect/subscribe log lines a beat to + # print before our prompts, so the two don't interleave on screen. + time.sleep(0.2) + click.echo( + f"\nCollecting LH2 calibration from {device.upper()}.\n" + "Stop the bot's app first (capture only runs in READY).\n" + ) + for corner in CORNERS: + click.prompt( + f"Place the DotBot at the {corner} corner, then press Enter", + default="", + show_default=False, + prompt_suffix="", + ) + try: + sample = session.capture( + lh_index=0, + timeout=timeout, + retries=retries, + on_attempt=lambda n, total: click.echo( + f" triggering capture (attempt {n}/{total}), " + f"waiting up to {timeout:g}s..." + ), + ) + except TimeoutError as exc: + click.echo(f" ! {exc}", err=True) + raise click.Abort() + samples.append(sample) + click.echo( + f" captured {corner}: " + f"count1={sample.count1} count2={sample.count2}" + ) + + manager = LighthouseManager(calibration_distance=distance, extra_lh_num=0) + try: + manager.compute_calibration(samples) + except Exception as exc: + click.echo(f"Failed to compute calibration: {exc}", err=True) + sys.exit(1) + path = manager.save_calibration(tag=tag) + click.echo(f"\nCalibration saved to {path}") + + if push: + payload = manager.calibration_output_path.read_bytes() + client.send_lh2_calibration(payload) + click.echo("Sent the calibration to the bot over the air.") + else: + click.echo( + "To send it to the bot over the air:\n" + f" dotbot swarm lh2-calibration push {path}" + ) + + +@cmd.command( + name="push", + help=( + "Send a saved LH2 calibration to the bot over the air. Forwards to " + "swarmit's `calibrate-lh2`, which picks the payload format (legacy " + "`.out` or `calibration-*.toml`) by file extension." + ), +) +@click.argument( + "path", + type=click.Path(exists=True, dir_okay=False), +) +@click.pass_context +def _push(ctx, path): + from dotbot.cli._swarm_inject import inject_config + from dotbot.cli.swarm import _load_swarmit_group, _run_swarmit + + swarmit_group = _load_swarmit_group() + final = inject_config(["calibrate-lh2", path], ctx.obj) + _run_swarmit(swarmit_group, final) diff --git a/dotbot/config.py b/dotbot/config.py new file mode 100644 index 00000000..250d048a --- /dev/null +++ b/dotbot/config.py @@ -0,0 +1,363 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Unified `dotbot` configuration: one file, one precedence chain. + +This is the resolver core for the single `dotbot` config file. It is +intentionally pure - no Click, no network, no global state - so the whole +precedence/discovery story is +exhaustively unit-testable without hardware. The CLI layer (a later phase) +feeds it the actual flags and `os.environ`. + +The file mirrors the four-namespace CLI: top-level shared keys plus `[fw]` / +`[device]` / `[swarm]` / `[run]` tables, and `[deployment.]` entries for the +physical deployments you switch between. + +```toml +default_deployment = "inria" +conn = "mqtts://broker.local:8883" # shared; sections/deployments override +swarm_id = "0001" + +[deployment.inria] # a named deployment - select, don't edit +conn = "mqtts://broker.inria.fr:8883" +swarm_id = "0001" + +[fw] +board = "dotbot-v3" + +[run.controller] +http_port = 8000 +``` + +Precedence for any value, highest wins: + + CLI flag > env (DOTBOT_
_, then shared DOTBOT_) + > file (section value > selected deployment > top-level) + > built-in default + +The selected deployment (`--deployment` > `DOTBOT_DEPLOYMENT` > `default_deployment`) +resolves first and slots into the file layer; an explicit flag/env still beats +it. Unknown keys are rejected (`extra='forbid'`) so a typo fails loud. +""" + +from __future__ import annotations + +import os +import tomllib +from pathlib import Path +from typing import Annotated, Any, Mapping, Optional + +from pydantic import ( + AfterValidator, + BaseModel, + ConfigDict, + Field, + ValidationError, +) + +# The four CLI namespaces, used to derive env-var names (DOTBOT_
_). +SECTIONS = ("fw", "device", "swarm", "run") + +# Where the user-level config lives. Geovane's call (2026-06-01): one dir, +# shared with the calibration data under ~/.dotbot/ - no XDG split. +USER_CONFIG_PATH = Path.home() / ".dotbot" / "config.toml" +# Project-level config, discovered in the current directory only. +PROJECT_CONFIG_NAME = "dotbot.toml" + + +class ConfigError(Exception): + """A config file is malformed, has an unknown key, or names a missing deployment.""" + + +def _check_conn(value: str | None) -> str | None: + """Validate a connection string with the same parser the `--conn` flag uses. + + One validator for the file path and the flag path, so they can't drift. + Imported lazily so merely importing this module doesn't pull in marilib. + """ + if value is None: + return value + from dotbot.cli._conn import ConnError, parse_connection + + try: + parse_connection(value) + except ConnError as exc: + raise ValueError(str(exc)) from exc + return value + + +# A connection string validated against `parse_connection` wherever it appears. +Conn = Annotated[Optional[str], AfterValidator(_check_conn)] + + +class _Strict(BaseModel): + """Base for every config section: reject unknown keys so typos fail loud.""" + + model_config = ConfigDict(extra="forbid") + + +# All fields are Optional and default to None: the model captures only what the +# file *explicitly* set, so the resolver can tell "unset" from "set to the +# default" and apply the precedence chain correctly. Built-in defaults live in +# code (dotbot/__init__.py), not here. + + +class Deployment(_Strict): + """One named physical deployment (Inria/100, La Poste/1000, ...). + + Holds only the environment-binding keys plus descriptive metadata. You + select a deployment; you never edit the file to switch. + """ + + conn: Conn = None + swarm_id: str | None = None + serial_port: str | None = None + location: str | None = None # descriptive, for `dotbot deployment list` + bots: int | None = None # descriptive + + +class FwSection(_Strict): + board: str | None = None + sandbox: bool | None = None + build_config: str | None = None # Debug | Release + segger_dir: str | None = None + firmware_repo: str | None = None # path to the DotBot-firmware clone + + +class DeviceSection(_Strict): + board: str | None = None + sn_starting_digits: str | None = None + build_config: str | None = None + + +class SwarmSection(_Strict): + conn: Conn = None + swarm_id: str | None = None + devices: str | None = None + + +class ControllerSection(_Strict): + http_port: int | None = None + map_size: str | None = None + background_map: str | None = None + log_output: str | None = None + csv_data_output: str | None = None + webbrowser: bool | None = None + gw_address: str | None = None + simulator_init_state: str | None = None + + +class GatewaySection(_Strict): + serial_port: str | None = None + mqtt: Conn = None + + +class RunSection(_Strict): + conn: Conn = None + swarm_id: str | None = None + controller: ControllerSection = Field(default_factory=ControllerSection) + gateway: GatewaySection = Field(default_factory=GatewaySection) + + +class DotbotConfig(_Strict): + """The whole file: top-level shared keys + the four section tables + deployments.""" + + default_deployment: str | None = None + log_level: str | None = None + conn: Conn = None + swarm_id: str | None = None + + fw: FwSection = Field(default_factory=FwSection) + device: DeviceSection = Field(default_factory=DeviceSection) + swarm: SwarmSection = Field(default_factory=SwarmSection) + run: RunSection = Field(default_factory=RunSection) + + # `[deployment.]` tables map to {name: Deployment}. + deployment: dict[str, Deployment] = Field(default_factory=dict) + + +# --- Discovery -------------------------------------------------------------- + + +def discover_config_path( + explicit: os.PathLike[str] | str | None = None, + *, + environ: Mapping[str, str] = os.environ, + start_dir: os.PathLike[str] | str | None = None, + include_user_file: bool = True, +) -> Path | None: + """Find the config file to load, highest priority first. + + 1. `explicit` (the `-c/--config PATH` flag) wins outright. + 2. `DOTBOT_CONFIG` env var (an explicit path by another name). + 3. A `dotbot.toml` in the current directory (the cwd only - no walking up to + parent directories, so the active config is always unambiguous). + 4. The user file `~/.dotbot/config.toml` (skipped when + `include_user_file=False` - used while the legacy `~/.dotbot/config.toml` + fw segger_dir reader still owns that file). + 5. None (caller uses built-in defaults). + """ + if explicit: + return Path(explicit) + env_path = environ.get("DOTBOT_CONFIG") + if env_path: + return Path(env_path) + + start = Path(start_dir or Path.cwd()).resolve() + candidate = start / PROJECT_CONFIG_NAME + if candidate.is_file(): + return candidate + + if include_user_file and USER_CONFIG_PATH.is_file(): + return USER_CONFIG_PATH + return None + + +def load_config(path: os.PathLike[str] | str | None) -> DotbotConfig: + """Load and validate a config file. `None` -> an empty config (all defaults). + + Raises `ConfigError` (with the file path) on bad TOML, an unknown key, a + wrong-typed value, or an invalid connection string. + """ + if path is None: + return DotbotConfig() + path = Path(path) + try: + with open(path, "rb") as handle: + data = tomllib.load(handle) + except (OSError, tomllib.TOMLDecodeError) as exc: + raise ConfigError(f"could not read config {path}: {exc}") from exc + try: + return DotbotConfig.model_validate(data) + except ValidationError as exc: + raise ConfigError(f"invalid config {path}:\n{exc}") from exc + + +def load_config_text(text: str, *, source: str = "") -> DotbotConfig: + """Validate a config TOML *string* (e.g. a fetched deployment fragment). + + Same validation as `load_config`, against the same model, so a published + fragment is held to the identical schema (`extra='forbid'` -> a typo fails + loud) before anything touches the local file. `source` names the origin in + error messages. + """ + try: + data = tomllib.loads(text) + except tomllib.TOMLDecodeError as exc: + raise ConfigError(f"invalid TOML from {source}: {exc}") from exc + try: + return DotbotConfig.model_validate(data) + except ValidationError as exc: + raise ConfigError(f"invalid config from {source}:\n{exc}") from exc + + +def load_discovered( + explicit: os.PathLike[str] | str | None = None, + *, + environ: Mapping[str, str] = os.environ, + start_dir: os.PathLike[str] | str | None = None, +) -> tuple[DotbotConfig, Path | None]: + """Discover + load in one step. Returns (config, source_path or None).""" + path = discover_config_path(explicit, environ=environ, start_dir=start_dir) + return load_config(path), path + + +# --- Deployment selection ------------------------------------------------------ + + +def select_deployment( + config: DotbotConfig, + *, + cli_name: str | None = None, + environ: Mapping[str, str] = os.environ, +) -> tuple[Deployment | None, str | None]: + """Resolve the active deployment: `--deployment` > `DOTBOT_DEPLOYMENT` > default_deployment. + + Returns (deployment, name), or (None, None) if none is selected. Raises + `ConfigError` if the selected name has no `[deployment.]` entry. + """ + name = cli_name or environ.get("DOTBOT_DEPLOYMENT") or config.default_deployment + if not name: + return None, None + if name not in config.deployment: + known = ", ".join(sorted(config.deployment)) or "(none defined)" + raise ConfigError(f"unknown deployment {name!r}; defined deployments: {known}") + return config.deployment[name], name + + +# --- Precedence resolution -------------------------------------------------- + + +def _env_candidates(section: str | None, key: str) -> tuple[str, ...]: + """Env-var names to check, in priority order (Cargo's mechanical mapping). + + Sectioned key -> `DOTBOT_
_`, then the shared `DOTBOT_` + alias. Top-level key -> just `DOTBOT_`. + """ + key_part = key.upper().replace("-", "_") + if section: + return (f"DOTBOT_{section.upper()}_{key_part}", f"DOTBOT_{key_part}") + return (f"DOTBOT_{key_part}",) + + +def _coerce(raw: str, like: Any) -> Any: + """Coerce an env-var string to the type of `like` (the default).""" + if isinstance(like, bool): + return raw.strip().lower() in ("1", "true", "yes", "on") + if isinstance(like, int): + try: + return int(raw) + except ValueError as exc: + raise ConfigError(f"expected an integer, got {raw!r}") from exc + return raw + + +def _file_value( + config: DotbotConfig | None, + section: str | None, + key: str, + deployment: Deployment | None, +) -> Any: + """The value this key has in the file layer: section > deployment > top-level.""" + if config is None: + return None + if section is not None: + section_obj = getattr(config, section, None) + value = getattr(section_obj, key, None) + if value is not None: + return value + if deployment is not None: + value = getattr(deployment, key, None) + if value is not None: + return value + return getattr(config, key, None) + + +def resolve( + key: str, + *, + section: str | None = None, + flag: Any = None, + config: DotbotConfig | None = None, + deployment: Deployment | None = None, + default: Any = None, + environ: Mapping[str, str] = os.environ, +) -> Any: + """Resolve one setting through the full precedence chain. + + `flag` > env (`DOTBOT_
_`, then shared `DOTBOT_`) > + file (section > deployment > top-level) > `default`. + + `section` is one of `SECTIONS` for a per-namespace key, or `None` for a + top-level shared key (e.g. `conn`, `swarm_id`). Env values are coerced to + the type of `default`. + """ + if flag is not None: + return flag + for name in _env_candidates(section, key): + if name in environ: + return _coerce(environ[name], default) + file_value = _file_value(config, section, key, deployment) + if file_value is not None: + return file_value + return default diff --git a/dotbot/controller.py b/dotbot/controller.py index 51349e2b..ac5dbdad 100644 --- a/dotbot/controller.py +++ b/dotbot/controller.py @@ -122,6 +122,8 @@ class ControllerSettings: mqtt_host: str = MQTT_HOST_DEFAULT mqtt_port: int = MQTT_PORT_DEFAULT mqtt_use_tls: bool = False + mqtt_username: Optional[str] = None + mqtt_password: Optional[str] = None gw_address: str = GATEWAY_ADDRESS_DEFAULT network_id: str = NETWORK_ID_DEFAULT controller_http_port: int = CONTROLLER_HTTP_PORT_DEFAULT @@ -691,6 +693,8 @@ async def _start_adapter(self): port=self.settings.mqtt_port, use_tls=self.settings.mqtt_use_tls, network_id=int(self.settings.network_id, 16), + username=self.settings.mqtt_username, + password=self.settings.mqtt_password, ) elif self.settings.adapter == "dotbot-simulator": self.adapter = DotBotSimulatorAdapter( diff --git a/dotbot/controller_app.py b/dotbot/controller_app.py index d36cf601..11066248 100644 --- a/dotbot/controller_app.py +++ b/dotbot/controller_app.py @@ -8,67 +8,159 @@ """Main module of the Dotbot controller command line tool.""" import asyncio +import os +import shutil import sys +from pathlib import Path import click import serial import toml from dotbot import ( - CONTROLLER_ADAPTER_DEFAULT, CONTROLLER_HTTP_PORT_DEFAULT, GATEWAY_ADDRESS_DEFAULT, MAP_SIZE_DEFAULT, - MQTT_HOST_DEFAULT, - MQTT_PORT_DEFAULT, - NETWORK_ID_DEFAULT, - SERIAL_BAUDRATE_DEFAULT, - SERIAL_PORT_DEFAULT, SIMULATOR_INIT_STATE_DEFAULT, pydotbot_version, ) +from dotbot.cli._cfg import from_config +from dotbot.cli._conn import ConnError, needs_swarm_id, parse_connection from dotbot.controller import Controller, ControllerSettings from dotbot.logger import setup_logging +# Old transport/identity config keys replaced by `conn` / `swarm_id`. +# Present-in-config triggers a warning and is dropped. +_LEGACY_TOML_KEYS = { + "adapter", + "mqtt_host", + "mqtt_port", + "mqtt_use_tls", + "network_id", + "swarmit_network_id", + "port", + "baudrate", +} + + +def _conn_to_settings(conn, swarm_id, sim_is_dotbot): + """Map `--conn` + `--swarm-id` into internal ControllerSettings fields. + + The internal `adapter` enum (`cloud`/`edge`/`dotbot-simulator`/…) is + an implementation detail; the CLI only ever sees `--conn`. Broker + credentials come from the environment (`DOTBOT_MQTT_USER` / + `DOTBOT_MQTT_PASS`), never the URL or a flag. + + Raises `click.ClickException` for a malformed `--conn` or a missing + `--swarm-id` on an mqtt connection. + """ + if conn is None: + raise click.ClickException( + "no connection given. Pass --conn (-n) with one of:\n" + " mqtts://host:port (an MQTT broker; also needs --swarm-id)\n" + " /dev/ttyACM0 (a serial gateway)\n" + " simulator (no hardware)" + ) + try: + parsed = parse_connection(conn) + except ConnError as exc: + raise click.ClickException(str(exc)) from exc + + if needs_swarm_id(parsed) and not swarm_id: + raise click.ClickException( + f"--conn {conn} needs --swarm-id: the broker carries multiple " + "swarms; --swarm-id selects yours." + ) + + if parsed.kind == "mqtt": + settings = { + "adapter": "cloud", + "mqtt_host": parsed.host, + "mqtt_port": parsed.port, + "mqtt_use_tls": parsed.use_tls, + "mqtt_username": os.environ.get("DOTBOT_MQTT_USER"), + "mqtt_password": os.environ.get("DOTBOT_MQTT_PASS"), + } + if swarm_id: + settings["network_id"] = swarm_id + return settings + if parsed.kind == "serial": + settings = {"adapter": "edge", "port": parsed.serial_port} + if swarm_id: + settings["network_id"] = swarm_id + return settings + # simulator + return {"adapter": "dotbot-simulator" if sim_is_dotbot else "sailbot-simulator"} + + +def _maybe_scaffold_sim_state(explicit_init_state): + """Offer to drop an editable example world in the current directory. + + `explicit_init_state` is the path set via `--simulator-init-state` or + the config file, or None when unspecified (the default world). Fires + only when nothing was specified and no `simulator_init_state.toml` is + here. An interactive run gets a [Y/n] prompt; declining — or a + non-interactive run (CI, a pipe) — leaves the cwd untouched and the + simulator falls back to the packaged world, so it always starts. + Writing the file lets the operator edit the simulated swarm + (positions, count, Mari vs default mode). + """ + if explicit_init_state is not None: + return # a path was set via --simulator-init-state or config + if Path(SIMULATOR_INIT_STATE_DEFAULT).is_file(): + return # a cwd file already exists; it'll be used as-is + if not sys.stdin.isatty(): + return # non-interactive: silently use the packaged default + + target = Path.cwd() / SIMULATOR_INIT_STATE_DEFAULT + if not click.confirm( + f"No {SIMULATOR_INIT_STATE_DEFAULT} in this directory. " + "Create an editable example here?", + default=True, + ): + return + + from dotbot.dotbot_simulator import packaged_init_state_path + + try: + shutil.copy(packaged_init_state_path(), target) + except OSError as exc: + click.echo( + f"Could not write {target}: {exc}; using the built-in world.", + err=True, + ) + return + click.echo(f"Created {target} — edit it to customize the simulated swarm.") + @click.command() @click.option( - "-a", - "--adapter", - type=click.Choice( - ["serial", "edge", "cloud", "dotbot-simulator", "sailbot-simulator"] - ), - help=f"Controller interface adapter. Defaults to {CONTROLLER_ADAPTER_DEFAULT}", -) -@click.option( - "-p", - "--port", + "-n", + "--conn", + "--connection", + "conn", type=str, - help=f"Serial port used by 'serial' and 'edge' adapters. Defaults to '{SERIAL_PORT_DEFAULT}'", -) -@click.option( - "-b", - "--baudrate", - type=int, - help=f"Serial baudrate used by 'serial' and 'edge' adapters. Defaults to {SERIAL_BAUDRATE_DEFAULT}", + help=( + "Connection to the swarm — one discriminated string: an MQTT " + "broker `mqtts://host:port`, a serial device path `/dev/ttyACM0`, " + "or `simulator`." + ), ) @click.option( - "-H", - "--mqtt-host", + "-s", + "--swarm-id", + "swarm_id", type=str, - help=f"MQTT host used by cloud adapter. Default: {MQTT_HOST_DEFAULT}.", + help=( + "Swarm id in hex. Required for an mqtt connection (the broker " + "carries many swarms); ignored for serial/simulator." + ), ) @click.option( - "-P", - "--mqtt-port", - type=int, - help=f"MQTT port used by cloud adapter. Default: {MQTT_PORT_DEFAULT}.", -) -@click.option( - "-T", - "--mqtt-use_tls/--no-mqtt-use_tls", - default=None, - help="Use TLS with MQTT (for cloud adapter).", + "--dotbot/--sailbot", + "sim_is_dotbot", + default=True, + help="With `--conn simulator`: which robot to simulate. Default: --dotbot.", ) @click.option( "-g", @@ -77,13 +169,6 @@ help=f"Gateway address in hex. Defaults to {GATEWAY_ADDRESS_DEFAULT:>0{16}}", ) @click.option( - "-s", - "--network-id", - type=str, - help=f"Network ID in hex. Defaults to {NETWORK_ID_DEFAULT:>0{4}}", -) -@click.option( - "-c", "--controller-http-port", type=int, help=f"Controller HTTP port of the REST API. Defaults to '{CONTROLLER_HTTP_PORT_DEFAULT}'", @@ -142,15 +227,13 @@ type=click.Path(dir_okay=False), help=f"Path to the simulator initial state .toml file. Defaults to '{SIMULATOR_INIT_STATE_DEFAULT}'.", ) +@click.pass_context def main( - adapter, - port, - baudrate, - mqtt_host, - mqtt_port, - mqtt_use_tls, + ctx, + conn, + swarm_id, + sim_is_dotbot, gw_address, - network_id, controller_http_port, map_size, background_map, @@ -166,16 +249,50 @@ def main( # welcome sentence print(f"Welcome to the DotBots controller (version: {pydotbot_version()}).") - # The priority order is CLI > ConfigFile (optional) > Defaults + # The priority order is CLI > ConfigFile (optional) > Defaults. + # The config file may carry `conn` / `swarm_id` too; CLI wins. + file_data = {} + if config_path: + file_data = toml.load(config_path) + + # Unified config / selected deployment, slotted in above the legacy + # `--config-path` file. Resolves CLI > env > unified-config (run > + # deployment > top-level) for each key; falls through to the param + # default (None) when no root context is present, preserving the + # legacy `--config-path` fallback that follows. + conn = from_config(ctx, "conn", "conn", "run") + swarm_id = from_config(ctx, "swarm_id", "swarm_id", "run") + + conn = conn if conn is not None else file_data.get("conn") + swarm_id = swarm_id if swarm_id is not None else file_data.get("swarm_id") + + # Warn (and drop) legacy transport keys in a config file — they're + # superseded by `conn` / `swarm_id` and silently flowing them through + # would mask a stale config. + legacy = sorted(_LEGACY_TOML_KEYS & set(file_data)) + if legacy: + click.echo( + f"warning: ignoring legacy config key(s) {legacy}; " + "use `conn` and `swarm_id` instead.", + err=True, + ) + + # Translate the single `--conn` connection string into the internal + # adapter + transport settings. The internal `adapter` enum stays an + # implementation detail — the CLI never exposes it. + conn_settings = _conn_to_settings(conn, swarm_id, sim_is_dotbot) + + # For a simulator connection with no init-state set (CLI default is + # None, so fold in any config value), offer to scaffold an editable + # world file in the cwd. resolve_init_state_path then picks up the + # freshly-written file (or the packaged world if declined/non-tty). + if conn_settings.get("adapter", "").endswith("simulator"): + _maybe_scaffold_sim_state( + simulator_init_state or file_data.get("simulator_init_state") + ) + cli_args = { - "adapter": adapter, - "port": port, - "baudrate": baudrate, - "mqtt_host": mqtt_host, - "mqtt_port": mqtt_port, - "mqtt_use_tls": mqtt_use_tls, "gw_address": gw_address, - "network_id": network_id, "controller_http_port": controller_http_port, "map_size": map_size, "background_map": background_map, @@ -187,11 +304,11 @@ def main( "csv_data_output": csv_data_output, } - data = {} - if config_path: - file_data = toml.load(config_path) - data.update(file_data) - + # Settings precedence: defaults < config-file (non-conn/legacy keys) < + # conn translation < other CLI flags. + dropped = _LEGACY_TOML_KEYS | {"conn", "swarm_id"} + data = {k: v for k, v in file_data.items() if k not in dropped} + data.update(conn_settings) data.update({k: v for k, v in cli_args.items() if v is not None}) controller_settings = ControllerSettings(**data) diff --git a/dotbot/dotbot_simulator.py b/dotbot/dotbot_simulator.py index bcac04b6..a9f2f58d 100644 --- a/dotbot/dotbot_simulator.py +++ b/dotbot/dotbot_simulator.py @@ -23,7 +23,7 @@ from dotbot_utils.protocol import Frame, Header, Packet from pydantic import BaseModel, Field, model_validator -from dotbot import GATEWAY_ADDRESS_DEFAULT +from dotbot import GATEWAY_ADDRESS_DEFAULT, SIMULATOR_INIT_STATE_DEFAULT from dotbot.logger import LOGGER from dotbot.protocol import ControlModeType, PayloadDotBotAdvertisement, PayloadType @@ -743,6 +743,29 @@ def _run(self): self._cond.wait(timeout=wait) +def packaged_init_state_path() -> Path: + """Absolute path to the default simulator world shipped in the package.""" + return Path(__file__).with_name(SIMULATOR_INIT_STATE_DEFAULT) + + +def resolve_init_state_path(path: str) -> str: + """Resolve the simulator init-state .toml to load. + + An existing file — an explicit ``--simulator-init-state`` path, or a + ``simulator_init_state.toml`` in the working directory — is used as + given. When the default is requested and no such file is present, + fall back to the world shipped inside the package, so the no-hardware + path (``dotbot run simulator`` / ``--conn simulator``) works from any directory + and from a pip-installed wheel. An explicit path that does not exist + is returned unchanged so the caller gets a clear FileNotFoundError. + """ + if Path(path).is_file(): + return path + if path == SIMULATOR_INIT_STATE_DEFAULT: + return str(packaged_init_state_path()) + return path + + class DotBotSimulatorCommunicationInterface: """Bidirectional serial interface to control simulated robots""" @@ -751,7 +774,9 @@ def __init__(self, on_frame_received: Callable, simulator_init_state: str): self.on_frame_received = on_frame_received self._stp_event = threading.Event() self.main_thread = threading.Thread(target=self.run, daemon=True) - init_state = InitStateToml(**toml.load(simulator_init_state)) + init_state = InitStateToml( + **toml.load(resolve_init_state_path(simulator_init_state)) + ) self._network = init_state.network self.dotbots = [ DotBotSimulator( diff --git a/dotbot/examples/charging_station/charging_station.py b/dotbot/examples/charging_station/charging_station.py index 28646ddd..ab44c215 100644 --- a/dotbot/examples/charging_station/charging_station.py +++ b/dotbot/examples/charging_station/charging_station.py @@ -31,12 +31,12 @@ BOT_RADIUS = 60 # Physical radius of a DotBot (unit), used for collision avoidance MAX_SPEED = 300 # Maximum allowed linear speed of a bot (mm/s) -(CHARGER_X, CHARGER_Y) = ( +CHARGER_X, CHARGER_Y = ( 500, 500, ) -(QUEUE_HEAD_X, QUEUE_HEAD_Y) = ( +QUEUE_HEAD_X, QUEUE_HEAD_Y = ( 500, 1500, ) # World-frame (X, Y) position of the charging queue head @@ -44,7 +44,7 @@ 300 # Spacing between consecutive bots in the charging queue (along X axis) ) -(PARK_X, PARK_Y) = (1700, 500) # World-frame (X, Y) position of the parking area origin +PARK_X, PARK_Y = (1700, 500) # World-frame (X, Y) position of the parking area origin PARK_SPACING = 300 # Spacing between parked bots (along Y axis) diff --git a/dotbot/examples/circle/__init__.py b/dotbot/examples/circle/__init__.py new file mode 100644 index 00000000..2eb0e0b9 --- /dev/null +++ b/dotbot/examples/circle/__init__.py @@ -0,0 +1,2 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause diff --git a/dotbot/examples/circle/circle.py b/dotbot/examples/circle/circle.py new file mode 100644 index 00000000..0cd53ca8 --- /dev/null +++ b/dotbot/examples/circle/circle.py @@ -0,0 +1,61 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""The simplest DotBot demo: drive one bot in a circle over the REST API. + +No pydotbot internals - just HTTP with `requests`, so it doubles as a +copy-pasteable template for your own scripts. Start a controller +(e.g. `dotbot run simulator -w`), then run `dotbot run demo circle`. +""" + +import time + +import click +import requests + + +@click.command(name="circle", help="Drive one bot in a circle (bare-REST demo).") +@click.option( + "--base", + default="http://localhost:8000", + show_default=True, + help="Controller REST base URL.", +) +@click.option( + "--seconds", + "-t", + default=5.0, + show_default=True, + help="How long to drive, in seconds.", +) +def main(base, seconds): + """Drive the first DotBot the controller sees in a circle, then stop.""" + try: + bots = requests.get(f"{base}/controller/dotbots", timeout=5).json() + except requests.RequestException as exc: + raise click.ClickException( + f"Cannot reach a controller at {base} ({exc}). " + "Start one first, e.g. `dotbot run simulator -w`." + ) from exc + if not bots: + raise click.ClickException( + f"No DotBots at {base}. The simulator (`dotbot run simulator -w`) " + "spawns one; on hardware, flash and connect a bot first." + ) + + address = bots[0]["address"] + move = f"{base}/controller/dotbots/{address}/0/move_raw" + click.echo(f"Driving {address} in a circle for {seconds:.0f}s (Ctrl-C to stop)...") + + # left_y and right_y are the two wheel speeds; unequal -> the bot turns. + end = time.time() + seconds + try: + while time.time() < end: + requests.put( + move, json={"left_x": 0, "left_y": 60, "right_x": 0, "right_y": 30} + ) + time.sleep(0.1) + finally: + # always stop the motors, even on Ctrl-C + requests.put(move, json={"left_x": 0, "left_y": 0, "right_x": 0, "right_y": 0}) + click.echo("Done.") diff --git a/dotbot/examples/qrkey_demo/README.md b/dotbot/examples/qrkey_demo/README.md index 998169b6..b95e698e 100644 --- a/dotbot/examples/qrkey_demo/README.md +++ b/dotbot/examples/qrkey_demo/README.md @@ -61,9 +61,6 @@ The controller is **completely agnostic** to the demo. Stop the demo and the controller is unaffected; if the broker is unreachable, the demo logs and retries — the controller never blocks on it. -See `plans/dotbot-access-levels.md` for the broader access-level -architecture sketch this example sits inside. - ## Configuration Settings come from env vars (or a `.env` file in the working diff --git a/dotbot/examples/qrkey_demo/__init__.py b/dotbot/examples/qrkey_demo/__init__.py index 727ec606..cfbd7684 100644 --- a/dotbot/examples/qrkey_demo/__init__.py +++ b/dotbot/examples/qrkey_demo/__init__.py @@ -7,8 +7,7 @@ This example consumes qrkey-decrypted MQTT commands from a phone and forwards them to a running PyDotBot controller via the controller's -REST API. The controller stays unaware of qrkey — see -plans/pydotbot-qrkey-example.md. +REST API. The controller stays unaware of qrkey. """ from dotbot.examples.qrkey_demo.client import ( # noqa: F401 diff --git a/dotbot/firmware/__init__.py b/dotbot/firmware/__init__.py new file mode 100644 index 00000000..c90f416e --- /dev/null +++ b/dotbot/firmware/__init__.py @@ -0,0 +1,9 @@ +"""DotBot firmware engine: fetch, flash, and read-back primitives. + +The hardware-facing library behind the `dotbot fw` (artifacts) and +`dotbot device` (one cabled device) CLI namespaces. Originally vendored +from the standalone `dotbot-provision` package; the `provision` *command* +has since dissolved into `dotbot device flash-swarmit-sandbox` / +`flash-mari-gateway` / `flash-programmer` / `info`, so this package is named +for what it is — the firmware engine — not the retired command. +""" diff --git a/dotbot/firmware/boards.py b/dotbot/firmware/boards.py new file mode 100644 index 00000000..43498d5c --- /dev/null +++ b/dotbot/firmware/boards.py @@ -0,0 +1,76 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Board → chip spec: the nrfjprog `-f` family and `--coprocessor` each board needs. + +Single source of truth for the flashable board targets and the nrfjprog flags +they require. `device flash` resolves the board passed via `-b/--board` to a +`BoardSpec` so it programs the right chip family and core, instead of assuming +nRF5340. + +Family/core values come from the DotBot-firmware `.emProject` device defines +(`arm_target_device_name`), verified 2026-05-31 — not from guesswork. nRF52 is +single-core (no `--coprocessor`); the nRF5340 is dual-core (app + net). + +Scope: the DotBot ecosystem is exclusively Nordic nRF (nRF52 / nRF5340) for +now, so this whole flash path assumes nrfjprog. Supporting a non-Nordic SoC +would mean a flasher-backend abstraction (this table would name the backend), +not just a new row. That may change if there's enough reason to — on no +particular timeline. +""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class BoardSpec: + """How nrfjprog must be invoked for a board. + + - ``family``: nrfjprog ``-f`` value — ``"NRF52"`` or ``"NRF53"``. + - ``coprocessor``: nrfjprog ``--coprocessor`` value — ``"CP_APPLICATION"`` / + ``"CP_NETWORK"`` on the (dual-core) nRF5340, or ``None`` on the + single-core nRF52 (where ``--coprocessor`` does not apply). + """ + + family: str + coprocessor: str | None + + +# Keyed by the build/flash target name. Comment = the chip per its .emProject. +BOARDS: dict[str, BoardSpec] = { + "dotbot-v1": BoardSpec("NRF52", None), # nRF52833 + "dotbot-v2": BoardSpec("NRF53", "CP_APPLICATION"), # nRF5340 (app core) + "dotbot-v3": BoardSpec("NRF53", "CP_APPLICATION"), # nRF5340 (app core) + "nrf52833dk": BoardSpec("NRF52", None), + "nrf52840dk": BoardSpec("NRF52", None), + "nrf5340dk-app": BoardSpec("NRF53", "CP_APPLICATION"), + "nrf5340dk-net": BoardSpec("NRF53", "CP_NETWORK"), + "sailbot-v1": BoardSpec("NRF52", None), # nRF52833 + "freebot-v1.0": BoardSpec("NRF52", None), # nRF52840 + "lh2-mini-mote": BoardSpec("NRF52", None), # nRF52833 + "xgo-v1": BoardSpec("NRF52", None), # nRF52833 + "xgo-v2": BoardSpec("NRF52", None), # nRF52833 +} + +# Fallback for a board not in the table: the historical nRF5340 app-core +# behavior, so an unlisted target keeps flashing as it did before. +DEFAULT_SPEC = BoardSpec("NRF53", "CP_APPLICATION") + + +def spec_for(board: str) -> BoardSpec: + """Return the `BoardSpec` for a target name (DEFAULT_SPEC if unlisted).""" + return BOARDS.get(board, DEFAULT_SPEC) + + +# Which nrfjprog families are multi-core — i.e. expose `--coprocessor` and have +# more than one core to reset. The nRF5340 is app + net; the nRF52 is +# single-core. A future multi-core family (e.g. nRF54H) is added here, in one +# place, instead of scattering `family == "NRF53"` checks through the engine. +MULTICORE_FAMILIES = frozenset({"NRF53"}) + + +def is_multicore_family(family: str) -> bool: + """True if `family` has multiple cores (so nrfjprog needs `--coprocessor`).""" + return family in MULTICORE_FAMILIES diff --git a/dotbot/firmware/config-sample.toml b/dotbot/firmware/config-sample.toml new file mode 100644 index 00000000..481199e7 --- /dev/null +++ b/dotbot/firmware/config-sample.toml @@ -0,0 +1,3 @@ +[provisioning] +network_id = "0100" +# firmware_version = "v0.6.0" diff --git a/dotbot/firmware/fetch.py b/dotbot/firmware/fetch.py new file mode 100644 index 00000000..cc3d1097 --- /dev/null +++ b/dotbot/firmware/fetch.py @@ -0,0 +1,302 @@ +"""Firmware release fetching + cache layout (no CLI, no flashing). + +The engine behind `dotbot fw fetch` and the auto-fetch hook in +`flash.flash_role`: decide which GitHub release to pull (pinned / latest / +an explicit tag), download every asset it publishes into the +source-qualified cache (`~/.dotbot/artifacts/-/`), and +record provenance in a `manifest.json`. Pure library code; the Click +surface lives in `dotbot/cli/fw.py`. + +Kept separate from `flash.py` (the hardware-facing flashing engine): this +module never touches a device, only the network + the cache. +""" + +from __future__ import annotations + +import json +import os +import shutil +import time +import urllib.error +import urllib.request +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timezone +from pathlib import Path + +import click + +GITHUB_API = "https://api.github.com/repos" +# Firmware release sources. swarmit ships the swarm system images (bootloader +# + mari netcore + mari gateway); DotBot-firmware ships the bare apps (.hex) +# and the sandbox apps (.bin). Each is cached in its own -/ +# subdir of the artifacts cache so versions and provenance never collide. +RELEASE_SOURCES = { + "swarmit": "DotBots/swarmit", + "dotbot-firmware": "DotBots/DotBot-firmware", +} +DEFAULT_FETCH_SOURCES = ("swarmit", "dotbot-firmware") +# The DotBot-firmware release this pydotbot is built and tested against. Unlike +# swarmit (a Python dependency, whose firmware release is tagged identically to +# the package, so we read it from importlib.metadata), DotBot-firmware is not a +# Python package - so the expected version is declared here and bumped +# deliberately when pydotbot adopts a new release. `dotbot fw fetch` (no -f) +# pulls exactly this; -f overrides it. +DOTBOT_FIRMWARE_VERSION = "1.22.0" + +# Transient HTTP statuses worth retrying (GitHub's asset CDN 502s now and +# then under concurrent load; 429 is rate-limiting). +_RETRY_STATUS = {429, 500, 502, 503, 504} + + +def resolve_fw_root(bin_dir: Path, source: str, fw_version: str) -> Path: + return bin_dir / f"{source}-{fw_version}" + + +def _human_size(num_bytes: int) -> str: + size = float(num_bytes) + for unit in ("B", "KB", "MB", "GB"): + if size < 1024 or unit == "GB": + return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}" + size /= 1024 + + +def _short_path(path: Path) -> str: + """Path relative to the cwd when that's shorter, else absolute.""" + try: + rel = os.path.relpath(path) + except ValueError: + # Windows: path and cwd on different drives have no relative form. + return str(path) + return rel if not rel.startswith("..") else str(path) + + +def download_file(url: str, dest: Path, *, retries: int = 3) -> int: + """Download ``url`` to ``dest``; return the number of bytes written. + + Retries transient failures (connection errors and HTTP 429/5xx) with + exponential backoff - GitHub's CDN occasionally 502s under concurrent + downloads, and a sporadic failure shouldn't abort the whole fetch. + """ + for attempt in range(retries + 1): + try: + with urllib.request.urlopen(url) as resp: + status = getattr(resp, "status", 200) + if status != 200: + raise click.ClickException(f"HTTP {status} while downloading {url}") + data = resp.read() + dest.write_bytes(data) + return len(data) + except urllib.error.HTTPError as exc: + if exc.code not in _RETRY_STATUS or attempt == retries: + raise click.ClickException( + f"HTTP error while downloading {url}: {exc}" + ) from exc + reason = f"HTTP {exc.code}" + except urllib.error.URLError as exc: + if attempt == retries: + raise click.ClickException( + f"Network error while downloading {url}: {exc}" + ) from exc + reason = str(exc.reason) + delay = 0.5 * (2**attempt) + click.echo( + f" [retry] {url.rsplit('/', 1)[-1]} ({reason}); retrying in {delay:.1f}s", + err=True, + ) + time.sleep(delay) + raise click.ClickException( # pragma: no cover - loop always returns/raises + f"Failed to download {url} after {retries} retries." + ) + + +def _github_get(url: str): + """Unauthenticated GitHub API GET (60 req/hour is plenty for a CLI).""" + request = urllib.request.Request( + url, + headers={"Accept": "application/vnd.github+json", "User-Agent": "dotbot"}, + ) + try: + with urllib.request.urlopen(request) as resp: + return json.load(resp) + except (urllib.error.HTTPError, urllib.error.URLError) as exc: + raise click.ClickException(f"GitHub API request failed ({url}): {exc}") from exc + + +def resolve_release(source: str, fw_version: str) -> dict: + """Return the GitHub release JSON for ``source`` at ``fw_version``. + + ``fw_version="latest"`` resolves the newest release via ``/releases`` + (prereleases included, unlike ``/releases/latest`` which skips them - the + current newest, e.g. swarmit 0.8.0rc2, is a prerelease). + """ + if source not in RELEASE_SOURCES: + raise click.ClickException( + f"Unknown firmware source '{source}'. Known: {', '.join(RELEASE_SOURCES)}." + ) + repo = RELEASE_SOURCES[source] + if fw_version == "latest": + releases = _github_get(f"{GITHUB_API}/{repo}/releases?per_page=1") + if not releases: + raise click.ClickException( + f"No releases found for {repo}; pass an explicit -f ." + ) + return releases[0] + return _github_get(f"{GITHUB_API}/{repo}/releases/tags/{fw_version}") + + +def resolve_latest_version(source: str = "swarmit") -> str: + """The newest release tag for ``source`` (prereleases included).""" + return resolve_release(source, "latest")["tag_name"] + + +def pinned_version(source: str) -> str: + """The exact firmware release this pydotbot pins for ``source``. + + swarmit is a Python dependency whose firmware release is tagged identically + to the package, so its pin is read from the installed package. DotBot-firmware + is not a Python package, so its pin is the declared ``DOTBOT_FIRMWARE_VERSION``. + `dotbot fw fetch` (no -f) resolves to these; pass -f to override. + """ + if source == "swarmit": + from importlib.metadata import PackageNotFoundError + from importlib.metadata import version as _pkg_version + + try: + return _pkg_version("swarmit") + except PackageNotFoundError as exc: + raise click.ClickException( + "Cannot infer the swarmit firmware version: the swarmit package " + "is not installed. Pass -f explicitly." + ) from exc + if source == "dotbot-firmware": + return DOTBOT_FIRMWARE_VERSION + raise click.ClickException( + f"Unknown firmware source '{source}'. Known: {', '.join(RELEASE_SOURCES)}." + ) + + +def fetch_assets( + source: str, fw_version: str, bin_dir: Path, local_root: Path | None = None +) -> Path: + """Fetch one source's firmware into ``bin_dir/-/``. + + For a released version, downloads every ``.hex``/``.bin`` asset the GitHub + release publishes (so it adapts to whatever the release ships, no hardcoded + asset list) and writes a ``manifest.json`` with provenance. For + ``fw_version="local"``, symlinks/copies from a local build tree into + ``-local/``. Used by `dotbot fw fetch` and the auto-fetch hook in + `flash_role`. + """ + if source not in RELEASE_SOURCES: + raise click.ClickException( + f"Unknown firmware source '{source}'. Known: {', '.join(RELEASE_SOURCES)}." + ) + if fw_version == "local" and not local_root: + raise click.ClickException("--local-root is required when --fw-version=local.") + if fw_version != "local" and local_root: + click.echo( + "[WARN] --local-root ignored when --fw-version is not 'local'.", + err=True, + ) + + if fw_version == "local": + return _link_local_assets(source, local_root, bin_dir) + + release = resolve_release(source, fw_version) + tag = release["tag_name"] + out_dir = resolve_fw_root(bin_dir, source, tag) + out_dir.mkdir(parents=True, exist_ok=True) + assets = [ + a for a in release.get("assets", []) if a["name"].endswith((".hex", ".bin")) + ] + if not assets: + raise click.ClickException( + f"{source} release {tag} publishes no .hex/.bin assets." + ) + click.echo( + f"Fetching {source} {tag} ({len(assets)} files) into {_short_path(out_dir)}" + ) + names: list[str] = [] + errors: list[str] = [] + # Downloads are I/O-bound, so a small thread pool overlaps them (urllib + # releases the GIL during the network read). Sources stay sequential. + # Kept modest: GitHub's asset CDN starts 502ing under heavier fan-out + # (download_file retries transient failures regardless). + with ThreadPoolExecutor(max_workers=4) as pool: + future_to_name = { + pool.submit( + download_file, asset["browser_download_url"], out_dir / asset["name"] + ): asset["name"] + for asset in assets + } + for future in as_completed(future_to_name): + name = future_to_name[future] + try: + size = future.result() + except click.ClickException as exc: + errors.append(f"{name}: {exc.format_message()}") + continue + names.append(name) + click.echo(f" {name} ({_human_size(size)})") + if errors: + raise click.ClickException( + f"{len(errors)} asset(s) failed to download:\n " + "\n ".join(errors) + ) + _write_manifest(out_dir, source, tag, RELEASE_SOURCES[source], names) + return out_dir + + +def _write_manifest( + out_dir: Path, source: str, version: str, repo: str, files: list[str] +) -> None: + """Record provenance next to the binaries (a cheap audit trail).""" + from dotbot import pydotbot_version + + manifest = { + "source": source, + "version": version, + "repo": repo, + "url": f"https://github.com/{repo}/releases/tag/{version}", + "pydotbot": pydotbot_version(), + "fetched_at": datetime.now(timezone.utc).isoformat(timespec="seconds"), + "files": sorted(files), + } + (out_dir / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n") + + +def _link_local_assets(source: str, local_root: Path, bin_dir: Path) -> Path: + """Symlink/copy a local build tree into ``bin_dir/-local/``.""" + if source != "swarmit": + raise click.ClickException( + f"--fw-version local is only wired for source 'swarmit' so far, " + f"not '{source}'." + ) + local_root = local_root.expanduser().resolve() + out_dir = resolve_fw_root(bin_dir, source, "local") + out_dir.mkdir(parents=True, exist_ok=True) + mapping = { + "bootloader-dotbot-v3.hex": local_root + / "device/bootloader/Output/dotbot-v3/Debug/Exe/bootloader-dotbot-v3.hex", + "netcore-nrf5340-net.hex": local_root + / "device/network_core/Output/nrf5340-net/Debug/Exe/netcore-nrf5340-net.hex", + "03app_gateway_app-nrf5340-app.hex": local_root + / "mari/firmware/app/03app_gateway_app/Output/nrf5340-app/Debug/Exe/03app_gateway_app-nrf5340-app.hex", + "03app_gateway_net-nrf5340-net.hex": local_root + / "mari/firmware/app/03app_gateway_net/Output/nrf5340-net/Debug/Exe/03app_gateway_net-nrf5340-net.hex", + } + missing = [name for name, src in mapping.items() if not src.exists()] + if missing: + raise click.ClickException( + f"Missing local build artifacts: {', '.join(missing)}" + ) + for name, src in mapping.items(): + dest = out_dir / name + if dest.exists() or dest.is_symlink(): + dest.unlink() + try: + os.symlink(src, dest) + click.echo(f"[LINK] {dest} -> {src}") + except OSError: + shutil.copy2(src, dest) + click.echo(f"[COPY] {dest} <- {src}") + return out_dir diff --git a/dotbot/firmware/flash.py b/dotbot/firmware/flash.py new file mode 100644 index 00000000..943592dc --- /dev/null +++ b/dotbot/firmware/flash.py @@ -0,0 +1,627 @@ +"""DotBot firmware flashing + provisioning engine (no CLI). + +The hardware-facing engine behind `dotbot device`: flash a role's system +firmware + config page (`flash_role`), flash a single app image +(`flash_app_image`), flash the debug-chip programmer (`flash_programmer`), +and read back provisioning state (`read_config_report`). Release fetching ++ the artifact cache live next door in `fetch.py`; the Click surface lives +in `dotbot/cli/device.py`. This module is pure library code. +""" + +from __future__ import annotations + +import json +import os +import time +from pathlib import Path + +import click + +from .fetch import fetch_assets, resolve_fw_root +from .nrf import ( + do_daplink, + do_daplink_if, + do_jlink, + flash_nrf_both_cores, + flash_nrf_one_core, + pick_last_jlink_snr, + pick_matching_jlink_snr, + read_device_id, + read_net_id, +) + +try: + from intelhex import IntelHex +except ModuleNotFoundError: # pragma: no cover - optional dependency + IntelHex = None +try: + import tomllib # Python 3.11+ +except ModuleNotFoundError: # pragma: no cover - fallback for older Pythons + tomllib = None + + +DEFAULT_BIN_DIR = Path("bin") +VALID_DEVICES = ("dotbot-v3", "gateway") +VALID_PROGRAMMERS = ("jlink", "daplink") +CONFIG_ADDR = 0x0103F800 +CONFIG_MAGIC = 0x5753524D +CONFIG_MANIFEST_NAME = "config-manifest.json" +# LH2 calibration is appended to the swarmit config page after (magic, net_id). +# Matches swarmit's swarmit_config_t and the format produced by +# dotbot-lh2-calibration (1-byte count + N matrices of 3x3 int32 LE). +LH2_MATRIX_BYTES = 3 * 3 * 4 # 3x3 int32 matrix +LH2_MAX_HOMOGRAPHIES = 16 +# Application images are linked after the bootloader. +APP_FLASH_BASE_ADDR = 0x00010000 +# Programmer bring-up files +GEEHY_PACK_NAME = "Geehy.APM32F1xx_DFP.1.1.0.pack" +JLINK_REQUIRED_FILES = ("JLink-ob.bin", "stm32f103xb_bl.hex", GEEHY_PACK_NAME) +DAPLINK_REQUIRED_FILES = ( + "stm32f103xb_bl.hex", + "stm32f103xb_if.hex", + GEEHY_PACK_NAME, +) +APM_DEVICE = "APM32F103CB" +# it seems to always start with 77 +DOTBOT_V3_SERIAL_PATTERN = r"77[0-9A-F]{7}" + +DEVICE_ASSETS: dict[str, dict[str, str]] = { + "dotbot-v3": { + "app": "bootloader-dotbot-v3.hex", + "net": "netcore-nrf5340-net.hex", + "examples": ["rgbled-dotbot-v3.bin", "dotbot-dotbot-v3.bin"], + }, + "gateway": { + "app": "03app_gateway_app-nrf5340-app.hex", + "net": "03app_gateway_net-nrf5340-net.hex", + "examples": [], + }, +} + + +def load_config(path: Path) -> dict: + if tomllib is None: + raise click.ClickException( + "tomllib not available; install Python 3.11+ or add tomli." + ) + try: + return tomllib.loads(path.read_text()) + except FileNotFoundError as exc: + raise click.ClickException(f"Config file not found: {path}") from exc + except Exception as exc: # noqa: BLE001 - surface parse errors + raise click.ClickException( + f"Failed to parse config file {path}: {exc}" + ) from exc + + +def normalize_network_id(raw: str | None) -> tuple[int, str] | None: + if raw is None: + return None + s = raw.strip().lower() + if s.startswith("0x"): + s = s[2:] + try: + value = int(s, 16) + except ValueError as exc: + raise click.ClickException( + f"Invalid network_id '{raw}' (expected hex)." + ) from exc + if not (0x0000 <= value <= 0xFFFF): + raise click.ClickException("network_id must be 16-bit (0x0000..0xFFFF).") + return value, f"{value:04X}" + + +def convert_bin_to_hex(bin_path: Path, base_addr: int) -> Path: + if IntelHex is None: + raise click.ClickException( + "intelhex not available; install it to convert .bin to .hex." + ) + if not bin_path.exists(): + raise click.ClickException(f"BIN file not found: {bin_path}") + hex_path = bin_path.with_suffix(".hex") + ih = IntelHex() + ih.frombytes(bin_path.read_bytes(), offset=base_addr) + ih.tofile(str(hex_path), "hex") + click.echo( + f"[OK ] converted {bin_path.name} -> {hex_path.name} @ 0x{base_addr:08X}" + ) + return hex_path + + +def find_existing_config_hex(fw_root: Path) -> Path | None: + candidates = sorted( + fw_root.glob("config-*.hex"), + key=lambda p: p.stat().st_mtime, + reverse=True, + ) + return candidates[0] if candidates else None + + +def make_config_hex_path( + fw_root: Path, device: str, fw_version: str, net_id_hex: str +) -> Path: + ts = time.strftime("%Y%b%d-%H%M%S") + return fw_root / f"config-{device}-{fw_version}-{net_id_hex}-{ts}.hex" + + +def load_calibration_file(path: Path) -> tuple[int, bytes]: + """Parse a swarmit LH2 calibration file. + + Accepts two formats: + + - **TOML** (`*.toml`, the modern record): schema-versioned, carries + metadata (timestamp, station count, calibration distance). The + `[calibration].data_hex` field is the same byte payload as the + legacy format, hex-encoded. + - **Legacy binary** (`calibration.out`): 1-byte count + N × 36 bytes. + + The flash path itself only needs the raw bytes; this loader just + extracts them from whichever envelope was provided. + """ + if path.suffix == ".toml": + if tomllib is None: + raise click.ClickException( + "Reading a .toml calibration file needs Python 3.11+ " + "(tomllib in the stdlib) or the tomli backport." + ) + try: + # Binary mode lets tomllib handle UTF-8 itself (TOML is + # spec'd as UTF-8); read_text() would pick up the platform + # default (cp1252 on Windows) and mangle the contents. + with open(path, "rb") as f: + parsed = tomllib.load(f) + data = bytes.fromhex(parsed["calibration"]["data_hex"]) + except (KeyError, ValueError) as exc: + raise click.ClickException( + f"Malformed TOML calibration file {path}: {exc}" + ) from exc + else: + data = path.read_bytes() + if len(data) < 1 or (len(data) - 1) % LH2_MATRIX_BYTES != 0: + raise click.ClickException( + f"Invalid calibration file size: expected 1+N*{LH2_MATRIX_BYTES} " + f"bytes (count byte + matrices), got {len(data)}" + ) + count = data[0] + matrices = data[1:] + expected = len(matrices) // LH2_MATRIX_BYTES + if count != expected: + raise click.ClickException( + f"Invalid calibration file: count byte ({count}) does not match " + f"matrix payload length ({expected})" + ) + if count == 0: + raise click.ClickException( + "Invalid calibration file: homography count cannot be zero" + ) + if count > LH2_MAX_HOMOGRAPHIES: + raise click.ClickException( + f"Invalid calibration file: homography count {count} exceeds " + f"LH2 limit ({LH2_MAX_HOMOGRAPHIES})" + ) + return count, matrices + + +def _write_word_le(ih, addr: int, word: int) -> None: + ih[addr + 0] = (word >> 0) & 0xFF + ih[addr + 1] = (word >> 8) & 0xFF + ih[addr + 2] = (word >> 16) & 0xFF + ih[addr + 3] = (word >> 24) & 0xFF + + +def create_config_hex( + dest: Path, + net_id_value: int, + calibration: tuple[int, bytes] | None = None, +) -> None: + if IntelHex is None: + raise click.ClickException( + "intelhex not available; install it to build config hex." + ) + ih = IntelHex() + # Layout matches swarmit_config_t in repos/swarmit/device/network_core/Source/main.c + # and mari_app_config_t in repos/mari/firmware/app/03app_gateway_net/main.c: + # offset 0: magic (uint32 LE) + # offset 4: has_net_id (uint32 LE) — 1 means the net_id below is provisioned + # offset 8: net_id (uint32 LE) + # offset 12: homography_count (uint32 LE) — swarmit only; meaningful only with --calibration + # offset 16: homographies[N][3][3] (int32 LE) — swarmit only + _write_word_le(ih, CONFIG_ADDR + 0, CONFIG_MAGIC) + _write_word_le(ih, CONFIG_ADDR + 4, 1) + _write_word_le(ih, CONFIG_ADDR + 8, net_id_value) + if calibration is not None: + count, matrices = calibration + _write_word_le(ih, CONFIG_ADDR + 12, count) + for i, b in enumerate(matrices): + ih[CONFIG_ADDR + 16 + i] = b + dest.parent.mkdir(parents=True, exist_ok=True) + ih.tofile(str(dest), "hex") + + +def load_config_manifest(path: Path) -> dict | None: + if not path.exists(): + return None + try: + return json.loads(path.read_text()) + except Exception as exc: # noqa: BLE001 - surface parse errors + raise click.ClickException( + f"Failed to parse config manifest {path}: {exc}" + ) from exc + + +def write_config_manifest(path: Path, payload: dict) -> None: + path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + + +def build_manifest_payload( + config_hex: Path, + device: str, + fw_version: str, + net_id_hex: str, + calibration_hex: str | None = None, +) -> dict: + created_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()) + return { + "config_hex": config_hex.name, + "device": device, + "fw_version": fw_version, + "network_id": net_id_hex, + "config_addr": f"0x{CONFIG_ADDR:08X}", + "magic": f"0x{CONFIG_MAGIC:08X}", + # Stored inline as hex (count byte + matrices, same bytes as the + # input file). Calibration data is small (typically <100 B, capped + # well under 1 kB at 16 matrices), so inlining keeps the manifest + # self-contained and human-inspectable. + "calibration": calibration_hex, + "created_at": created_at, + } + + +def manifest_matches( + payload: dict, + device: str, + fw_version: str, + net_id_hex: str, + calibration_hex: str | None = None, +) -> bool: + if not isinstance(payload, dict): + return False + return ( + payload.get("device") == device + and payload.get("fw_version") == fw_version + and payload.get("network_id") == net_id_hex + and payload.get("config_addr") == f"0x{CONFIG_ADDR:08X}" + and payload.get("magic") == f"0x{CONFIG_MAGIC:08X}" + and payload.get("calibration") == calibration_hex + and isinstance(payload.get("config_hex"), str) + ) + + +def flash_role( + role: str, + *, + net_id: tuple[int, str], + fw_version: str, + calibration_path: Path | None = None, + bin_dir: Path = DEFAULT_BIN_DIR, + sn_starting_digits: str | None = None, + default_app_name: str | None = None, +) -> None: + """Flash a device's role: system firmware bundle (app+net cores) + config. + + Backend for `dotbot device flash-swarmit-sandbox` (role='dotbot-v3') and + `dotbot device flash-mari-gateway` (role='gateway'). Selects the J-Link, + flashes both cores, writes the config page (magic + has_net_id + + net_id [+ calibration, dotbot-v3 only]), then best-effort reads back + net_id/device_id (never raises on readback failure). If the role's + images are absent from ``bin_dir//``, fetches the release + first (the "run fetch under the hood" behaviour). + """ + assets = DEVICE_ASSETS[role] + net_id_val, net_id_hex = net_id + + if sn_starting_digits: + snr = pick_matching_jlink_snr(sn_starting_digits) + else: + snr = pick_last_jlink_snr() + if snr is None: + raise click.ClickException( + "Unable to auto-select J-Link; provide --snr explicitly." + ) + click.echo(f"[INFO] using J-Link with serial number: {snr}") + + if role == "dotbot-v3" and not snr.startswith("77"): + click.secho( + f"[WARN] Serial number {snr} seems to not be a DotBot, but you are trying to flash a {role} firmware to it.", + fg="yellow", + ) + if not click.confirm( + "Do you want to continue? (you can check or plug the right board)", + default=True, + ): + raise click.ClickException("Aborting.") + elif role == "gateway" and snr.startswith("77"): + click.secho( + f"[WARN] Serial number {snr} seems to be a DotBot, but you are trying to flash a {role} firmware to it.", + fg="yellow", + ) + if not click.confirm( + "Do you want to continue? (you can check or plug the right board)", + default=True, + ): + raise click.ClickException("Aborting.") + + calibration_data: tuple[int, bytes] | None = None + calibration_hex: str | None = None + if calibration_path is not None: + if role != "dotbot-v3": + raise click.ClickException( + "--calibration is only valid for the sandbox host (dotbot-v3); " + "gateway firmware does not have LH2 homographies." + ) + count, matrices = load_calibration_file(calibration_path) + calibration_data = (count, matrices) + calibration_hex = (bytes([count]) + matrices).hex() + click.echo(f"[INFO] calibration: {count} matrices from {calibration_path}") + + fw_root = resolve_fw_root(bin_dir, "swarmit", fw_version) + # Auto-fetch: if the role's images aren't already present, pull the + # swarmit release into bin_dir/swarmit-/ before flashing. + pre_app = fw_root / assets["app"] + pre_net = fw_root / assets["net"] + if fw_version != "local" and not (pre_app.exists() and pre_net.exists()): + click.echo(f"[INFO] firmware {fw_version} not found in {fw_root}; fetching...") + fetch_assets("swarmit", fw_version, bin_dir) + if not fw_root.exists(): + raise click.ClickException(f"Firmware root not found: {fw_root}") + + device = role + + default_app_hex: Path | None = None + if device == "dotbot-v3": + if default_app_name: + name = default_app_name.strip() + if not name: + raise click.ClickException("--app cannot be empty.") + candidate = fw_root / f"{name}-{device}.bin" + if candidate.exists(): + default_app_hex = convert_bin_to_hex(candidate, APP_FLASH_BASE_ADDR) + else: + raise click.ClickException(f"App firmware not found: {candidate}") + else: + # default to dotbot app if no name is provided + candidate = fw_root / "dotbot-dotbot-v3.bin" + if candidate.exists(): + default_app_hex = convert_bin_to_hex(candidate, APP_FLASH_BASE_ADDR) + else: + if default_app_name: + click.echo( + "[WARN] --app is only supported for dotbot-v3; skipping.", + err=True, + ) + + app_hex = fw_root / assets["app"] + net_hex = fw_root / assets["net"] + manifest_path = fw_root / CONFIG_MANIFEST_NAME + manifest = load_config_manifest(manifest_path) + config_hex = None + if manifest: + click.echo( + f"[INFO] loaded manifest {manifest_path}: {json.dumps(manifest, indent=2)}" + ) + if manifest_matches(manifest, device, fw_version, net_id_hex, calibration_hex): + candidate = fw_root / manifest["config_hex"] + if candidate.exists(): + config_hex = candidate + click.secho( + f"[NOTE] using config hex from manifest: {config_hex}", + fg="yellow", + ) + else: + click.secho( + "[INFO] manifest does not match, will create new config hex", + fg="yellow", + ) + + if config_hex is None: + config_hex = make_config_hex_path(fw_root, device, fw_version, net_id_hex) + click.secho(f"[INFO] created new config hex: {config_hex}", fg="green") + + missing = [] + for p in (app_hex, net_hex): + if p.exists(): + continue + if p.is_symlink(): + # Path.exists() follows symlinks; a dangling symlink reports + # missing without surfacing the broken target. Re-running + # `dotbot fw fetch -f --local-root ` typically + # refreshes these. + missing.append(f"{p} (broken symlink → {os.readlink(p)})") + else: + missing.append(str(p)) + if missing: + missing_list = ", ".join(missing) + raise click.ClickException(f"Missing firmware files: {missing_list}") + + click.echo(f"[INFO] device: {device}") + click.echo(f"[INFO] fw_version: {fw_version}") + click.echo(f"[INFO] network_id: 0x{net_id_hex}") + click.echo(f"[INFO] app hex: {app_hex}") + click.echo(f"[INFO] net hex: {net_hex}") + click.echo(f"[INFO] config hex: {config_hex}") + + if not config_hex.exists(): + create_config_hex(config_hex, net_id_val, calibration=calibration_data) + click.echo(f"[OK ] wrote config hex: {config_hex}") + manifest_payload = build_manifest_payload( + config_hex, + device, + fw_version, + net_id_hex, + calibration_hex=calibration_hex, + ) + write_config_manifest(manifest_path, manifest_payload) + click.echo(f"[OK ] wrote config manifest: {manifest_path}") + click.echo(f"[INFO] manifest: {json.dumps(manifest_payload, indent=2)}") + else: + click.echo(f"[INFO] using existing config hex: {config_hex}") + click.echo() + flash_nrf_both_cores(app_hex, net_hex, nrfjprog_opt=None, snr_opt=snr) + flash_nrf_one_core(net_hex=config_hex, nrfjprog_opt=None, snr_opt=snr) + if default_app_hex is not None: + click.echo(f"[INFO] default app hex: {default_app_hex}") + flash_nrf_one_core(app_hex=default_app_hex, nrfjprog_opt=None, snr_opt=snr) + elif device == "dotbot-v3": + click.echo("[INFO] default app hex not found; skipping.") + click.secho("\n[INFO] ==== Flash Complete ====\n", fg="green") + time.sleep(0.2) + try: + readback_net_id = read_net_id(snr=snr) + readback_device_id = read_device_id(snr=snr) + except RuntimeError as exc: + click.echo(f"[WARN] readback failed: {exc}", err=True) + return + click.echo("[INFO] readback values:") + click.echo(f"[INFO] net_id: {readback_net_id}") + last_6_digits_spaced = " ".join( + readback_device_id[-6:][i : i + 2] + for i in range(0, len(readback_device_id[-6:]), 2) + ) + click.echo( + f"[INFO] device_id: {readback_device_id} (last 6 digits: {last_6_digits_spaced})" + ) + click.secho( + "[NOTE] you may need to press the reset button on the DotBot " + "for it to join the network", + fg="yellow", + ) + + +def flash_app_image( + image: Path, *, board: str = "dotbot-v3", sn_starting_digits: str | None = None +) -> None: + """Flash a single firmware image to one cabled device (a whole-chip program). + + Backend for `dotbot device flash `. Accepts a `.hex` (flashed + as-is) or a `.bin` (converted at APP_FLASH_BASE_ADDR first). `board` + selects the nrfjprog family + core via `boards.spec_for`: an nRF52 board + flashes its single core; on the nRF5340 a `*-net` board programs the + network core, otherwise the application core. No sandbox host required. + """ + from .boards import spec_for + + if not image.exists(): + raise click.ClickException(f"Firmware image not found: {image}") + spec = spec_for(board) + if sn_starting_digits: + snr = pick_matching_jlink_snr(sn_starting_digits) + else: + snr = pick_last_jlink_snr() + if snr is None: + raise click.ClickException( + "Unable to auto-select J-Link; provide --snr explicitly." + ) + click.echo(f"[INFO] using J-Link with serial number: {snr}") + image_hex = ( + convert_bin_to_hex(image, APP_FLASH_BASE_ADDR) + if image.suffix == ".bin" + else image + ) + click.echo(f"[INFO] flashing {board} ({spec.family}) image: {image_hex}") + if spec.coprocessor == "CP_NETWORK": + flash_nrf_one_core(net_hex=image_hex, family=spec.family, snr_opt=snr) + else: + flash_nrf_one_core(app_hex=image_hex, family=spec.family, snr_opt=snr) + click.secho("\n[INFO] ==== Flash Complete ====\n", fg="green") + + +def read_config_report(sn_starting_digits: str | None = None) -> tuple[str, str]: + """Read back (net_id, device_id) from a connected device. + + Backend for `dotbot device info`. Returns net_id (or the string + "unprovisioned" when the config page has no valid magic) and the + 64-bit device id. Raises RuntimeError only on a genuine nrfjprog + communication failure — a blank/unprovisioned board is not an error. + """ + if sn_starting_digits: + snr = pick_matching_jlink_snr(sn_starting_digits) + else: + snr = pick_last_jlink_snr() + if snr is None: + raise click.ClickException( + "Unable to auto-select J-Link; provide --snr explicitly." + ) + click.echo(f"[INFO] using J-Link with serial number: {snr}", err=True) + return read_net_id(snr=snr), read_device_id(snr=snr) + + +def flash_programmer( + programmer_firmware: str, files_dir: Path, probe_uid: str | None = None +) -> None: + """Flash J-Link OB / DAPLink firmware to the on-board debug chip. + + Backend for `dotbot device flash-programmer` (was + `provision flash-bringup`). Programs the APM32F103 programmer chip + itself — an obscure, one-time-per-board bring-up step. + """ + files_dir = files_dir.expanduser().resolve() + if not files_dir.exists(): + raise click.ClickException(f"files-dir does not exist: {files_dir}") + + required = { + "jlink": JLINK_REQUIRED_FILES, + "daplink": DAPLINK_REQUIRED_FILES, + }[programmer_firmware] + + missing = [name for name in required if not (files_dir / name).exists()] + if missing: + missing_list = ", ".join(missing) + raise click.ClickException( + f"Missing required files in {files_dir}: {missing_list}" + ) + + click.echo(f"[INFO] programmer: {programmer_firmware}") + click.echo(f"[INFO] files-dir: {files_dir}") + if probe_uid: + click.echo(f"[INFO] probe uid: {probe_uid}") + if programmer_firmware == "jlink": + jlink_bin = (files_dir / "JLink-ob.bin").resolve() + bl_hex = (files_dir / "stm32f103xb_bl.hex").resolve() + pack_path = str((files_dir / GEEHY_PACK_NAME).resolve()) + do_jlink( + jlink_bin, + bl_hex, + apm_device=APM_DEVICE, + jlinktool=None, + pack_path=pack_path, + probe_uid=probe_uid, + ) + elif programmer_firmware == "daplink": + bl_hex = (files_dir / "stm32f103xb_bl.hex").resolve() + if_hex = (files_dir / "stm32f103xb_if.hex").resolve() + pack_path = str((files_dir / GEEHY_PACK_NAME).resolve()) + do_daplink( + bl_hex, + apm_device=APM_DEVICE, + jlinktool=None, + pack_path=pack_path, + probe_uid=probe_uid, + ) + time.sleep(1.0) + do_daplink_if( + if_hex, + apm_device=APM_DEVICE, + pack_path=pack_path, + probe_uid=probe_uid, + ) + else: + raise click.ClickException( + f"Invalid programmer firmware: {programmer_firmware}" + ) + + # small delay to let the target settle if needed + time.sleep(1.0) + click.secho( + f"[OK ] ==== {programmer_firmware} programmer firmware flashed ====", + fg="green", + ) diff --git a/dotbot/firmware/nrf.py b/dotbot/firmware/nrf.py new file mode 100644 index 00000000..db432a24 --- /dev/null +++ b/dotbot/firmware/nrf.py @@ -0,0 +1,480 @@ +#!/usr/bin/env python3 +import os +import shutil +import subprocess +import tempfile +import time +from pathlib import Path + +from .boards import is_multicore_family + +# Timings +POLL_INTERVAL = 1.0 +TIMEOUT_JLINK_SEC = 120 +TIMEOUT_BUILD_SEC = 900 +TIMEOUT_MAINTENANCE_SEC = 300 + +DEFAULT_SWD_SPEED_KHZ = 4000 + + +def run(cmd, timeout=None, cwd=None): + print(f"[CMD] {' '.join(cmd)}") + proc = subprocess.run( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + timeout=timeout, + cwd=cwd, + ) + print(proc.stdout) + return proc.returncode, proc.stdout + + +def run_capture(cmd): + proc = subprocess.run( + cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True + ) + if proc.returncode != 0: + raise RuntimeError(proc.stdout.strip() or f"Command failed: {' '.join(cmd)}") + return proc.stdout + + +def which_tool(exe_name, user_supplied=None, candidates=None): + if user_supplied: + return user_supplied + p = shutil.which(exe_name) + if p: + return p + for c in candidates or []: + if Path(c).exists(): + return c + return exe_name + + +_NRFJPROG_CANDIDATES = ( + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", +) + + +def nrfjprog_available() -> bool: + """True if the `nrfjprog` Nordic command-line tool can be located. + + Checks PATH (both `nrfjprog` and the Windows `nrfjprog.exe`) plus the + well-known install locations. Lets the device commands fail fast with + a friendly install hint instead of a late, cryptic subprocess error. + """ + if shutil.which("nrfjprog") or shutil.which("nrfjprog.exe"): + return True + return any(Path(c).exists() for c in _NRFJPROG_CANDIDATES) + + +# ---------- JLink / DAPLink (APM32F103) ---------- +def make_jlink_script(device, speed_khz, hex_path): + lines = [] + lines.append(f"device {device}") + lines.append("si SWD") + if speed_khz: + lines.append(f"speed {speed_khz}") + lines.append("connect") + lines.append("h") + lines.append("r") + lines.append("erase") + lines.append(f"loadfile {hex_path}") + lines.append("verify") + lines.append("r") + lines.append("g") + lines.append("exit") + return "\n".join(lines) + + +def jlink_flash_hex(jlink_exe, device, image_hex, timeout=TIMEOUT_JLINK_SEC): + speed_khz = DEFAULT_SWD_SPEED_KHZ + with tempfile.NamedTemporaryFile("w", delete=False, suffix=".jlink") as tf: + tf.write(make_jlink_script(device, speed_khz, str(image_hex))) + script_path = tf.name + try: + rc, out = run([jlink_exe, "-CommanderScript", script_path], timeout=timeout) + finally: + try: + os.unlink(script_path) + except OSError: + pass + if rc != 0 or "ERROR" in out.upper() or "FAILED" in out.upper(): + raise RuntimeError("J-Link flash failed; see log above.") + + +def pyocd_flash_hex(jlink_bin, device, pack_path: str, probe_uid: str | None = None): + erase_args = [ + "pyocd", + "erase", + "--chip", + "--pack", + pack_path, + "-t", + str(device), + ] + if probe_uid: + erase_args += ["--uid", probe_uid] + rc, out = run(erase_args, timeout=60) + args = ["pyocd", "flash", str(jlink_bin)] + args += ["--pack", pack_path] + args += ["-t", str(device)] + if probe_uid: + args += ["--uid", probe_uid] + rc, out = run(args, timeout=120) + + +def do_daplink( + bl_hex: Path, + apm_device: str, + jlinktool: str | None, + pack_path: str, + probe_uid: str | None = None, +): + """Flash STM32 bootloader (DAPLink) using external J-Link.""" + jlink_tool = which_tool( + "JLink.exe", + jlinktool, + candidates=[ + # r"C:\Program Files\SEGGER\JLink_V818\JLink.exe", + "/usr/local/bin/JLinkExe", + "/usr/bin/JLinkExe", + ], + ) + if not bl_hex.exists(): + raise FileNotFoundError(f"Bootloader image not found: {bl_hex}") + + print("== Flashing STM32 bootloader (DAPLink) to APM32F103CB ==") + jlink_flash_hex(jlink_tool, apm_device, bl_hex) + print("[OK] DAPLink bootloader programmed.") + + +def do_daplink_if( + if_hex: Path, apm_device: str, pack_path: str, probe_uid: str | None = None +): + """Flash DAPLink interface firmware over SWD using pyOCD.""" + if not if_hex.exists(): + raise FileNotFoundError(f"DAPLink interface image not found: {if_hex}") + + print("== Flashing DAPLink interface image via pyOCD ==") + pyocd_flash_hex(if_hex, apm_device, pack_path, probe_uid=probe_uid) + print("[OK] DAPLink interface programmed.") + + +def do_jlink( + jlink_bin: Path, + bl_hex: Path, + apm_device: str, + jlinktool: str | None, + pack_path: str, + probe_uid: str | None = None, +): + """Flash STM32 bootloader, then J-Link OB image (overwrites BL).""" + if not jlink_bin.exists(): + raise FileNotFoundError(f"J-Link OB image not found: {jlink_bin}") + + do_daplink( + bl_hex=bl_hex, + apm_device=apm_device, + jlinktool=jlinktool, + pack_path=pack_path, + ) + + print("[INFO] Waiting 5 seconds for STM32 bootloader to enumerate...") + time.sleep(5) + + print("== Flashing J-Link OB image via pyOCD ==") + pyocd_flash_hex(jlink_bin, apm_device, pack_path, probe_uid=probe_uid) + print("[OK] J-Link OB programmed.") + + +# ---------- Flash nRF5340 with nrfjprog ---------- +def pick_last_jlink_snr(nrfjprog_opt=None): + nrfjprog = which_tool( + "nrfjprog.exe", + nrfjprog_opt, + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + + rc2, out2 = run([nrfjprog, "--ids"], timeout=10) + ids = ( + [line.strip() for line in out2.splitlines() if line.strip().isdigit()] + if rc2 == 0 + else [] + ) + print(f"[DEBUG] Found J-Link IDs: {ids}") + if ids: + return ids[-1] + raise RuntimeError("Unable to auto-select J-Link; provide --snr explicitly.") + + +def pick_matching_jlink_snr(sn_starting_digits: str, nrfjprog_opt: str | None = None): + nrfjprog = which_tool( + "nrfjprog.exe", + nrfjprog_opt, + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + rc2, out2 = run([nrfjprog, "--ids"], timeout=10) + ids = ( + [ + line.strip() + for line in out2.splitlines() + if line.strip().isdigit() and line.strip().startswith(sn_starting_digits) + ] + if rc2 == 0 + else [] + ) + print(f"[DEBUG] Found J-Link IDs: {ids}") + if not ids: + raise RuntimeError( + f"No J-Link found with serial number starting with {sn_starting_digits}" + ) + return ids[0] + + +def nrfjprog_recover(nrfjprog, snr=None): + args = [nrfjprog, "-f", "NRF53"] + if snr: + args += ["-s", str(snr)] + print(f"[INFO] Recovering both cores of nRF5340 (SNR={snr})...") + rc, out = run(args + ["--recover", "--coprocessor", "CP_APPLICATION"], timeout=120) + rc, out = run(args + ["--recover", "--coprocessor", "CP_NETWORK"], timeout=120) + print(f"[INFO] Erasing both cores of nRF5340 (SNR={snr})...") + rc, out = run(args + ["-e"], timeout=120) + + +def nrfjprog_program( + nrfjprog, + hex_path, + network=False, + family="NRF53", + verify=True, + reset=True, + chiperase=True, + sectorerase=False, + snr=None, +): + if chiperase and sectorerase: + raise ValueError("Use only one of chiperase or sectorerase.") + args = [nrfjprog, "-f", family] + if snr: + args += ["-s", str(snr)] + # --coprocessor only applies to multi-core families (the nRF5340); a + # single-core family (nRF52) has no coprocessor and rejects the flag. + if is_multicore_family(family): + args += ["--coprocessor", "CP_NETWORK" if network else "CP_APPLICATION"] + args += ["--program", str(hex_path)] + if verify: + args += ["--verify"] + if chiperase: + args += ["--chiperase"] + elif sectorerase: + args += ["--sectorerase"] + if reset: + args += ["--reset"] + rc, out = run(args, timeout=120) + if rc != 0 or "ERROR" in out.upper() or "failed" in out.lower(): + raise RuntimeError("nrfjprog programming failed; see log above.") + + +def _parse_memrd_words(output: str) -> list[str]: + line = output.strip().splitlines()[0] if output.strip() else "" + if ":" not in line: + raise RuntimeError(f"Unexpected memrd output: {output.strip()}") + _, rest = line.split(":", 1) + words = [w for w in rest.strip().split() if not w.startswith(("0x", "0X"))] + return words + + +def read_device_id(snr: str | None = None) -> str: + nrfjprog = which_tool( + "nrfjprog.exe", + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + args = [nrfjprog, "-f", "NRF53"] + args += ["--coprocessor", "CP_NETWORK"] + args += ["--memrd", "0x01FF0204"] + args += ["--n", "8"] + if snr: + args += ["-s", str(snr)] + out = run_capture(args) + words = _parse_memrd_words(out) + if len(words) < 2: + raise RuntimeError(f"Unexpected device ID output: {out.strip()}") + return f"{words[1]}{words[0]}" + + +def read_net_id(snr: str | None = None) -> str: + nrfjprog = which_tool( + "nrfjprog.exe", + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + args = [nrfjprog, "-f", "NRF53"] + args += ["--coprocessor", "CP_NETWORK"] + # Read both has_net_id (offset 4) and net_id (offset 8) from the swarmit + # config page; layout matches swarmit_config_t (see create_config_hex + # in cli.py and DotBots/swarmit network_core/Source/main.c). + args += ["--memrd", "0x0103F804"] + args += ["--n", "8"] + if snr: + args += ["-s", str(snr)] + out = run_capture(args) + words = _parse_memrd_words(out) + if len(words) < 2: + raise RuntimeError(f"Unexpected net ID output: {out.strip()}") + has_net_id, net_id = words[0], words[1] + if int(has_net_id, 16) != 1: + return "unprovisioned" + return f"{net_id[-4:]}" + + +def flash_nrf_both_cores( + app_hex: Path, net_hex: Path, nrfjprog_opt: str | None, snr_opt: str | None +): + """Flash nRF5340 application and network cores with full recover + chiperase.""" + if not app_hex.exists(): + raise FileNotFoundError(f"App hex not found: {app_hex}") + if not net_hex.exists(): + raise FileNotFoundError(f"Net hex not found: {net_hex}") + + nrfjprog = which_tool( + "nrfjprog.exe", + nrfjprog_opt, + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + + snr = snr_opt or pick_last_jlink_snr(nrfjprog) + print(f"[INFO] Using J-Link with serial number: {snr}") + + nrfjprog_recover(nrfjprog, snr=snr) + + print("== Flashing nRF5340 application core with nrfjprog ==") + nrfjprog_program( + nrfjprog, + app_hex, + network=False, + verify=True, + reset=True, + chiperase=True, + snr=snr, + ) + print("[OK] Application core programmed.") + + print("== Flashing nRF5340 network core with nrfjprog ==") + nrfjprog_program( + nrfjprog, + net_hex, + network=True, + verify=True, + reset=True, + chiperase=True, + snr=snr, + ) + print("[OK] Network core programmed.") + + +def flash_nrf_one_core( + app_hex: Path | None = None, + net_hex: Path | None = None, + family: str = "NRF53", + nrfjprog_opt: str | None = None, + snr_opt: str | None = None, +): + """Flash only one core; no recover and no chiperase. + + `family` is the nrfjprog `-f` value ("NRF53" / "NRF52"). On nRF52 there is + no network core, so `net_hex` and the per-core reset are nRF5340-only. + """ + if app_hex is None and net_hex is None: + raise FileNotFoundError("Provide app_hex or net_hex.") + if app_hex is not None and net_hex is not None: + raise FileNotFoundError("Provide only one of app_hex or net_hex.") + if app_hex is not None and not app_hex.exists(): + raise FileNotFoundError(f"App hex not found: {app_hex}") + if net_hex is not None and not net_hex.exists(): + raise FileNotFoundError(f"Net hex not found: {net_hex}") + + nrfjprog = which_tool( + "nrfjprog.exe", + nrfjprog_opt, + candidates=[ + # r"C:\Program Files\Nordic Semiconductor\nrf-command-line-tools\bin\nrfjprog.exe" + "/usr/local/bin/nrfjprog", + "/usr/bin/nrfjprog", + ], + ) + + snr = snr_opt or pick_last_jlink_snr(nrfjprog) + print(f"[INFO] Using J-Link with serial number: {snr}") + + if app_hex is not None: + print(f"== Flashing {family} application core with nrfjprog ==") + nrfjprog_program( + nrfjprog, + app_hex, + network=False, + family=family, + verify=True, + reset=True, + chiperase=False, + sectorerase=True, + snr=snr, + ) + print("[OK] Application core programmed.") + else: + print(f"== Flashing {family} network core with nrfjprog ==") + nrfjprog_program( + nrfjprog, + net_hex, + network=True, + family=family, + verify=True, + reset=True, + chiperase=False, + sectorerase=True, + snr=snr, + ) + print("[OK] Network core programmed.") + time.sleep(0.5) + if is_multicore_family(family): + # reset every core + nrfjprog_reset_core(nrfjprog, snr=snr, core="CP_NETWORK", family=family) + nrfjprog_reset_core(nrfjprog, snr=snr, core="CP_APPLICATION", family=family) + else: + # single-core family: one reset, no --coprocessor + nrfjprog_reset_core(nrfjprog, snr=snr, core=None, family=family) + + +def nrfjprog_reset_core(nrfjprog, snr=None, core="CP_APPLICATION", family="NRF53"): + args = [nrfjprog, "-f", family] + if snr: + args += ["-s", str(snr)] + args += ["--reset"] + # --coprocessor is multi-core-only; a single-core family resets directly. + if is_multicore_family(family) and core: + args += ["--coprocessor", core] + rc, out = run(args, timeout=120) + if rc != 0 or "ERROR" in out.upper() or "failed" in out.lower(): + raise RuntimeError("nrfjprog reset failed; see log above.") diff --git a/dotbot/server.py b/dotbot/server.py index 1105e70e..0e64a3bd 100644 --- a/dotbot/server.py +++ b/dotbot/server.py @@ -364,4 +364,14 @@ async def ws_dotbots(websocket: WebSocket): # Mount static files after all routes are defined FRONTEND_DIR = os.path.join(os.path.dirname(__file__), "frontend", "build") -api.mount("/PyDotBot", StaticFiles(directory=FRONTEND_DIR, html=True), name="PyDotBot") +if os.path.isdir(FRONTEND_DIR): + api.mount( + "/PyDotBot", StaticFiles(directory=FRONTEND_DIR, html=True), name="PyDotBot" + ) +else: + LOGGER.warning( + "Frontend build not found at %s; the web UI will be unavailable. " + "Install the published wheel (pip install --pre pydotbot) or build the " + "frontend: cd dotbot/frontend && npm install && npm run build", + FRONTEND_DIR, + ) diff --git a/dotbot/simulator_init_state.toml b/dotbot/simulator_init_state.toml new file mode 100644 index 00000000..3df1a766 --- /dev/null +++ b/dotbot/simulator_init_state.toml @@ -0,0 +1,35 @@ +# This file is used to create and configure simulated DotBot instances for the simulator. +# It is only used when the flag `-p dotbot-simulator` is set, otherwise it is ignored. + +[network] +pdr = 100 # packet delivery ratio for "default" network mode (0-100) +uplink_pdr = 100 # PDR for DotBot→gateway direction in "mari" mode +downlink_pdr = 100 # PDR for gateway→DotBot direction in "mari" mode +slot_duration_ms = 1.236 # Mari TSCH slot duration in ms (from mac.h) +mqtt_latency_ms = 0.0 # one-way MQTT broker latency in ms (applied in both modes) + +[[dotbots]] +address = "BADCAFE111111111" # DotBot unique address +calibrated = 0xff # optional, defaults to only first lighthouse calibrated +pos_x = 1000 # [0, 2_000] in mm +pos_y = 250 # [0, 2_000] +direction = 0 # [0, 360] in degrees, 0 is facing north, increasing clockwise +network_mode = "default" # optional, defaults to "default", can be set to "mari" + +[[dotbots]] +address = "DEADBEEF22222222" +pos_x = 1500 +pos_y = 200 +direction = 180 + +[[dotbots]] +address = "B0B0F00D33333333" +pos_x = 1_500 +pos_y = 1_500 +direction = 180 + +[[dotbots]] +address = "BADC0DE444444444" +pos_x = 1_000 +pos_y = 1_000 +direction = 360 diff --git a/dotbot/tests/test_adapter.py b/dotbot/tests/test_adapter.py index 76b226b2..7e542d0d 100644 --- a/dotbot/tests/test_adapter.py +++ b/dotbot/tests/test_adapter.py @@ -1,9 +1,11 @@ import asyncio +from pathlib import Path from unittest.mock import patch import pytest from dotbot_utils.hdlc import hdlc_encode from dotbot_utils.protocol import Frame, Header, Packet +from marilib.mari_protocol import NextProto from dotbot.adapter import ( DotBotSimulatorAdapter, @@ -80,7 +82,9 @@ async def start_task(): adapter.send_payload(frame.header.destination, payload) adapter.mari.send_frame.assert_called_once_with( - dst=frame.header.destination, payload=frame.packet.to_bytes() + dst=frame.header.destination, + payload=frame.packet.to_bytes(), + next_proto=NextProto.DOTBOT_APP, ) adapter.close() adapter.mari.close.assert_called_once() @@ -115,7 +119,9 @@ async def start_task(): adapter.send_payload(frame.header.destination, payload) adapter.mari.send_frame.assert_called_once_with( - dst=frame.header.destination, payload=frame.packet.to_bytes() + dst=frame.header.destination, + payload=frame.packet.to_bytes(), + next_proto=NextProto.DOTBOT_APP, ) adapter.close() @@ -186,3 +192,33 @@ async def start_task(): adapter.simulator.write.assert_called_once_with(frame.to_bytes()) adapter.close() adapter.simulator.stop.assert_called_once() + + +def test_simulator_adapter_close_before_start_is_noop(): + """close() must not raise when start() never assigned `simulator` + (e.g. start() failed loading the init state).""" + adapter = DotBotSimulatorAdapter() + adapter.close() # no AttributeError + + +def test_resolve_init_state_path_falls_back_to_packaged(tmp_path, monkeypatch): + """The default init-state resolves to the packaged world when no file + exists in the cwd, so `dotbot run simulator` works from any directory.""" + from dotbot import SIMULATOR_INIT_STATE_DEFAULT + from dotbot.dotbot_simulator import resolve_init_state_path + + monkeypatch.chdir(tmp_path) # a dir without simulator_init_state.toml + resolved = resolve_init_state_path(SIMULATOR_INIT_STATE_DEFAULT) + assert Path(resolved).is_file() + assert Path(resolved).name == SIMULATOR_INIT_STATE_DEFAULT + + # A cwd file is preferred over the packaged copy. + local = tmp_path / SIMULATOR_INIT_STATE_DEFAULT + local.write_text('[[dotbots]]\naddress = "0000000000000001"\n') + assert ( + resolve_init_state_path(SIMULATOR_INIT_STATE_DEFAULT) + == SIMULATOR_INIT_STATE_DEFAULT + ) + + # An explicit, missing path is returned unchanged (caller gets the error). + assert resolve_init_state_path("nope/missing.toml") == "nope/missing.toml" diff --git a/dotbot/tests/test_calibration_lighthouse2.py b/dotbot/tests/test_calibration_lighthouse2.py new file mode 100644 index 00000000..d069ffbd --- /dev/null +++ b/dotbot/tests/test_calibration_lighthouse2.py @@ -0,0 +1,153 @@ +"""Tests for the LH2 calibration math + persistence. + +Carried over from dotbot-lh2-calibration's tests/test_lighthouse2.py. +The original test called calculate_camera_point with positional args +matching an older signature (count1, count2, lh_index); the function +now takes an LH2Counts dataclass. Fixed during the fold; kept the +golden values. +""" + +import tomllib + +import numpy as np +import pytest + +from dotbot.calibration import lighthouse2 +from dotbot.calibration.lighthouse2 import ( + LH2Counts, + LH2Homography, + LighthouseManager, + calculate_camera_point, +) + + +def test_camera_points(): + counts = LH2Counts(lh_index=1, count1=49341, count2=85887) + x, y = calculate_camera_point(counts) + assert x == pytest.approx(-0.43435315273542) + assert y == pytest.approx(0.1512338330873567) + + +def _seed_homography(value: float) -> LH2Homography: + h = LH2Homography() + h.matrix = np.full((3, 3), value, dtype=np.float64) + return h + + +def test_save_calibration_writes_toml_and_legacy_out(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + mgr = LighthouseManager(calibration_distance=750, extra_lh_num=1) + mgr.calibration_output_path = tmp_path / lighthouse2.CALIBRATION_LEGACY_OUT + mgr.homographies = [_seed_homography(1.5), _seed_homography(2.5)] + + mgr.save_calibration() + + toml_files = list(tmp_path.glob("calibration-*.toml")) + assert len(toml_files) == 1, f"expected exactly one TOML file, got {toml_files}" + assert ( + tmp_path / "calibration.out" + ).exists(), "legacy .out should still be written" + + with open(toml_files[0], "rb") as f: + parsed = tomllib.load(f) + assert parsed["schema_version"] == lighthouse2.CALIBRATION_SCHEMA_VERSION + assert parsed["metadata"]["calibration_distance_mm"] == 750 + assert parsed["metadata"]["num_lh_stations"] == 2 + assert parsed["metadata"]["created_at"].endswith("Z") + + payload = bytes.fromhex(parsed["calibration"]["data_hex"]) + assert payload[0] == 2 + assert len(payload) == 1 + 2 * 36 + assert payload == (tmp_path / "calibration.out").read_bytes() + + +def test_save_calibration_tag_in_filename_and_metadata(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + mgr = LighthouseManager(extra_lh_num=0) + mgr.calibration_output_path = tmp_path / lighthouse2.CALIBRATION_LEGACY_OUT + mgr.homographies = [_seed_homography(1.0)] + + path = mgr.save_calibration(tag="office-2x2m") + + assert path.name.startswith("calibration-office-2x2m-") + with open(path, "rb") as f: + parsed = tomllib.load(f) + assert parsed["metadata"]["tag"] == "office-2x2m" + + +def test_save_calibration_sanitizes_and_omits_empty_tag(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + mgr = LighthouseManager(extra_lh_num=0) + mgr.calibration_output_path = tmp_path / lighthouse2.CALIBRATION_LEGACY_OUT + mgr.homographies = [_seed_homography(1.0)] + + # Unsafe characters collapse to dashes and the leading ".." is trimmed; + # the slug stays a single filename component inside ~/.dotbot. + path = mgr.save_calibration(tag="../lab room/A") + assert path.parent == tmp_path + assert path.name.startswith("calibration-lab-room-A-") + + # A tag that reduces to nothing is treated as absent (no stray dashes). + path = mgr.save_calibration(tag="///") + assert path.name.startswith("calibration-2") # the timestamp year + with open(path, "rb") as f: + assert "tag" not in tomllib.load(f)["metadata"] + + +def test_slug_tag_rules(): + assert lighthouse2._slug_tag("office-2x2m") == "office-2x2m" + assert lighthouse2._slug_tag(" a b ") == "a-b" + assert lighthouse2._slug_tag("a/b\\c:d") == "a-b-c-d" + assert lighthouse2._slug_tag("--keep_me.v2--") == "keep_me.v2" + assert lighthouse2._slug_tag("..") == "" + assert lighthouse2._slug_tag("***") == "" + + +def test_load_calibration_prefers_newest_toml(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + mgr = LighthouseManager(extra_lh_num=0) + mgr.calibration_output_path = tmp_path / lighthouse2.CALIBRATION_LEGACY_OUT + + mgr.homographies = [_seed_homography(3.0)] + mgr.save_calibration() + first = list(tmp_path.glob("calibration-*.toml"))[0] + first.stat() # touch to avoid mtime tie + import os + import time + + older = time.time() - 60 + os.utime(first, (older, older)) + + mgr.homographies = [_seed_homography(7.0)] + mgr.save_calibration() + + matrices = mgr.load_calibration() + assert len(matrices) == 1 + # The newest save wrote 7.0; legacy .out would also be 7.0 (last + # write wins), so this test specifically pins that the loader picks + # a TOML file at all by checking the matrix matches the in-memory + # value packed via homography_as_bytes. + expected = lighthouse2.homography_as_bytes(np.full((3, 3), 7.0)) + assert matrices[0] == expected + + +def test_load_calibration_falls_back_to_legacy_out(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + legacy = tmp_path / "calibration.out" + # 1 homography, all-zero matrix + legacy.write_bytes(b"\x01" + (b"\x00" * 36)) + + mgr = LighthouseManager() + mgr.calibration_output_path = legacy + matrices = mgr.load_calibration() + assert matrices == [b"\x00" * 36] + + +def test_load_calibration_rejects_unknown_schema(monkeypatch, tmp_path): + monkeypatch.setattr(lighthouse2, "CALIBRATION_DIR", tmp_path) + (tmp_path / "calibration-2099-01-01T00-00-00Z.toml").write_text( + 'schema_version = 999\n[calibration]\ndata_hex = "00"\n' + ) + mgr = LighthouseManager() + with pytest.raises(ValueError, match="schema_version 999"): + mgr.load_calibration() diff --git a/dotbot/tests/test_calibration_ota.py b/dotbot/tests/test_calibration_ota.py new file mode 100644 index 00000000..856ff958 --- /dev/null +++ b/dotbot/tests/test_calibration_ota.py @@ -0,0 +1,109 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for the over-the-air LH2 capture decoding + collection logic. + +These exercise our host-side orchestration (payload decode, trigger/wait/ +retry) with a fake client. They are NOT a substitute for hardware-in-the- +loop validation of the actual swarmit transport - the fake stands in only +for the SwarmitClient surface, never for Mari/MQTT/serial behavior. +""" + +import threading + +from dotbot.calibration.ota import ( + CaptureSession, + parse_capture_payload, +) + +_TAG = 0xCA + + +def _record(lh_index: int, count1: int, count2: int) -> bytes: + return ( + bytes([lh_index]) + count1.to_bytes(4, "little") + count2.to_bytes(4, "little") + ) + + +def _payload(*records: bytes) -> bytes: + return bytes([_TAG]) + b"".join(records) + + +def test_parse_empty_or_untagged_returns_nothing(): + assert parse_capture_payload(b"", _TAG) == [] + # A regular text log line: first byte is not the tag. + assert parse_capture_payload(b"hello world", _TAG) == [] + + +def test_parse_single_sample(): + samples = parse_capture_payload(_payload(_record(0, 49341, 85887)), _TAG) + assert len(samples) == 1 + assert samples[0].lh_index == 0 + assert samples[0].count1 == 49341 + assert samples[0].count2 == 85887 + + +def test_parse_multiple_samples(): + samples = parse_capture_payload( + _payload(_record(0, 1, 2), _record(1, 3, 4), _record(2, 5, 6)), + _TAG, + ) + assert [(s.lh_index, s.count1, s.count2) for s in samples] == [ + (0, 1, 2), + (1, 3, 4), + (2, 5, 6), + ] + + +def test_parse_ignores_trailing_partial_record(): + # Tag + one full 9-byte record + 3 stray bytes that can't form a record. + data = _payload(_record(0, 7, 8)) + b"\x01\x02\x03" + samples = parse_capture_payload(data, _TAG) + assert len(samples) == 1 + assert (samples[0].count1, samples[0].count2) == (7, 8) + + +class _FakeClient: + """Minimal SwarmitClient stand-in: emits one tagged event per trigger. + + Mirrors the real firmware contract (samples only arrive in reply to a + capture request), so CaptureSession's drain-then-trigger ordering is + exercised the same way it is against a bot. + """ + + def __init__(self, device: str, records: bytes): + self._device = device.upper() + self._records = records + self._triggered = threading.Event() + + def request_lh2_capture(self, device: str) -> None: + self._triggered.set() + + def watch_log_events(self): + while True: + if self._triggered.wait(timeout=0.05): + self._triggered.clear() + yield { + "addr": self._device, + "data_hex": _payload(self._records).hex(), + } + + +def test_capture_session_returns_triggered_sample(): + client = _FakeClient("ABCD", _record(0, 111, 222)) + with CaptureSession(client, "abcd", _TAG) as session: + sample = session.capture(lh_index=0, timeout=2.0, retries=2) + assert sample.lh_index == 0 + assert (sample.count1, sample.count2) == (111, 222) + + +def test_capture_session_ignores_other_devices(): + # Event addressed to a different bot must not satisfy the capture. + client = _FakeClient("FFFF", _record(0, 1, 2)) + with CaptureSession(client, "ABCD", _TAG) as session: + try: + session.capture(lh_index=0, timeout=0.3, retries=0) + except TimeoutError: + pass + else: + raise AssertionError("expected TimeoutError for mismatched addr") diff --git a/dotbot/tests/test_cli_cfg_helper.py b/dotbot/tests/test_cli_cfg_helper.py new file mode 100644 index 00000000..9e214a80 --- /dev/null +++ b/dotbot/tests/test_cli_cfg_helper.py @@ -0,0 +1,137 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for the `from_config` option/config bridge (Phase 3). + +`from_config` decides, per Click option, whether the value came from the +command line (user wins) or should fall through the config resolver +(config > env > the option's default). These tests drive it through a tiny +throwaway Click command so the parameter-source machinery is exercised for +real, plus one integration check via `dotbot fw artifacts --print-path` that +a config-set `[fw].board` reaches the printed artifact path. +""" + +from pathlib import Path + +import click +import pytest +from click.testing import CliRunner + +from dotbot.cli._cfg import from_config +from dotbot.cli.fw import cmd as fw_cmd +from dotbot.config import DotbotConfig + + +@pytest.fixture +def runner(): + return CliRunner() + + +def _probe_command(): + """A throwaway command whose single option reads through `from_config`.""" + + @click.command() + @click.option("--board", "-b", default="dotbot-v3") + @click.pass_context + def probe(ctx, board): + resolved = from_config(ctx, "board", "board", "fw") + click.echo(resolved) + + return probe + + +def test_flag_on_commandline_wins_over_config(runner): + """An explicit `--board` beats a config that sets `[fw].board`.""" + cfg = DotbotConfig.model_validate({"fw": {"board": "from-config"}}) + result = runner.invoke( + _probe_command(), + ["--board", "from-flag"], + obj={"config": cfg, "deployment": None}, + ) + assert result.exit_code == 0, result.output + assert result.output.strip() == "from-flag" + + +def test_no_flag_falls_to_config(runner): + """No `--board` on the command line -> the config value is used.""" + cfg = DotbotConfig.model_validate({"fw": {"board": "from-config"}}) + result = runner.invoke( + _probe_command(), + [], + obj={"config": cfg, "deployment": None}, + ) + assert result.exit_code == 0, result.output + assert result.output.strip() == "from-config" + + +def test_no_config_falls_to_option_default(runner): + """No flag and no config -> the option's own default flows through.""" + result = runner.invoke(_probe_command(), [], obj={"config": DotbotConfig()}) + assert result.exit_code == 0, result.output + assert result.output.strip() == "dotbot-v3" + + +def test_no_ctx_obj_falls_to_option_default(runner): + """`ctx.obj` is None when a command runs without the root group -> default.""" + result = runner.invoke(_probe_command(), []) + assert result.exit_code == 0, result.output + assert result.output.strip() == "dotbot-v3" + + +def test_env_beats_config(runner, monkeypatch): + """Env var (`DOTBOT_FW_BOARD`) beats the file layer, loses to the flag.""" + monkeypatch.setenv("DOTBOT_FW_BOARD", "from-env") + cfg = DotbotConfig.model_validate({"fw": {"board": "from-config"}}) + result = runner.invoke( + _probe_command(), [], obj={"config": cfg, "deployment": None} + ) + assert result.exit_code == 0, result.output + assert result.output.strip() == "from-env" + + +# --- integration: config-set [fw].board reaches the artifact path ----------- + + +@pytest.fixture +def fake_firmware_repo(tmp_path, monkeypatch): + """Point `DOTBOT_FIRMWARE_REPO` at a tmp dir with a Makefile so + `artifact_path` can resolve a repo without a real DotBot-firmware clone.""" + repo = tmp_path / "fake-dotbot-firmware" + repo.mkdir() + (repo / "Makefile").write_text("# fake\n") + monkeypatch.setenv("DOTBOT_FIRMWARE_REPO", str(repo)) + return repo + + +def test_fw_artifacts_print_path_reflects_config_board(runner, fake_firmware_repo): + """`fw artifacts --print-path --app dotbot` with `-t` omitted uses the + config-set `[fw].board` in the printed path; `-t` overrides it.""" + cfg = DotbotConfig.model_validate({"fw": {"board": "nrf5340dk-app"}}) + + # -t omitted: the config board lands in the path. + from_cfg = runner.invoke( + fw_cmd, + ["artifacts", "--print-path", "--app", "dotbot"], + obj={"config": cfg, "deployment": None}, + ) + assert from_cfg.exit_code == 0, from_cfg.output + expected = str( + Path("Output") + / "nrf5340dk-app" + / "Release" + / "Exe" + / "dotbot-nrf5340dk-app.hex" + ) + assert from_cfg.output.strip().endswith(expected) + + # -t overrides the config board. + overridden = runner.invoke( + fw_cmd, + ["artifacts", "--print-path", "--app", "dotbot", "-t", "dotbot-v3"], + obj={"config": cfg, "deployment": None}, + ) + assert overridden.exit_code == 0, overridden.output + expected_override = str( + Path("Output") / "dotbot-v3" / "Release" / "Exe" / "dotbot-dotbot-v3.hex" + ) + assert overridden.output.strip().endswith(expected_override) diff --git a/dotbot/tests/test_cli_config.py b/dotbot/tests/test_cli_config.py new file mode 100644 index 00000000..34ae5618 --- /dev/null +++ b/dotbot/tests/test_cli_config.py @@ -0,0 +1,87 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Phase-2 wiring: the root `-c/--config` + `--deployment` flags, and the +`fw`/`device` `--config` -> `--build-config` rename. Headless (CliRunner).""" + +import pytest +from click.testing import CliRunner + +from dotbot.cli.main import cli + + +@pytest.fixture +def runner(): + return CliRunner() + + +def _write(tmp_path, text): + path = tmp_path / "dotbot.toml" + path.write_text(text) + return path + + +# --- root config loading ---------------------------------------------------- + + +def test_root_accepts_valid_config(runner, tmp_path): + cfg = _write( + tmp_path, 'swarm_id = "0001"\n[deployment.inria]\nconn = "simulator"\n' + ) + result = runner.invoke(cli, ["-c", str(cfg), "fw", "--help"]) + assert result.exit_code == 0, result.output + + +def test_root_bad_config_errors(runner, tmp_path): + cfg = _write(tmp_path, 'swrm_id = "x"\n') # unknown key -> extra=forbid + result = runner.invoke(cli, ["-c", str(cfg), "fw", "--help"]) + assert result.exit_code != 0 + assert "config" in result.output.lower() + + +def test_root_missing_config_errors(runner, tmp_path): + result = runner.invoke(cli, ["-c", str(tmp_path / "nope.toml"), "fw", "--help"]) + assert result.exit_code != 0 + + +def test_root_selects_deployment(runner, tmp_path): + cfg = _write(tmp_path, '[deployment.inria]\nconn = "simulator"\n') + result = runner.invoke( + cli, ["-c", str(cfg), "--deployment", "inria", "fw", "--help"] + ) + assert result.exit_code == 0, result.output + + +def test_root_unknown_deployment_errors(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["--deployment", "nope", "fw", "--help"]) + assert result.exit_code != 0 + assert "deployment" in result.output.lower() + + +def test_root_no_config_is_fine(runner): + # No -c, no dotbot.toml, user-file fallback off -> empty config, no error. + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["fw", "--help"]) + assert result.exit_code == 0, result.output + + +# --- build-config rename ---------------------------------------------------- + + +def test_fw_build_uses_build_config(runner): + result = runner.invoke(cli, ["fw", "build", "--help"]) + assert result.exit_code == 0 + assert "--build-config" in result.output + + +def test_fw_build_rejects_old_short_flag(runner): + # Clean break: `-c` no longer sets the build config (it's the root flag now). + result = runner.invoke(cli, ["fw", "build", "-c", "Debug"]) + assert result.exit_code != 0 + + +def test_device_flash_uses_build_config(runner): + result = runner.invoke(cli, ["device", "flash", "--help"]) + assert result.exit_code == 0 + assert "--build-config" in result.output diff --git a/dotbot/tests/test_cli_deployment_fetch.py b/dotbot/tests/test_cli_deployment_fetch.py new file mode 100644 index 00000000..4b8be888 --- /dev/null +++ b/dotbot/tests/test_cli_deployment_fetch.py @@ -0,0 +1,186 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot deployment fetch` - pull published `[deployment.*]` tables and merge. + +Headless: SOURCE is given as a local file path (the command reads a path or an +http(s) URL), so no network is touched. The merge/diff/confirm/comment-preserving +logic is exercised end to end through the root group. +""" + +import pytest +from click.testing import CliRunner + +import dotbot.cli.deployment_cmd as dcmd +import dotbot.config as cfg +from dotbot.cli.main import cli + +_REGISTRY = """\ +[deployment.inria] +conn = "mqtts://broker.inria:8883" +swarm_id = "0001" +location = "Inria Paris" + +[deployment.bench] +conn = "simulator" +""" + + +@pytest.fixture +def runner(): + return CliRunner() + + +def _registry(tmp_path, text=_REGISTRY): + path = tmp_path / "deployments.toml" + path.write_text(text) + return path + + +def _user_config(tmp_path, monkeypatch, text=None): + """Point the user config at a tmp path; optionally seed it.""" + target = tmp_path / "home" / ".dotbot" / "config.toml" + monkeypatch.setattr(dcmd, "USER_CONFIG_PATH", target) + if text is not None: + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(text) + return target + + +def test_fetch_adds_into_user_config(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + target = _user_config(tmp_path, monkeypatch) + result = runner.invoke(cli, ["deployment", "fetch", str(reg)]) + assert result.exit_code == 0, result.output + assert "+ inria" in result.output + assert "+ bench" in result.output + loaded = cfg.load_config(target) + assert set(loaded.deployment) == {"inria", "bench"} + assert loaded.deployment["inria"].conn == "mqtts://broker.inria:8883" + + +def test_fetch_no_source_uses_default_registry_url(runner, tmp_path, monkeypatch): + _user_config(tmp_path, monkeypatch) + seen = {} + + def fake_read(source): + seen["url"] = source + return _REGISTRY + + monkeypatch.setattr(dcmd, "_read_source", fake_read) + result = runner.invoke(cli, ["deployment", "fetch"]) + assert result.exit_code == 0, result.output + assert seen["url"] == dcmd._DEFAULT_REGISTRY_URL + + +def test_fetch_into_project_writes_local_dotbot_toml(runner, tmp_path): + reg = _registry(tmp_path) + with runner.isolated_filesystem(): + result = runner.invoke( + cli, ["deployment", "fetch", str(reg), "--into", "project"] + ) + assert result.exit_code == 0, result.output + from pathlib import Path + + written = Path("dotbot.toml") + assert written.is_file() + assert set(cfg.load_config(written).deployment) == {"inria", "bench"} + + +def test_fetch_dry_run_writes_nothing(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + target = _user_config(tmp_path, monkeypatch) + result = runner.invoke(cli, ["deployment", "fetch", str(reg), "--dry-run"]) + assert result.exit_code == 0, result.output + assert "+ inria" in result.output + assert "dry run" in result.output + assert not target.exists() + + +def test_fetch_idempotent_reports_same(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + _user_config(tmp_path, monkeypatch) + runner.invoke(cli, ["deployment", "fetch", str(reg)]) + again = runner.invoke(cli, ["deployment", "fetch", str(reg)]) + assert again.exit_code == 0, again.output + assert "= inria" in again.output + assert "up to date" in again.output.lower() + + +def test_fetch_changed_prompts_and_aborts_on_no(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + # Seed the user file with a DIFFERENT inria conn -> a "changed" entry. + target = _user_config( + tmp_path, + monkeypatch, + text='[deployment.inria]\nconn = "mqtts://old:8883"\n', + ) + result = runner.invoke(cli, ["deployment", "fetch", str(reg)], input="n\n") + assert result.exit_code != 0 # aborted + assert "~ inria" in result.output + # File untouched: the old conn is still there. + assert cfg.load_config(target).deployment["inria"].conn == "mqtts://old:8883" + + +def test_fetch_changed_with_yes_replaces(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + target = _user_config( + tmp_path, + monkeypatch, + text='[deployment.inria]\nconn = "mqtts://old:8883"\n', + ) + result = runner.invoke(cli, ["deployment", "fetch", str(reg), "--yes"]) + assert result.exit_code == 0, result.output + assert ( + cfg.load_config(target).deployment["inria"].conn == "mqtts://broker.inria:8883" + ) + + +def test_fetch_preserves_comments_and_other_content(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path) + seed = ( + "# my notes\n" + 'log_level = "debug"\n' + "\n" + "[fw]\n" + 'board = "dotbot-v3"\n' + "\n" + "[deployment.local]\n" + 'conn = "simulator"\n' + ) + target = _user_config(tmp_path, monkeypatch, text=seed) + result = runner.invoke(cli, ["deployment", "fetch", str(reg), "--yes"]) + assert result.exit_code == 0, result.output + written = target.read_text() + assert "# my notes" in written # comment survives + assert "[fw]" in written # other section survives + loaded = cfg.load_config(target) + assert loaded.fw.board == "dotbot-v3" + assert loaded.log_level == "debug" + # local kept, inria/bench added + assert set(loaded.deployment) == {"local", "inria", "bench"} + + +def test_fetch_rejects_invalid_fragment(runner, tmp_path, monkeypatch): + # Unknown key -> extra='forbid' -> validation error before any write. + reg = _registry(tmp_path, text="[deployment.x]\nbogus_key = 1\n") + target = _user_config(tmp_path, monkeypatch) + result = runner.invoke(cli, ["deployment", "fetch", str(reg)]) + assert result.exit_code != 0 + assert "invalid config" in result.output.lower() + assert not target.exists() + + +def test_fetch_rejects_fragment_without_deployments(runner, tmp_path, monkeypatch): + reg = _registry(tmp_path, text='[fw]\nboard = "dotbot-v3"\n') + _user_config(tmp_path, monkeypatch) + result = runner.invoke(cli, ["deployment", "fetch", str(reg)]) + assert result.exit_code != 0 + assert "no [deployment" in result.output + + +def test_fetch_rejects_bad_source(runner, tmp_path, monkeypatch): + _user_config(tmp_path, monkeypatch) + result = runner.invoke(cli, ["deployment", "fetch", "not-a-file-or-url"]) + assert result.exit_code != 0 + assert "not a URL or an existing file" in result.output diff --git a/dotbot/tests/test_cli_dispatcher.py b/dotbot/tests/test_cli_dispatcher.py new file mode 100644 index 00000000..338ad9fd --- /dev/null +++ b/dotbot/tests/test_cli_dispatcher.py @@ -0,0 +1,340 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for the `dotbot` CLI dispatcher. + +Goal: lock the discovery surface (the four top-level groups + the `run` +process group + --help) so a future refactor doesn't silently drop a +command. We're NOT testing the underlying subcommand behavior here — that +lives in each subcommand's own test module (test_controller_app.py etc.). +""" + +import os +import subprocess +import sys + +import pytest +from click.testing import CliRunner + +from dotbot.cli import _lazy +from dotbot.cli.main import _SUBCOMMANDS, cli +from dotbot.cli.run import _RUN_SUBCOMMANDS + +# Importing dotbot.controller (transitively, dotbot.server) blows up at +# module-import time if the React UI hasn't been built — FastAPI's +# StaticFiles mount asserts the directory exists. That's a pre-existing +# import-time side effect, not something the CLI scaffold introduced. +# Skip the subcommands whose lazy import triggers it when the bundle +# isn't built (typical for fresh editable installs). +_FRONTEND_BUILD = os.path.join( + os.path.dirname(os.path.dirname(os.path.abspath(__file__))), + "frontend", + "build", +) +_FRONTEND_PRESENT = os.path.isdir(_FRONTEND_BUILD) + + +# The top level is the four object-namespaces plus the read-only +# management commands (config, deployment). +EXPECTED_SUBCOMMANDS = { + "fw", + "device", + "swarm", + "run", + "config", + "deployment", +} + +# `run` groups the host-side processes (the former flat top-level verbs). +EXPECTED_RUN_SUBCOMMANDS = { + "controller", + "gateway", + "simulator", + "lh2-calibration", + "demo", + "keyboard", + "joystick", +} + +# Top-level groups whose --help backend lives in OTHER packages with their +# own protocol registries (swarmit). When pytest pre-loads dotbot.protocol +# via test_controller etc., importing swarmit in the same process triggers +# a duplicate payload-type registration (ValueError 0x81 already +# registered). This is the known cross-package protocol duplication +# captured in the consolidation roadmap §1; it never happens in real +# `dotbot ` invocations (each shell run is a fresh process). We verify +# these in a subprocess. +_CROSS_PACKAGE_SUBS = {"swarm"} + +# `run` subcommands whose lazy import is hostile to an in-process headless +# test: keyboard/joystick import pygame/pynput at module load; +# controller/simulator trigger dotbot.server's StaticFiles import-time mount. +_TELEOP_SUBS = {"keyboard", "joystick"} +_FRONTEND_DEPENDENT = {"controller", "simulator"} + + +@pytest.fixture +def runner(): + return CliRunner() + + +def test_root_help_lists_the_four_namespaces(runner): + result = runner.invoke(cli, ["--help"]) + assert result.exit_code == 0, result.output + # Check the rendered "Commands:" section specifically — the root help + # prose also names the four groups, so a bare full-output substring + # check would pass even if a command were dropped from the list. + commands = result.output.split("Commands:", 1)[1] + for name in EXPECTED_SUBCOMMANDS: + assert name in commands, f"namespace `{name}` missing from rendered list" + + +def test_subcommand_table_matches_expected_set(): + """The static top-level `_SUBCOMMANDS` tuple is the wiring contract.""" + declared = {name for name, _, _ in _SUBCOMMANDS} + assert declared == EXPECTED_SUBCOMMANDS + + +def test_run_help_lists_every_process(runner): + result = runner.invoke(cli, ["run", "--help"]) + assert result.exit_code == 0, result.output + # Same as the root: assert against the rendered command list, not the + # prose (which contains "controller"/"gateway"/"simulator"/"demo" as words). + commands = result.output.split("Commands:", 1)[1] + for name in EXPECTED_RUN_SUBCOMMANDS: + assert name in commands, f"`run {name}` missing from rendered list" + + +def test_run_subcommand_table_matches_expected_set(): + """The static `_RUN_SUBCOMMANDS` tuple is the `run`-group contract.""" + declared = {name for name, _, _ in _RUN_SUBCOMMANDS} + assert declared == EXPECTED_RUN_SUBCOMMANDS + + +def test_no_flat_process_verbs_at_top_level(runner): + """The host-process verbs must NOT be reachable at the top level — + they moved under `run`. This is the regression guard for the reorg.""" + for name in EXPECTED_RUN_SUBCOMMANDS: + result = runner.invoke(cli, [name, "--help"]) + assert result.exit_code != 0, f"`dotbot {name}` should no longer exist" + # Assert it failed because the command is GONE, not because a + # re-added verb's backend errored at import (which also exits != 0). + assert ( + "No such command" in result.output + ), f"`dotbot {name}` failed for the wrong reason:\n{result.output}" + + +def test_version_flag(runner): + result = runner.invoke(cli, ["--version"]) + assert result.exit_code == 0 + assert "dotbot" in result.output + + +@pytest.mark.parametrize( + "subcommand", + sorted(EXPECTED_SUBCOMMANDS - _CROSS_PACKAGE_SUBS), +) +def test_top_level_group_help_works(runner, subcommand): + """Every in-process top-level group's --help runs cleanly. + + swarm is excluded (its swarmit backend collides with PyDotBot's + protocol registry inside a single pytest process — covered separately + by the subprocess test). fw/device/run import no heavy backend at + --help time. + """ + result = runner.invoke(cli, [subcommand, "--help"]) + assert result.exit_code == 0, result.output + + +@pytest.mark.parametrize( + "subcommand", + sorted(EXPECTED_RUN_SUBCOMMANDS - _TELEOP_SUBS), +) +def test_run_subcommand_help_works(runner, subcommand): + """Every in-process `run` subcommand's --help runs cleanly. + + keyboard/joystick are excluded because they import pygame/pynput at + module load time (headless-CI hostile). controller/sim trigger + dotbot.server's StaticFiles import-time mount; skipped if the frontend + bundle hasn't been built. + """ + if subcommand in _FRONTEND_DEPENDENT and not _FRONTEND_PRESENT: + pytest.skip( + "frontend bundle missing; run `cd dotbot/frontend && npm run build`" + ) + result = runner.invoke(cli, ["run", subcommand, "--help"]) + assert result.exit_code == 0, result.output + + +@pytest.mark.parametrize("subcommand", sorted(_CROSS_PACKAGE_SUBS)) +def test_cross_package_subcommand_help_works(subcommand): + """`dotbot swarm --help` in a clean process. + + A subprocess avoids the swarmit vs PyDotBot protocol-registry + collision that only manifests inside pytest's shared-process test + session. See _CROSS_PACKAGE_SUBS comment above. + """ + result = subprocess.run( + [sys.executable, "-m", "dotbot.cli", subcommand, "--help"], + capture_output=True, + text=True, + timeout=30, + ) + assert result.returncode == 0, f"stdout:\n{result.stdout}\nstderr:\n{result.stderr}" + # Sanity: the help text should mention the subcommand or its purpose. + combined = result.stdout + result.stderr + assert "Usage" in combined + + +def test_run_help_does_not_import_controller_app(): + """`dotbot run --help` must NOT eagerly import the heavy controller + backend — that's the whole point of the lazy `run` group. + + Run in a fresh subprocess so sys.modules is clean (other test modules + in the shared pytest process may already have imported it). + """ + code = ( + "import sys;" + "from click.testing import CliRunner;" + "from dotbot.cli.main import cli;" + "r = CliRunner().invoke(cli, ['run', '--help']);" + "assert r.exit_code == 0, r.output;" + "heavy = [m for m in ('dotbot.controller_app','dotbot.server'," + "'pygame','pynput','dotbot.calibration') if m in sys.modules];" + "assert not heavy, f'run --help eagerly imported: {heavy}';" + "print('OK')" + ) + result = subprocess.run( + [sys.executable, "-c", code], capture_output=True, text=True, timeout=30 + ) + assert result.returncode == 0, f"stdout:\n{result.stdout}\nstderr:\n{result.stderr}" + + +def test_fw_make_is_mounted_under_fw(runner): + """The make escape hatch lives at `dotbot fw make`, not top-level.""" + result = runner.invoke(cli, ["fw", "make", "--help"]) + assert result.exit_code == 0, result.output + assert "SEGGER_DIR" in result.output + # And it is gone from the top level. + assert runner.invoke(cli, ["make", "--help"]).exit_code != 0 + + +def test_fw_mock_exits_nonzero(runner): + """fw stubs must surface that they're not implemented (exit 2).""" + result = runner.invoke(cli, ["fw", "new", "myapp"]) + assert result.exit_code == 2 + assert "not implemented" in result.output.lower() + + +def test_demo_list(runner): + """`dotbot run demo --list` enumerates demos including `qr`.""" + result = runner.invoke(cli, ["run", "demo", "--list"]) + assert result.exit_code == 0 + assert "qr" in result.output + + +def test_demo_default_lists(runner): + """`dotbot run demo` with no subcommand also lists (discoverability).""" + result = runner.invoke(cli, ["run", "demo"]) + assert result.exit_code == 0 + assert "qr" in result.output + + +def test_lazy_subcommand_missing_extra_exits_with_hint(): + """A subcommand whose backend isn't installed prints an install hint.""" + + def loader(): + raise ImportError("simulated missing dep") + + stub = _lazy.lazy_subcommand( + name="fake", + extra="fake-extra", + package="fake-pkg", + help="A fake subcommand for the test.", + loader=loader, + ) + + runner = CliRunner() + result = runner.invoke(stub, []) + assert result.exit_code == 1 + assert "pip install dotbot[fake-extra]" in result.output + assert "fake-pkg" in result.output + + +def test_python_m_dotbot_cli_entrypoint(runner): + """`python -m dotbot.cli` must dispatch through the same group.""" + # In-process check that the __main__ module routes to the same group. + from dotbot.cli import __main__ as cli_main_module + + assert cli_main_module.cli is cli + + +def test_python_m_dotbot_cli_help_subprocess(): + """End-to-end: `python -m dotbot.cli --help` runs in a fresh process.""" + result = subprocess.run( + [sys.executable, "-m", "dotbot.cli", "--help"], + capture_output=True, + text=True, + timeout=30, + ) + assert result.returncode == 0, f"stdout:\n{result.stdout}\nstderr:\n{result.stderr}" + for name in EXPECTED_SUBCOMMANDS: + assert name in result.stdout, f"`{name}` missing from `python -m` help" + + +def test_python_m_dotbot_cli_version_subprocess(): + """End-to-end: `python -m dotbot.cli --version` prints a version line.""" + result = subprocess.run( + [sys.executable, "-m", "dotbot.cli", "--version"], + capture_output=True, + text=True, + timeout=30, + ) + assert result.returncode == 0, f"stdout:\n{result.stdout}\nstderr:\n{result.stderr}" + assert "dotbot" in result.stdout + + +def test_lh2_calibration_missing_extras_prints_hint(runner, monkeypatch): + """When [calibrate] extras aren't installed, `dotbot run lh2-calibration` + (default `collect`) exits 1 with a pip-install hint instead of a + traceback.""" + # Simulate the dotbot.calibration.cli module being unavailable. + # `monkeypatch.setitem(sys.modules, name, None)` makes + # `from name import ...` raise ImportError per CPython's import + # protocol — same condition as a real missing extra. + monkeypatch.setitem(sys.modules, "dotbot.calibration.cli", None) + result = runner.invoke(cli, ["run", "lh2-calibration"]) + assert result.exit_code == 1, result.output + assert "pip install dotbot[calibrate]" in result.output + + +def test_lh2_calibration_collect_missing_extras_prints_hint(runner, monkeypatch): + """`dotbot run lh2-calibration collect` is the explicit alias for the + default; same install-hint fallback when extras are missing.""" + monkeypatch.setitem(sys.modules, "dotbot.calibration.cli", None) + result = runner.invoke(cli, ["run", "lh2-calibration", "collect"]) + assert result.exit_code == 1, result.output + assert "pip install dotbot[calibrate]" in result.output + + +def test_lh2_calibration_apply_missing_extras_prints_hint(runner, monkeypatch): + """`dotbot run lh2-calibration apply` falls back to the install hint + when the calibration runtime deps aren't available.""" + monkeypatch.setitem(sys.modules, "dotbot.calibration.exporter", None) + monkeypatch.setitem(sys.modules, "dotbot.calibration.lighthouse2", None) + result = runner.invoke(cli, ["run", "lh2-calibration", "apply", "/tmp/lh2.h"]) + assert result.exit_code == 1, result.output + assert "pip install dotbot[calibrate]" in result.output + + +def test_lh2_calibration_apply_no_saved_calibration(runner, tmp_path, monkeypatch): + """`apply` exits 1 with a clear message when no saved calibration + exists at the expected location.""" + # Point LighthouseManager at an empty tmp dir so load_calibration + # finds nothing. + monkeypatch.setattr("dotbot.calibration.lighthouse2.CALIBRATION_DIR", tmp_path) + result = runner.invoke( + cli, ["run", "lh2-calibration", "apply", str(tmp_path / "out.h")] + ) + assert result.exit_code == 1, result.output + assert "No saved calibration" in result.output diff --git a/dotbot/tests/test_cli_helpers.py b/dotbot/tests/test_cli_helpers.py new file mode 100644 index 00000000..c7f271eb --- /dev/null +++ b/dotbot/tests/test_cli_helpers.py @@ -0,0 +1,290 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Phase-5 management commands: `dotbot config` + `dotbot deployment`. + +Read-only inspectors over the config the root group already resolved onto +`ctx.obj`. Headless (CliRunner), invoked through the root so the context is +populated (a bare `runner.invoke(show)` would have `ctx.obj is None`). +""" + +from pathlib import Path + +import pytest +from click.testing import CliRunner + +import dotbot.config as cfg +from dotbot.cli.main import cli + +# A small config with two named deployments and a default selection. +_CONFIG = """\ +default_deployment = "inria" +swarm_id = "0001" +conn = "simulator" + +[fw] +board = "dotbot-v3" + +[deployment.inria] +conn = "simulator" +swarm_id = "0001" +location = "Inria Paris" +bots = 100 + +[deployment.laposte] +conn = "mqtts://broker.local:8883" +location = "La Poste" +bots = 1000 +""" + + +@pytest.fixture +def runner(): + return CliRunner() + + +def _write(tmp_path, text=_CONFIG): + path = tmp_path / "dotbot.toml" + path.write_text(text) + return path + + +# --- config path ------------------------------------------------------------ + + +def test_config_path_with_config(runner, tmp_path): + cfg = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg), "config", "path"]) + assert result.exit_code == 0, result.output + assert str(cfg) in result.output + + +def test_config_path_without_config(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "path"]) + assert result.exit_code == 0, result.output + assert "none" in result.output.lower() + assert "built-in defaults" in result.output + + +# --- config show ------------------------------------------------------------ + + +def test_config_show_with_config(runner, tmp_path): + cfg = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg), "config", "show"]) + assert result.exit_code == 0, result.output + assert str(cfg) in result.output + # The selected (default) deployment is reported. + assert "inria" in result.output + # A top-level scalar and a nested section both render. + assert "swarm_id" in result.output + assert "[fw]" in result.output + assert "board" in result.output + + +def test_config_show_skips_none_values(runner, tmp_path): + cfg = _write(tmp_path, 'swarm_id = "0001"\n') + result = runner.invoke(cli, ["-c", str(cfg), "config", "show"]) + assert result.exit_code == 0, result.output + # `log_level` is unset (None) and must not appear. + assert "log_level" not in result.output + + +def test_config_show_without_config(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "show"]) + assert result.exit_code == 0, result.output + assert "(none)" in result.output # no deployment selected + assert "built-in defaults" in result.output + + +# --- deployment list ----------------------------------------------------------- + + +def test_deployment_list_shows_names_and_active_marker(runner, tmp_path): + cfg = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg), "deployment", "list"]) + assert result.exit_code == 0, result.output + assert "inria" in result.output + assert "laposte" in result.output + # The active (default_deployment) one is marked with `*`. + active_line = next(line for line in result.output.splitlines() if "inria" in line) + assert active_line.lstrip().startswith("*") + # Descriptive fields render. + assert "Inria Paris" in result.output + assert "1000" in result.output + + +def test_deployment_list_honors_deployment_flag(runner, tmp_path): + cfg = _write(tmp_path) + result = runner.invoke( + cli, ["-c", str(cfg), "--deployment", "laposte", "deployment", "list"] + ) + assert result.exit_code == 0, result.output + active_line = next(line for line in result.output.splitlines() if "laposte" in line) + assert active_line.lstrip().startswith("*") + + +def test_deployment_list_empty(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["deployment", "list"]) + assert result.exit_code == 0, result.output + assert "no deployments configured" in result.output.lower() + + +# --- deployment show ----------------------------------------------------------- + + +def test_deployment_show_known(runner, tmp_path): + cfg = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg), "deployment", "show", "inria"]) + assert result.exit_code == 0, result.output + assert "inria" in result.output + assert "Inria Paris" in result.output + assert "conn" in result.output + # It is the active deployment. + assert "active" in result.output + + +def test_deployment_show_unknown_errors(runner, tmp_path): + cfg_file = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "show", "nope"]) + assert result.exit_code != 0 + assert "nope" in result.output + # Lists the defined deployments in the error. + assert "inria" in result.output + + +# --- deployment use ---------------------------------------------------------- + + +def test_deployment_use_sets_default(runner, tmp_path): + # _CONFIG defaults to "inria"; switch it to "laposte". + cfg_file = _write(tmp_path) + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "use", "laposte"]) + assert result.exit_code == 0, result.output + assert "laposte" in result.output + assert cfg.load_config(cfg_file).default_deployment == "laposte" + + +def test_deployment_use_preserves_comments(runner, tmp_path): + text = ( + "# my deployments\n" + '# default_deployment = "old"\n' + "\n" + "[deployment.inria]\n" + 'conn = "simulator"\n' + ) + cfg_file = _write(tmp_path, text) + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "use", "inria"]) + assert result.exit_code == 0, result.output + written = cfg_file.read_text() + assert "# my deployments" in written # comment survives + assert 'default_deployment = "inria"' in written + assert cfg.load_config(cfg_file).default_deployment == "inria" + + +def test_deployment_use_inserts_when_absent(runner, tmp_path): + # No default_deployment line at all -> the key is inserted before the table. + text = '[deployment.inria]\nconn = "simulator"\n' + cfg_file = _write(tmp_path, text) + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "use", "inria"]) + assert result.exit_code == 0, result.output + assert cfg.load_config(cfg_file).default_deployment == "inria" + + +def test_deployment_use_unknown_leaves_file_untouched(runner, tmp_path): + cfg_file = _write(tmp_path) + before = cfg_file.read_text() + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "use", "nope"]) + assert result.exit_code != 0 + assert "nope" in result.output + assert "inria" in result.output # lists known deployments + assert cfg_file.read_text() == before + + +def test_deployment_use_without_config_hints_init(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["deployment", "use", "inria"]) + assert result.exit_code != 0 + assert "config init" in result.output + + +def test_deployment_use_then_list_marks_it_active(runner, tmp_path): + cfg_file = _write(tmp_path) + runner.invoke(cli, ["-c", str(cfg_file), "deployment", "use", "laposte"]) + result = runner.invoke(cli, ["-c", str(cfg_file), "deployment", "list"]) + active_line = next(line for line in result.output.splitlines() if "laposte" in line) + assert active_line.lstrip().startswith("*") + + +# --- config init ------------------------------------------------------------ + + +def test_config_init_writes_valid_starter(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "init"]) + assert result.exit_code == 0, result.output + written = Path("dotbot.toml") + assert written.is_file() + # The starter is all-commented, so it loads as a valid empty config. + loaded = cfg.load_config(written) + assert loaded.conn is None + assert loaded.deployment == {} + + +def test_config_init_refuses_overwrite_without_force(runner): + with runner.isolated_filesystem(): + assert runner.invoke(cli, ["config", "init"]).exit_code == 0 + again = runner.invoke(cli, ["config", "init"]) + assert again.exit_code != 0 + assert "already exists" in again.output + forced = runner.invoke(cli, ["config", "init", "--force"]) + assert forced.exit_code == 0, forced.output + + +def test_config_init_global(runner, tmp_path, monkeypatch): + import dotbot.cli.config_cmd as ccmd + + user = tmp_path / "home" / ".dotbot" / "config.toml" + monkeypatch.setattr(ccmd, "USER_CONFIG_PATH", user) + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "init", "--global"]) + assert result.exit_code == 0, result.output + assert user.is_file() + + +def test_config_show_without_config_hints_init(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "show"]) + assert "config init" in result.output + + +def test_config_init_prefills_conn_and_swarm_id(runner): + with runner.isolated_filesystem(): + result = runner.invoke( + cli, + ["config", "init", "--conn", "mqtts://broker:8883", "--swarm-id", "0001"], + ) + assert result.exit_code == 0, result.output + loaded = cfg.load_config(Path("dotbot.toml")) + assert loaded.conn == "mqtts://broker:8883" + assert loaded.swarm_id == "0001" + + +def test_config_init_conn_only_leaves_swarm_id_unset(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "init", "--conn", "simulator"]) + assert result.exit_code == 0, result.output + loaded = cfg.load_config(Path("dotbot.toml")) + assert loaded.conn == "simulator" + assert loaded.swarm_id is None + + +def test_config_init_rejects_bad_conn(runner): + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["config", "init", "--conn", "http://nope"]) + assert result.exit_code != 0 + assert "invalid --conn" in result.output + assert not Path("dotbot.toml").exists() diff --git a/dotbot/tests/test_cli_swarm_inject.py b/dotbot/tests/test_cli_swarm_inject.py new file mode 100644 index 00000000..ab053f73 --- /dev/null +++ b/dotbot/tests/test_cli_swarm_inject.py @@ -0,0 +1,78 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""`dotbot swarm` config -> swarmit flag injection. + +Tests the pure helper (`_swarm_inject`) so swarmit itself is never imported - +its protocol registry collides with PyDotBot's in a shared test process (the +full `dotbot swarm` invocation is covered by the subprocess test in +`test_cli_dispatcher`). +""" + +import pytest + +from dotbot.cli._swarm_inject import inject_config +from dotbot.config import DotbotConfig + + +@pytest.fixture(autouse=True) +def _clean_conn_env(monkeypatch): + # The resolver also reads env; clear the swarm/conn vars for determinism. + for var in ( + "DOTBOT_CONN", + "DOTBOT_SWARM_CONN", + "DOTBOT_SWARM_ID", + "DOTBOT_SWARM_SWARM_ID", + ): + monkeypatch.delenv(var, raising=False) + + +def _obj(**kw): + return {"config": DotbotConfig(**kw), "deployment": None} + + +def test_injects_conn_and_swarm_id(): + out = inject_config(["status"], _obj(conn="mqtts://b:8883", swarm_id="1234")) + assert out == ["--conn", "mqtts://b:8883", "--swarm-id", "1234", "status"] + + +def test_swarm_id_only(): + out = inject_config(["status"], _obj(swarm_id="1234")) + assert out == ["--swarm-id", "1234", "status"] + + +def test_explicit_conn_flag_wins(): + out = inject_config( + ["--conn", "mqtts://x:1", "status"], + _obj(conn="mqtts://b:8883", swarm_id="1234"), + ) + # conn not re-injected; swarm_id still filled in. + assert out.count("--conn") == 1 + assert out[-3:] == ["--conn", "mqtts://x:1", "status"] + assert "--swarm-id" in out and "1234" in out + + +def test_short_conn_flag_wins(): + out = inject_config(["-n", "simulator", "status"], _obj(conn="mqtts://b:8883")) + assert "mqtts://b:8883" not in out + + +def test_config_path_flag_skips_injection(): + out = inject_config( + ["-c", "other.toml", "status"], + _obj(conn="mqtts://b:8883", swarm_id="1234"), + ) + assert out == ["-c", "other.toml", "status"] + + +def test_help_skips_injection(): + assert inject_config(["--help"], _obj(conn="mqtts://b:8883")) == ["--help"] + assert inject_config(["status", "-h"], _obj(conn="mqtts://b:8883")) == [ + "status", + "-h", + ] + + +def test_no_config_is_noop(): + assert inject_config(["status"], None) == ["status"] + assert inject_config(["status"], _obj()) == ["status"] diff --git a/dotbot/tests/test_config.py b/dotbot/tests/test_config.py new file mode 100644 index 00000000..da23f020 --- /dev/null +++ b/dotbot/tests/test_config.py @@ -0,0 +1,311 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Headless tests for the unified config resolver (dotbot/config.py). + +Pure {flags, env, file} -> resolved value; no hardware, no network. Covers the +precedence chain, discovery order, deployment selection, and strict validation. +""" + +import pytest + +import dotbot.config as cfg + +# --- discovery -------------------------------------------------------------- + + +def test_discover_explicit_wins(tmp_path, monkeypatch): + explicit = tmp_path / "given.toml" + explicit.write_text("") + monkeypatch.setenv("DOTBOT_CONFIG", str(tmp_path / "env.toml")) + (tmp_path / cfg.PROJECT_CONFIG_NAME).write_text("") + assert cfg.discover_config_path(explicit, start_dir=tmp_path) == explicit + + +def test_discover_env_var(tmp_path, monkeypatch): + env_file = tmp_path / "env.toml" + monkeypatch.setenv("DOTBOT_CONFIG", str(env_file)) + assert ( + cfg.discover_config_path(None, environ={"DOTBOT_CONFIG": str(env_file)}) + == env_file + ) + + +def test_discover_project_cwd_only(tmp_path): + # A dotbot.toml in the cwd is discovered. + project = tmp_path / cfg.PROJECT_CONFIG_NAME + project.write_text("") + assert cfg.discover_config_path(None, environ={}, start_dir=tmp_path) == project + + +def test_discover_ignores_parent_dirs(tmp_path, monkeypatch): + # A dotbot.toml in a PARENT is NOT discovered - the cwd only, no walking up. + (tmp_path / cfg.PROJECT_CONFIG_NAME).write_text("") + nested = tmp_path / "a" / "b" + nested.mkdir(parents=True) + monkeypatch.setattr(cfg, "USER_CONFIG_PATH", tmp_path / "nope.toml") + assert cfg.discover_config_path(None, environ={}, start_dir=nested) is None + + +def test_discover_user_fallback(tmp_path, monkeypatch): + user = tmp_path / "home.toml" + user.write_text("") + monkeypatch.setattr(cfg, "USER_CONFIG_PATH", user) + empty = tmp_path / "empty" + empty.mkdir() + assert cfg.discover_config_path(None, environ={}, start_dir=empty) == user + + +def test_discover_none(tmp_path, monkeypatch): + monkeypatch.setattr(cfg, "USER_CONFIG_PATH", tmp_path / "missing.toml") + empty = tmp_path / "empty" + empty.mkdir() + assert cfg.discover_config_path(None, environ={}, start_dir=empty) is None + + +def test_discover_user_file_skipped(tmp_path, monkeypatch): + # include_user_file=False ignores ~/.dotbot/config.toml (Phase-2 behavior). + user = tmp_path / "home.toml" + user.write_text("") + monkeypatch.setattr(cfg, "USER_CONFIG_PATH", user) + empty = tmp_path / "empty" + empty.mkdir() + got = cfg.discover_config_path( + None, environ={}, start_dir=empty, include_user_file=False + ) + assert got is None + + +# --- loading + validation --------------------------------------------------- + + +def test_load_none_is_empty(): + config = cfg.load_config(None) + assert config.conn is None + assert config.deployment == {} + + +def test_load_valid(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text( + """ +default_deployment = "inria" +conn = "mqtts://broker.local:8883" +swarm_id = "0001" + +[deployment.inria] +conn = "mqtts://broker.inria.fr:8883" +swarm_id = "0001" +location = "Inria Paris" +bots = 100 + +[fw] +board = "dotbot-v3" + +[run.controller] +http_port = 8000 +""" + ) + config = cfg.load_config(path) + assert config.default_deployment == "inria" + assert config.fw.board == "dotbot-v3" + assert config.run.controller.http_port == 8000 + assert config.deployment["inria"].bots == 100 + + +def test_load_unknown_top_level_key_rejected(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text('swrm_id = "0001"\n') # typo + with pytest.raises(cfg.ConfigError): + cfg.load_config(path) + + +def test_load_unknown_section_key_rejected(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text('[fw]\nbord = "x"\n') # typo in a section + with pytest.raises(cfg.ConfigError): + cfg.load_config(path) + + +def test_load_bad_conn_rejected(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text('conn = "ftp://nope"\n') # unrecognized scheme + with pytest.raises(cfg.ConfigError): + cfg.load_config(path) + + +def test_load_accepts_valid_conn_forms(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text( + '[deployment.sim]\nconn = "simulator"\n' + '[deployment.cable]\nconn = "/dev/ttyACM0"\n' + '[deployment.mqtt]\nconn = "mqtts://h:8883"\n' + ) + config = cfg.load_config(path) + assert set(config.deployment) == {"sim", "cable", "mqtt"} + + +def test_load_bad_type_rejected(tmp_path): + path = tmp_path / "dotbot.toml" + path.write_text('[run.controller]\nhttp_port = "not-an-int"\n') + with pytest.raises(cfg.ConfigError): + cfg.load_config(path) + + +# --- deployment selection ------------------------------------------------------ + + +def _two_deployments(): + return cfg.DotbotConfig( + default_deployment="inria", + deployment={ + "inria": cfg.Deployment(swarm_id="0001"), + "laposte": cfg.Deployment(swarm_id="002a"), + }, + ) + + +def test_select_deployment_cli_beats_env_and_default(): + config = _two_deployments() + tb, name = cfg.select_deployment( + config, cli_name="laposte", environ={"DOTBOT_DEPLOYMENT": "inria"} + ) + assert name == "laposte" + assert tb.swarm_id == "002a" + + +def test_select_deployment_env_beats_default(): + config = _two_deployments() + _, name = cfg.select_deployment(config, environ={"DOTBOT_DEPLOYMENT": "laposte"}) + assert name == "laposte" + + +def test_select_deployment_default(): + config = _two_deployments() + _, name = cfg.select_deployment(config, environ={}) + assert name == "inria" + + +def test_select_deployment_none_when_unset(): + config = cfg.DotbotConfig() + assert cfg.select_deployment(config, environ={}) == (None, None) + + +def test_select_deployment_unknown_raises(): + config = _two_deployments() + with pytest.raises(cfg.ConfigError): + cfg.select_deployment(config, cli_name="nope", environ={}) + + +# --- precedence resolution -------------------------------------------------- + + +def test_resolve_flag_wins(): + config = cfg.DotbotConfig(conn="mqtts://file:8883") + got = cfg.resolve( + "conn", + flag="mqtts://flag:8883", + config=config, + environ={"DOTBOT_CONN": "mqtts://env:8883"}, + default="mqtts://default:8883", + ) + assert got == "mqtts://flag:8883" + + +def test_resolve_env_beats_file_and_default(): + config = cfg.DotbotConfig(swarm_id="file") + got = cfg.resolve( + "swarm_id", + config=config, + environ={"DOTBOT_SWARM_ID": "env"}, + default="default", + ) + assert got == "env" + + +def test_resolve_sectioned_env_name(): + got = cfg.resolve( + "board", + section="fw", + environ={"DOTBOT_FW_BOARD": "nrf5340dk-app"}, + default="dotbot-v3", + ) + assert got == "nrf5340dk-app" + + +def test_resolve_shared_env_alias_for_section_key(): + # A sectioned key falls back to the shared DOTBOT_ alias. + got = cfg.resolve( + "swarm_id", section="swarm", environ={"DOTBOT_SWARM_ID": "abcd"}, default="0000" + ) + assert got == "abcd" + + +def test_resolve_file_only_then_default(): + config = cfg.DotbotConfig(log_level="debug") + assert ( + cfg.resolve("log_level", config=config, environ={}, default="info") == "debug" + ) + assert ( + cfg.resolve("log_level", config=cfg.DotbotConfig(), environ={}, default="info") + == "info" + ) + + +def test_resolve_section_beats_top_level(): + config = cfg.DotbotConfig( + swarm_id="top", swarm=cfg.SwarmSection(swarm_id="section") + ) + got = cfg.resolve( + "swarm_id", section="swarm", config=config, environ={}, default="d" + ) + assert got == "section" + + +def test_resolve_deployment_beats_top_level(): + config = cfg.DotbotConfig(conn="mqtts://top:8883") + tb = cfg.Deployment(conn="mqtts://inria:8883") + got = cfg.resolve("conn", config=config, deployment=tb, environ={}, default=None) + assert got == "mqtts://inria:8883" + + +def test_resolve_section_beats_deployment(): + # Documented order: section value > selected deployment > top-level. + config = cfg.DotbotConfig(swarm=cfg.SwarmSection(swarm_id="section")) + tb = cfg.Deployment(swarm_id="deployment") + got = cfg.resolve( + "swarm_id", + section="swarm", + config=config, + deployment=tb, + environ={}, + default="d", + ) + assert got == "section" + + +def test_resolve_env_coercion_int(): + got = cfg.resolve( + "http_port", + section="run", + environ={"DOTBOT_RUN_HTTP_PORT": "9000"}, + default=8000, + ) + assert got == 9000 and isinstance(got, int) + + +def test_resolve_env_coercion_bool(): + got = cfg.resolve( + "webbrowser", + section="run", + environ={"DOTBOT_RUN_WEBBROWSER": "true"}, + default=False, + ) + assert got is True + + +def test_resolve_bad_int_env_raises(): + with pytest.raises(cfg.ConfigError): + cfg.resolve( + "http_port", section="run", environ={"DOTBOT_HTTP_PORT": "x"}, default=8000 + ) diff --git a/dotbot/tests/test_conn.py b/dotbot/tests/test_conn.py new file mode 100644 index 00000000..e8d52b63 --- /dev/null +++ b/dotbot/tests/test_conn.py @@ -0,0 +1,81 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for the `--conn` connection-string parser (`dotbot.cli._conn`).""" + +import pytest + +from dotbot.cli._conn import ConnError, needs_swarm_id, parse_connection + + +def test_parse_mqtt_tls(): + c = parse_connection("mqtts://argus.paris.inria.fr:8883") + assert c.kind == "mqtt" + assert c.host == "argus.paris.inria.fr" + assert c.port == 8883 + assert c.use_tls is True + + +def test_parse_mqtt_plain(): + c = parse_connection("mqtt://localhost:1883") + assert c.kind == "mqtt" + assert c.host == "localhost" + assert c.port == 1883 + assert c.use_tls is False + + +def test_parse_mqtt_default_ports(): + # No explicit port → 8883 for TLS, 1883 for plain. + assert parse_connection("mqtts://host").port == 8883 + assert parse_connection("mqtt://host").port == 1883 + + +def test_parse_mqtt_missing_host_errors(): + with pytest.raises(ConnError): + parse_connection("mqtts://:8883") + + +def test_parse_serial_device_path(): + c = parse_connection("/dev/ttyACM0") + assert c.kind == "serial" + assert c.serial_port == "/dev/ttyACM0" + + +def test_parse_serial_macos_usbmodem(): + c = parse_connection("/dev/tty.usbmodem0007745943981") + assert c.kind == "serial" + assert c.serial_port == "/dev/tty.usbmodem0007745943981" + + +def test_parse_serial_windows_com(): + c = parse_connection("COM3") + assert c.kind == "serial" + assert c.serial_port == "COM3" + + +def test_parse_simulator_both_spellings(): + assert parse_connection("simulator").kind == "simulator" + assert parse_connection("sim").kind == "simulator" + + +def test_parse_simulator_case_insensitive(): + assert parse_connection("Simulator").kind == "simulator" + + +def test_parse_unknown_scheme_errors(): + # A scheme we don't recognize is an error, not a silent serial path. + with pytest.raises(ConnError): + parse_connection("http://example.com:8000") + with pytest.raises(ConnError): + parse_connection("serial:///dev/ttyACM0") # scheme deliberately unsupported + + +def test_parse_empty_errors(): + with pytest.raises(ConnError): + parse_connection("") + + +def test_needs_swarm_id_only_for_mqtt(): + assert needs_swarm_id(parse_connection("mqtts://host:8883")) is True + assert needs_swarm_id(parse_connection("/dev/ttyACM0")) is False + assert needs_swarm_id(parse_connection("simulator")) is False diff --git a/dotbot/tests/test_controller.py b/dotbot/tests/test_controller.py index 61987464..9c0370ab 100644 --- a/dotbot/tests/test_controller.py +++ b/dotbot/tests/test_controller.py @@ -194,7 +194,6 @@ async def test_controller_get_dotbots_query(query, length, controller): assert len(dotbots) == length -@pytest.mark.filterwarnings("ignore::DeprecationWarning") def test_controller_sailbot_simulator(): """Check controller called for sailbot simulator.""" @@ -211,11 +210,9 @@ async def start_simulator(): except asyncio.TimeoutError: pass - loop = asyncio.get_event_loop() - loop.run_until_complete(start_simulator()) + asyncio.run(start_simulator()) -@pytest.mark.filterwarnings("ignore::DeprecationWarning") def test_controller_dotbot_simulator(): """Check controller called for dotbot simulator.""" @@ -232,8 +229,7 @@ async def start_simulator(): except asyncio.TimeoutError: pass - loop = asyncio.get_event_loop() - loop.run_until_complete(start_simulator()) + asyncio.run(start_simulator()) @pytest.mark.parametrize( diff --git a/dotbot/tests/test_controller_app.py b/dotbot/tests/test_controller_app.py index 9d7d3e22..613a9209 100644 --- a/dotbot/tests/test_controller_app.py +++ b/dotbot/tests/test_controller_app.py @@ -10,59 +10,20 @@ from dotbot.controller_app import main -MAIN_HELP_EXPECTED = """Usage: main [OPTIONS] - - DotBotController, universal SailBot and DotBot controller. - -Options: - -a, --adapter [serial|edge|cloud|dotbot-simulator|sailbot-simulator] - Controller interface adapter. Defaults to - serial - -p, --port TEXT Serial port used by 'serial' and 'edge' - adapters. Defaults to '/dev/ttyACM0' - -b, --baudrate INTEGER Serial baudrate used by 'serial' and 'edge' - adapters. Defaults to 1000000 - -H, --mqtt-host TEXT MQTT host used by cloud adapter. Default: - localhost. - -P, --mqtt-port INTEGER MQTT port used by cloud adapter. Default: - 1883. - -T, --mqtt-use_tls / --no-mqtt-use_tls - Use TLS with MQTT (for cloud adapter). - -g, --gw-address TEXT Gateway address in hex. Defaults to - 0000000000000000 - -s, --network-id TEXT Network ID in hex. Defaults to 0000 - -c, --controller-http-port INTEGER - Controller HTTP port of the REST API. Defaults - to '8000' - -w, --webbrowser / --no-webbrowser - Open a web browser automatically - -v, --verbose Run in verbose mode (all payloads received are - printed in terminal) - --log-level [debug|info|warning|error] - Logging level. Defaults to info - --log-output PATH Filename where logs are redirected - --csv-data-output PATH Filename where CSV data logs are stored. If - not set, CSV data logging is disabled. - --config-path FILE Path to a .toml configuration file. - -m, --map-size TEXT Map size in mm. Defaults to '2000x2000' - -M, --background-map FILE Path to a background map image file in png - format. The image shouldbe a top-down view of - the environment, with 1024 pixels width and a - height proportional to the real map size. The - map size should be set with the --map-size - option (default: 2000x2000). - --simulator-init-state FILE Path to the simulator initial state .toml - file. Defaults to 'simulator_init_state.toml'. - --help Show this message and exit. -""" - -@pytest.mark.skipif(sys.platform != "linux", reason="Serial port is different") def test_main_help(): + """Help advertises the new `--conn` / `--swarm-id` surface and no + longer the dropped `--adapter` / `-H/-P/-T` flags.""" runner = CliRunner() result = runner.invoke(main, ["--help"]) assert result.exit_code == 0 - assert result.output == MAIN_HELP_EXPECTED + assert "--conn" in result.output + assert "--swarm-id" in result.output + assert "--sailbot" in result.output + # Dropped flags must be gone. + assert "--adapter" not in result.output + assert "--mqtt-host" not in result.output + assert "--network-id" not in result.output @patch("dotbot_utils.serial_interface.serial.Serial.open") @@ -71,32 +32,76 @@ def test_main_help(): def test_main(run, version, _): version.return_value = "test" runner = CliRunner() - result = runner.invoke(main) + # A connection is now required; `simulator` needs no hardware/swarm-id. + result = runner.invoke(main, ["--conn", "simulator"]) assert result.exit_code == 0 assert "Welcome to the DotBots controller (version: test)." in result.output run.assert_called_once() version.side_effect = PackageNotFoundError - result = runner.invoke(main) + result = runner.invoke(main, ["--conn", "simulator"]) assert result.exit_code == 0 assert "Welcome to the DotBots controller (version: unknown)." in result.output +@pytest.mark.skipif(sys.platform == "win32", reason="Doesn't work on Windows") +@patch("dotbot.controller_app.asyncio.run") +@patch("dotbot.controller_app.Controller") +def test_run_controller_uses_selected_deployment(controller, _asyncio_run, tmp_path): + """Through the root group: a selected deployment supplies `conn` so + `run controller` starts without a CLI `--conn` and consumes + `deployment.sim.conn = "simulator"`.""" + from dotbot.cli.main import cli + + config_file = tmp_path / "dotbot.toml" + config_file.write_text( + """ +default_deployment = "sim" + +[deployment.sim] +conn = "simulator" +""" + ) + + runner = CliRunner() + result = runner.invoke(cli, ["-c", str(config_file), "run", "controller"]) + assert result.exit_code == 0, result.output + # The deployment's conn=simulator was consumed: no "no connection" error, + # and the adapter resolves to the simulator. + settings = controller.call_args.args[0] + assert settings.adapter == "dotbot-simulator" + + +def test_main_without_conn_errors(): + """No `--conn` → a clear error listing the connection forms.""" + runner = CliRunner() + result = runner.invoke(main, []) + assert result.exit_code != 0 + assert "mqtts://" in result.output and "simulator" in result.output + + +def test_main_mqtt_without_swarm_id_errors(): + runner = CliRunner() + result = runner.invoke(main, ["--conn", "mqtts://argus:8883"]) + assert result.exit_code != 0 + assert "swarm-id" in result.output + + @patch("dotbot_utils.serial_interface.serial.Serial.open") @patch("dotbot.controller.Controller.run") def test_main_interrupts(run, _): runner = CliRunner() run.side_effect = KeyboardInterrupt - result = runner.invoke(main) + result = runner.invoke(main, ["--conn", "simulator"]) assert result.exit_code == 0 runner = CliRunner() run.side_effect = SystemExit - result = runner.invoke(main) + result = runner.invoke(main, ["--conn", "simulator"]) assert result.exit_code == 0 run.side_effect = serial.serialutil.SerialException("serial test error") - result = runner.invoke(main) + result = runner.invoke(main, ["--conn", "/dev/ttyACM0"]) assert result.exit_code != 0 assert "Serial error: serial test error" in result.output @@ -105,12 +110,13 @@ def test_main_interrupts(run, _): @patch("dotbot_utils.serial_interface.serial.Serial.open") @patch("dotbot.controller_app.Controller") def test_main_with_config(controller, _, tmp_path): + """Config file carries `conn` + `swarm_id` (new keys); CLI absent.""" log_file = tmp_path / "logfile.log" config_file = tmp_path / "config.toml" config_file.write_text( f""" -adapter = "serial" -network_id = "AA26" +conn = "mqtts://argus:8883" +swarm_id = "AA26" log_level = "debug" log_output = "{log_file}" """ @@ -118,7 +124,88 @@ def test_main_with_config(controller, _, tmp_path): runner = CliRunner() runner.invoke(main, ["--config-path", config_file.as_posix()]) - assert controller.call_args.args[0].network_id == "AA26" - assert controller.call_args.args[0].adapter == "serial" - assert controller.call_args.args[0].log_level == "debug" - assert controller.call_args.args[0].log_output == str(log_file) + settings = controller.call_args.args[0] + assert settings.network_id == "AA26" + assert settings.adapter == "cloud" + assert settings.mqtt_host == "argus" + assert settings.log_level == "debug" + assert settings.log_output == str(log_file) + + +@pytest.mark.skipif(sys.platform == "win32", reason="Doesn't work on Windows") +@patch("dotbot_utils.serial_interface.serial.Serial.open") +@patch("dotbot.controller_app.Controller") +def test_main_warns_on_legacy_config_keys(controller, _, tmp_path): + """A config file with old transport keys (adapter/mqtt_host/...) gets a + warning, and those keys are dropped (conn/swarm_id drive it).""" + config_file = tmp_path / "cfg.toml" + config_file.write_text( + 'conn = "simulator"\nadapter = "serial"\nmqtt_host = "stale"\n' + ) + runner = CliRunner() + result = runner.invoke(main, ["--config-path", config_file.as_posix()]) + assert "legacy config key" in result.output + settings = controller.call_args.args[0] + # conn=simulator wins; the stale adapter/mqtt_host are ignored. + assert settings.adapter == "dotbot-simulator" + assert settings.mqtt_host != "stale" + + +def test_scaffold_sim_state_creates_example_when_accepted(tmp_path, monkeypatch): + """Interactive simulator run with nothing specified + `y` writes an + editable `simulator_init_state.toml` in the cwd.""" + from dotbot import SIMULATOR_INIT_STATE_DEFAULT + from dotbot.controller_app import _maybe_scaffold_sim_state + + monkeypatch.chdir(tmp_path) + with patch("sys.stdin") as stdin, patch("click.confirm", return_value=True): + stdin.isatty.return_value = True + _maybe_scaffold_sim_state(None) # the value main() passes by default + created = tmp_path / SIMULATOR_INIT_STATE_DEFAULT + assert created.is_file() + assert "[[dotbots]]" in created.read_text() + + +def test_scaffold_sim_state_declined_writes_nothing(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + from dotbot import SIMULATOR_INIT_STATE_DEFAULT + from dotbot.controller_app import _maybe_scaffold_sim_state + + with patch("sys.stdin") as stdin, patch("click.confirm", return_value=False): + stdin.isatty.return_value = True + _maybe_scaffold_sim_state(None) + assert not (tmp_path / SIMULATOR_INIT_STATE_DEFAULT).exists() + + +def test_scaffold_sim_state_noninteractive_never_prompts(tmp_path, monkeypatch): + """No TTY (CI, a pipe) → no prompt, no file; the packaged world is used.""" + monkeypatch.chdir(tmp_path) + from dotbot import SIMULATOR_INIT_STATE_DEFAULT + from dotbot.controller_app import _maybe_scaffold_sim_state + + with patch("sys.stdin") as stdin, patch("click.confirm") as confirm: + stdin.isatty.return_value = False + _maybe_scaffold_sim_state(None) + confirm.assert_not_called() + assert not (tmp_path / SIMULATOR_INIT_STATE_DEFAULT).exists() + + +def test_scaffold_sim_state_skips_when_explicit_path_given(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + from dotbot.controller_app import _maybe_scaffold_sim_state + + with patch("sys.stdin") as stdin, patch("click.confirm") as confirm: + stdin.isatty.return_value = True + _maybe_scaffold_sim_state("my_world.toml") # explicit path → no prompt + confirm.assert_not_called() + + +@patch("dotbot_utils.serial_interface.serial.Serial.open") +@patch("dotbot.controller.Controller.run") +@patch("dotbot.controller_app._maybe_scaffold_sim_state") +def test_main_simulator_offers_scaffold_with_none(scaffold, _run, _serial): + """Regression: `--conn simulator` with no flag/config must reach the + scaffold with None (the option default), not a sentinel string.""" + runner = CliRunner() + runner.invoke(main, ["--conn", "simulator"]) + scaffold.assert_called_once_with(None) diff --git a/dotbot/tests/test_controller_conn.py b/dotbot/tests/test_controller_conn.py new file mode 100644 index 00000000..c19b52f8 --- /dev/null +++ b/dotbot/tests/test_controller_conn.py @@ -0,0 +1,86 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for the controller's `--conn` / `--swarm-id` CLI surface. + +These exercise `_conn_to_settings` (pure translation + validation) and +the Click wiring, without starting a real controller / MQTT / serial. +""" + +import click +import pytest + +from dotbot.controller_app import _conn_to_settings + + +def test_mqtt_conn_maps_to_cloud_adapter(): + s = _conn_to_settings("mqtts://argus:8883", "1234", sim_is_dotbot=True) + assert s["adapter"] == "cloud" + assert s["mqtt_host"] == "argus" + assert s["mqtt_port"] == 8883 + assert s["mqtt_use_tls"] is True + assert s["network_id"] == "1234" + + +def test_mqtt_conn_without_swarm_id_errors(): + with pytest.raises(click.ClickException) as exc: + _conn_to_settings("mqtts://argus:8883", None, sim_is_dotbot=True) + assert "swarm-id" in str(exc.value) + + +def test_serial_conn_maps_to_edge_adapter_no_swarm_id_needed(): + s = _conn_to_settings("/dev/ttyACM0", None, sim_is_dotbot=True) + assert s["adapter"] == "edge" + assert s["port"] == "/dev/ttyACM0" + # No swarm-id required, and none injected when absent. + assert "network_id" not in s + + +def test_serial_conn_keeps_swarm_id_when_given(): + s = _conn_to_settings("/dev/ttyACM0", "00aa", sim_is_dotbot=True) + assert s["adapter"] == "edge" + assert s["network_id"] == "00aa" + + +def test_simulator_conn_maps_to_dotbot_simulator(): + s = _conn_to_settings("simulator", None, sim_is_dotbot=True) + assert s["adapter"] == "dotbot-simulator" + + +def test_simulator_conn_sailbot(): + s = _conn_to_settings("simulator", None, sim_is_dotbot=False) + assert s["adapter"] == "sailbot-simulator" + + +def test_sim_alias_spelling(): + assert _conn_to_settings("sim", None, sim_is_dotbot=True)["adapter"] == ( + "dotbot-simulator" + ) + + +def test_no_conn_errors_with_guidance(): + with pytest.raises(click.ClickException) as exc: + _conn_to_settings(None, None, sim_is_dotbot=True) + msg = str(exc.value) + assert "mqtts://" in msg and "simulator" in msg + + +def test_malformed_conn_errors(): + with pytest.raises(click.ClickException): + _conn_to_settings("http://nope:1", "1234", sim_is_dotbot=True) + + +def test_env_credentials_threaded_into_mqtt_settings(monkeypatch): + monkeypatch.setenv("DOTBOT_MQTT_USER", "alice") + monkeypatch.setenv("DOTBOT_MQTT_PASS", "s3cret") + s = _conn_to_settings("mqtts://argus:8883", "1234", sim_is_dotbot=True) + assert s["mqtt_username"] == "alice" + assert s["mqtt_password"] == "s3cret" + + +def test_env_credentials_absent_are_none(monkeypatch): + monkeypatch.delenv("DOTBOT_MQTT_USER", raising=False) + monkeypatch.delenv("DOTBOT_MQTT_PASS", raising=False) + s = _conn_to_settings("mqtts://argus:8883", "1234", sim_is_dotbot=True) + assert s["mqtt_username"] is None + assert s["mqtt_password"] is None diff --git a/dotbot/tests/test_device.py b/dotbot/tests/test_device.py new file mode 100644 index 00000000..551176b2 --- /dev/null +++ b/dotbot/tests/test_device.py @@ -0,0 +1,539 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for `dotbot device` — CLI surface, config-hex bytes, read-and-report. + +Hardware-free: the actual J-Link flashing is monkeypatched. What's +verified here is the command/option shape, the config-page bytes +`create_config_hex` emits (inspectable via IntelHex, no device needed), +the `device info` read-and-report contract (never fails on a blank +board), and the friendly nrfjprog-missing error. +""" + +from pathlib import Path + +import click +import pytest +from click.testing import CliRunner + +import dotbot.firmware.fetch as fetch +import dotbot.firmware.flash as flash +from dotbot.cli.device import _looks_like_path +from dotbot.cli.device import cmd as device_cmd + + +@pytest.fixture +def runner(): + return CliRunner() + + +@pytest.fixture +def _no_nrfjprog_gate(monkeypatch): + """Make ensure_nrfjprog() a no-op so commands reach their backend.""" + monkeypatch.setattr("dotbot.cli.device.ensure_nrfjprog", lambda: None) + + +def test_device_help_lists_commands(runner): + result = runner.invoke(device_cmd, ["--help"]) + assert result.exit_code == 0 + for sub in ( + "flash", + "flash-swarmit-sandbox", + "flash-mari-gateway", + "flash-programmer", + "info", + ): + assert sub in result.output + + +def test_flash_swarmit_sandbox_accepts_calibration(runner): + """flash-swarmit-sandbox has --calibration (LH2 lives on dotbot-v3).""" + result = runner.invoke(device_cmd, ["flash-swarmit-sandbox", "--help"]) + assert result.exit_code == 0 + assert "--calibration" in result.output + + +def test_flash_mari_gateway_rejects_calibration(runner): + """flash-mari-gateway has no --calibration option (gateway has no LH2).""" + result = runner.invoke(device_cmd, ["flash-mari-gateway", "--help"]) + assert result.exit_code == 0 + assert "--calibration" not in result.output + # Passing it is an unknown-option error. + bad = runner.invoke( + device_cmd, + ["flash-mari-gateway", "--swarm-id", "1234", "-f", "0.8.0rc1", "-l", "x.out"], + ) + assert bad.exit_code != 0 + + +def test_flash_swarmit_sandbox_requires_swarm_id(runner): + """flash-swarmit-sandbox needs a swarm id (flag or config); -f is now + optional and defaults to the latest release (so no network in this test: + swarm_id is checked before the version resolves).""" + with runner.isolated_filesystem(): + result = runner.invoke(device_cmd, ["flash-swarmit-sandbox", "-f", "0.8.0rc1"]) + assert result.exit_code != 0 + assert "no swarm id" in result.output + + +def test_flash_mari_gateway_help_disambiguates_from_bridge(runner): + """`device flash-mari-gateway` help points away from the `dotbot run gateway` bridge.""" + result = runner.invoke(device_cmd, ["flash-mari-gateway", "--help"]) + assert result.exit_code == 0 + assert "dotbot run gateway" in result.output # the "use the bridge instead" note + + +def test_flash_swarmit_sandbox_calls_engine(runner, _no_nrfjprog_gate, monkeypatch): + calls = {} + + def fake_flash_role(role, **kw): + calls["role"] = role + calls["kw"] = kw + + monkeypatch.setattr("dotbot.firmware.flash.flash_role", fake_flash_role) + result = runner.invoke( + device_cmd, + ["flash-swarmit-sandbox", "--swarm-id", "0100", "-f", "0.8.0rc1", "-s", "77"], + ) + assert result.exit_code == 0, result.output + assert calls["role"] == "dotbot-v3" + assert calls["kw"]["net_id"] == (0x0100, "0100") + assert calls["kw"]["fw_version"] == "0.8.0rc1" + assert calls["kw"]["sn_starting_digits"] == "77" + + +def test_flash_swarmit_sandbox_defaults_to_pinned_version( + runner, _no_nrfjprog_gate, monkeypatch +): + """With no -f, the role flash uses the pinned swarmit version (matching + `fw fetch`), not the latest release - and resolves it without the network.""" + + calls = {} + monkeypatch.setattr( + "dotbot.firmware.flash.flash_role", + lambda role, **kw: calls.update(role=role, kw=kw), + ) + result = runner.invoke( + device_cmd, ["flash-swarmit-sandbox", "--swarm-id", "0100", "-s", "77"] + ) + assert result.exit_code == 0, result.output + assert calls["kw"]["fw_version"] == fetch.pinned_version("swarmit") + + +def test_flash_mari_gateway_calls_engine_with_gateway_role( + runner, _no_nrfjprog_gate, monkeypatch +): + calls = {} + monkeypatch.setattr( + "dotbot.firmware.flash.flash_role", + lambda role, **kw: calls.update(role=role, kw=kw), + ) + result = runner.invoke( + device_cmd, ["flash-mari-gateway", "--swarm-id", "1234", "-f", "0.8.0rc1"] + ) + assert result.exit_code == 0, result.output + assert calls["role"] == "gateway" + # gateway carries no calibration. + assert "calibration_path" not in calls["kw"] + + +# ── swarm id defaults from the selected deployment's swarm_id ───────── + + +def _write_cfg(tmp_path, text): + path = tmp_path / "dotbot.toml" + path.write_text(text) + return path + + +def test_flash_mari_gateway_net_id_from_deployment( + runner, _no_nrfjprog_gate, tmp_path, monkeypatch +): + """No --swarm-id + a selected deployment -> net_id derived from its swarm_id.""" + from dotbot.cli.main import cli + + calls = {} + monkeypatch.setattr( + "dotbot.firmware.flash.flash_role", + lambda role, **kw: calls.update(role=role, kw=kw), + ) + cfg = _write_cfg( + tmp_path, + 'default_deployment = "lab"\n[deployment.lab]\nswarm_id = "1234"\n', + ) + result = runner.invoke( + cli, + ["-c", str(cfg), "device", "flash-mari-gateway", "-s", "10", "-f", "0.8.0rc1"], + ) + assert result.exit_code == 0, result.output + assert calls["role"] == "gateway" + assert calls["kw"]["net_id"] == (0x1234, "1234") + + +def test_flash_mari_gateway_explicit_net_id_overrides_deployment( + runner, _no_nrfjprog_gate, tmp_path, monkeypatch +): + """An explicit --swarm-id beats the deployment's swarm_id.""" + from dotbot.cli.main import cli + + calls = {} + monkeypatch.setattr( + "dotbot.firmware.flash.flash_role", + lambda role, **kw: calls.update(role=role, kw=kw), + ) + cfg = _write_cfg( + tmp_path, + 'default_deployment = "lab"\n[deployment.lab]\nswarm_id = "1234"\n', + ) + result = runner.invoke( + cli, + [ + "-c", + str(cfg), + "device", + "flash-mari-gateway", + "--swarm-id", + "0099", + "-f", + "0.8.0rc1", + ], + ) + assert result.exit_code == 0, result.output + assert calls["kw"]["net_id"] == (0x0099, "0099") + + +def test_flash_mari_gateway_no_swarm_id_no_config_errors(runner, _no_nrfjprog_gate): + """No --swarm-id and no swarm_id/deployment -> a clean ClickException, not a crash.""" + from dotbot.cli.main import cli + + with runner.isolated_filesystem(): + result = runner.invoke(cli, ["device", "flash-mari-gateway", "-f", "0.8.0rc1"]) + assert result.exit_code != 0 + assert "no swarm id" in result.output + + +def test_flash_swarmit_sandbox_net_id_from_deployment( + runner, _no_nrfjprog_gate, tmp_path, monkeypatch +): + """flash-swarmit-sandbox also defaults net_id from the deployment's swarm_id.""" + from dotbot.cli.main import cli + + calls = {} + monkeypatch.setattr( + "dotbot.firmware.flash.flash_role", + lambda role, **kw: calls.update(role=role, kw=kw), + ) + cfg = _write_cfg( + tmp_path, + 'default_deployment = "lab"\n[deployment.lab]\nswarm_id = "1234"\n', + ) + result = runner.invoke( + cli, + [ + "-c", + str(cfg), + "device", + "flash-swarmit-sandbox", + "-s", + "10", + "-f", + "0.8.0rc1", + ], + ) + assert result.exit_code == 0, result.output + assert calls["role"] == "dotbot-v3" + assert calls["kw"]["net_id"] == (0x1234, "1234") + + +# ── device info: read-and-report, never fails on a blank board ────────── + + +def test_info_reports_provisioned(runner, _no_nrfjprog_gate, monkeypatch): + monkeypatch.setattr( + "dotbot.firmware.flash.read_config_report", + lambda sn=None: ("1234", "BDF2B04BC00D2725"), + ) + result = runner.invoke(device_cmd, ["info", "-s", "77"]) + assert result.exit_code == 0, result.output + assert "provisioned" in result.output + assert "0x1234" in result.output + assert "BDF2B04BC00D2725" in result.output + + +def test_info_reports_unprovisioned_without_failing( + runner, _no_nrfjprog_gate, monkeypatch +): + """A blank board is a normal state — exit 0, report + fix hint.""" + monkeypatch.setattr( + "dotbot.firmware.flash.read_config_report", + lambda sn=None: ("unprovisioned", "BDF2B04BC00D2725"), + ) + result = runner.invoke(device_cmd, ["info"]) + assert result.exit_code == 0, result.output + assert "not provisioned" in result.output + assert "flash-swarmit-sandbox" in result.output + + +def test_info_surfaces_comms_failure(runner, _no_nrfjprog_gate, monkeypatch): + def boom(sn=None): + raise RuntimeError("no probe") + + monkeypatch.setattr("dotbot.firmware.flash.read_config_report", boom) + result = runner.invoke(device_cmd, ["info"]) + assert result.exit_code != 0 + assert "Could not read the device" in result.output + + +def test_nrfjprog_missing_gives_friendly_error(runner, monkeypatch): + """No nrfjprog → a clear install hint, not a stack trace.""" + monkeypatch.setattr("dotbot.firmware.nrf.nrfjprog_available", lambda: False) + result = runner.invoke(device_cmd, ["info"]) + assert result.exit_code != 0 + assert "nrfjprog" in result.output + + +# ── _looks_like_path discrimination (app name vs file) ────────────────── + + +@pytest.mark.parametrize( + "value,is_path", + [ + ("dotbot", False), + ("spin", False), + ("dotbot-dotbot-v3.hex", True), + ("spin-sandbox-dotbot-v3.bin", True), + ("./artifacts/dotbot-dotbot-v3.hex", True), + ("/tmp/x.bin", True), + ], +) +def test_looks_like_path(value, is_path): + assert _looks_like_path(value) is is_path + + +# ── Config-hex bytes (unit-testable without hardware) ─────────────────── + + +def _read_word_le(ih, addr): + return ih[addr] | (ih[addr + 1] << 8) | (ih[addr + 2] << 16) | (ih[addr + 3] << 24) + + +def test_create_config_hex_writes_magic_and_net_id(tmp_path): + from dotbot.firmware.flash import CONFIG_ADDR, CONFIG_MAGIC, create_config_hex + + pytest.importorskip("intelhex") + from intelhex import IntelHex + + dest = tmp_path / "config.hex" + create_config_hex(dest, 0x1234) + ih = IntelHex(str(dest)) + assert _read_word_le(ih, CONFIG_ADDR + 0) == CONFIG_MAGIC + assert _read_word_le(ih, CONFIG_ADDR + 4) == 1 # has_net_id + assert _read_word_le(ih, CONFIG_ADDR + 8) == 0x1234 + + +def test_create_config_hex_appends_calibration(tmp_path): + from dotbot.firmware.flash import CONFIG_ADDR, create_config_hex + + pytest.importorskip("intelhex") + from intelhex import IntelHex + + # 2 homography matrices, 36 bytes each (3x3 int32). + matrices = bytes(range(72)) + dest = tmp_path / "config-cal.hex" + create_config_hex(dest, 0x00AA, calibration=(2, matrices)) + ih = IntelHex(str(dest)) + assert _read_word_le(ih, CONFIG_ADDR + 12) == 2 # homography_count + got = bytes(ih[CONFIG_ADDR + 16 + i] for i in range(72)) + assert got == matrices + + +def test_intelhex_is_a_core_dependency(): + """intelhex was folded into core deps (the [provision] extra is gone), + so config-hex building works on a default `pip install pydotbot`.""" + + assert flash.IntelHex is not None + + +def test_fetch_assets_downloads_release_into_source_version_dir(tmp_path, monkeypatch): + """fetch_assets pulls every .hex/.bin the release lists into + -/, skips .elf/.map, and writes a manifest.""" + import json as _json + + fake_release = { + "tag_name": "0.8.0rc2", + "assets": [ + {"name": "bootloader-dotbot-v3.hex", "browser_download_url": "u1"}, + {"name": "netcore-nrf5340-net.hex", "browser_download_url": "u2"}, + {"name": "bootloader-dotbot-v3.elf", "browser_download_url": "u3"}, + ], + } + monkeypatch.setattr(fetch, "resolve_release", lambda source, version: fake_release) + + def fake_download(url, dest): + dest.write_bytes(b"\x00") + return 1 + + monkeypatch.setattr(fetch, "download_file", fake_download) + out = fetch.fetch_assets("swarmit", "latest", tmp_path) + assert out == tmp_path / "swarmit-0.8.0rc2" + assert (out / "bootloader-dotbot-v3.hex").exists() + assert (out / "netcore-nrf5340-net.hex").exists() + assert not (out / "bootloader-dotbot-v3.elf").exists() # .elf skipped + manifest = _json.loads((out / "manifest.json").read_text()) + assert manifest["source"] == "swarmit" + assert manifest["version"] == "0.8.0rc2" + assert "bootloader-dotbot-v3.hex" in manifest["files"] + assert manifest["pydotbot"] # provenance: which pydotbot fetched this + + +def test_fetch_assets_unknown_source_errors(tmp_path): + """An unknown source is a clear error, not a KeyError.""" + + with pytest.raises(click.ClickException): + fetch.fetch_assets("not-a-source", "latest", tmp_path) + + +def test_resolve_latest_version_returns_newest_tag(monkeypatch): + """Returns the first (newest, prereleases included) tag from the API.""" + import io + import json + + payload = json.dumps([{"tag_name": "0.8.0rc2"}, {"tag_name": "0.8.0rc1"}]).encode() + monkeypatch.setattr( + fetch.urllib.request, "urlopen", lambda req: io.BytesIO(payload) + ) + assert fetch.resolve_latest_version() == "0.8.0rc2" + + +def test_resolve_latest_version_no_releases_errors(monkeypatch): + """An empty release list is a clear error, not an IndexError.""" + import io + + monkeypatch.setattr(fetch.urllib.request, "urlopen", lambda req: io.BytesIO(b"[]")) + with pytest.raises(click.ClickException): + fetch.resolve_latest_version() + + +def test_resolve_latest_version_network_error_errors(monkeypatch): + """A network failure surfaces as a friendly ClickException.""" + + def boom(req): + raise fetch.urllib.error.URLError("offline") + + monkeypatch.setattr(fetch.urllib.request, "urlopen", boom) + with pytest.raises(click.ClickException): + fetch.resolve_latest_version() + + +def test_download_file_retries_transient_5xx(tmp_path, monkeypatch): + """A sporadic 502 (GitHub's CDN under concurrent load) is retried, then + succeeds - one bad gateway shouldn't abort the whole fetch.""" + import io + + calls = {"n": 0} + + def flaky_urlopen(url): + calls["n"] += 1 + if calls["n"] == 1: + raise fetch.urllib.error.HTTPError(url, 502, "Bad Gateway", {}, None) + return io.BytesIO(b"\xde\xad") + + monkeypatch.setattr(fetch.urllib.request, "urlopen", flaky_urlopen) + monkeypatch.setattr(fetch.time, "sleep", lambda _delay: None) # skip real backoff + + dest = tmp_path / "spin-dotbot-v3.hex" + size = fetch.download_file("http://x/spin-dotbot-v3.hex", dest, retries=3) + assert size == 2 + assert dest.read_bytes() == b"\xde\xad" + assert calls["n"] == 2 # one retry + + +def test_download_file_gives_up_on_non_transient(tmp_path, monkeypatch): + """A 404 is not transient - it surfaces immediately, with no backoff.""" + + def not_found(url): + raise fetch.urllib.error.HTTPError(url, 404, "Not Found", {}, None) + + sleeps: list[float] = [] + monkeypatch.setattr(fetch.urllib.request, "urlopen", not_found) + monkeypatch.setattr(fetch.time, "sleep", lambda d: sleeps.append(d)) + + with pytest.raises(click.ClickException): + fetch.download_file("http://x/missing.hex", tmp_path / "missing.hex", retries=3) + assert sleeps == [] # never retried + + +def test_short_path_falls_back_to_absolute_across_drives(monkeypatch): + """On Windows os.path.relpath raises ValueError when the path and cwd are + on different drives (C: vs D:); _short_path must return the absolute path, + not crash.""" + + def boom(_p): + raise ValueError("path is on mount 'C:', start on mount 'D:'") + + monkeypatch.setattr(fetch.os.path, "relpath", boom) + p = Path("/x/swarmit-1.2.3") + assert fetch._short_path(p) == str(p) + + +def test_pinned_version_dotbot_firmware_is_declared(): + """DotBot-firmware (not a Python dep) pins to the declared constant.""" + + assert fetch.pinned_version("dotbot-firmware") == fetch.DOTBOT_FIRMWARE_VERSION + + +def test_pinned_version_swarmit_from_installed_package(): + """swarmit's firmware version is inferred from the installed package.""" + import importlib.metadata as md + + assert fetch.pinned_version("swarmit") == md.version("swarmit") + + +def test_pinned_version_unknown_source_errors(): + """An unknown source is a clear error, not a KeyError.""" + + with pytest.raises(click.ClickException): + fetch.pinned_version("not-a-source") + + +def test_fetch_no_args_resolves_pinned_versions(monkeypatch): + """`dotbot fw fetch` with no flags fetches the pinned version per source, + not 'latest'.""" + from dotbot.cli.fw import cmd as fw_cmd + + calls: list[tuple[str, str]] = [] + monkeypatch.setattr(fetch, "pinned_version", lambda src: f"PIN-{src}") + monkeypatch.setattr( + fetch, + "fetch_assets", + lambda src, version, bin_dir, local_root=None: ( + calls.append((src, version)) or Path(f"/x/{src}-{version}") + ), + ) + res = CliRunner().invoke(fw_cmd, ["fetch"]) + assert res.exit_code == 0, res.output + assert calls == [ + ("swarmit", "PIN-swarmit"), + ("dotbot-firmware", "PIN-dotbot-firmware"), + ] + assert "latest" not in res.output # the pinned path never says "latest" + + +def test_fetch_explicit_version_overrides_pin(monkeypatch): + """-f with --source bypasses the pin and passes through verbatim.""" + from dotbot.cli.fw import cmd as fw_cmd + + calls: list[tuple[str, str]] = [] + pin_called: list[str] = [] + monkeypatch.setattr(fetch, "pinned_version", lambda src: pin_called.append(src)) + monkeypatch.setattr( + fetch, + "fetch_assets", + lambda src, version, bin_dir, local_root=None: ( + calls.append((src, version)) or Path(f"/x/{src}-{version}") + ), + ) + res = CliRunner().invoke(fw_cmd, ["fetch", "-S", "dotbot-firmware", "-f", "1.21.0"]) + assert res.exit_code == 0, res.output + assert calls == [("dotbot-firmware", "1.21.0")] + assert pin_called == [] # explicit -f never consults the pin diff --git a/dotbot/tests/test_flash_boards.py b/dotbot/tests/test_flash_boards.py new file mode 100644 index 00000000..b4a971e5 --- /dev/null +++ b/dotbot/tests/test_flash_boards.py @@ -0,0 +1,133 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Board → nrfjprog family/core resolution and `device flash` routing. + +Locks the fix for the nRF53-only flash bug: `device flash` must program the +right chip family (`-f NRF52` vs `NRF53`) and core (`--coprocessor`) for the +board, instead of always assuming nRF5340. Hardware-free — `run` (the nrfjprog +subprocess) and the J-Link picker are monkeypatched. +""" + +from pathlib import Path + +import pytest + +from dotbot.cli._fw_helpers import BARE_TARGETS +from dotbot.firmware import boards, flash, nrf + +# ── board spec table ──────────────────────────────────────────────────── + + +def test_spec_for_known_boards(): + assert boards.spec_for("nrf52840dk") == boards.BoardSpec("NRF52", None) + assert boards.spec_for("nrf52833dk") == boards.BoardSpec("NRF52", None) + assert boards.spec_for("dotbot-v3") == boards.BoardSpec("NRF53", "CP_APPLICATION") + assert boards.spec_for("nrf5340dk-app") == boards.BoardSpec( + "NRF53", "CP_APPLICATION" + ) + assert boards.spec_for("nrf5340dk-net") == boards.BoardSpec("NRF53", "CP_NETWORK") + + +def test_spec_for_unknown_board_falls_back_to_default(): + assert boards.spec_for("totally-made-up") == boards.DEFAULT_SPEC + assert boards.DEFAULT_SPEC.family == "NRF53" + + +def test_bare_targets_is_the_board_table(): + """BARE_TARGETS is derived from BOARDS — one source of truth.""" + assert set(BARE_TARGETS) == set(boards.BOARDS) + + +def test_is_multicore_family(): + assert boards.is_multicore_family("NRF53") is True + assert boards.is_multicore_family("NRF52") is False + + +def test_board_family_and_coprocessor_are_consistent(): + """A board carries a coprocessor iff its family is multi-core — so a bad + row (NRF52 with a coprocessor, or NRF53 without one) fails here.""" + for name, spec in boards.BOARDS.items(): + if boards.is_multicore_family(spec.family): + assert spec.coprocessor is not None, name + else: + assert spec.coprocessor is None, name + + +# ── nrfjprog arg construction (the actual bug) ────────────────────────── + + +@pytest.fixture +def capture_nrfjprog(monkeypatch): + calls = [] + + def fake_run(cmd, timeout=None, cwd=None): + calls.append(cmd) + return 0, "OK" + + monkeypatch.setattr("dotbot.firmware.nrf.run", fake_run) + return calls + + +def test_nrfjprog_program_nrf52_uses_nrf52_and_no_coprocessor(capture_nrfjprog): + nrf.nrfjprog_program("nrfjprog", Path("x.hex"), family="NRF52", chiperase=True) + args = capture_nrfjprog[0] + assert args[args.index("-f") + 1] == "NRF52" + assert "--coprocessor" not in args # nRF52 is single-core + + +def test_nrfjprog_program_nrf53_app_uses_cp_application(capture_nrfjprog): + nrf.nrfjprog_program("nrfjprog", Path("x.hex"), network=False, family="NRF53") + args = capture_nrfjprog[0] + assert args[args.index("-f") + 1] == "NRF53" + assert "CP_APPLICATION" in args + + +def test_nrfjprog_program_nrf53_net_uses_cp_network(capture_nrfjprog): + nrf.nrfjprog_program("nrfjprog", Path("x.hex"), network=True, family="NRF53") + assert "CP_NETWORK" in capture_nrfjprog[0] + + +# ── device flash routing: board → family/core ────────────────────────── + + +@pytest.fixture +def capture_one_core(monkeypatch): + calls = [] + + def fake_one_core( + app_hex=None, net_hex=None, family="NRF53", nrfjprog_opt=None, snr_opt=None + ): + calls.append({"app_hex": app_hex, "net_hex": net_hex, "family": family}) + + monkeypatch.setattr("dotbot.firmware.flash.flash_nrf_one_core", fake_one_core) + monkeypatch.setattr( + "dotbot.firmware.flash.pick_last_jlink_snr", lambda *a, **k: "777" + ) + return calls + + +def test_flash_app_image_nrf52_board_flashes_app_core_nrf52(tmp_path, capture_one_core): + img = tmp_path / "dotbot_gateway-nrf52840dk.hex" + img.write_text(":00000001FF\n") + flash.flash_app_image(img, board="nrf52840dk") + call = capture_one_core[0] + assert call["family"] == "NRF52" + assert call["app_hex"] == img and call["net_hex"] is None + + +def test_flash_app_image_net_board_flashes_net_core(tmp_path, capture_one_core): + img = tmp_path / "nrf5340_net-nrf5340dk-net.hex" + img.write_text(":00000001FF\n") + flash.flash_app_image(img, board="nrf5340dk-net") + call = capture_one_core[0] + assert call["family"] == "NRF53" + assert call["net_hex"] == img and call["app_hex"] is None + + +def test_flash_app_image_default_board_is_nrf53_app_core(tmp_path, capture_one_core): + img = tmp_path / "dotbot-dotbot-v3.hex" + img.write_text(":00000001FF\n") + flash.flash_app_image(img) # default board dotbot-v3 + call = capture_one_core[0] + assert call["family"] == "NRF53" and call["app_hex"] == img diff --git a/dotbot/tests/test_fw.py b/dotbot/tests/test_fw.py new file mode 100644 index 00000000..93d9eeb4 --- /dev/null +++ b/dotbot/tests/test_fw.py @@ -0,0 +1,757 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for `dotbot fw` (bare firmware build/clean/targets/artifacts). + +These tests stub `subprocess.call` / `subprocess.run` so they don't +need a SEGGER install or a DotBot-firmware checkout — they verify the +CLI's argument shape, validations, and the command line passed to +make, not the actual build. + +The single exception is `test_bare_targets_match_makefile_list_targets`, +which shells out to `make list-targets` in the real DotBot-firmware +repo to catch silent drift between the CLI's hardcoded enums and the +Makefile. It self-skips if the workspace layout or the `list-targets` +rule isn't available. +""" + +import subprocess +from pathlib import Path + +import click +import pytest +from click.testing import CliRunner + +from dotbot.cli import _fw_helpers +from dotbot.cli.fw import cmd as fw_cmd + + +@pytest.fixture +def runner(): + return CliRunner() + + +@pytest.fixture +def fake_repo(tmp_path, monkeypatch): + """Pretend repos/DotBot-firmware exists at a tmp path with a Makefile.""" + repo = tmp_path / "fake-dotbot-firmware" + repo.mkdir() + (repo / "Makefile").write_text("# fake\n") + monkeypatch.setenv("DOTBOT_FIRMWARE_REPO", str(repo)) + return repo + + +@pytest.fixture +def fake_segger(tmp_path, monkeypatch): + """Pretend SES is installed at a tmp path with a runnable emBuild.""" + segger = tmp_path / "fake-segger" + (segger / "bin").mkdir(parents=True) + embuild = segger / "bin" / "emBuild" + embuild.write_text("#!/bin/sh\nexit 0\n") + embuild.chmod(0o755) + monkeypatch.setenv("SEGGER_DIR", str(segger)) + return segger + + +@pytest.fixture +def capture_make(monkeypatch): + """Stub `subprocess.call` (the actual `make` invocation) and + `subprocess.run` (used by `list_projects` to enumerate buildable + apps) so the test never touches a real Makefile. + """ + calls = [] + + def fake_call(cmd, cwd=None, env=None): + calls.append({"cmd": cmd, "cwd": cwd, "env": env}) + return 0 + + def fake_run(cmd, cwd=None, env=None, **kw): + # Mimic `make -s list-projects` returning a small default set + # so build() can enumerate "apps to build" without erroring. + class _R: + returncode = 0 + stdout = "dotbot\nlh2_calibration\nlog_dump\n" + stderr = "" + + return _R() + + monkeypatch.setattr("dotbot.cli._fw_helpers.subprocess.call", fake_call) + monkeypatch.setattr("dotbot.cli._fw_helpers.subprocess.run", fake_run) + return calls + + +def test_fw_help_lists_real_subcommands(runner): + result = runner.invoke(fw_cmd, ["--help"]) + assert result.exit_code == 0 + for sub in ("build", "clean", "targets", "artifacts", "fetch", "list"): + assert sub in result.output + # Sandbox is a flavor flag now, not a separate namespace: + assert "--sandbox" in result.output + + +def test_fw_targets_lists_bare_targets_one_per_line(runner): + result = runner.invoke(fw_cmd, ["targets"]) + assert result.exit_code == 0 + lines = [ln for ln in result.output.splitlines() if ln.strip()] + assert "dotbot-v3" in lines + assert "sailbot-v1" in lines + # No sandbox-* targets under the bare namespace: + assert not any(ln.startswith("sandbox-") for ln in lines) + # One target per line, no decoration: + assert all(ln == ln.strip() for ln in lines) + + +def test_fw_build_rejects_sandbox_target_with_redirect_hint(runner): + """A `sandbox-` prefixed bare target points at the `--sandbox` flag.""" + result = runner.invoke(fw_cmd, ["build", "--target", "sandbox-dotbot-v3"]) + assert result.exit_code != 0 + assert "fw build -t dotbot-v3 --sandbox" in result.output + + +def test_fw_build_rejects_unknown_target_with_suggestion(runner): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbotv3"]) # missing dash + assert result.exit_code != 0 + assert "dotbot-v3" in result.output # didyoumean suggestion + + +def test_fw_build_default_target_is_dotbot_v3( + runner, fake_repo, fake_segger, capture_make +): + """No-arg build defaults to dotbot-v3 (Geovane's daily target).""" + result = runner.invoke(fw_cmd, ["build"]) + assert result.exit_code == 0, result.output + assert len(capture_make) == 1 + cmd = capture_make[0]["cmd"] + assert "BUILD_TARGET=dotbot-v3" in cmd + assert "BUILD_CONFIG=Release" in cmd # default per the plan + + +def test_fw_build_passes_incremental_by_default( + runner, fake_repo, fake_segger, capture_make +): + """Default is `BUILD_MODE=` (empty → emBuild's natural incremental + mode) for fast edit/build loop. SES 8.22a has no `-build` flag; the + only valid action flag is `-rebuild`.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_MODE=" in cmd + assert "BUILD_MODE=-rebuild" not in cmd + + +def test_fw_build_rebuild_flag_forces_full_rebuild( + runner, fake_repo, fake_segger, capture_make +): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "--rebuild"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_MODE=-rebuild" in cmd + + +def test_fw_build_quiet_by_default(runner, fake_repo, fake_segger, capture_make): + """Default is `QUIET=1` to suppress SES `-verbose -echo` flood.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "QUIET=1" in cmd + + +def test_fw_build_verbose_drops_quiet(runner, fake_repo, fake_segger, capture_make): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "-v"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "QUIET=1" not in cmd + + +def test_fw_build_with_app_appends_project_name( + runner, fake_repo, fake_segger, capture_make, monkeypatch +): + """`--app NAME` appends the project so make builds only that one.""" + monkeypatch.setattr( + "dotbot.cli.fw.list_projects", lambda target: ["dotbot", "lh2_calibration"] + ) + result = runner.invoke( + fw_cmd, ["build", "--target", "dotbot-v3", "--app", "dotbot"] + ) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert cmd[-1] == "dotbot" + + +def test_fw_build_rejects_unavailable_project( + runner, fake_repo, fake_segger, monkeypatch +): + """Project not in the post-filter list is rejected pre-make.""" + monkeypatch.setattr("dotbot.cli.fw.list_projects", lambda target: ["dotbot"]) + result = runner.invoke( + fw_cmd, ["build", "--target", "dotbot-v1", "--app", "dotbot_gateway"] + ) + assert result.exit_code != 0 + assert "not available" in result.output + + +def test_fw_clean_invokes_make_clean(runner, fake_repo, fake_segger, capture_make): + result = runner.invoke(fw_cmd, ["clean", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_TARGET=dotbot-v3" in cmd + assert "clean" in cmd + + +def test_fw_artifacts_builds_then_collects_to_user_dir( + runner, fake_repo, fake_segger, capture_make, tmp_path +): + """`dotbot fw artifacts` no longer runs `make artifacts` (whose path + formula is buggy for sandbox and writes to the firmware repo's + `artifacts/`). It does a regular build, then copies the produced + artifacts to the user-chosen out dir (default `./artifacts/`).""" + out = tmp_path / "user-artifacts" + result = runner.invoke( + fw_cmd, ["artifacts", "--target", "dotbot-v3", "--out", str(out)] + ) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + # Builds (no explicit make target), doesn't invoke `make artifacts`. + assert "artifacts" not in cmd + assert "BUILD_TARGET=dotbot-v3" in cmd + # The user-chosen out dir was created. + assert out.is_dir() + + +def test_fw_artifacts_print_path_requires_app(runner, fake_repo, fake_segger): + """`--print-path` without `--app` exits with a hint.""" + result = runner.invoke( + fw_cmd, ["artifacts", "--target", "dotbot-v3", "--print-path"] + ) + assert result.exit_code != 0 + assert "--app" in result.output + + +def test_fw_artifacts_print_path_returns_makefile_formula( + runner, fake_repo, fake_segger +): + result = runner.invoke( + fw_cmd, + ["artifacts", "--target", "dotbot-v3", "--app", "dotbot", "--print-path"], + ) + assert result.exit_code == 0, result.output + out = result.output.strip() + expected = str( + Path("apps") + / "dotbot" + / "Output" + / "dotbot-v3" + / "Release" + / "Exe" + / "dotbot-dotbot-v3.hex" + ) + assert out.endswith(expected) + + +def test_fw_new_still_not_implemented(runner): + """`new` is deferred to a separate templates plan.""" + result = runner.invoke(fw_cmd, ["new", "my-experiment"]) + assert result.exit_code == 2 + assert "not implemented" in result.output.lower() + + +# ── Sandbox flavor (`dotbot fw --sandbox`) ──────────────────────── +# Sandbox apps are no longer a separate `swarm fw` subgroup; they're the +# `--sandbox` flavor of the same `dotbot fw` commands (sandbox-, +# emits .bin, OTA-flashed via `dotbot swarm flash`). + + +def test_sandbox_targets_lists_boards(runner): + result = runner.invoke(fw_cmd, ["targets", "--sandbox"]) + assert result.exit_code == 0 + lines = [ln for ln in result.output.splitlines() if ln.strip()] + assert "dotbot-v3" in lines + assert "nrf5340dk" in lines + # User-facing names — no `sandbox-` prefix: + assert not any(ln.startswith("sandbox-") for ln in lines) + + +def test_sandbox_build_rejects_sandbox_prefix(runner): + """User shouldn't pass `sandbox-dotbot-v3` — drop the prefix.""" + result = runner.invoke( + fw_cmd, ["build", "--target", "sandbox-dotbot-v3", "--sandbox"] + ) + assert result.exit_code != 0 + assert "Drop the `sandbox-` prefix" in result.output + + +def test_sandbox_build_rejects_unknown_board(runner): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v9", "--sandbox"]) + assert result.exit_code != 0 + assert "Unknown sandbox board" in result.output + + +def test_sandbox_build_prepends_sandbox_prefix_to_target( + runner, fake_repo, fake_segger, capture_make +): + """`--sandbox --target dotbot-v3` becomes `BUILD_TARGET=sandbox-dotbot-v3`.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "--sandbox"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_TARGET=sandbox-dotbot-v3" in cmd + + +def test_sandbox_build_default_board(runner, fake_repo, fake_segger, capture_make): + result = runner.invoke(fw_cmd, ["build", "--sandbox"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_TARGET=sandbox-dotbot-v3" in cmd + assert "BUILD_CONFIG=Release" in cmd + + +def test_sandbox_artifacts_print_path_uses_bin_extension( + runner, fake_repo, fake_segger +): + """Sandbox artifacts are `.bin` (what swarmit OTA flashes), not `.hex`.""" + result = runner.invoke( + fw_cmd, + [ + "artifacts", + "--target", + "dotbot-v3", + "--app", + "dotbot", + "--sandbox", + "--print-path", + ], + ) + assert result.exit_code == 0, result.output + out = result.output.strip() + # SES's `$(BuildTarget)` macro now matches the make-level BUILD_TARGET + # (including the `sandbox-` prefix), so Output paths are flavor-distinct. + expected = str( + Path("apps-sandbox") + / "dotbot" + / "Output" + / "sandbox-dotbot-v3" + / "Release" + / "Exe" + / "dotbot-sandbox-dotbot-v3.bin" + ) + assert out.endswith(expected) + + +def test_sandbox_clean_invokes_make_clean(runner, fake_repo, fake_segger, capture_make): + result = runner.invoke(fw_cmd, ["clean", "--target", "dotbot-v3", "--sandbox"]) + assert result.exit_code == 0, result.output + cmd = capture_make[0]["cmd"] + assert "BUILD_TARGET=sandbox-dotbot-v3" in cmd + assert "clean" in cmd + + +def test_sandbox_artifacts_collected_filename_distinct_from_bare( + runner, fake_repo, fake_segger, capture_make, tmp_path, monkeypatch +): + """Sandbox artifacts collect with a filename naturally distinct from + any bare equivalent — `dotbot-sandbox-dotbot-v3.bin` vs bare + `dotbot-dotbot-v3.hex` — because SES's `$(BuildTarget)` macro now + includes the `sandbox-` prefix. No CLI-side mangling required.""" + src_dir = ( + fake_repo + / "apps-sandbox" + / "dotbot" + / "Output" + / "sandbox-dotbot-v3" + / "Release" + / "Exe" + ) + src_dir.mkdir(parents=True) + (src_dir / "dotbot-sandbox-dotbot-v3.bin").write_bytes(b"\xde\xad\xbe\xef") + monkeypatch.setattr("dotbot.cli.fw.list_projects", lambda target: ["dotbot"]) + out = tmp_path / "user-artifacts" + result = runner.invoke( + fw_cmd, + ["artifacts", "--target", "dotbot-v3", "--sandbox", "--out", str(out)], + ) + assert result.exit_code == 0, result.output + collected = list(out.iterdir()) + assert len(collected) == 1 + assert collected[0].name == "dotbot-sandbox-dotbot-v3.bin" + + +# ── Output polish: preamble, timing, gated make-line echo ─────────────── + + +def test_fw_build_quiet_does_not_echo_make_line( + runner, fake_repo, fake_segger, capture_make +): + """Default (no -v): make command line stays out of output.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + assert "$ make" not in result.output + + +def test_fw_build_verbose_echoes_make_line( + runner, fake_repo, fake_segger, capture_make +): + """-v echoes the full make command so it's copy-pasteable.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "-v"]) + assert result.exit_code == 0, result.output + assert "$ make" in result.output + assert "BUILD_TARGET=dotbot-v3" in result.output + + +def test_fw_build_prints_preamble_and_success( + runner, fake_repo, fake_segger, capture_make +): + """Happy path: preamble before make, success line with timing after.""" + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + assert "Building" in result.output + assert "dotbot-v3" in result.output + assert "Release" in result.output + assert "incremental" in result.output + # Success line uses a check mark + timing. + assert "✓" in result.output + assert "Built dotbot-v3" in result.output + + +def test_fw_build_rebuild_says_rebuild_in_preamble( + runner, fake_repo, fake_segger, capture_make +): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "--rebuild"]) + assert result.exit_code == 0, result.output + assert "rebuild" in result.output + assert "incremental" not in result.output + + +def test_fw_clean_prints_cleaned_success_line( + runner, fake_repo, fake_segger, capture_make +): + result = runner.invoke(fw_cmd, ["clean", "--target", "dotbot-v3"]) + assert result.exit_code == 0, result.output + assert "Cleaning dotbot-v3" in result.output + assert "✓ Cleaned" in result.output + + +def test_fw_artifacts_prints_collected_success_line( + runner, fake_repo, fake_segger, capture_make, tmp_path +): + result = runner.invoke( + fw_cmd, + ["artifacts", "--target", "dotbot-v3", "--out", str(tmp_path / "out")], + ) + assert result.exit_code == 0, result.output + assert "Building + collecting artifacts" in result.output + assert "✓ Collected" in result.output + + +def test_run_make_returns_elapsed_seconds(fake_repo, fake_segger, monkeypatch): + """`run_make` must return a float so subcommands can format the timing.""" + monkeypatch.setattr("dotbot.cli._fw_helpers.subprocess.call", lambda *a, **kw: 0) + elapsed = _fw_helpers.run_make("dotbot-v3", "Release", "dotbot") + assert isinstance(elapsed, float) + assert elapsed >= 0 + + +def test_sandbox_build_prints_preamble(runner, fake_repo, fake_segger, capture_make): + result = runner.invoke(fw_cmd, ["build", "--target", "dotbot-v3", "--sandbox"]) + assert result.exit_code == 0, result.output + assert "Building" in result.output + assert "sandbox" in result.output.lower() + assert "✓ Built" in result.output + + +# ── `dotbot fw make` escape hatch ─────────────────────────────────────── + + +from dotbot.cli.make import cmd as make_cmd # noqa: E402 + + +@pytest.fixture +def capture_make_passthrough(monkeypatch): + """Capture `subprocess.call` in dotbot.cli.make (the escape hatch). + + Distinct from `capture_make` (which patches `_fw_helpers.subprocess`) + because `make.py` imports `subprocess` directly. + """ + calls = [] + + def fake_call(cmd, cwd=None, env=None): + calls.append({"cmd": cmd, "cwd": cwd, "env": env}) + return 0 + + monkeypatch.setattr("dotbot.cli.make.subprocess.call", fake_call) + return calls + + +def test_dotbot_make_help_lists_examples(runner): + result = runner.invoke(make_cmd, ["--help"]) + assert result.exit_code == 0 + # Help should call out the workspace-resolved SEGGER_DIR — that's the + # entire point vs. raw `cd repos/DotBot-firmware && make ...`. + assert "SEGGER_DIR" in result.output + + +def test_dotbot_make_forwards_args_verbatim( + runner, fake_repo, fake_segger, capture_make_passthrough +): + """`dotbot fw make foo bar BAZ=qux` invokes `make foo bar BAZ=qux`.""" + result = runner.invoke( + make_cmd, ["help", "BUILD_TARGET=dotbot-v3", "PACKAGES_DIR_OPT=-p /opt"] + ) + assert result.exit_code == 0 + assert len(capture_make_passthrough) == 1 + cmd = capture_make_passthrough[0]["cmd"] + assert cmd[0] == "make" + assert "help" in cmd + assert "BUILD_TARGET=dotbot-v3" in cmd + assert "PACKAGES_DIR_OPT=-p /opt" in cmd + + +def test_dotbot_make_runs_in_firmware_repo( + runner, fake_repo, fake_segger, capture_make_passthrough +): + result = runner.invoke(make_cmd, ["list-targets"]) + assert result.exit_code == 0 + assert capture_make_passthrough[0]["cwd"] == fake_repo + + +def test_dotbot_make_injects_segger_dir( + runner, fake_repo, fake_segger, capture_make_passthrough +): + """SEGGER_DIR is set in the make env regardless of what the user passes.""" + result = runner.invoke(make_cmd, ["help"]) + assert result.exit_code == 0 + env = capture_make_passthrough[0]["env"] + assert env["SEGGER_DIR"] == str(fake_segger) + + +def test_dotbot_make_propagates_make_exit_code( + runner, fake_repo, fake_segger, monkeypatch +): + monkeypatch.setattr("dotbot.cli.make.subprocess.call", lambda *a, **kw: 7) + result = runner.invoke(make_cmd, ["bogus-target"]) + assert result.exit_code == 7 + + +# ── Help-text footer pointing at the escape hatch ─────────────────────── + + +def test_fw_help_points_at_dotbot_make(runner): + result = runner.invoke(fw_cmd, ["--help"]) + assert result.exit_code == 0 + assert "dotbot fw make" in result.output + + +# ── Helper-level tests ────────────────────────────────────────────────── + + +@pytest.fixture +def isolated_home(tmp_path, monkeypatch): + """Point the unified config's user file at a tmp dir and run in a clean + cwd, so fw config tests don't see the real `~/.dotbot/config.toml` or a + stray `dotbot.toml`.""" + home = tmp_path / "home" + (home / ".dotbot").mkdir(parents=True) + monkeypatch.setattr( + "dotbot.config.USER_CONFIG_PATH", + home / ".dotbot" / "config.toml", + ) + work = tmp_path / "work" + work.mkdir() + monkeypatch.chdir(work) + return home + + +def _write_config(home, toml_body): + (home / ".dotbot" / "config.toml").write_text(toml_body) + + +def test_resolve_segger_dir_uses_env_first(tmp_path, monkeypatch, isolated_home): + """Env var beats config file beats glob.""" + _write_config(isolated_home, '[fw]\nsegger_dir = "/from/config"\n') + monkeypatch.setenv("SEGGER_DIR", str(tmp_path)) + assert _fw_helpers.resolve_segger_dir() == tmp_path + + +def test_resolve_segger_dir_falls_back_to_config(monkeypatch, isolated_home): + """When SEGGER_DIR is unset, `[fw].segger_dir` from the config wins.""" + _write_config(isolated_home, '[fw]\nsegger_dir = "/from/config"\n') + monkeypatch.delenv("SEGGER_DIR", raising=False) + assert _fw_helpers.resolve_segger_dir() == Path("/from/config") + + +def test_resolve_segger_dir_uses_macos_glob_when_no_env_or_config( + tmp_path, monkeypatch, isolated_home +): + """macOS fallback: glob `/Applications/SEGGER/SEGGER Embedded Studio*`.""" + monkeypatch.delenv("SEGGER_DIR", raising=False) + fake_install = tmp_path / "SEGGER Embedded Studio 9.99" + (fake_install / "bin").mkdir(parents=True) + (fake_install / "bin" / "emBuild").touch() + monkeypatch.setattr("dotbot.cli._fw_helpers.sys.platform", "darwin") + monkeypatch.setattr( + "dotbot.cli._fw_helpers._SEGGER_MACOS_GLOB", + str(tmp_path / "SEGGER Embedded Studio*"), + ) + assert _fw_helpers.resolve_segger_dir() == fake_install + + +def test_resolve_segger_dir_picks_latest_glob_match( + tmp_path, monkeypatch, isolated_home +): + """Multiple SES installs → lexicographically-latest wins (newer version).""" + monkeypatch.delenv("SEGGER_DIR", raising=False) + for v in ("8.22a", "8.30a", "9.10"): + d = tmp_path / f"SEGGER Embedded Studio {v}" / "bin" + d.mkdir(parents=True) + (d / "emBuild").touch() + monkeypatch.setattr("dotbot.cli._fw_helpers.sys.platform", "darwin") + monkeypatch.setattr( + "dotbot.cli._fw_helpers._SEGGER_MACOS_GLOB", + str(tmp_path / "SEGGER Embedded Studio*"), + ) + picked = _fw_helpers.resolve_segger_dir() + assert picked.name == "SEGGER Embedded Studio 9.10" + + +def test_resolve_segger_dir_errors_when_nothing_found(monkeypatch, isolated_home): + monkeypatch.delenv("SEGGER_DIR", raising=False) + monkeypatch.setattr("dotbot.cli._fw_helpers.sys.platform", "linux") + with pytest.raises(click.ClickException) as excinfo: + _fw_helpers.resolve_segger_dir() + # Error message must surface BOTH escape hatches so the user can fix + # whichever they prefer. + msg = str(excinfo.value) + assert "SEGGER_DIR" in msg + assert "~/.dotbot/config.toml" in msg + + +def test_resolve_firmware_repo_finds_sibling_clone( + tmp_path, monkeypatch, isolated_home +): + """The fallback lookup path: `/DotBot-firmware/Makefile`.""" + repo = tmp_path / "DotBot-firmware" + repo.mkdir() + (repo / "Makefile").touch() + monkeypatch.chdir(tmp_path) + monkeypatch.delenv("DOTBOT_FIRMWARE_REPO", raising=False) + assert _fw_helpers.resolve_firmware_repo() == repo + + +def test_resolve_firmware_repo_env_var_wins(tmp_path, monkeypatch, isolated_home): + """Env var overrides the config and the CWD-sibling default.""" + sibling = tmp_path / "DotBot-firmware" + sibling.mkdir() + (sibling / "Makefile").touch() + elsewhere = tmp_path / "elsewhere" / "DotBot-firmware" + elsewhere.mkdir(parents=True) + (elsewhere / "Makefile").touch() + monkeypatch.chdir(tmp_path) + monkeypatch.setenv("DOTBOT_FIRMWARE_REPO", str(elsewhere)) + assert _fw_helpers.resolve_firmware_repo() == elsewhere + + +def test_resolve_firmware_repo_falls_back_to_config( + tmp_path, monkeypatch, isolated_home +): + """No env var and no ./DotBot-firmware -> `[fw].firmware_repo` from config wins.""" + repo = tmp_path / "fw-clone" + repo.mkdir() + (repo / "Makefile").touch() + _write_config(isolated_home, f'[fw]\nfirmware_repo = "{repo.as_posix()}"\n') + monkeypatch.delenv("DOTBOT_FIRMWARE_REPO", raising=False) + monkeypatch.chdir(tmp_path) # no ./DotBot-firmware here + assert _fw_helpers.resolve_firmware_repo() == repo + + +def test_resolve_firmware_repo_errors_when_nothing_found(tmp_path, monkeypatch): + """No env var, no `/DotBot-firmware/` → clear error with both + escape hatches in the message.""" + monkeypatch.chdir(tmp_path) + monkeypatch.delenv("DOTBOT_FIRMWARE_REPO", raising=False) + with pytest.raises(click.ClickException) as excinfo: + _fw_helpers.resolve_firmware_repo() + msg = str(excinfo.value) + assert "DOTBOT_FIRMWARE_REPO" in msg + assert "cd" in msg # the "cd to the directory containing your clone" hint + + +def test_resolve_firmware_repo_env_var_pointing_at_no_makefile_errors( + tmp_path, monkeypatch +): + """Bad env-var path fails loudly rather than silently falling back.""" + bad = tmp_path / "no-makefile-here" + bad.mkdir() + monkeypatch.setenv("DOTBOT_FIRMWARE_REPO", str(bad)) + with pytest.raises(click.ClickException) as excinfo: + _fw_helpers.resolve_firmware_repo() + assert "DOTBOT_FIRMWARE_REPO" in str(excinfo.value) + assert "Makefile" in str(excinfo.value) + + +def test_malformed_config_raises_with_path(monkeypatch, isolated_home): + """A malformed config surfaces a clean error naming the file, not a traceback.""" + _write_config(isolated_home, "this is not [valid toml\n") + monkeypatch.delenv("SEGGER_DIR", raising=False) + with pytest.raises(click.ClickException) as excinfo: + _fw_helpers.resolve_segger_dir() + assert "config.toml" in str(excinfo.value) + + +# ── Parity guard against silent drift ─────────────────────────────────── + + +def _real_firmware_repo_or_skip(): + """Find the real DotBot-firmware repo for the parity test, or skip.""" + import os + from pathlib import Path + + env = os.environ.get("DOTBOT_FIRMWARE_REPO") + if env and (Path(env) / "Makefile").is_file(): + return Path(env) + here = Path(__file__).resolve() + for parent in here.parents: + candidate = parent / "repos" / "DotBot-firmware" + if (candidate / "Makefile").is_file(): + return candidate + pytest.skip( + "Could not locate the real DotBot-firmware repo; set " + "DOTBOT_FIRMWARE_REPO or run from inside the workspace." + ) + + +def test_targets_match_makefile_list_targets(): + """`set(BARE_TARGETS) | set('sandbox-'+SANDBOX_BOARDS)` must equal what + the Makefile reports via `make list-targets`. + + Catches the silent drift case where someone adds e.g. dotbot-v4 to + the Makefile and forgets to update the CLI's hardcoded enum. + + Self-skips if the real DotBot-firmware repo or the `list-targets` + Make rule isn't available (older checkout pre-dating that commit). + """ + repo = _real_firmware_repo_or_skip() + try: + result = subprocess.run( + ["make", "-s", "list-targets"], + cwd=repo, + capture_output=True, + text=True, + timeout=10, + ) + except (subprocess.SubprocessError, FileNotFoundError): + pytest.skip("`make list-targets` not runnable in this environment.") + if result.returncode != 0: + pytest.skip( + "`make list-targets` rule not present in this DotBot-firmware " + "checkout. Bump the submodule / pull a newer Makefile to enable " + "this parity guard." + ) + makefile_targets = { + line.strip() for line in result.stdout.splitlines() if line.strip() + } + cli_targets = set(_fw_helpers.BARE_TARGETS) | { + f"sandbox-{b}" for b in _fw_helpers.SANDBOX_BOARDS + } + assert makefile_targets == cli_targets, ( + f"CLI hardcoded targets drifted from Makefile.\n" + f"In CLI but not Makefile: {cli_targets - makefile_targets}\n" + f"In Makefile but not CLI: {makefile_targets - cli_targets}" + ) diff --git a/dotbot/tests/test_gateway.py b/dotbot/tests/test_gateway.py new file mode 100644 index 00000000..08f44c6f --- /dev/null +++ b/dotbot/tests/test_gateway.py @@ -0,0 +1,101 @@ +# SPDX-FileCopyrightText: 2026-present Inria +# SPDX-License-Identifier: BSD-3-Clause + +"""Tests for `dotbot run gateway` — the CLI surface, not the live bridge. + +The bridge itself (`_run_gateway`) needs a real serial gateway, so it's +mocked here; we check flag parsing and that the command forwards +`--port` / `--mqtt-url` correctly. +""" + +from unittest.mock import patch + +from click.testing import CliRunner + +from dotbot.cli.gateway import cmd as gateway_cmd +from dotbot.cli.main import cli + + +def _write_config(tmp_path, text): + path = tmp_path / "dotbot.toml" + path.write_text(text) + return path + + +def test_gateway_help_mentions_print_and_broker(): + result = CliRunner().invoke(gateway_cmd, ["--help"]) + assert result.exit_code == 0 + assert "--port" in result.output + assert "--mqtt-url" in result.output + assert "--no-print" in result.output + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_forwards_port_mqtt_url_and_print(run): + result = CliRunner().invoke( + gateway_cmd, + ["--port", "/dev/ttyACM0", "--mqtt-url", "mqtts://argus:8883"], + ) + assert result.exit_code == 0, result.output + # print defaults to True. + run.assert_called_once_with("/dev/ttyACM0", "mqtts://argus:8883", True) + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_no_mqtt_defaults_print_on(run): + result = CliRunner().invoke(gateway_cmd, ["--port", "/dev/ttyACM0"]) + assert result.exit_code == 0, result.output + run.assert_called_once_with("/dev/ttyACM0", None, True) + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_no_print_flag(run): + result = CliRunner().invoke(gateway_cmd, ["--port", "/dev/ttyACM0", "--no-print"]) + assert result.exit_code == 0, result.output + run.assert_called_once_with("/dev/ttyACM0", None, False) + + +# --- deployment fallback (through the root group) --------------------------- + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_falls_back_to_deployment_broker(run, tmp_path): + """No --mqtt-url -> the selected deployment's MQTT conn reaches the bridge.""" + cfg = _write_config( + tmp_path, + 'default_deployment = "lab"\n' + "[deployment.lab]\n" + 'conn = "mqtts://broker:8883"\n', + ) + result = CliRunner().invoke(cli, ["-c", str(cfg), "run", "gateway"]) + assert result.exit_code == 0, result.output + run.assert_called_once_with(None, "mqtts://broker:8883", True) + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_cli_mqtt_url_beats_deployment(run, tmp_path): + """An explicit --mqtt-url wins over the deployment's conn.""" + cfg = _write_config( + tmp_path, + 'default_deployment = "lab"\n' + "[deployment.lab]\n" + 'conn = "mqtts://broker:8883"\n', + ) + result = CliRunner().invoke( + cli, + ["-c", str(cfg), "run", "gateway", "--mqtt-url", "mqtts://override:8883"], + ) + assert result.exit_code == 0, result.output + run.assert_called_once_with(None, "mqtts://override:8883", True) + + +@patch("dotbot.cli.gateway._run_gateway") +def test_gateway_non_mqtt_deployment_conn_stays_print_only(run, tmp_path): + """A serial/simulator deployment conn is not a broker -> mqtt_url stays None.""" + cfg = _write_config( + tmp_path, + 'default_deployment = "lab"\n' "[deployment.lab]\n" 'conn = "simulator"\n', + ) + result = CliRunner().invoke(cli, ["-c", str(cfg), "run", "gateway"]) + assert result.exit_code == 0, result.output + run.assert_called_once_with(None, None, True) diff --git a/dotbot/tests/test_protocol.py b/dotbot/tests/test_protocol.py index 838fa29a..1896e672 100644 --- a/dotbot/tests/test_protocol.py +++ b/dotbot/tests/test_protocol.py @@ -56,8 +56,11 @@ class PayloadWithBytesFixedLengthTest(Payload): data: bytes = b"" -register_parser(0x81, PayloadWithBytesTest) -register_parser(0x82, PayloadWithBytesFixedLengthTest) +# Test-only payload types, deliberately clear of dotbot's real types (<= 0xfa) +# and swarmit's (0x80-0xa1): both register into this shared dotbot_utils registry +# and swarmit (a core dep) may be imported in the same process. +register_parser(0xFB, PayloadWithBytesTest) +register_parser(0xFC, PayloadWithBytesFixedLengthTest) @pytest.mark.parametrize( @@ -235,20 +238,20 @@ def test_parse_header(bytes_, expected): ), pytest.param( b"\x01\x10\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00" # header - b"\x81" # payload type + b"\xfb" # payload type b"\x08" # count b"abcdefgh", # data Header(), - 0x81, + 0xFB, PayloadWithBytesTest(count=8, data=b"abcdefgh"), id="PayloadWithBytesTest", ), pytest.param( b"\x01\x10\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00" # header - b"\x82" # payload type + b"\xfc" # payload type b"abcdefgh", # data Header(), - 0x82, + 0xFC, PayloadWithBytesFixedLengthTest(data=b"abcdefgh"), id="PayloadWithBytesFixedLengthTest", ), @@ -491,7 +494,7 @@ def test_frame_parser(bytes_, header, payload_type, payload): ), ), b"\x01\x10\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00" # header - b"\x81" # payload type + b"\xfb" # payload type b"\x08" # count b"abcdefgh", # data id="PayloadWithBytesTest", @@ -504,7 +507,7 @@ def test_frame_parser(bytes_, header, payload_type, payload): ), ), b"\x01\x10\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00" # header - b"\x82" # payload type + b"\xfc" # payload type b"abcdefgh", # data id="PayloadWithBytesFixedLengthTest", ), @@ -735,7 +738,7 @@ def test_payload_to_bytes(frame, expected): ( " +------+------+--------------------+--------------------+------+\n" " CUSTOM_DATA | ver. | type | dst | src | type |\n" - " (28 Bytes) | 0x01 | 0x10 | 0xffffffffffffffff | 0x0000000000000000 | 0x81 |\n" + " (28 Bytes) | 0x01 | 0x10 | 0xffffffffffffffff | 0x0000000000000000 | 0xfb |\n" " +------+------+--------------------+--------------------+------+\n" " +------+--------------------+\n" " | len. | data |\n" @@ -755,7 +758,7 @@ def test_payload_to_bytes(frame, expected): ( " +------+------+--------------------+--------------------+------+\n" " CUSTOM_DATA | ver. | type | dst | src | type |\n" - " (27 Bytes) | 0x01 | 0x10 | 0xffffffffffffffff | 0x0000000000000000 | 0x82 |\n" + " (27 Bytes) | 0x01 | 0x10 | 0xffffffffffffffff | 0x0000000000000000 | 0xfc |\n" " +------+------+--------------------+--------------------+------+\n" " +--------------------+\n" " | data |\n" diff --git a/pyproject.toml b/pyproject.toml index a4f8b274..09097996 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,6 +7,7 @@ build-backend = "hatchling.build" [tool.hatch.build] include = [ "dotbot/frontend/*", + "dotbot/*.toml", "*.py" ] exclude = [ @@ -22,7 +23,7 @@ path = "utils/hooks/sdist.py" [project] name = "pydotbot" -version = "0.27.0" +version = "0.29.0" authors = [ { name="Alexandre Abadie", email="alexandre.abadie@inria.fr" }, { name="Theo Akbas", email="theo.akbas@inria.fr" }, @@ -30,6 +31,7 @@ authors = [ { name="Said Alvarado-Marin", email="said-alexander.alvarado-marin@inria.fr" }, { name="Mališa Vučinić", email="malisa.vucinic@inria.fr" }, { name="Diego Badillo", email="diego.badillo@sansano.usm.cl" }, + { name="Geovane Fedrecheski", email="geovane.fedrecheski@inria.fr" }, ] dependencies = [ "click >= 8.1.7", @@ -40,14 +42,18 @@ dependencies = [ "pynput >= 1.8.1", "pyserial >= 3.5", "qrkey >= 0.12.2", + "requests >= 2.31.0", "rich >= 14.0.0", "structlog >= 24.4.0", "uvicorn >= 0.32.0", "websockets >= 13.1.0", "gmqtt >= 0.7.0", - "marilib-pkg >= 0.8.0", + "intelhex >= 2.3.0", + "marilib-pkg >= 0.9.0", "pydotbot-utils >= 0.3.0", + "swarmit >= 0.8.0", "toml >= 0.10.2", + "tomlkit >= 0.13.0", ] description = "Package to easily control your DotBots and SailBots." readme = "README.md" @@ -67,13 +73,31 @@ classifiers = [ "Bug Tracker" = "https://github.com/DotBots/PyDotBot/issues" [project.scripts] -dotbot-controller = "dotbot.controller_app:main" -dotbot-edge-gateway = "dotbot.edge_gateway_app:main" -dotbot-keyboard = "dotbot.keyboard:main" -dotbot-joystick = "dotbot.joystick:main" -# dotbot-qrkey console script removed — the demo is now an example, -# run via `python -m dotbot.examples.qrkey_demo`. See -# dotbot/examples/qrkey_demo/README.md. +# The unified dispatcher is the only console script. Every workflow is a +# `dotbot ` subcommand — there are no per-command `dotbot-*` +# binaries. +dotbot = "dotbot.cli.main:cli" + +[project.optional-dependencies] +# Optional subcommand backends. Keep the core install lean; opt in to +# the bits you actually use. +calibrate = [ + "opencv-python >= 4.12.0.88", + "textual >= 6.4.0", +] +all = [ + "opencv-python >= 4.12.0.88", + "textual >= 6.4.0", +] +dev = [ + "pytest", + "pytest-asyncio", + "pytest-cov", + "ruff", + "black", + "isort", + "pre-commit", +] [tool.ruff] lint.select = ["E", "F"] diff --git a/setup.cfg b/setup.cfg index 8b489609..9c997d80 100644 --- a/setup.cfg +++ b/setup.cfg @@ -7,9 +7,29 @@ addopts = -vv -s --cov-report=term --cov-report=term-missing --cov-report=xml + --ignore=dotbot/calibration testpaths = dotbot asyncio_default_fixture_loop_scope = function +# Why --ignore dotbot/calibration: --doctest-modules walks every .py file +# to find doctests, importing each. dotbot/calibration/* imports textual + +# cv2, gated behind the [calibrate] extra for size reasons; the CI test env +# doesn't install it, so doctest discovery crashes on import. It has no +# doctests, so the ignore only skips import-failure noise; its real tests +# live in dotbot/tests/. (dotbot/firmware/* needs no ignore — its sole +# optional dep, intelhex, is now a core dependency.) + +[coverage:run] +# dotbot/firmware/* is the hardware flash engine vendored from the +# standalone dotbot-provision package (and dotbot/calibration from +# dotbot-lh2-calibration); both shipped with ~0% host-test coverage and +# most of their logic is HIL-only. Excluded from coverage so the vendored +# bytes don't tank the signal for the rest of the codebase. Real host +# tests for the unit-testable parts are tracked as a separate follow-up. +omit = + dotbot/calibration/* + dotbot/firmware/* + [tool.black] line-length = 79 skip-string-normalization = true diff --git a/tox.ini b/tox.ini index f2e9a9fe..193a7eef 100644 --- a/tox.ini +++ b/tox.ini @@ -7,7 +7,6 @@ isolated_build = true allowlist_externals = cli: {[testenv:cli]allowlist_externals} web: {[testenv:web]allowlist_externals} - pin_code: {[testenv:pin_code]allowlist_externals} commands= tests: {[testenv:tests]commands} check: {[testenv:check]commands} @@ -50,7 +49,8 @@ allowlist_externals= /bin/bash /usr/bin/bash commands= - bash -exc "dotbot-controller --help" + bash -exc "dotbot --help" + bash -exc "dotbot run controller --help" [testenv:web] allowlist_externals= @@ -58,11 +58,8 @@ allowlist_externals= /usr/bin/bash commands = bash -exc "cd dotbot/frontend && npm run lint" -[testenv:pin_code] -allowlist_externals= - /bin/bash - /usr/bin/bash -commands = bash -exc "cd dotbot/pin_code_ui && npm test && npm run lint" +# pin_code env removed: dotbot/pin_code_ui never existed in the tree — +# silent dead config flagged in workspace AGENTS.md. [testenv:format] deps=