Spiking Control

This repository contains research code for end-to-end trained spiking neural networks (SNNs) for continuous robotic control, developed between 2022 and 2026 by Justus Hübotter.

The project was initiated as part of the
SPIKEFERENCE project,
co-funded by the Human Brain Project (HBP) – SGA3 (Grant 945539) and the
Donders Institute for Brain, Cognition and Behaviour, Radboud University.

Over several iterations, this work evolved from early transition-model-based control ideas into a fully spiking, model-based predictive control framework capable of learning continuous low-level control (velocity, acceleration, and torque) for robotic manipulators.

---

Overview

The core idea explored in this repository is that fully spiking neural networks can be trained end-to-end—using surrogate gradients—to solve continuous control tasks, without ANN pretraining, conversion, or hand-designed controllers.

The approach follows a predictive-control architecture composed of two spiking networks:

• Prediction model (forward model)
  Learns system dynamics by predicting future state changes given current state and action.

• Policy model (inverse model)
  Learns goal-directed actions by backpropagating task errors through the learned prediction model, using imagined rollouts.

This structure enables model-based learning in the spiking domain, closely related to world models and predictive coding, while remaining compatible with modern deep-learning-style optimization.

---

📄 Associated Publication

The final and most complete description of the method, experiments, and ablation studies is provided in the following paper:

Justus Hübotter, Pablo Lanillos, Marcel van Gerven, Serge Thill (2025)
Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learning
arXiv:2509.05356
https://arxiv.org/abs/2509.05356
:contentReference[oaicite:0]{index=0}

This paper presents:

• A fully spiking Pred-Control SNN architecture
• End-to-end surrogate-gradient training
• Extensive ablations on:
  • time constants
  • adaptive thresholds (ALIF neurons)
  • initialization
  • recurrence
  • low-rank compression
• Continuous control of:
  • a 2D planar reaching task
  • a 6-DoF Franka Emika Panda robot (torque control)

Earlier conceptual groundwork appeared in:

J. Hübotter, S. Thill, M. van Gerven, P. Lanillos (2022)
Learning Policies for Continuous Control via Transition Models
3rd International Workshop on Active Inference
https://arxiv.org/abs/2209.08033

---

Code Structure & Status

This repository reflects an active research codebase, not a minimal benchmark implementation.
Some parts are experimental, optimized for flexibility and ablation studies rather than simplicity.

• Core training logic is stable and used for all reported results
• Configuration is managed via Hydra
• Logging and experiment tracking is done with Weights & Biases
• Some scripts may assume familiarity with the paper

New visualizations (gifs / videos of trained agents) will be added in a future update.

---

Installation

Clone the repository and install dependencies (preferably in a fresh environment):

git clone https://github.com/jhuebotter/spiking_control.git
cd spiking_control
pip install -r requirements.txt

Control Stork (required)

All spiking network components are implemented using Control Stork, a PyTorch-based framework for spiking control models.

git clone https://github.com/jhuebotter/control_stork.git
cd control_stork
pip install -r requirements.txt
pip install -e .

---

Experiment Configuration

We use Hydra to manage experiment settings.

• All configs live in the conf/ directory
• Model variants, environments, and ablations are controlled via YAML
• Experiments are fully reproducible via config overrides

Logging with Weights & Biases (optional)

To enable experiment tracking:

1. Create a Weights & Biases account
2. Ensure wandb: true is set in config.yaml
3. Authenticate locally:

wandb login

---

Supported Environments

Currently supported continuous control environments include:

Planar Tasks

• plane Simple linear planar system

  • velocity or acceleration control

• reacher 2-DoF planar robot arm

  • velocity or acceleration control
  • static or moving targets

3D Robotic Arm

• franka 6-DoF Franka Emika Panda (Isaac Sim / Isaac Lab)

  • torque, velocity, or acceleration control
  • optional gravity

---

Example Results

(Existing gifs are shown below; updated visualizations will follow.)

Planar Control

Planar Robot Arm

Learned Dynamics Model

The prediction network learns accurate autoregressive state rollouts:

---

Citation

If you use this code or build upon it, please cite:

@article{huebotter2025spiking,
  title={Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learning},
  author={H{\"u}botter, Justus and Lanillos, Pablo and van Gerven, Marcel and Thill, Serge},
  journal={arXiv preprint arXiv:2509.05356},
  year={2025}
}

---

License & Acknowledgments

This work was partially supported by the SPIKEFERENCE project (Human Brain Project SGA3, Grant Agreement No. 945539).

The code is released for research purposes. Please see the LICENSE file for details.

---

Questions, issues, or contributions are welcome — but please note that this repository reflects an evolving research project rather than a polished library.