jetfx ( a personal Audio DSP + Guitar NAM that runs on the Jetson Platform )

jetfx is a low-latency, JACK-based live audio processor for vocal + guitar workflows. It runs two processing workers in parallel:

• Guitar path: optional Neural Amp Model (NAM) inference + optional cabinet model inference.
• Vocal path: voice processing modes (live and fx) with a configurable vocal strip (HPF, compressor, de-esser, limiter).

At runtime, you control the engine through stdin commands (no restart required), and it prints level meters once per second. It also starts a local Web UI + REST/WebSocket control server on port 7878 by default.

---

Startup Guide

1) Prerequisites

• Linux system with JACK installed (jackd). jetfx now asks JACK to auto-start the server if it is not already running.
• Rust toolchain (edition 2024 capable).
• Audio device routes available as JACK system:capture_* and system:playback_* ports.

The app attempts auto-connection on startup by taking the first two discovered system:capture* and system:playback* JACK ports. In a standard setup this maps to:

• system:capture_1 -> jetfx:mic_in
• system:capture_2 -> jetfx:gtr_in
• jetfx:out_l -> system:playback_1
• jetfx:out_r -> system:playback_2

2) Build

cargo build

Optional NAM inference backend (ort)

By default, the project is built without ONNX Runtime inference and NAM processing falls back to bypass.

Build with:

cargo build --features ort

Notes:

• with ort enabled, NAM is configured for TensorRT execution provider only (JETFX_NAM_EP=tensorrt). If TensorRT cannot initialize, NAM/IR stay disabled (no CPU fallback).
• the dependency is configured with default-features = false and load-dynamic, so Cargo does not need a system OpenSSL install just to compile. You still need ONNX Runtime libraries available at runtime when using ort.
• if you see Failed to load ONNX Runtime dylib at runtime, set ORT_DYLIB_PATH (full path to libonnxruntime.so) or add that directory to LD_LIBRARY_PATH.

3) Configure via jetfx.toml (recommended) or environment variables

You can run without exports by placing a jetfx.toml file in the project root (or .jetfx.toml). A ready-to-edit template is provided at jetfx.toml.example.

[env]
JETFX_NAM_MODEL = "./models/nam_rt_1024.onnx"
JETFX_CAB_MODEL = "./models/CabinetIR_1024_from_wav.onnx"
JETFX_NAM_WINDOW = 256
JETFX_NAM_EP = "tensorrt"
JETFX_TRT_FP16 = true
JETFX_TRT_ENGINE_CACHE = true
JETFX_TRT_ENGINE_CACHE_PATH = ".jetfx_trt_cache"
JETFX_TRT_TIMING_CACHE = true
JETFX_TRT_TIMING_CACHE_PATH = ".jetfx_trt_cache"
JETFX_FS = 48000
ORT_DYLIB_PATH = "/full/path/to/libonnxruntime.so"

Environment variables still work and override defaults when needed:

export JETFX_NAM_MODEL="/path/to/model.onnx"
export JETFX_CAB_MODEL="/path/to/cab_model.onnx"
export JETFX_NAM_WINDOW="256"
export JETFX_NAM_EP="tensorrt"   # tensorrt|trt
export JETFX_TRT_FP16="1"       # 1|0, default 1
export JETFX_TRT_ENGINE_CACHE="1"  # 1|0, default 1
export JETFX_TRT_ENGINE_CACHE_PATH=".jetfx_trt_cache"
export JETFX_TRT_TIMING_CACHE="1"  # 1|0, default 1
export JETFX_TRT_TIMING_CACHE_PATH=".jetfx_trt_cache"
export JETFX_FS="48000"
export ORT_DYLIB_PATH="/full/path/to/libonnxruntime.so"

• JETFX_NAM_MODEL: initial NAM model path.
• JETFX_CAB_MODEL: initial cabinet model path (TensorRT-capable ONNX).
• JETFX_NAM_WINDOW: NAM window size (clamped in code).
• JETFX_NAM_EP: execution provider selector for NAM (tensorrt/trt only; other values are ignored).
• JETFX_TRT_FP16: enables TensorRT FP16 kernels when supported (1 default).
• JETFX_TRT_ENGINE_CACHE: enables TensorRT engine caching (1 default).
• JETFX_TRT_ENGINE_CACHE_PATH: engine cache directory (default .jetfx_trt_cache).
• JETFX_TRT_TIMING_CACHE: enables TensorRT timing cache (1 default).
• JETFX_TRT_TIMING_CACHE_PATH: timing cache directory (default .jetfx_trt_cache).
• JETFX_FS: sample rate used by VC worker internals.
• ORT_DYLIB_PATH: path to libonnxruntime.so when using dynamic ORT loading.

Cabinet model contract (required)

The cabinet stage now runs as a TensorRT-backed ONNX model (not text FIR taps). For reliable behavior:

• input must be static audio shape compatible with [1, N] where N is fixed,
• output must match the input window length,
• model should be stateless (no recurrent state tensors),
• window size should be in 32..16384.

If contract checks fail at load time, cabinet processing is disabled and the runtime logs indicate will indicate.

4) Run

cargo run

That is enough for a default local setup (no env var exports needed). If JACK is not already running, jetfx now attempts to launch:

jackd -P95 -dalsa -dhw:0 -r48000 -p1024 -n3 -s

Or with ORT feature:

cargo run --features ort

On startup you should see:

• a controls help list,
• periodic meters ... lines,
• jetfx running. Ctrl+C to quit.
• web UI running at http://0.0.0.0:7878

Web remote control (phone/laptop on same network)

Open http://<jetson-ip>:7878 from any device on the same LAN.

• GET /api/state current control snapshot.
• PUT /api/state patch controls (bools, floats, texts).
• GET /api/presets list scene names.
• POST /api/presets/save/:name save scene.
• POST /api/presets/load/:name load scene.
• GET /ws/meters websocket stream at ~10 Hz.

Set JETFX_WEB_PORT to change the listen port.

5) Stop

Press Ctrl+C in the terminal.

---

User Guide

Audio routing behavior

• Inputs:
  • mic_in receives vocal source.
  • gtr_in receives guitar source.
• Outputs:
  • out_l, out_r.
• Routing modes:
  • route mono: vocal+guitar summed to both L/R.
  • route split: vocal to left, guitar to right.

Real-time controls (stdin)

Type commands directly into the running terminal.

Enable/disable blocks

strip on|off
hpf on|off
comp on|off
lim on|off
deess on|off
invert on|off
mute on|off
nam_bypass on|off
cab_bypass on|off
vc_bypass on|off

• nam_bypass bypasses only NAM inference; cabinet IR still runs unless cab_bypass is on.
• cab_bypass controls cabinet IR independently.

Set parameters

set vocal_gain <0..4>
set guitar_gain <0..4>
set vc_mix <0..1>
set vc_live_pitch <-7..7>
set vc_live_formant <-1..1>
set vc_live_slap_ms <0..220>
set vc_live_reverb <0..1>
set nam_mix <0..1>
set hpf_hz <20..1000>
set comp_thresh <-60..0>
set comp_ratio <1..20>
set comp_attack_ms <0.1..200>
set comp_release_ms <5..2000>
set comp_makeup_db <-12..24>
set limiter_thresh <0.1..1.0>

Modes and model/IR switching

vc_mode live
route mono|split
nam_model <path>
cab_model <path>

• vc_mode live: sample-by-sample vocal FX path.
• vc_mode fx is currently hidden/disabled until a TensorRT VC backend is implemented.
• nam_model <path> and cab_model <path> can be changed while running.
• cab_model expects an ONNX model that satisfies the cabinet contract above.

Scene (preset) management

scene save <name>
scene load <name>
scene list

Scenes are persisted under:

./scenes/<name>.json

Status dump

status

Prints current route, toggles, gains, mix values, vocal-strip settings, VC settings, and selected NAM/CAB paths.

---

Practical quick recipes

Dry vocal + processed guitar in mono

route mono
set vc_mix 0
set nam_mix 1

Vocal left / guitar right for FOH split

route split

Bypass all neural-ish processing quickly

nam_bypass on
vc_bypass on
cab_bypass on

Save a working live setup

scene save rehearsal_a

Then later:

scene load rehearsal_a

---

Troubleshooting

• If JACK connection fails, ensure JACK is running and your system ports exist.
• If NAM appears inactive:
  • build with --features ort,
  • set a valid JETFX_NAM_MODEL or use nam_model <path> at runtime.
• If NAM sounds like digital fluttering/chopped audio:
  • watch stderr for NAM: output underrun (...) warnings,
  • if warnings appear, inference is not keeping up in real time,
  • reduce JETFX_NAM_WINDOW (try 64 or 128), use a lighter model, or increase JACK period/buffer size.
• On Jetson (unified memory), this build targets TensorRT for NAM. Ensure TensorRT-enabled ONNX Runtime is available; there is no CPU fallback for NAM/IR/VC FX.
• To verify TensorRT caching + FP16 are engaged, look for startup logs like:
  • NAM: TensorRT config fp16=true engine_cache=true ... timing_cache=true ...
  • CAB: TensorRT config fp16=true engine_cache=true ... timing_cache=true ... and confirm cache files are created under JETFX_TRT_ENGINE_CACHE_PATH after first run.
• If cabinet stage sounds unchanged, verify your cabinet ONNX model path, TensorRT compatibility, and contract compliance (static [1,N] I/O, stateless, valid window).
• If levels clip, reduce vocal_gain / guitar_gain and check meter output.

Migration note for legacy IR files

Legacy text-based IR tap files are not valid cabinet inputs anymore. Convert/export cabinet processing to a TensorRT-compatible ONNX model and load it via JETFX_CAB_MODEL / cab_model <path>.

---

Development

Basic checks

cargo fmt
cargo check

Run tests

cargo test

(There may be no tests yet; this still validates build/test harness.)