Rust Code generation

AGENTS.md

@@ -6,26 +6,26 @@ Welcome! This document provides concise, high-leverage context for AI agents wor
 `fsdr-cli` is a Rust-based CLI tool leveraging FutureSDR for digital signal processing, acting as an advanced replacement for `csdr`. The goal is to produce reliable, high-performance DSP flowgraphs.
 It translates CSDR-style command pipelines into an intermediate graph structure (GRC) that is strictly compatible with the **GNU Radio Companion (.grc)** file format.
 
-### GRC Compatibility
-The intermediate graph must be 100% compatible with GNU Radio Companion. This means:
-- Block IDs and parameter names must match GRC definitions.
-- Parameter values and enumerations (e.g., window types, item types) must follow GRC standards.
-- This compatibility allows `fsdr-cli` to execute `.grc` files directly using the FutureSDR runtime.
+### Dual-Backend Architecture (Futamura Projection)
+The project supports two execution modes through an abstract backend:
+1. **Interpreted (Runtime)**: Builds a live FutureSDR flowgraph for immediate execution.
+2. **Compiled (Codegen)**: Generates optimized Rust source code where all complex parameters (filter taps, windows) are pre-computed and baked into the binary.
 
-- **Tech Stack**: Rust (edition 2021), FutureSDR, anynow, pest (Grc/Command grammar).
-- **Core Dependencies**: `futuresdr`, `fsdr-blocks`, `cpal` (audio).
+- **Tech Stack**: Rust (edition 2021), FutureSDR, anynow, pest, **proc-macros (fsdr-cli-macros)**.
+- **Core Dependencies**: `futuresdr`, `fsdr-blocks`, `quote`, `proc-macro2`.
 
 ## Critical Commands
 - **Build:** `cargo build` / `cargo build --release`
-- **Test:** `make test` (runs both `cargo test` and `csdr` verification checks), `cargo test`
+- **Test:** `make test`, `cargo test`, `cargo test -p fsdr-cli-macros`
 - **Typecheck & Lint:** `cargo clippy -- -D warnings`, `cargo fmt`
 
 ## Directory Map
-- `src/`: Core logic (`main.rs`, `lib.rs`) and CLI parsing (`cmd_line.pest`, `cmd_grammar.rs`, `cmd_line.rs`).
+- `src/`: Core logic and CLI parsing.
+- `src/grc/backend.rs`: The `FsdrBackend` trait and its Runtime/Codegen implementations.
+- `src/grc/converter/`: Generic block converters supporting both backends.
+- `fsdr-cli-macros/`: Procedural macro crate providing `#[fsdr_instantiate]`.
 - `src/blocks/`: Custom FutureSDR DSP blocks specific to this CLI.
-- `src/csdr_cmd/`: Parsers and mapping logic to translate `csdr` commands to FutureSDR blocks.
-- `src/grc/`: GNU Radio Companion YAML/layout support and standard generation (`builder.rs`).
-- `src/grc/converter/`: Specialized logic for mapping GRC blocks back into FutureSDR kernels.
+- `src/csdr_cmd/`: Parsers and mapping logic to translate `csdr` commands to GRC instances.
 - `tests/`: Integration tests, data files, and benchmarking.
 - `Makefile`: Heavily used for end-to-end `csdr` output comparison checks.
 

Cargo.toml

@@ -10,6 +10,7 @@ description = "`fsdr-cli` is a command line tool to carry out DSP tasks for Soft
 authors=["Loïc Fejoz <loic@fejoz.net>"]
 
 [package.metadata.wix]
+
 upgrade-guid = "DE2DF6DA-AD53-48D5-A4B2-F4020ECC3592"
 path-guid = "66D9F37F-A1D4-44C6-B9B1-508B9CEF4577"
 license = false
@@ -50,6 +51,10 @@ tokio = { version = "1.33.0", features = ["full"], optional = true }
 tower = {version="0.4.13", optional = true }
 tower-http = { version = "0.5.2", features = ["cors"], optional = true  }
 itertools = "0.14.0"
+fsdr-cli-macros = { path = "fsdr-cli-macros" }
+quote = "1.0"
+proc-macro2 = "1.0"
+num-traits = "0.2"
 
 [profile.release]
 codegen-units = 1
@@ -64,6 +69,11 @@ inherits = "release"
 lto = "fat"
 
 # Config for 'cargo dist'
+[workspace]
+members = [
+    "fsdr-cli-macros"
+]
+
 [workspace.metadata.dist]
 # The preferred cargo-dist version to use in CI (Cargo.toml SemVer syntax)
 cargo-dist-version = "0.17.0"

README.md

@@ -115,6 +115,22 @@ Yet, thanks to newly added `weaver_lsb_cf` and `weaver_usb_cf` we may have bette
 fsdr-cli csdr load_c tests/ssb_lsb_256k_complex2.dat ! shift_addition_cc "(-51500/256000)" ! rational_resampler_cc 48000 256000 ! weaver_usb_cf "(1500/48000)" ! gain_ff 0.00000008 ! limit_ff ! audio 48_000 1
 ```
 
+## Rust Code Generation (Futamura Projection)
+
+`fsdr-cli` can export any DSP flowgraph (whether from `csdr` commands or a `.grc` file) as a standalone, optimized Rust source code file. This feature leverages the [Futamura projection](https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_projections) concept: the tool partially evaluates the flowgraph, resolves all parameters (like filter taps), and bakes them directly into the generated Rust code.
+
+To generate Rust code, add the `--generate <filepath>` flag:
+
+```bash
+# Generate optimized Rust code from a csdr pipeline
+fsdr-cli csdr --generate my_dsp_app.rs load_f input.bin ! gain_ff 2.0 ! dump_f
+
+# Generate optimized Rust code from a GRC flowgraph
+fsdr-cli grc tests/chain1.grc --generate generated_app.rs
+```
+
+The resulting file is a standard Rust program that uses the `futuresdr` runtime. All complex math (like windowing and tap generation) is already performed, resulting in a standalone binary with zero runtime overhead for parameter resolution.
+
 ## IQEngine plugins
 
 ```bash

agent_docs/architecture.md

@@ -7,19 +7,29 @@ It acts as an intelligent command-line parser that constructs and executes `Futu
 1. **Parsing**: `pest` grammar in `src/cmd_line.pest` parses command lines.
 2. **Dispatch**: `src/csdr_cmd/mod.rs` and `src/csdr_cmd/any_cmd.rs` route commands to specific builders.
 3. **Builder**: `src/grc/builder.rs` manages the `GrcBuilder` state, constructing a graph that complies with GRC standards (block IDs, parameter names, and enumerations).
-4. **Conversion**: Individual block converters in `src/grc/converter/` (e.g., `blocks_file_sink.rs`) map these GRC-defined blocks and parameters to specific FutureSDR kernels.
-5. **Execution:** The `Runtime` starts the flowgraph, usually connecting `stdin` to `stdout` processing.
+4. **Backend Selection**: Based on CLI flags (`--generate`), either `RuntimeBackend` or `CodegenBackend` is selected.
+5. **Conversion**: Individual block converters in `src/grc/converter/` (e.g., `fir_filter_xx.rs`) use the selected backend to instantiate blocks.
+6. **Execution/Generation**:
+   - **Runtime**: The `Flowgraph` is executed by the FutureSDR runtime.
+   - **Codegen**: Optimized Rust source code is emitted to a file.
 
-## Key Modules
-- `src/csdr_cmd/`: Builders for various `csdr` compatible commands.
-- `src/grc/`: GRC parsing and the core `GrcBuilder` logic.
-- `src/grc/converter/`: specialized logic for mapping GRC blocks to FutureSDR kernels.
-- `src/iqengine_blockconverter.rs`: Bridge for IQEngine plugin integration.
+## Dual-Backend Architecture (Futamura Projection)
+The core of `fsdr-cli`'s efficiency is the separation of **semantic resolution** from **physical instantiation**. 
+Block converters perform all complex mathematical operations (like calculating filter taps for a given transition bandwidth) once. The resulting data is then passed to the `FsdrBackend` trait.
+
+### `#[fsdr_instantiate]` Macro
+To ensure "One Source of Truth", we use the `#[fsdr_instantiate]` procedural macro. This macro takes a standard Rust block instantiation function and automatically generates:
+- A compiled version for the `RuntimeBackend`.
+- A `TokenStream` emitting version for the `CodegenBackend`.
 
-## Important Patterns
-- **Pest Parsing:** See `src/cmd_grammar.rs` and `src/cmd_line.rs` for how the rules defined in `.pest` are evaluated.
-- **csdr equivalence:** `src/csdr_cmd/` is the central hub mapping legacy `csdr` commands to modern SDR block representations. Look at existing definitions for implementing a new command.
+## Key Modules
+- `src/grc/backend.rs`: The abstraction layer for flowgraph construction.
+- `fsdr-cli-macros/`: The engine behind the Futamura projection.
+- `src/grc/converter/`: Generic converters that work across both backends.
 
 ## Extension Patterns
-- **New CSDR Command**: Add a rule to `cmd_line.pest`, create a new module in `src/csdr_cmd/`, and update `AnyCmd` in `src/csdr_cmd/any_cmd.rs`.
-- **New GRC Block**: Implement `BlockConverter` in `src/grc/converter/` and register it in `src/grc/converter/mod.rs`.
+- **New CSDR Command**: Add a rule to `cmd_line.pest`, create a new module in `src/csdr_cmd/`, and update `AnyCmd` in `src/csdr_cmd/mod.rs`.
+- **New GRC Block**: 
+  1. Add a method to `FsdrBackend` in `src/grc/backend.rs`.
+  2. Implement the instantiation logic once using `#[fsdr_instantiate]`.
+  3. Implement `BlockConverter<B>` in `src/grc/converter/` using the new backend method.

agent_docs/conventions.md

@@ -25,7 +25,21 @@ The intermediate graph is the source of truth for block definitions.
 - **TryInto**: Conversion from strings to `GrcItemType` must use `try_into()?` for safe propagation.
 
 ## Parameter Extraction
-Use `Grc2FutureSdr::parameter_as_f64` and similar helpers in `src/grc/converter/mod.rs` to extract block parameters with expression evaluation support.
+Use `parameter_as_f64` and similar standalone helpers in `src/grc/converter/mod.rs` to extract block parameters with expression evaluation support.
+
+## Backend Abstraction
+All block conversion logic must be generic over `B: FsdrBackend`.
+- **Do not** use `Flowgraph` directly in block converters.
+- **Do not** use `BlockId` directly in return types; use `B::BlockRef`.
+- Return `Box<dyn ConnectorAdapter<B::BlockRef>>`.
+
+## Futamura Projection (Code Generation)
+When adding new blocks, you must ensure they work for both runtime execution and source code generation.
+- **Single Source of Truth**: Use the `#[fsdr_instantiate]` macro in `src/grc/backend.rs`.
+- **ToTokens Wrappers**: If a parameter is a complex type (like `Vec<f32>` or `Complex32`), use one of the `Codegen...` wrappers (e.g., `CodegenTaps`) to ensure it can be baked into the generated source code.
+- **Explicit Types**: In `#[fsdr_instantiate]` functions, always use explicit type annotations for `Apply` or `Sink` closures to aid the compiler in the generated code.
 
 ## Testing
-Always run `cargo test` after modifying building logic. Complex commands often have dedicated tests (e.g., `tests/grc_parse.rs`).
+Always run `cargo test` after modifying building logic. 
+- Use `cargo test -p fsdr-cli-macros` for changes to the macro system.
+- Use `tests/codegen.rs` to verify that new blocks generate valid, compilable Rust code.

agent_docs/testing_guidelines.md

@@ -14,3 +14,9 @@ The most critical part of `fsdr-cli` is perfectly replicating or replacing `csdr
 ## Mocking
 - Avoid over-mocking the Runtime. 
 - You can test individual blocks by creating a manual `Flowgraph`, instantiating a `futuresdr::blocks::VectorSource` with known byte slices, your block under test, and a `futuresdr::blocks::VectorSink` to capture the output and assert against expected results.
+
+## Code Generation Testing
+When adding or modifying block generation:
+1. **Verification**: Run `cargo run --bin fsdr-cli -- csdr --generate test.rs [your command]` and verify `test.rs` is valid Rust.
+2. **Integration Tests**: Add a test case to `tests/codegen.rs`.
+3. **Optimized Baking**: Ensure that pre-computed values (like filter taps) appear as literals in the generated code, confirming that the Futamura projection is actually baking the data.

fsdr-cli-macros/Cargo.toml

@@ -0,0 +1,16 @@
+[package]
+name = "fsdr-cli-macros"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+proc-macro = true
+
+[dependencies]
+syn = { version = "2.0", features = ["full", "fold"] }
+quote = "1.0"
+proc-macro2 = "1.0"
+
+[dev-dependencies]
+futuresdr = { git = "https://github.com/FutureSDR/FutureSDR", branch = "main" }
+num-complex = "0.4"

fsdr-cli-macros/src/lib.rs

@@ -0,0 +1,102 @@
+use proc_macro::TokenStream;
+use quote::quote;
+use syn::fold::{self, Fold};
+use syn::{parse_macro_input, BinOp, Expr, ExprBinary, ExprLit, FnArg, Ident, Lit, Pat};
+
+/// A very basic macro to evaluate a math expression at compile time.
+#[proc_macro]
+pub fn eval_math(input: TokenStream) -> TokenStream {
+    let expr = parse_macro_input!(input as Expr);
+
+    fn eval(expr: &Expr) -> i64 {
+        match expr {
+            Expr::Lit(ExprLit {
+                lit: Lit::Int(i), ..
+            }) => i.base10_parse().unwrap(),
+            Expr::Binary(ExprBinary {
+                left, op, right, ..
+            }) => {
+                let l = eval(left);
+                let r = eval(right);
+                match op {
+                    BinOp::Add(_) => l + r,
+                    BinOp::Sub(_) => l - r,
+                    BinOp::Mul(_) => l * r,
+                    BinOp::Div(_) => l / r,
+                    _ => panic!("Unsupported operator"),
+                }
+            }
+            Expr::Group(g) => eval(&g.expr),
+            Expr::Paren(p) => eval(&p.expr),
+            _ => panic!("Unsupported expression"),
+        }
+    }
+
+    let result = eval(&expr);
+    quote! { #result }.into()
+}
+
+struct Interpolator {
+    arg_names: Vec<Ident>,
+}
+
+impl Fold for Interpolator {
+    fn fold_expr(&mut self, i: Expr) -> Expr {
+        if let Expr::Path(ref p) = i {
+            if let Some(ident) = p.path.get_ident() {
+                if self.arg_names.contains(ident) {
+                    // Verbatim is a way to tell syn "don't parse this, just emit these tokens".
+                    // Here we emit '#' then the identifier, which quote! in the codegen
+                    // function will interpret as interpolation.
+                    let pound = proc_macro2::Punct::new('#', proc_macro2::Spacing::Alone);
+                    let mut tokens = proc_macro2::TokenStream::new();
+                    tokens.extend(quote!(#pound #ident));
+                    return Expr::Verbatim(tokens);
+                }
+            }
+        }
+        fold::fold_expr(self, i)
+    }
+}
+
+/// The fsdr_instantiate macro.
+#[proc_macro_attribute]
+pub fn fsdr_instantiate(_attr: TokenStream, item: TokenStream) -> TokenStream {
+    let input_fn = parse_macro_input!(item as syn::ItemFn);
+
+    // 1. Keep the original function (for RuntimeBackend)
+    let original_fn = &input_fn;
+
+    // 2. Identify function arguments
+    let mut arg_names = Vec::new();
+    for arg in &input_fn.sig.inputs {
+        if let FnArg::Typed(pat_type) = arg {
+            if let Pat::Ident(pat_ident) = &*pat_type.pat {
+                arg_names.push(pat_ident.ident.clone());
+            }
+        }
+    }
+
+    // 3. Transform the function body for codegen
+    let mut interpolator = Interpolator { arg_names };
+    let transformed_block = interpolator.fold_block(*input_fn.block.clone());
+
+    // 4. Generate the Codegen version
+    let fn_name = &input_fn.sig.ident;
+    let codegen_fn_name = quote::format_ident!("{}_codegen", fn_name);
+    let fn_args = &input_fn.sig.inputs;
+    let stmts = &transformed_block.stmts;
+
+    let expanded = quote! {
+        #original_fn
+
+        #[allow(unused_variables)]
+        pub fn #codegen_fn_name(#fn_args) -> proc_macro2::TokenStream {
+            // By using transformed_block, which contains #ident sequences,
+            // this quote! will interpolate the values of the arguments.
+            quote::quote! { #(#stmts)* }
+        }
+    };
+
+    expanded.into()
+}

fsdr-cli-macros/tests/test_macros.rs

@@ -0,0 +1,72 @@
+use fsdr_cli_macros::{eval_math, fsdr_instantiate};
+use futuresdr::num_complex::Complex32;
+use proc_macro2::TokenStream;
+use quote::{quote, ToTokens};
+
+#[test]
+fn test_eval_math() {
+    let result = eval_math!(2 + 3 * 4);
+    assert_eq!(result, 14);
+}
+
+// 1. Handling simple types (f32) works natively because f32: ToTokens
+#[fsdr_instantiate]
+pub fn multiply_by_two(val: f32) -> f32 {
+    val * 2.0
+}
+
+#[test]
+fn test_simple_interpolation() {
+    let input = 21.0;
+    let codegen_tokens = multiply_by_two_codegen(input);
+    let codegen_string = codegen_tokens.to_string();
+    println!("Simple generated tokens: {}", codegen_string);
+    // Flexible check: ignore spaces and literal suffixes
+    let compact = codegen_string.replace(' ', "");
+    assert!(compact.contains("21"));
+}
+
+// 2. Handling Complex types: We need a wrapper that implements ToTokens
+#[derive(Clone, Copy)]
+pub struct CodegenComplex(pub Complex32);
+
+impl ToTokens for CodegenComplex {
+    fn to_tokens(&self, tokens: &mut TokenStream) {
+        let re = self.0.re;
+        let im = self.0.im;
+        tokens.extend(quote!(futuresdr::num_complex::Complex32::new(#re, #im)));
+    }
+}
+
+// 3. Handling Vec/Taps: We need a wrapper that implements ToTokens
+#[derive(Clone)]
+pub struct CodegenTaps(pub Vec<f32>);
+
+impl ToTokens for CodegenTaps {
+    fn to_tokens(&self, tokens: &mut TokenStream) {
+        let values = &self.0;
+        tokens.extend(quote!(vec![ #(#values),* ]));
+    }
+}
+
+// Use a NON-MACRO body to confirm interpolation works for complex types
+#[fsdr_instantiate]
+pub fn get_fir_metadata(taps: CodegenTaps, center: CodegenComplex) -> (usize, f32) {
+    (taps.0.len(), center.0.re)
+}
+
+#[test]
+fn test_complex_type_handling() {
+    let taps = CodegenTaps(vec![0.1, 0.2, 0.3]);
+    let center = CodegenComplex(Complex32::new(1.0, -1.0));
+
+    let codegen_tokens = get_fir_metadata_codegen(taps, center);
+    let codegen_string = codegen_tokens.to_string();
+    println!("Complex generated tokens: {}", codegen_string);
+
+    // Flexible check by removing spaces
+    let compact = codegen_string.replace(' ', "");
+
+    assert!(compact.contains("vec![0.1f32,0.2f32,0.3f32]"));
+    assert!(compact.contains("Complex32::new(1f32,-1f32)"));
+}