Foundation-1

AI audio generation and transformation plugin for DAWs. Uses the Foundation-1 model to generate and transform audio loops from text prompts with BPM, key, and bar conditioning.

Available as VST3, Audio Unit, and Standalone on macOS.

Generating audio from a text prompt, then transforming it in Transform mode (~20s per generation on M4).

preview_video.mp4

How It Works

Foundation-1 is a JUCE C++ plugin that communicates with a local Python inference server. The plugin manages the full server lifecycle automatically:

1. Install the .pkg -- copies plugin binaries and backend source to ~/.foundation/backend
2. Open the plugin in your DAW -- first launch creates a Python venv and installs dependencies
3. Wait for model download (~1 GB from HuggingFace) and server startup
4. Generate audio by typing a text prompt and clicking Generate
5. Transform existing audio by switching to Transform mode and adjusting the transformation amount

The backend runs locally on localhost:8765. No data is sent to external services (other than the initial model download from HuggingFace).

Requirements

• macOS 12 (Monterey) or later
• Python 3.9+ (ships with macOS 12+, uses /usr/bin/python3)
• ~3 GB disk space (Python deps + model weights)
• Internet connection for first launch
• Apple Silicon recommended (uses MPS acceleration); also supports CUDA and CPU

Install (Pre-built Package)

Download Foundation-1-1.0.0.pkg from the releases page.

The package is unsigned -- right-click the .pkg and select Open to bypass Gatekeeper.

The installer includes:

• Foundation-1.vst3 -> /Library/Audio/Plug-Ins/VST3/
• Foundation-1.component -> /Library/Audio/Plug-Ins/Components/
• Foundation-1.app -> /Applications/
• Backend server source -> ~/.foundation/backend/

Build from Source

Prerequisites

• CMake 3.22+
• C++17 compiler (Xcode command line tools)
• libcurl (ships with macOS)

Build

mkdir build && cd build
cmake ..
cmake --build . --config Release

The build fetches JUCE 7.0.12 automatically via CMake's FetchContent. Plugin binaries are copied to the system plugin directories after build (COPY_PLUGIN_AFTER_BUILD is enabled).

Set Up the Backend

cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
pip install --no-deps https://github.com/RoyalCities/RC-stable-audio-tools/archive/f1f13af77cb9bc34dd6eb001b2430b3c85375ea2.zip

The --no-deps flag on stable-audio-tools is required to avoid a pip dependency resolution issue with boto3/s3fs.

Run the Backend Manually

cd backend
source .venv/bin/activate
python run.py

The server starts on http://127.0.0.1:8765. The first run downloads the Foundation-1 model weights (~1 GB).

Build the Installer

bash scripts/package.sh

Produces dist/Foundation-1-1.0.0.pkg.

Architecture

foundation-1/
  plugin/Source/
    PluginProcessor.cpp/h    -- JUCE audio processor
    PluginEditor.cpp/h       -- Main UI layout, timer polling, overlay
    Network/
      BackendClient.cpp/h    -- Server lifecycle, HTTP client, PID management
    UI/
      DesignTokens.h         -- Colors, spacing, radii, typography
      FoundationLookAndFeel   -- Custom JUCE widget rendering
      HeaderBar               -- Top bar: title, status indicator, retry button
      PromptPanel             -- Text prompt input + generate button
      ControlStrip            -- BPM, key, scale, bars, duration controls
      WaveformDisplay         -- Waveform view, playhead, play/DAW buttons, drag-to-DAW
      SampleBrowser           -- Right sidebar: generation history
      SetupOverlay            -- Modal overlay during setup/download
  backend/
    run.py                   -- Server entry point (uvicorn)
    config/settings.py       -- Server settings (host, port, model ID)
    app/
      main.py                -- FastAPI endpoints: /health, /status, /generate, /audio2audio, /detect-key, /models
      inference.py           -- Model loading + audio generation via stable-audio-tools
  scripts/
    setup.sh                 -- Dev setup script
    package.sh               -- macOS .pkg builder

Plugin-Backend Communication

The plugin communicates with the backend over HTTP REST:

Endpoint	Method	Purpose
/health	GET	Connection check, model loaded status
/status	GET	Detailed progress (downloading, loading, error)
/generate	POST	Generate audio from prompt + parameters
/audio2audio	POST	Transform existing audio with prompt + transformation amount
/detect-key	POST	Detect musical key and scale from audio
/models	GET	List available models

Backend State Machine

The plugin tracks server state as:

offline -> settingUp -> starting -> downloadingModel -> ready

• offline: Server not running
• settingUp: Creating venv, installing pip dependencies
• starting: Server process launched, waiting for /health response
• downloadingModel: Server up, model weights downloading from HuggingFace
• ready: Model loaded, ready to generate

Server Lifecycle

• The plugin auto-launches the server when the editor opens
• PID file at ~/.foundation/server.pid prevents duplicate servers
• Server processes persist across plugin close (shared across DAW instances)
• If the server is in an error state, the plugin kills and restarts it

API

POST /generate

Request:

{
  "prompt": "warm analog synth pad",
  "bpm": 120.0,
  "bars": 4,
  "key": "C Major",
  "duration": 8.0
}

Response:

{
  "success": true,
  "file_path": "/Users/you/.foundation/outputs/foundation_a1b2c3d4.wav",
  "sample_rate": 44100,
  "duration": 8.0,
  "generation_time": 12.34
}

GET /health

{
  "status": "ok",
  "model_loaded": true,
  "device": "mps"
}

GET /status

{
  "status": "downloading",
  "detail": "Downloading model weights (~1GB)...",
  "model_loaded": false,
  "device": "mps",
  "error": null
}

File Locations

Path	Purpose
~/.foundation/backend/	Backend server source (installed by .pkg)
~/.foundation/backend/.venv/	Python virtual environment
~/.foundation/outputs/	Generated audio files
~/.foundation/server.pid	Running server PID
~/.foundation/server.log	Server stdout/stderr
~/.foundation/setup.log	First-time setup log
~/.foundation/.setup_complete	Marker file: setup finished

Troubleshooting

Plugin shows "Setting Up" for a long time First-time setup installs PyTorch and other dependencies. Check ~/.foundation/setup.log for progress.

Plugin shows "Server Error" Check ~/.foundation/server.log for details. Common causes:

• Port 8765 already in use by another process
• Python dependency installation failed
• Disk space insufficient for model weights

Server won't start after a failed setup Delete the incomplete setup and retry:

rm -rf ~/.foundation/backend/.venv ~/.foundation/.setup_complete

Then reopen the plugin.

Audio sounds wrong or has mismatched sample rate The model's native sample rate is used. If your DAW session uses a different rate, you may need to resample.

Dependencies

Plugin (C++)

• JUCE 7.0.12 -- Audio plugin framework
• libcurl -- HTTP client

Backend (Python)

• torch, torchaudio -- PyTorch for inference
• transformers, safetensors -- Model loading
• huggingface-hub -- Model downloads
• fastapi, uvicorn -- HTTP server
• soundfile -- Audio file I/O
• stable-audio-tools (RC fork) -- Diffusion-based audio generation
• k-diffusion, einops, x-transformers, descript-audio-codec -- Model runtime dependencies

License

This project is licensed under the GNU General Public License v3.0.

JUCE is used under its GPLv3 license.

Known Limitations

• macOS only (uses fork()/execl() for server management)
• Port 8765 is hardcoded
• Package installer is unsigned (requires Gatekeeper bypass)
• First-time setup requires internet and takes several minutes
• Single model support (Foundation-1)