CondenseFlow

CondenseFlow: Scalable Latent Space Collaboration via Semantic Compression for Multi-Agent Systems

Accepted at ACL 2026 Findings

Overview

CondenseFlow is an efficient latent space collaboration framework for LLM-based multi-agent systems. It introduces the Latent Thought Condenser (LTC), a lightweight module that compresses KV caches into fixed-size semantic representations via learnable semantic probes, achieving O(1) communication complexity regardless of context length or collaboration depth.

Key results:

• >99% KV cache memory reduction compared to dense latent transfer
• ~20% inference latency reduction under extended interaction
• <2% accuracy degradation across 7 benchmarks and 6 models

Installation

pip install -e .
# or
pip install -r requirements.txt

Quick Start

from src.models.ltc_wrapper import LTCWrapper
from src.pipelines.standard_pipeline import StandardPipeline

# Load model with LTC
model = LTCWrapper(
    model_name_or_path="Qwen/Qwen3-8B-Instruct",
    ltc_checkpoint="./checkpoints/ltc.pt"
)

# Run standard 4-agent pipeline (Planner -> Critic -> Refiner -> Solver)
pipeline = StandardPipeline(model, communication_mode="condenseflow")
result = pipeline.run("Solve this math problem...")
print(result["answer"])

Scripts

1. Train LTC Module

python scripts/train_ltc.py \
    --model_config configs/model/qwen3_14b.yaml \
    --training_config configs/training/ltc_default.yaml \
    --output_dir ./outputs/ltc_training

2. Evaluate on Benchmarks

python scripts/evaluate.py \
    --model_config configs/model/qwen3_14b.yaml \
    --ltc_checkpoint ./outputs/ltc_training/checkpoint.pt \
    --benchmarks aime2024 aime2025 hmmt2025 gpqa mbpp medqa livecodebench \
    --communication_modes text dense condenseflow \
    --num_runs 5

3. Run Standard Protocol

python scripts/run_standard_protocol.py \
    --model_config configs/model/qwen3_14b.yaml \
    --ltc_checkpoint ./outputs/ltc_training/checkpoint.pt \
    --question "Find the sum of all integer bases b > 9 for which 17_b is a divisor of 97_b." \
    --communication_mode condenseflow

4. Run Stress Test

python scripts/run_stress_test.py \
    --model_config configs/model/qwen3_14b.yaml \
    --ltc_checkpoint ./outputs/ltc_training/checkpoint.pt \
    --benchmark aime2025 \
    --max_rounds 20 \
    --communication_mode condenseflow

5. Analyze Compression

python scripts/analyze_compression.py \
    --model_config configs/model/qwen3_14b.yaml \
    --ltc_checkpoint ./outputs/ltc_training/checkpoint.pt \
    --output_dir ./analysis

Supported Models

Scale	Models
Small (~8B)	Qwen3-8B-Instruct, LLaMA-3.1-8B-Instruct, Gemma-2-9b-it
Mid (~14B)	Qwen3-14B, DeepSeek-R1-Distill-Qwen-14B, Ring-mini-2.0

Communication Modes

Mode	Description	Memory Scaling
text	Natural language exchange	O(R * T)
dense	Full KV cache transfer	O(R * T * L * d_h)
condenseflow	LTC-compressed representations	O(K * L * d_h) = O(1)

Project Structure

condenseflow/
├── src/
│   ├── models/          # LTC module and model wrappers
│   ├── agents/          # Agent implementations (Planner, Critic, Refiner, Solver)
│   ├── pipelines/       # Standard 4-agent and stress test pipelines
│   ├── training/        # LTC training (losses, dataset, trainer)
│   ├── evaluation/      # Benchmarks and metrics
│   └── utils/           # Configuration, logging, memory tracking
├── scripts/             # Entry point scripts
├── configs/             # Model and evaluation configurations
└── tests/               # Unit tests

Citation

@inproceedings{chen2026condenseflow,
    title={CondenseFlow: Scalable Latent Space Collaboration via Semantic Compression for Multi-Agent Systems},
    author={Xiaoyu Chen and Fengge Wu and Junsuo Zhao and Yun Fan},
    booktitle={Findings of the Association for Computational Linguistics: ACL 2026},
    year={2026}
}

License

MIT License