Skip to content

AutoChunking + Performance Enhancements #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
techbuzzz merged 2 commits into from
Jun 27, 2026
Merged

AutoChunking + Performance Enhancements #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6c7b728
Add AutoChunking: progressive document splitting
techbuzzz Jun 27, 2026
a4ca886
Optimize text splitting, caching, and allocations
techbuzzz Jun 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 83 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,89 @@ All notable changes to **Delibera** are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [10.2.2] - 2026
## [10.2.3] - 2026

### Added — AutoChunking (progressive disclosure for large documents)

- **AutoChunking** — automatic splitting of large knowledge documents (contracts, reports, articles)
into context-window-sized chunks distributed across debate rounds via progressive disclosure.
When a document exceeds the smallest model's context window, the orchestrator creates a
`ChunkingPlan` and distributes chunks evenly across rounds so every model receives a complete
view of the document by the final round.

- **`Chunking` namespace** (`Delibera.Core.Chunking`):
- `AutoChunker` — static planner with 3 strategies: `SemanticBoundary` (respects Markdown
headers → paragraphs → sentences), `FixedSize`, `SlidingWindow` (50% overlap).
- `AutoChunkingOrchestrator` — analyses model capabilities, calculates overhead, creates
the plan, and distributes chunks per round.
- `AutoChunkingOptions` — configuration: strategy, safety margin, max chunks/round,
Map-Reduce toggle, progressive disclosure toggle.
- `ChunkingPlan` / `DocumentChunk` — immutable records describing the split.

- **`ModelCapabilities`** record — context window, max output tokens, vision/tool support,
model family. Obtained from providers via the new `ILLMProvider.GetModelCapabilitiesAsync()`.

- **`ModelContextWindowRegistry`** — static registry of 40+ popular models (Llama, Qwen,
DeepSeek, Phi, Mistral, Gemma, GPT, Claude, YandexGPT, …) with their context window sizes.
`Register()` for custom models. Case-insensitive substring matching.

- **`ILLMProvider.GetModelCapabilitiesAsync(string model, CancellationToken)`** — new
default interface method (returns `null`). Overridden by:
- `OllamaProvider` — queries `/api/show` and extracts `num_ctx` from Modelfile parameters
via regex. Falls back to `ModelContextWindowRegistry`.
- `ChatClientLLMProvider` / `YandexGptProvider` — fall back to `ModelContextWindowRegistry`.

- **`PromptContext` extended** — new fields: `ChunkingPlan`, `AutoChunkingEnabled`,
`MinContextWindow`. New method `GetChunkedUserPrompt(roundNumber, totalRounds, previousRounds)`
returns round-appropriate chunks with `[Chunk X/Y]` markers and section titles.

- **`DebateScenario.BuildChunkedPrompt()`** — protected helper used by all built-in strategies
(`StandardDebate`, `CritiqueDebate`, `ConsensusDebate`). Replaces `GetFullUserPrompt()` calls
with round-aware chunk distribution.

- **`CouncilBuilder` bulk configuration**:
- `WithOptions(CouncilOptions options)` — apply a pre-built options snapshot.
- `WithOptions(Action<CouncilOptions> configure)` — inline lambda configuration.
- `CouncilBuilder(CouncilOptions options)` constructor — one-shot setup.
- `WithAutoChunking(AutoChunkingOptions?)` — enable chunking via fluent API.
- `WithModelContextWindow(pattern, tokens)` — register custom model context windows.
- `ApplyOptions()` — transfers all non-default `CouncilOptions` fields to the builder.

- **`ICouncilBuilder` updated** — new methods: `WithOptions(CouncilOptions)`,
`WithOptions(Action<CouncilOptions>)`, `WithAutoChunking(...)`, `WithModelContextWindow(...)`.

- **`AutoChunkingConfig`** in `CouncilOptions` — bound from `Delibera:AutoChunking`
configuration section. Fields: `Enabled`, `Strategy`, `SafetyMargin`, `MaxChunksPerRound`,
`EnableMapReduce`, `EnableProgressiveDisclosure`, `ModelContextWindows` (dictionary).
`ToOptions()` converts to `AutoChunkingOptions` and registers custom model windows.

- **DI auto-wiring** — `AddDelibera(IConfiguration, ILoggerFactory, ...)` now resolves
`IOptions<CouncilOptions>` and passes it to `new CouncilBuilder(options)`, so all
settings (strategy, rounds, temperature, compression, auto-chunking, etc.) are applied
automatically. Explicit builder calls take precedence.

- **Console example** `AutoChunkingExample` (run with `--autochunking`) demonstrating:
- 3 configuration paths: fluent API, options snapshot, lambda.
- Offline chunking plan demo for 4K/8K/32K/128K context windows.
- Model context window registry dump.
- Synthetic contract document (~15K+ chars) that triggers chunking on small-context models.

### Changed

- Bumped `Delibera.Core` package version `10.2.2` → `10.2.3`.
- `CouncilExecutor` constructor now accepts optional `AutoChunkingOptions?` parameter.
- `CouncilExecutor.ExecuteAsync()` invokes `AutoChunkingOrchestrator.PrepareContextAsync()`
before the debate when AutoChunking is enabled.
- `CouncilExecutor.GetInfo()` displays AutoChunking configuration when active.
- `appsettings.json` updated with `AutoChunking` section.

### Compatibility

- **No breaking changes.** AutoChunking is opt-in — disabled by default. All existing
`ILLMProvider` implementations continue to work (default `GetModelCapabilitiesAsync`
returns `null`). `PromptContext` is a `record` with `with`-expression support, so
existing code that constructs it directly is unaffected. The new `CouncilExecutor`
constructor parameter is optional.

### Added — Polly v8 resilience via Microsoft.Extensions.Http.Resilience

Expand Down
93 changes: 92 additions & 1 deletion README-RU.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

### ⚖️ Продуманные решения с помощью ИИ

**Коллективное принятие решений через структурированное обсуждение ИИ — с RAG, pgvector, Knowledge Keeper, 🛠️ Operator (MCP-инструменты), Chairman, 🔥 сжатием контекста, 💉 Dependency Injection и 📋 журналированием выполнения**
**Коллективное принятие решений через структурированное обсуждение ИИ — с RAG, pgvector, Knowledge Keeper, 🛠️ Operator (MCP-инструменты), Chairman, 🔥 сжатием контекста, ✂️ AutoChunking, 💉 Dependency Injection и 📋 журналированием выполнения**

[![NuGet](https://img.shields.io/nuget/v/Delibera.Core.svg)](https://www.nuget.org/packages/Delibera.Core)
[![License: MIT](https://img.shields.io/badge/License-MIT-10B981.svg)](LICENSE)
Expand Down Expand Up @@ -44,6 +44,7 @@
| **🛠️ Operator (MCP-инструменты)** | Микроагент, делегирующий задачи MCP-серверам (веб, файлы, Marp, Notion, …) по запросу в ходе дебатов |
| **🐘 Qdrant + pgvector** | Подключаемые векторные хранилища — отдельная БД или ваш существующий PostgreSQL |
| **🗜️ Сжатие контекста** | 4 стратегии (Semantic, Deduplication, Summarization, Hybrid) экономят 30–70% токенов |
| **✂️ AutoChunking** | Прогрессивное раскрытие больших документов по раундам — с учётом контекстных окон моделей |
| **💉 Dependency Injection** | Расширение `AddDelibera()` для `IServiceCollection` с полной привязкой опций |
| **📋 Журналирование выполнения** | Модель `ExecutionLog` с `LogLevel` — события Chairman, KK, сжатия и участников |
| **📁 Раздельный вывод файлов** | Экспорт `result.md`, `statistics.md` и `logs.md` по отдельности |
Expand All @@ -63,6 +64,7 @@
- [Dependency Injection](#-dependency-injection)
- [Operator (MCP-инструменты)](#️-operator-mcp-инструменты)
- [Сжатие контекста](#️-сжатие-контекста)
- [AutoChunking](#-autochunking)
- [Интеграция RAG](#-интеграция-rag)
- [Стратегии дебатов](#️-стратегии-дебатов)
- [Структура выходных файлов](#-структура-выходных-файлов)
Expand Down Expand Up @@ -452,6 +454,93 @@ TokenCounter ← Эвристическая оценка ток

---

## ✂️ AutoChunking

Автоматическое разбиение больших документов (договоры, отчёты, статьи) на чанки,
соответствующие размеру контекстного окна моделей, с распределением по раундам дебатов
через **прогрессивное раскрытие** (progressive disclosure). Если документ превышает
контекстное окно самой маленькой модели, оркестратор создаёт план чанкинга и равномерно
распределяет чанки — каждая модель получает полное представление к финальному раунду.

### Как это работает

1. **Определение моделей** — запрос контекстного окна каждой модели через
`GetModelCapabilitiesAsync()` (Ollama `/api/show` или встроенный реестр из 40+ моделей).
2. **Расчёт накладных расходов** — системный промпт + вопрос + буфер ответа + история.
3. **План чанкинга** — разбивка документа по семантическим границам (Markdown-заголовки →
параграфы → предложения), каждый чанк ≤ `minWindow − overhead − safetyMargin`.
4. **Прогрессивное раскрытие** — Раунд 1 получает чанки 1..N/3, Раунд 2 — N/3+1..2N/3 и т.д.
Каждый раунд видит маркеры `[Chunk X/Y] SectionTitle` + сводку предыдущих раундов.

### Три способа конфигурации

```csharp
// Способ 1: Fluent API
var executor = new CouncilBuilder()
.WithAutoChunking(new AutoChunkingOptions
{
Strategy = ChunkingStrategy.SemanticBoundary,
SafetyMargin = 0.15,
MaxChunksPerRound = 3
})
/* …участники, chairman, знания… */
.Build();

// Способ 2: Объект CouncilOptions (из DI или вручную)
var options = new CouncilOptions
{
AutoChunking = new AutoChunkingConfig { Enabled = true, Strategy = "SemanticBoundary" }
};
var executor = new CouncilBuilder(options) // или .WithOptions(options)
/* … */
.Build();

// Способ 3: Лямбда-конфигурация
var executor = new CouncilBuilder()
.WithOptions(o =>
{
o.AutoChunking.Enabled = true;
o.AutoChunking.MaxChunksPerRound = 2;
})
/* … */
.Build();
```

### Конфигурация (`appsettings.json`)

```json
{
"Delibera": {
"AutoChunking": {
"Enabled": true,
"Strategy": "SemanticBoundary",
"SafetyMargin": 0.15,
"MaxChunksPerRound": 3,
"EnableMapReduce": true,
"EnableProgressiveDisclosure": true,
"ModelContextWindows": {
"my-custom-model": 65536
}
}
}
}
```

### Реестр контекстных окон моделей

```csharp
// Запрос известных моделей
var window = ModelContextWindowRegistry.GetContextWindow("llama3.2"); // → 131072
var window = ModelContextWindowRegistry.GetContextWindow("phi3:mini"); // → 4096

// Регистрация своих моделей
ModelContextWindowRegistry.Register("my-fine-tuned-model", 65536);
```

> ▶️ Запустите демо: `dotnet run --project src/Delibera.ConsoleApp -- --autochunking`

---

## 📚 Интеграция RAG

Используйте выделенный экземпляр **Qdrant** или вашу существующую базу **PostgreSQL/pgvector** в
Expand Down Expand Up @@ -567,6 +656,7 @@ dotnet run -- --rag # RAG на базе Qdrant с Knowledge Keepe
dotnet run -- --pgvector # RAG на базе pgvector
dotnet run -- --operator # Основы роли Operator (MCP-инструменты)
dotnet run -- --operator-mcp # 🆕 Operator с MCP-серверами browser + Marp
dotnet run -- --autochunking # 🆕 Демо AutoChunking (большие документы)

# Или запустить полное демо по умолчанию (читает appsettings.json)
dotnet run
Expand Down Expand Up @@ -736,6 +826,7 @@ Delibera.Core
├── Council/ ← CouncilBuilder, CouncilExecutor, Chairman, KnowledgeKeeper, Operator
├── Debate/ ← StandardDebate, CritiqueDebate, ConsensusDebate
├── Compression/ ← Semantic / Deduplication / Summarization / Hybrid
├── Chunking/ ← AutoChunker, AutoChunkingOrchestrator, AutoChunkingOptions
├── Providers/
│ ├── LLM/ ← OllamaProvider, ChatClientLLMProvider, EmbeddingGeneratorProvider
│ ├── RAG/ ← QdrantRagProvider, PgVectorRagProvider
Expand Down
93 changes: 92 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

### ⚖️ Thoughtful AI Decisions

**Collective decision making through structured AI deliberation — with RAG, pgvector, Knowledge Keeper, 🛠️ Operator (MCP tools), Chairman, 🔥 Context Compression, 💉 Dependency Injection & 📋 Execution Logging**
**Collective decision making through structured AI deliberation — with RAG, pgvector, Knowledge Keeper, 🛠️ Operator (MCP tools), Chairman, 🔥 Context Compression, ✂️ AutoChunking, 💉 Dependency Injection & 📋 Execution Logging**

[![NuGet](https://img.shields.io/nuget/v/Delibera.Core.svg)](https://www.nuget.org/packages/Delibera.Core)
[![License: MIT](https://img.shields.io/badge/License-MIT-10B981.svg)](LICENSE)
Expand Down Expand Up @@ -43,6 +43,7 @@ well-reasoned outcomes** rather than single-model guesses.
| **🛠️ Operator (MCP Tools)** | A micro-agent that delegates tasks to MCP servers (web, files, Marp, Notion, …) on demand during the debate |
| **🐘 Qdrant + pgvector** | Pluggable vector stores — use a dedicated DB or your existing PostgreSQL |
| **🗜️ Context Compression** | 4 strategies (Semantic, Deduplication, Summarization, Hybrid) save 30–70% of tokens |
| **✂️ AutoChunking** | Progressive disclosure of large documents across rounds — respects model context windows |
| **💉 Dependency Injection** | `AddDelibera()` extension for `IServiceCollection` with full options binding |
| **📋 Execution Logging** | `ExecutionLog` model with `LogLevel` — Chairman, KK, Compression & participant events |
| **📁 Separate File Output** | Export `result.md`, `statistics.md`, and `logs.md` independently |
Expand All @@ -62,6 +63,7 @@ well-reasoned outcomes** rather than single-model guesses.
- [Dependency Injection](#-dependency-injection)
- [Operator (MCP Tools)](#️-operator-mcp-tools)
- [Context Compression](#-context-compression)
- [AutoChunking](#-autochunking)
- [RAG Integration](#-rag-integration)
- [Debate Strategies](#-debate-strategies)
- [Output Files Structure](#-output-files-structure)
Expand Down Expand Up @@ -448,6 +450,93 @@ TokenCounter ← Heuristic token estimation

---

## ✂️ AutoChunking

Automatically split large knowledge documents (contracts, reports, articles) into
context-window-sized chunks distributed across debate rounds via **progressive disclosure**.
When a document exceeds the smallest model's context window, the orchestrator creates a
chunking plan and distributes chunks evenly — every model receives a complete view by the
final round.

### How it works

1. **Model discovery** — queries each model's context window via `GetModelCapabilitiesAsync()`
(Ollama `/api/show`, or the built-in registry of 40+ models).
2. **Overhead calculation** — system prompt + user question + response buffer + history.
3. **Chunking plan** — splits the document on semantic boundaries (Markdown headers →
paragraphs → sentences), each chunk ≤ `minWindow − overhead − safetyMargin`.
4. **Progressive disclosure** — Round 1 gets chunks 1..N/3, Round 2 gets N/3+1..2N/3, etc.
Each round sees `[Chunk X/Y] SectionTitle` markers + a summary of previous rounds.

### Three configuration paths

```csharp
// Path 1: Fluent API
var executor = new CouncilBuilder()
.WithAutoChunking(new AutoChunkingOptions
{
Strategy = ChunkingStrategy.SemanticBoundary,
SafetyMargin = 0.15,
MaxChunksPerRound = 3
})
/* …members, chairman, knowledge… */
.Build();

// Path 2: CouncilOptions snapshot (from DI or manual)
var options = new CouncilOptions
{
AutoChunking = new AutoChunkingConfig { Enabled = true, Strategy = "SemanticBoundary" }
};
var executor = new CouncilBuilder(options) // or .WithOptions(options)
/* … */
.Build();

// Path 3: Lambda configuration
var executor = new CouncilBuilder()
.WithOptions(o =>
{
o.AutoChunking.Enabled = true;
o.AutoChunking.MaxChunksPerRound = 2;
})
/* … */
.Build();
```

### Configuration (`appsettings.json`)

```json
{
"Delibera": {
"AutoChunking": {
"Enabled": true,
"Strategy": "SemanticBoundary",
"SafetyMargin": 0.15,
"MaxChunksPerRound": 3,
"EnableMapReduce": true,
"EnableProgressiveDisclosure": true,
"ModelContextWindows": {
"my-custom-model": 65536
}
}
}
}
```

### Model Context Window Registry

```csharp
// Query known models
var window = ModelContextWindowRegistry.GetContextWindow("llama3.2"); // → 131072
var window = ModelContextWindowRegistry.GetContextWindow("phi3:mini"); // → 4096

// Register custom models
ModelContextWindowRegistry.Register("my-fine-tuned-model", 65536);
```

> ▶️ Run the demo: `dotnet run --project src/Delibera.ConsoleApp -- --autochunking`

---

## 📚 RAG Integration

Use a dedicated **Qdrant** instance or your existing **PostgreSQL/pgvector** database as a vector store.
Expand Down Expand Up @@ -562,6 +651,7 @@ dotnet run -- --rag # Qdrant-backed RAG with Knowledge Keeper
dotnet run -- --pgvector # pgvector-backed RAG
dotnet run -- --operator # Operator role basics (MCP tools)
dotnet run -- --operator-mcp # 🆕 Operator with browser + Marp MCP servers
dotnet run -- --autochunking # 🆕 AutoChunking demo (large documents)

# Or run the full default demo (reads appsettings.json)
dotnet run
Expand Down Expand Up @@ -731,6 +821,7 @@ Delibera.Core
├── Council/ ← CouncilBuilder, CouncilExecutor, Chairman, KnowledgeKeeper, Operator
├── Debate/ ← StandardDebate, CritiqueDebate, ConsensusDebate
├── Compression/ ← Semantic / Deduplication / Summarization / Hybrid
├── Chunking/ ← AutoChunker, AutoChunkingOrchestrator, AutoChunkingOptions
├── Providers/
│ ├── LLM/ ← OllamaProvider, ChatClientLLMProvider, EmbeddingGeneratorProvider
│ ├── RAG/ ← QdrantRagProvider, PgVectorRagProvider
Expand Down
Loading
Loading