✨(back) make summarization async

CHANGELOG.md

@@ -13,6 +13,7 @@ and this project adheres to
 - ✨(back) add model fallback mechanism
 - ✨(back) add celery for running background tasks
 - 🧱(helm) add celery worker and beat deployments
+- ✨(conversation) summarize messages
 
 ### Changed
 

docs/attachments.md

@@ -517,11 +517,12 @@ Notes:
 
 The decision of which documents are inlined as `full-context` vs left as `tool_call_only` is made by `chat/document_context_builder.py:build_documents_listing` on each turn (called via `_build_document_context_instruction` in `chat/clients/pydantic_ai.py`):
 
-1. Compute the `document_budget` in tokens:
+1. Compute budgets in tokens (`chat/tokens.py` splits `max_token_context` by ratio, then subtracts the security buffer from **each** share independently):
    ```text
-   document_budget = max(int(model.max_token_context * DOCUMENT_CONTEXT_BUDGET_RATIO)
-                         - DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS, 0)
+   document_budget      = max(int(max_token_context * DOCUMENT_CONTEXT_BUDGET_RATIO) - DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS, 0)
+   message_token_budget = max(int(max_token_context * (1 - DOCUMENT_CONTEXT_BUDGET_RATIO)) - DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS, 0)
    ```
+   The document share (`document_budget`, via `compute_document_budget`) inlines conversation documents; the history share (`message_token_budget`, via `compute_conversation_budget`) is the conversation-summarization trigger. `build_documents_listing` receives the raw `max_token_context`, `budget_ratio`, and `security_buffer_tokens` and applies the formula above.
 2. Load all text attachments from object storage **in parallel** (`asyncio.gather`). Attachments that fail to load are marked `tool_call_only` with their failure logged; other documents are not affected.
 3. Iterate documents oldest-first (`order_by("created_at", "id")`). For each document:
    - If its token count exceeds the whole budget alone → keep `tool_call_only`.
@@ -536,6 +537,18 @@ Token estimation uses `tiktoken` with the `cl100k_base` encoding (GPT-4 tokenize
 The assembled listing is **cached** per turn (in `_build_documents_listing`, `pydantic_ai.py`) keyed on:
 `conversation_id`, `user_id`, `model_hrid`, `model.max_token_context`, `DOCUMENT_CONTEXT_BUDGET_RATIO`, `DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS`, and a fingerprint of `(attachment.id, attachment.updated_at)` for every text attachment — **conversation and project text attachments both contribute to the fingerprint**. Any attachment add / remove / edit (including project files), or any settings change, invalidates the cache. TTL is 30 minutes (`CACHE_TIMEOUT`).
 
+#### Conversation history summarization
+
+When `message_token_budget` is exceeded, `chat/agents/history_processors.py` calls a separate summarization model (`LLM_SUMMARIZATION_MODEL_HRID`) and stores the result on the conversation (`history_summary`, `history_summary_checkpoint`). This runs at the **start of a new user turn**, before `agent.iter`, against **stored** `pydantic_messages` from previous turns only (the current user prompt is extracted separately and, by design, is not counted in the budget check — see ADR 0002). That stored history usually ends on an assistant `ModelResponse`.
+
+1. **Trigger**: estimated tokens in the active history slice exceed `message_token_budget` (see formulas above) and there are new messages after the last checkpoint.
+2. **After a summary**: the model receives the stored summary text (dynamic instruction) plus the last `CONVERSATION_SUMMARY_CONTEXT_MESSAGES` `ModelMessage` entries before the checkpoint. Use an **even** value so the retained window starts on a user `ModelRequest` in a plain user/assistant alternation (tool messages can break parity).
+3. **Summary length**: capped by `CONVERSATION_SUMMARY_MAX_TOKENS` on the summarization LLM call.
+4. **Disable summarization**: zero the `message_token_budget`, by setting `max_token_context=0` on the chat model in `LLM_CONFIGURATIONS`, `DOCUMENT_CONTEXT_BUDGET_RATIO=1` (reallocates the whole context to the document share), or a `DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS` large enough to consume the history share. Note: *omitting* `max_token_context` does **not** disable summarization — unlike document inlining, `resolve_conversation_budget` falls back to `DEFAULT_MAX_TOKEN_CONTEXT`, so the history budget stays non-zero.
+5. **Visible blocking generation**: at the start of a turn whose active history (previous turns only; the new user message is not counted — see ADR 0002) exceeds the history budget, the turn enqueues a Celery task (`chat/tasks.py:summarize_conversation_history`) and waits for it on the open SSE stream, emitting `keep_alive` data parts; the client shows an in-thread progress indicator anchored on the `summarize` tool events. Concurrency is controlled by a time-bounded claim (`ChatConversation.history_summary_claimed_at`, TTL = task hard time limit 120s + 60s): if the task never claims within a 10s grace window or its claim dies and no summary has landed, the turn fails with a `summarization_failed` stream error — there is no inline fallback; generation stays Celery-only (3 task retries with exponential backoff). A turn is never answered with truncated or over-budget context — see `docs/adr/0002-summarization-visible-blocking-phase.md` and ADR 0001.
+
+The security buffer is **not** a dedicated reserve for system prompts, tool schemas, or completion tokens; those are added on top of the planned document/history split. Size the buffer (and/or plan `max_token_context` below the model nominal window) to leave headroom for that overhead.
+
 #### Targeted document operations (`document_id`)
 
 Three tools accept an optional `document_id` argument, each with its own IDOR boundary:
@@ -614,8 +627,11 @@ A `READY` attachment whose `rag_document_id` is null (e.g. parse succeeded but t
 | `RAG_DOCUMENT_SEARCH_BACKEND`                | `AlbertRagBackend` | Import path of the vector-search backend used for indexing and search (Albert or Find) |
 | `PROJECT_FILES_MAX_COUNT`                    | `10`           | Max non-image attachments per project (excludes hidden markdown companions). Enforced at upload-time in `ChatProjectAttachmentViewSet`. Bounds per-turn system-prompt token cost (every entry contributes to `project_documents` on every conversation turn). |
 | `PROJECT_IMAGES_MAX_COUNT`                   | `3`            | Max image attachments per project. Enforced at upload-time. Bounds per-turn vision token cost - every project image is pinned to every turn alongside conversation-message images, and provider request-level image caps (Anthropic ~20/request) clip the trailing entries first. |
-| `DOCUMENT_CONTEXT_BUDGET_RATIO`              | `0.5`          | Fraction of `model.max_token_context` reserved for inlined documents (0 disables full-context inlining; everything stays `tool_call_only`) |
-| `DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS`    | `1000`         | Tokens subtracted from the inlining budget to absorb tokenizer drift on non-OpenAI models |
+| `DOCUMENT_CONTEXT_BUDGET_RATIO`              | `0.5`          | Fraction of `model.max_token_context` reserved for inlined documents; `(1 - ratio)` goes to the conversation history budget (summarization trigger). The security buffer is subtracted from each share afterwards. `0` disables full-context inlining (everything stays `tool_call_only`). |
+| `DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS`    | `1000`         | Tokens subtracted from **each** of the document and history budgets (independently), to absorb tokenizer drift on non-OpenAI models and leave headroom beyond the planned split |
+| `CONVERSATION_SUMMARY_CONTEXT_MESSAGES`      | `10`           | Number of pydantic-ai `ModelMessage` entries kept in runtime history after a conversation summary (in addition to the stored summary text). Use an even value (see [Conversation history summarization](#conversation-history-summarization)) |
+| `CONVERSATION_SUMMARY_MAX_TOKENS`            | `2048`         | `max_tokens` for the summarization LLM call when generating or updating `history_summary` |
+| `LLM_SUMMARIZATION_MODEL_HRID`               | `default-summarization-model` | HRID in `LLM_CONFIGURATIONS` for the conversation summarization agent (see [LLM Configuration](llm-configuration.md)) |
 | `RAG_COLLECTION_INACTIVITY_DAYS`             | `30`           | Conversations inactive for this many days have their RAG collection de-indexed by `deindex_inactive_collections`. Resets on re-index. |
 
 #### RAG_FILES_ACCEPTED_FORMATS
@@ -664,7 +680,7 @@ RAG_FILES_ACCEPTED_FORMATS = [
 
 ### Per-model setting: `max_token_context`
 
-Each entry in `LLM_CONFIGURATIONS` accepts a `max_token_context` integer field declaring the model's context window size. When set, it drives the inlining budget for the documents listing (`document_budget = max_token_context * DOCUMENT_CONTEXT_BUDGET_RATIO - DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS`).
+Each entry in `LLM_CONFIGURATIONS` accepts a `max_token_context` integer field declaring the model's context window size. When set, it drives document inlining and conversation summarization budgets: `max_token_context` is split by `DOCUMENT_CONTEXT_BUDGET_RATIO`, and `DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS` is subtracted from **each** share independently (see the formulas under [Inlining policy and FIFO eviction](#inlining-policy-and-fifo-eviction)).
 
 If a model has no `max_token_context`, all of its documents are kept `tool_call_only` regardless of size and a warning is logged on every chat turn. Setting the field accurately matters: too low and small documents get pushed to RAG-only when they could be inlined; too high and the LLM may exceed its real window.
 

docs/env.md

@@ -82,6 +82,10 @@ These are the environment variables you can set for the `conversations-backend`
 | LLM_CONFIGURATION_FILE_PATH                     | Path to the LLM configuration JSON file. See [LLM Configuration](llm-configuration.md) for details                                | <BASE_DIR>/conversations/configuration/llm/default.json |
 | LLM_DEFAULT_MODEL_HRID                          | HRID of the model used for conversations                                                                                          | default-model                                           |
 | LLM_SUMMARIZATION_MODEL_HRID                    | HRID of the model used for summarization                                                                                          | default-summarization-model                             |
+| DOCUMENT_CONTEXT_BUDGET_RATIO                   | Fraction of `max_token_context` for inlined conversation documents; `(1 - ratio)` is the conversation history share (security buffer subtracted from each afterwards). See [attachments.md](attachments.md) | `0.5`                                                   |
+| DOCUMENT_CONTEXT_SECURITY_BUFFER_TOKENS         | Tokens subtracted from each of the document and history budgets independently                                                              | `1000`                                                  |
+| CONVERSATION_SUMMARY_CONTEXT_MESSAGES           | `ModelMessage` count kept after a conversation summary (use an even value). See [attachments.md](attachments.md)                               | `10`                                                    |
+| CONVERSATION_SUMMARY_MAX_TOKENS                 | Max tokens for the conversation summarization LLM output                                                                                        | `2048`                                                  |
 | AI_API_KEY                                      | AI API key to be used for the default provider (used in default LLM configuration, not for production use)                        |                                                         |
 | AI_BASE_URL                                     | OpenAI compatible AI base URL (used in default LLM configuration, not for production use)                                         |                                                         |
 | AI_MODEL                                        | AI Model name to use (used in default LLM configuration, not for production use)                                                  |                                                         |