Add v0.3.0/v0.4.0 cookbooks for @betterdb/agent-cache and new Python track

content/betterdb-agent-cache-py/01-getting-started.md

@@ -0,0 +1,160 @@
+`betterdb-agent-cache` is a multi-tier exact-match cache for AI agent workloads backed by Valkey. It combines three cache tiers behind one connection:
+
+- **LLM tier** — caches full LLM responses by exact match on model + messages + params
+- **Tool tier** — caches tool/function call results by tool name and argument hash
+- **Session tier** — key-value store for agent state with per-field TTL
+
+Works on vanilla Valkey 7+, Amazon ElastiCache, Google Cloud Memorystore, and Amazon MemoryDB with no modules required.
+
+## Prerequisites
+
+- **Valkey 7+** (no modules needed)
+- Python 3.11+
+
+## Step 1: Start Valkey
+
+```bash
+docker run -d --name valkey -p 6379:6379 valkey/valkey:latest
+```
+
+## Step 2: Install
+
+```bash
+pip install betterdb-agent-cache
+```
+
+For provider-specific adapters, install the relevant extra:
+
+```bash
+pip install "betterdb-agent-cache[openai]"       # OpenAI
+pip install "betterdb-agent-cache[anthropic]"    # Anthropic
+pip install "betterdb-agent-cache[langchain]"    # LangChain
+pip install "betterdb-agent-cache[langgraph]"    # LangGraph
+pip install "betterdb-agent-cache[llamaindex]"   # LlamaIndex
+```
+
+## Step 3: Create an AgentCache
+
+```python
+import asyncio
+import valkey.asyncio as valkey_client
+from betterdb_agent_cache import AgentCache, TierDefaults
+from betterdb_agent_cache.types import AgentCacheOptions
+
+async def main():
+    client = valkey_client.Valkey(host="localhost", port=6379)
+
+    cache = AgentCache(AgentCacheOptions(
+        client=client,
+        tier_defaults={
+            "llm":     TierDefaults(ttl=3600),   # LLM responses: 1 hour
+            "tool":    TierDefaults(ttl=300),     # Tool results: 5 minutes
+            "session": TierDefaults(ttl=1800),    # Session state: 30 min (sliding)
+        },
+        # cost_table is optional — 1,900+ models covered by default
+    ))
+```
+
+No `initialize()` call needed — all tiers use plain Valkey commands (SET, GET, HINCRBY) with no index creation.
+
+## Step 4: LLM Tier
+
+```python
+params = {
+    "model": "gpt-4o-mini",
+    "messages": [{"role": "user", "content": "What is Valkey?"}],
+    "temperature": 0,
+}
+
+# Check for a cached response
+llm_result = await cache.llm.check(params)
+
+if llm_result.hit:
+    print("LLM cache HIT:", llm_result.response)
+else:
+    print("LLM cache MISS → calling LLM...")
+    response = await call_llm(params)  # your LLM call
+
+    from betterdb_agent_cache.types import LlmStoreOptions
+    await cache.llm.store(params, response, LlmStoreOptions(
+        tokens={"input": 14, "output": 62},  # enables cost tracking
+    ))
+```
+
+The cache key is a SHA-256 hash of the canonicalized params. `{"messages": [...], "model": "gpt-4o"}` and `{"model": "gpt-4o", "messages": [...]}` produce the same key.
+
+## Step 5: Tool Tier
+
+```python
+tool_name = "get_weather"
+tool_args = {"city": "Sofia", "units": "metric"}
+
+tool_result = await cache.tool.check(tool_name, tool_args)
+
+if tool_result.hit:
+    print("Tool cache HIT:", tool_result.response)
+else:
+    print("Tool cache MISS → calling API...")
+    import json
+    data = await get_weather(tool_args)  # your tool call
+
+    from betterdb_agent_cache.types import ToolStoreOptions
+    await cache.tool.store(tool_name, tool_args, json.dumps(data), ToolStoreOptions(
+        cost=0.001,  # API call cost in dollars, for tracking
+    ))
+```
+
+Tool args are serialized with recursively sorted keys before hashing, so `{"city": "Sofia", "units": "metric"}` and `{"units": "metric", "city": "Sofia"}` hit the same cache entry.
+
+## Step 6: Session Tier
+
+```python
+thread_id = "user-42-session-1"
+
+# Store agent state fields
+await cache.session.set(thread_id, "last_intent", "book_flight")
+await cache.session.set(thread_id, "destination", "Sofia")
+await cache.session.set(thread_id, "departure_date", "2026-06-01")
+
+# Retrieve a field (also refreshes its TTL)
+intent = await cache.session.get(thread_id, "last_intent")
+# "book_flight"
+
+# Get all fields for a thread
+state = await cache.session.get_all(thread_id)
+# {"last_intent": "book_flight", "destination": "Sofia", "departure_date": "2026-06-01"}
+```
+
+Each `get()` refreshes the TTL on that field (sliding window). The session stays alive as long as the agent is actively using it.
+
+## Key Formats
+
+All keys are prefixed with the `name` option (default: `betterdb_ac`):
+
+| Tier | Key Pattern |
+|------|-------------|
+| LLM cache | `betterdb_ac:llm:{sha256hash}` |
+| Tool cache | `betterdb_ac:tool:{toolName}:{sha256hash}` |
+| Session field | `betterdb_ac:session:{threadId}:{field}` |
+| Stats hash | `betterdb_ac:__stats` |
+
+```python
+cache = AgentCache(AgentCacheOptions(
+    client=client,
+    name="myapp_prod",  # all keys prefixed with 'myapp_prod:'
+))
+```
+
+## Cleanup
+
+```python
+# Destroy all state for a thread
+await cache.session.destroy_thread(thread_id)
+
+# Invalidate all LLM cache entries for a model
+await cache.llm.invalidate_by_model("gpt-4o-mini")
+
+# Close the client when done
+await cache.shutdown()
+await client.aclose()
+```

content/betterdb-agent-cache-py/02-llm-and-tool-cache.md

@@ -0,0 +1,211 @@
+## LLM Cache Tier
+
+The LLM cache stores full LLM responses by exact match on all parameters that affect the output.
+
+### What Gets Hashed
+
+The cache key is a SHA-256 hash of these fields (with recursively sorted dict keys for determinism):
+
+| Field | Notes |
+|-------|-------|
+| `model` | `'gpt-4o'`, `'claude-opus-4-6'`, etc. |
+| `messages` | Full message list including roles and content |
+| `temperature` | `None` is treated as unset |
+| `top_p` | `None` is treated as unset |
+| `max_tokens` | `None` is treated as unset |
+| `tools` | Tool definitions if present |
+| `tool_choice` | `None` is treated as unset |
+| `seed` | `None` is treated as unset |
+| `stop` | `None` is treated as unset |
+| `response_format` | `None` is treated as unset |
+| `reasoning_effort` | For models supporting extended thinking |
+| `prompt_cache_key` | Pass-through for provider-level prompt caching |
+
+### Check and Store
+
+```python
+import valkey.asyncio as valkey_client
+from openai import AsyncOpenAI
+from betterdb_agent_cache import AgentCache, TierDefaults
+from betterdb_agent_cache.types import AgentCacheOptions, LlmStoreOptions
+
+client = valkey_client.Valkey(host="localhost", port=6379)
+openai = AsyncOpenAI()
+
+# cost_table is optional — 1,900+ models covered by default (see "Default Cost Table" below)
+cache = AgentCache(AgentCacheOptions(
+    client=client,
+    tier_defaults={"llm": TierDefaults(ttl=3600)},
+))
+
+async def cached_completion(params: dict) -> str:
+    cached = await cache.llm.check(params)
+    if cached.hit:
+        return cached.response or ""
+
+    response = await openai.chat.completions.create(**params)
+    text = response.choices[0].message.content or ""
+    usage = response.usage
+
+    await cache.llm.store(params, text, LlmStoreOptions(
+        tokens={
+            "input":  usage.prompt_tokens     if usage else 0,
+            "output": usage.completion_tokens if usage else 0,
+        },
+    ))
+    return text
+```
+
+Token counts in `store()` enable cost savings tracking via the cost table. Without `tokens`, cost tracking is skipped for that entry.
+
+### Invalidating by Model
+
+```python
+deleted = await cache.llm.invalidate_by_model("gpt-4o-mini")
+print(f"Deleted {deleted} entries")
+```
+
+---
+
+## Tool Cache Tier
+
+The tool cache stores tool/function call results. Valuable for expensive external API calls — weather, geocoding, database queries, web search — that return stable results for the same inputs.
+
+### Check and Store
+
+```python
+import json
+from betterdb_agent_cache.types import ToolStoreOptions
+
+async def cached_tool(name: str, args: dict) -> dict:
+    cached = await cache.tool.check(name, args)
+    if cached.hit:
+        return json.loads(cached.response)
+
+    result = await call_tool(name, args)  # your tool implementation
+
+    await cache.tool.store(name, args, json.dumps(result), ToolStoreOptions(
+        ttl=300,    # per-call TTL override
+        cost=0.005, # API call cost in dollars
+    ))
+    return result
+```
+
+### Per-Tool TTL Policies
+
+```python
+from betterdb_agent_cache.types import ToolPolicy
+
+# Weather data: short TTL, changes frequently
+await cache.tool.set_policy("get_weather",    ToolPolicy(ttl=300))    # 5 min
+
+# Stock prices: very short TTL
+await cache.tool.set_policy("get_stock_price", ToolPolicy(ttl=60))    # 1 min
+
+# Geocoding: long TTL, addresses don't change
+await cache.tool.set_policy("geocode_address", ToolPolicy(ttl=86400)) # 24h
+
+# Web search: medium TTL
+await cache.tool.set_policy("web_search",     ToolPolicy(ttl=3600))   # 1h
+```
+
+TTL precedence: per-call `ttl` > tool policy > `tier_defaults["tool"].ttl` > `default_ttl`.
+
+### Invalidation
+
+```python
+# Invalidate all results for a specific tool
+deleted = await cache.tool.invalidate_by_tool("get_weather")
+print(f"Deleted {deleted} weather cache entries")
+
+# Invalidate one specific call
+existed = await cache.tool.invalidate("get_weather", {"city": "Sofia"})
+```
+
+---
+
+## Cost Tracking and Stats
+
+### Aggregate Stats
+
+```python
+stats = await cache.stats()
+print(stats.llm.hits, stats.llm.misses, f"{stats.llm.hit_rate:.0%}")
+# 150 50 75%
+
+print(f"Cost saved: ${stats.cost_saved_micros / 1_000_000:.4f}")
+# Cost saved: $12.5000
+
+# Per-tool breakdown
+for tool_name, tool_stats in stats.per_tool.items():
+    print(f"{tool_name}: {tool_stats.hit_rate:.0%} hit rate")
+```
+
+Cost savings are stored as microdollars (integers) to avoid floating-point precision issues. Divide by `1_000_000` to get dollars.
+
+### Tool Effectiveness Recommendations
+
+```python
+effectiveness = await cache.tool_effectiveness()
+for entry in effectiveness:
+    print(f"{entry.tool}: {entry.hit_rate:.0%} hit rate — {entry.recommendation}")
+# get_weather:   85% hit rate — increase_ttl
+# web_search:    62% hit rate — optimal
+# rare_api_call:  8% hit rate — decrease_ttl_or_disable
+```
+
+| Recommendation | Condition | Action |
+|---|---|---|
+| `increase_ttl` | Hit rate > 80% and TTL < 1 hour | Results are stable and reused frequently — extend TTL |
+| `optimal` | Hit rate 40–80% | No change needed |
+| `decrease_ttl_or_disable` | Hit rate < 40% | Results change too fast or rarely repeated |
+
+---
+
+## Default Cost Table
+
+`betterdb-agent-cache` ships a bundled cost table sourced from [LiteLLM's model pricing data](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) and refreshed on every release. Cost tracking works out of the box for 1,900+ models — no `cost_table` configuration required.
+
+```python
+# No cost_table needed — GPT-4o, Claude, Gemini, and 1,900+ others are covered
+cache = AgentCache(AgentCacheOptions(client=client))
+```
+
+### Overriding Specific Models
+
+User-supplied `cost_table` entries merge on top of the defaults:
+
+```python
+from betterdb_agent_cache import ModelCost
+
+cache = AgentCache(AgentCacheOptions(
+    client=client,
+    cost_table={
+        "my-fine-tuned-gpt4o": ModelCost(input_per_1k=0.005, output_per_1k=0.015),
+    },
+))
+```
+
+### Inspecting the Bundled Table
+
+```python
+from betterdb_agent_cache import DEFAULT_COST_TABLE
+
+print(DEFAULT_COST_TABLE.get("gpt-4o-mini"))
+# ModelCost(input_per_1k=0.00015, output_per_1k=0.0006)
+
+print(len(DEFAULT_COST_TABLE))
+# 1900+
+```
+
+### Disabling the Default Table
+
+```python
+cache = AgentCache(AgentCacheOptions(
+    client=client,
+    use_default_cost_table=False,
+    cost_table={
+        "gpt-4o": ModelCost(input_per_1k=0.0025, output_per_1k=0.010),
+    },
+))
+```