feat: optional TwelveLabs content-aware video search & analysis (Marengo + Pegasus)

[mohit-twelvelabs]

Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).

What this adds

An opt-in agent tool that searches the actual video content of a channel — what is shown and said on screen (objects, scenes, actions, on-screen text, spoken audio) — using TwelveLabs' Marengo model, plus a Pegasus-backed helper for grounded single-video analysis.

Today the chat agent searches video transcripts (PGVector semantic + BM25). That's great for what was said, but misses purely-visual moments and anything the transcript drops. This new twelvelabs_video_search tool complements the existing search so the agent can answer questions like "find the part where they show the dashboard" or "when do they demo the product".

Why it helps this project

• Searches video content directly, not just transcripts — surfaces visual moments transcripts can't.
• Slots into the existing ReAct agent as just another tool, so the LLM picks it when a query is visual/temporal.
• Returns clip-level results (video_id, start, end, score, thumbnail_url) that map cleanly onto the existing timestamped-results UI.

Opt-in / non-breaking

• Disabled unless TWELVELABS_API_KEY is set. With no key, TwelveLabsService.is_enabled() is False, the tool is not registered in the agent, and the transcript-only pipeline runs exactly as before.
• The SDK is an optional dependency (pip install -e ".[twelvelabs]") and is imported lazily, so nothing changes for existing installs.
• Follows the repo's existing tool conventions: a service class + a Tools.tool() classmethod returning a StructuredTool with a Pydantic args_schema, mirroring VectorDatabaseTools / SQLTools.

How it was tested

• Added tests/test_twelvelabs.py (the repo's first tests — also on the roadmap):
  • no-network unit tests: disabled without a key, client raises TwelveLabsNotConfiguredError when unconfigured, enabled with a key;
  • a live, key-gated test asserting a Marengo text embedding is 512-dim (auto-skips without a key).
• All 4 tests pass locally against the repo's pinned langchain/langgraph.
• Verified the agent wiring: with a key set, twelvelabs_video_search is appended to the tool list with a single query arg.
• Verified the Marengo search.query request shape end-to-end against the live API (the server accepts and validates the call; full search results require a Marengo-enabled index of the channel's videos).
• Ran the repo's formatters/linter (ruff, black, isort) on all changed files — clean.

You can grab a free API key at https://twelvelabs.io — there's a generous free tier.