Protocol-first Live2D multi-agent animation prototype for the 2026 agent ecosystem.
L2MAS explores how creative agents can plan, generate, voice, animate, review, and render Live2D animation through interoperable protocols. It uses A2A for agent collaboration, MCP 2025-11-25 + Streamable HTTP + Tasks for tool access, and a provider registry so cloud models and local models are first-class peers.
Qwen3.7-Max, Gemini Omni, Eleven v3, Textoon, and Live2D Cubism 5.3 are treated as 2026 capability baselines, not hard-coded dependencies.
| Signal | What to know |
|---|---|
| Purpose | Prototype a protocol-first, multi-agent Live2D animation pipeline. |
| Search keywords | Live2D, multi-agent AI, AI agents, MCP, Model Context Protocol, A2A, Agent2Agent, local AI, ComfyUI, Ollama, vLLM, FFmpeg, VTuber. |
| Runs today | Deterministic mock MVP plus a local FFmpeg video.compose smoke path. |
| Extension model | Provider registry and capability routing; agents call capabilities, not fixed model names. |
| Best for | AI agent builders, Live2D/VTube tooling researchers, local AI workflow developers, and animation automation experiments. |
English is the canonical documentation entry. Localized READMEs are limited to the top language set for Live2D technical development and community distribution: English, Simplified Chinese, Korean, Spanish, and Japanese.
| Language | README |
|---|---|
| English | README.md |
| 简体中文 | README.zh-CN.md |
| 한국어 | README.ko.md |
| Español | README.es.md |
| 日本語 | README.ja.md |
Translation policy: docs/i18n/README.md.
L2MAS is an early open-source prototype. The repository is designed around a two-stage roadmap:
| Stage | Goal | Status |
|---|---|---|
| MVP prototype | Run a local end-to-end path: script -> storyboard -> model -> voice -> motion -> render |
In progress |
| v2.0 architecture | Evolve into distributed A2A agents, MCP tool clusters, streaming task progress, Kubernetes, observability, security, and multi-tenant provider routing | Planned |
Current agent skeletons:
| Agent | MVP role | v2.0 direction |
|---|---|---|
| Director | Storyboard planning and orchestration | Cross-agent task routing and quality gates |
| Modeling | Sample model, Textoon local pipeline, or mock Live2D model path | Text/image-to-Live2D provider routing |
| Voice | Cloud/local TTS or mock audio artifact | Emotional TTS, voice conversion, STT integration |
| Animation | Motion and expression parameter planning | Motion generation, lip-sync aware shot animation |
| Renderer | FFmpeg local composition | Distributed MCP render service |
Planned v2.0 agents include Writer, Artist, LipSync, and QA.
Most AI animation experiments bind directly to one model, one tool API, or one workflow graph. L2MAS instead separates the system into:
- Generic protocol layer: A2A, MCP, task state, artifact schema, provider registry, and capability routing.
- Specialized creative layer: Live2D, Textoon, VTube/Live2D runtime, FFmpeg, TTS/STT, video generation, and video editing providers.
- Cloud/local parity: local providers are not a fallback afterthought; they are a supported deployment mode for privacy, cost control, offline work, and experimentation.
Agents call capabilities such as voice.generate or motion.generate. They do not call a fixed vendor model directly.
flowchart TB
user["User / App / API"] --> director["Director Agent"]
director --> writer["Writer Agent v2.0"]
director --> artist["Artist Agent v2.0"]
director --> modeling["Modeling Agent"]
director --> voice["Voice Agent"]
director --> animation["Animation Agent"]
director --> renderer["Renderer Agent"]
voice --> lipsync["LipSync Agent v2.0"]
animation --> qa["QA Agent v2.0"]
subgraph protocol["Protocol and Routing Layer"]
a2a["A2A Agent Cards and Tasks"]
mcp["MCP 2025-11-25 Streamable HTTP and Tasks"]
registry["Provider Registry"]
artifacts["Artifact Schema"]
end
director --> protocol
modeling --> protocol
voice --> protocol
animation --> protocol
renderer --> protocol
subgraph providers["Cloud, Local, and Hybrid Providers"]
llm["LLM / Agent Providers"]
visual["Image, Video, Character Providers"]
live2d["Live2D / Textoon Tooling"]
speech["TTS / STT / Voice Conversion"]
ffmpeg["FFmpeg Render Pipeline"]
end
protocol --> providers
The project standardizes on capability names that can be routed to cloud, local, or hybrid providers:
| Capability | Purpose |
|---|---|
script.plan |
script planning, storyboard structure, shot metadata |
character.generate |
character concepts, visual references, style exploration |
model.live2d.generate |
Live2D model generation or model artifact selection |
voice.generate |
dialogue voice generation |
speech.transcribe |
speech-to-text or phoneme preparation |
voice.convert |
voice conversion or cloning workflows |
lip_sync.align |
phoneme, viseme, and mouth-shape alignment |
motion.generate |
expression, pose, parameter, and motion sequencing |
video.compose |
scene composition and final render |
video.edit |
post-generation video editing |
quality.review |
script, motion, audio, render, and policy review |
Provider registry is the central contract for model and tool routing. Example: config/provider_registry.example.json.
Required fields:
| Field | Meaning |
|---|---|
provider_id |
stable provider identifier |
locality |
cloud, local, or hybrid |
protocol |
openai-compatible, ollama, mcp, comfyui, a2a, or custom-rest |
capabilities |
supported capability names |
endpoint |
cloud API, local service URL, or MCP/A2A endpoint |
models |
available model identifiers or workflow names |
hardware_profile |
expected hardware or runtime profile |
priority |
routing priority; lower is preferred |
fallbacks |
ordered fallback provider IDs |
privacy_mode |
remote, local-only, or hybrid |
status |
verified, experimental, template, or mock |
live_test_env |
optional environment variable that enables live provider tests |
auth_env |
optional API key environment variable |
healthcheck |
optional HTTP or binary probe metadata |
verification_evidence |
optional evidence record required when status is verified |
Provider availability is intentionally conservative. As of the current development state, local-ffmpeg is the only live-verified non-mock provider, with evidence recorded in docs/verification/local-ffmpeg.json. Other real adapters are contract-tested as experimental or held as template entries until a live service is validated.
L2MAS treats local inference and local media pipelines as first-class runtime targets.
| Category | Cloud baseline examples | Local/self-hosted compatibility |
|---|---|---|
| LLM / Agent | Qwen3.7-Max, Claude, GPT, Gemini | OpenAI-compatible endpoint, Ollama, vLLM, LM Studio, llama.cpp server |
| Image / video / character | Gemini Omni, specialized image/video APIs | ComfyUI local API, Diffusers worker, Textoon local pipeline |
| TTS / STT | Eleven v3, cloud STT/TTS APIs | local TTS, Whisper, whisper.cpp |
| Voice conversion | cloud voice conversion APIs | RVC-like and SeedVC-like providers |
| Embedding / rerank | cloud embedding/rerank APIs | local embedding services, OpenAI-compatible embedding endpoints |
| Render / compose | hosted media processing | FFmpeg local, FFmpeg MCP server |
Validate the current prototype configuration:
cp .env.example .env
docker compose configValidate JSON configuration:
python3 -m json.tool config/a2a_config.json > /dev/null
python3 -m json.tool config/mcp_config.json > /dev/null
python3 -m json.tool config/provider_registry.example.json > /dev/nullRun the deterministic local MVP smoke tests:
python3 -m unittest discover -s tests -vGenerate a provider verification probe report without enabling live network probes:
python3 examples/probe_providers.py --output output/provider-probe.jsonIf FFmpeg is available, the non-mock path can produce a real local MP4 container for video.compose while earlier generation stages remain deterministic prototype artifacts.
Use a local LLM by starting any compatible endpoint, then prioritizing that provider in the registry:
- Ollama:
http://localhost:11434 - vLLM OpenAI-compatible server
- LM Studio local server
- llama.cpp server
- Any OpenAI-compatible endpoint
The MVP path must remain runnable with mock or local providers when cloud API keys are absent.
| Document | Purpose |
|---|---|
| docs/architecture/two-stage-roadmap.md | MVP to v2.0 architecture roadmap |
| deployment_guide.md | English deployment and evolution guide |
| deployment_guide.zh-CN.md | Simplified Chinese deployment guide |
| config/provider_registry.example.json | provider registry reference example |
| docs/provider-verification.md | provider status, live verification, and disclosure policy |
| docs/i18n/README.md | localization policy |
| docs/github/repository-launch-checklist.md | GitHub publishing checklist and metadata |
| docs/github/discovery-profile.md | GitHub discovery profile, topics, labels, and community funnel |
| docs/releases/v0.1.0.md | v0.1.0 release notes |
| docs/releases/v0.2.0-draft.md | v0.2.0 draft notes and verification policy |
L2MAS is licensed under Apache-2.0.
| Community file | Purpose |
|---|---|
| CONTRIBUTING.md | contribution workflow and validation |
| CODE_OF_CONDUCT.md | community behavior expectations |
| SECURITY.md | private vulnerability reporting |
| SUPPORT.md | support channels and issue guidance |
| CHANGELOG.md | notable changes |
| GOVERNANCE.md | maintainer-led governance |
| CITATION.cff | citation metadata for GitHub |
Do not commit API keys, private endpoints, proprietary model weights, commercial media, or unauthorized Live2D assets.
This project is not affiliated with Live2D Inc. Live2D, Cubism, and related names are trademarks or registered trademarks of their respective owners.
Suggested repository description:
Live2D multi-agent animation prototype with MCP, A2A, provider routing, local AI, ComfyUI/Ollama/vLLM, and FFmpeg.
Suggested topics:
live2d, vtuber, animation-generation, text-to-animation, generative-ai, ai-agents, multi-agent, mcp, model-context-protocol, a2a, agent-to-agent, provider-registry, local-ai, openai-compatible, ollama, vllm, comfyui, diffusers, ffmpeg, python
More launch details: docs/github/discovery-profile.md.
Live2D animation generation, multi-agent AI, AI agents, MCP, Model Context Protocol, A2A, Agent2Agent, provider registry, capability routing, local AI, Ollama, vLLM, LM Studio, llama.cpp, ComfyUI, Diffusers, FFmpeg, Textoon, TTS, STT, lip sync, VTuber automation, cloud local hybrid AI.