Hermes Agent Architecture

Public-safe architecture notes for my local Hermes-based multi-agent system, memory layer, and operations workflow.

This repository documents my personal/local Hermes Agent architecture at a portfolio-safe level: role-specialized profiles/agents, orchestration, Mixture-of-Agents / LLM council patterns, judge-based review for higher-risk decisions, and durable memory using JSONL archives, SQLite FTS5, semantic/hybrid retrieval, and RRF-style ranking.

It also documents an important operational control: memory recall and memory ingestion can be turned on or off. This lets a session read from memory, write new memory, both, or neither, depending on the task sensitivity and desired behavior.

This is a public architecture/documentation repository. It does not claim ownership of any upstream project named Hermes, and it does not expose private code, local paths, secrets, logs, or personal memory data.

Why this exists

Agent systems become difficult to trust when everything is handled by one undifferentiated assistant. My goal with this architecture is to separate work by role, memory scope, model capability, tool boundary, and review level.

The core idea:

• different profiles/agents for different work
• orchestration instead of one-shot prompting
• durable memory with controlled recall and controlled ingestion
• LLM council + judge for higher-risk decisions
• upstream-safe customization where local extensions stay separate from core platform code

Current status

• Implementation status: active local/private system and extension layer
• Public status: architecture notes and sanitized examples
• Primary use case: agentic workflows, memory/retrieval, model routing, audit/review patterns
• Target direction: production-minded applied AI and field-engineering-style agent systems

Architecture at a glance

flowchart TD
    A[User request] --> B[Orchestrator]
    B --> C{Task classification}
    C --> D1[Generalist profile]
    C --> D2[Coder profile]
    C --> D3[Auditor profile]
    C --> D4[Orchestrator profile]
    C --> D5[Ops profile]
    C --> D6[PR Scout profile]
    C --> D7[Researcher profile]

    B --> M[Memory control]
    M --> R{Recall enabled?}
    R -->|On| E[Memory router]
    R -->|Off| N[No memory injected]

    E --> F1[Append-only JSONL archive]
    E --> F2[SQLite FTS5 keyword search]
    E --> F3[Semantic retrieval]
    F2 --> G[RRF / hybrid ranking]
    F3 --> G
    G --> H[Context budget]
    H --> B

    D1 --> I[LLM council / Mixture-of-Agents]
    D2 --> I
    D3 --> I
    D4 --> I
    D5 --> I
    D6 --> I
    D7 --> I
    I --> J[Judge / final review]
    J --> K[Output / report]

    K --> P{Ingest enabled?}
    P -->|On| Q[Persist safe metadata / memory]
    P -->|Off| S[Do not write session memory]

Loading

Profiles / agents

Profile / agent	Purpose
Generalist	Broad assistant tasks, clarification, synthesis, summaries, and default support
Coder	Implementation planning, repo-aware changes, tests, debugging support, and patch review preparation
Auditor	Risk checks, hidden side effects, consistency review, public/private boundary checks, and safety review
Orchestrator	Breaks down work, routes tasks, coordinates profiles/agents, and decides when council or human review is needed
Ops	Environment checks, commands, service/log/system state, deployment hygiene, and operational troubleshooting
PR Scout	Open-source contribution scanning, issue/PR opportunity triage, repo-fit checks, and contribution planning
Researcher	Background research, technical comparisons, source review, documentation support, and decision context

Memory recall and ingestion controls

The memory system is intentionally not always a single always-on behavior.

At the session/workflow level, two controls matter:

Control	Meaning
Recall On	Relevant memory can be retrieved and injected into the current task context
Recall Off	The agent works without pulling prior memory into the prompt/context
Ingest On	Safe session output/metadata can be written into memory after the task
Ingest Off	The session does not write new memory, useful for sensitive, temporary, or experimental work

This creates four useful modes:

Mode	Use case
Recall On + Ingest On	Normal long-running work where continuity is useful
Recall On + Ingest Off	Use prior context but do not store the current session
Recall Off + Ingest On	Start fresh but allow selected new learnings to be stored
Recall Off + Ingest Off	Clean/private/temporary session with no memory read or write

Memory model

The memory layer is designed around controlled recall, not “remember everything all the time.”

• Append-only JSONL archive for provenance and raw chronological history
• SQLite FTS5 for precise keyword recall
• Semantic retrieval for conceptual similarity
• Hybrid retrieval + RRF-style ranking to combine signals
• Context budgeting to avoid prompt bloat
• Recall control to decide whether memory is used in the current session
• Ingest control to decide whether current-session data is written back
• Dashboard visibility for memory/retrieval diagnostics where applicable

What this demonstrates

This project demonstrates my work around:

• role-specialized multi-agent workflows
• profile-based routing and orchestration
• LLM council / Mixture-of-Agents patterns
• durable agent memory architecture
• session-level recall and ingestion controls
• retrieval quality and context budgeting
• model-role separation
• upstream-safe extension design
• agentic workflow review and audit gates

Public artifacts in this repo

• docs/architecture.md — detailed architecture overview
• docs/memory-model.md — durable memory, recall, ingestion, and retrieval design
• docs/orchestration-flow.md — profile-based orchestration flow
• docs/public-vs-private.md — public/private boundary
• examples/sample-agent-run.md — sanitized example run
• diagrams/hermes-agent-architecture.mmd — architecture diagram
• diagrams/memory-retrieval-flow.mmd — memory retrieval and ingestion flow

Related work

• cmc-os-architecture — local-first agentic workflow control plane architecture
• local-llm-lab — local/cloud LLM experimentation and evaluation notes
• odysseus — fork used for upstream contribution work and agent-workflow exploration

Public/private boundary

This repository is intentionally a public architecture artifact. It does not include private implementation code, private memory data, local logs, secrets, credentials, private paths, or sensitive workflows.