Skip to content

fix(part20): replace fabricated config blocks with real Hermes schemas#14

Merged
OnlyTerp merged 1 commit into
mainfrom
fix/observability-routing-schema
May 27, 2026
Merged

fix(part20): replace fabricated config blocks with real Hermes schemas#14
OnlyTerp merged 1 commit into
mainfrom
fix/observability-routing-schema

Conversation

@OnlyTerp

@OnlyTerp OnlyTerp commented May 27, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #13. The reporter flagged that the model_routing: / intent: / complexity: config block in part20-observability.md ("Rule 1: Route by Task Complexity") doesn't exist in Hermes. They're right — and on audit, that's the tip of the iceberg. The previous rev of part20 invented a stack of YAML blocks the Hermes loader does not read.

Verified against Hermes v0.14.0 source (/home/terp/.hermes/hermes-agent, commit cea87d913), the upstream cli-config.yaml.example, and the bundled plugins/observability/langfuse plugin.

Fabricated keys removed → real ones used

Old (invented) New (real)
model_routing: with routes / intent / complexity / match / tokens_gte auxiliary: per-task models (vision, web_extract, compression, summarization) + provider_routing: for OpenRouter + hermes fallback chain
prompt_caching: with enabled / providers / cache_system_prompt / cache_skills / cache_memory_digest / min_cache_tokens One-key prompt_caching.cache_ttl ("5m" or "1h"), plus the separate openrouter.response_cache layer
telemetry: spans: browser_cdp: true / computer_use: true Grep hermes logs tail -f --level WARNING or use a Langfuse alert on the existing browser_use / computer_use tool spans
fast_mode: with per-gateway defaults + user_override /fast on|off|status slash command — it's a runtime toggle, no YAML block
compression.auto.at_tokens / preserve.tool_results_matching / topics_from Real keys: compression.{enabled, threshold, target_ratio, protect_last_n}
alerts: cost_spike: / token_anomaly: block Alerts live in the tracing backend (Langfuse / Helicone / Phoenix) or in a small log-tailer that fires through hermes send telegram
observability: langfuse: block in config.yaml HERMES_LANGFUSE_PUBLIC_KEY / HERMES_LANGFUSE_SECRET_KEY / HERMES_LANGFUSE_BASE_URL / HERMES_LANGFUSE_SAMPLE_RATE / etc. in ~/.hermes/.env, matching the bundled plugin's README and __init__.py
observability: otel: block with endpoint / protocol / headers / attributes Standard OTEL_EXPORTER_OTLP_* env vars in ~/.hermes/.env
hermes evals init / dataset / run / compare subcommands Langfuse Datasets & Experiments workflow (Hermes has no built-in evals subcommand)

Heads-up callout for smart_model_routing

Added a paragraph at the top of the "Cost Routing Playbook" noting that the older smart_model_routing: block (cheap-vs-strong by message length) was removed upstream in commit 424e9f36b ("refactor: remove smart_model_routing feature", #12732, 2026-04-19). Readers who saw it referenced in older Hermes docs or upstream issue #16211 won't try to bring it back.

Verification

  • Hermes v0.14.0 installed locally; ~/.hermes/hermes-agent/hermes_cli/config.py is the loader of record.
  • grep -rn smart_model_routing /home/terp/.hermes/hermes-agent --include="*.py" returns zero hits in current source (only the removal commit in git log).
  • ~/.hermes/hermes-agent/plugins/observability/langfuse/{plugin.yaml,README.md,__init__.py} confirms the env-var contract.
  • cli-config.yaml.example confirms the real prompt_caching, compression, openrouter, and provider_routing shapes.
  • git diff --check clean; only part20-observability.md (rewrite of Rules 1/2/2B/3/4/5 + Langfuse + OTel + eval sections) and CHANGELOG.md (dated entry) changed.

Fixes #13.


Open in Devin Review

Issue #13 flagged that the 'Route by Task Complexity' rule referenced a
model_routing: DSL with intent/complexity/match keys that doesn't exist
in Hermes. On audit, that's the tip of the iceberg: the previous rev of
part20 invented a stack of config blocks that the Hermes loader does
not read.

Verified against Hermes v0.14.0 source
(/home/terp/.hermes/hermes-agent, commit cea87d913) and the upstream
cli-config.yaml.example.

Fabricated keys removed and replaced with real ones:

- model_routing: with intent/complexity/match/routes/tokens_gte
  -> auxiliary: per-task models + provider_routing: for OpenRouter
  + hermes fallback chain. Heads-up callout cites commit 424e9f36b
  ('refactor: remove smart_model_routing feature', #12732,
  2026-04-19) so readers don't try to bring back the older
  smart_model_routing block either.
- prompt_caching: with enabled/providers/cache_system_prompt/cache_skills/
  cache_memory_digest/min_cache_tokens
  -> real one-key block prompt_caching.cache_ttl ('5m' or '1h'),
  plus the separate openrouter.response_cache layer.
- telemetry: spans: browser_cdp / computer_use
  -> grep hermes logs or set a Langfuse alert on the existing
  browser_use/computer_use tool spans.
- fast_mode: with per-gateway defaults and user_override
  -> /fast on|off|status slash command (it's a runtime toggle, no
  YAML block).
- compression.auto.at_tokens / preserve / topics_from
  -> real compression.{enabled, threshold, target_ratio,
  protect_last_n} from cli-config.yaml.example.
- alerts.cost_spike / alerts.token_anomaly
  -> alerts live in the tracing backend (Langfuse / Helicone /
  Phoenix) or in a small log-tailer that fires through hermes send
  telegram; there is no alerts: config block.
- observability.langfuse.* (host/public_key/secret_key/sample_rate/
  traced_tools/redact_payloads)
  -> HERMES_LANGFUSE_PUBLIC_KEY / HERMES_LANGFUSE_SECRET_KEY /
  HERMES_LANGFUSE_BASE_URL / HERMES_LANGFUSE_SAMPLE_RATE / etc. in
  ~/.hermes/.env, matching the bundled plugin's README and
  __init__.py.
- observability.otel.endpoint / protocol / headers / attributes
  -> standard OTEL_EXPORTER_OTLP_* env vars in ~/.hermes/.env.
- hermes evals init / dataset / run / compare subcommands
  -> Langfuse Datasets & Experiments workflow (Hermes has no
  built-in evals subcommand).

Fixes #13.
@OnlyTerp OnlyTerp merged commit 43e3c8e into main May 27, 2026
4 of 5 checks passed

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] some features don't seem to be available in Hermes

1 participant