diff --git a/README.md b/README.md index a4455ac..c240950 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,53 @@ Andromeda is a tools-first financial QA system over SEC filings. It combines planner-routed tool calls, hybrid retrieval, reranking, and eval-governed iteration so numeric and narrative answers are grounded in explicit evidence. +## Latest Status (as of 2026-02-20) + +Recent repo changes and benchmark results to know first: + +- Planner characteristics quality improved substantially in the latest planner eval run: + - exact match `0.98` + - macro F1 `0.9960` + - micro F1 `0.9919` + - see `BENCHMARK_PLANNER_v3.md` + +## UI Snapshot Walkthrough + +### 1) Main workspace (planner + progress + source viewer) + +![Andromeda main workspace](docs/full_snapshot.png) + +The main UI combines conversation history, planner stage timings, tool results, reranked evidence, and a synchronized source viewer. + +### 2) Tools-first evidence collection + +![Tool snapshot](docs/tool_snapshot.png) + +Tool cards surface profile/valuation, recent news, price history, and SEC financial metrics before final synthesis. + +### 3) Reranked chunks used for answer grounding + +![Reranked chunks panel](docs/reranked_chunks.png) + +The retrieval panel shows ranked chunks, filing metadata, and direct "open source / view in app" links for inspection. + +### 4) Final answer structure + +![Answer example](docs/answer_example.png) + +Answers are organized as thesis points with direct quotes and additional context tied to evidence. + +### 5) Citation list for traceability + +![Answer citations](docs/answer_citations.png) + +Each response includes a consolidated cited-sources block with filing tags and tool outputs. + +### 6) Eval review interface + +![Eval review UI](docs/eval_review_UI.png) + +The review UI supports case filtering, pass/fail labeling, timing inspection, and targeted export/reload workflows. ## Architecture diff --git a/README_EVAL.md b/README_EVAL.md index 3a83495..3ba8216 100644 --- a/README_EVAL.md +++ b/README_EVAL.md @@ -180,9 +180,9 @@ Per run directory (`planner_eval_run.*`): ## 3) Current Metrics Snapshot ### Full single-ticker suite (100 queries, all non-comparison kinds) -TO BE UPDATED +TO BE UPDATED ### Multi-ticker comparison snapshot -TO BE UPDATED +TO BE UPDATED ### Open-ended stress diff --git a/docs/answer_citations.png b/docs/answer_citations.png new file mode 100644 index 0000000..b2d35ce Binary files /dev/null and b/docs/answer_citations.png differ diff --git a/docs/answer_example.png b/docs/answer_example.png new file mode 100644 index 0000000..9521ee5 Binary files /dev/null and b/docs/answer_example.png differ diff --git a/docs/answer_goog_chunk_citation.png b/docs/answer_goog_chunk_citation.png deleted file mode 100644 index a2b53dd..0000000 Binary files a/docs/answer_goog_chunk_citation.png and /dev/null differ diff --git a/docs/answer_goog_citations.png b/docs/answer_goog_citations.png deleted file mode 100644 index 3e5d839..0000000 Binary files a/docs/answer_goog_citations.png and /dev/null differ diff --git a/docs/answer_goog_example.png b/docs/answer_goog_example.png deleted file mode 100644 index 73decf6..0000000 Binary files a/docs/answer_goog_example.png and /dev/null differ diff --git a/docs/answer_goog_snippet.png b/docs/answer_goog_snippet.png deleted file mode 100644 index e19ab03..0000000 Binary files a/docs/answer_goog_snippet.png and /dev/null differ diff --git a/docs/answer_tesla_example.png b/docs/answer_tesla_example.png deleted file mode 100644 index 886b588..0000000 Binary files a/docs/answer_tesla_example.png and /dev/null differ diff --git a/docs/eval_judge_1.png b/docs/eval_judge_1.png deleted file mode 100644 index 7813951..0000000 Binary files a/docs/eval_judge_1.png and /dev/null differ diff --git a/docs/eval_judge_2.png b/docs/eval_judge_2.png deleted file mode 100644 index 0e90292..0000000 Binary files a/docs/eval_judge_2.png and /dev/null differ diff --git a/docs/eval_overview.png b/docs/eval_overview.png deleted file mode 100644 index 405d835..0000000 Binary files a/docs/eval_overview.png and /dev/null differ diff --git a/docs/eval_reranked_chunks.png b/docs/eval_reranked_chunks.png deleted file mode 100644 index eb28ee4..0000000 Binary files a/docs/eval_reranked_chunks.png and /dev/null differ diff --git a/docs/eval_review_UI.png b/docs/eval_review_UI.png new file mode 100644 index 0000000..310cb11 Binary files /dev/null and b/docs/eval_review_UI.png differ diff --git a/docs/full_snapshot.png b/docs/full_snapshot.png new file mode 100644 index 0000000..c5db2cc Binary files /dev/null and b/docs/full_snapshot.png differ diff --git a/docs/reranked_chunks.png b/docs/reranked_chunks.png new file mode 100644 index 0000000..188e47a Binary files /dev/null and b/docs/reranked_chunks.png differ diff --git a/docs/tool_snapshot.png b/docs/tool_snapshot.png new file mode 100644 index 0000000..685b996 Binary files /dev/null and b/docs/tool_snapshot.png differ