-
Notifications
You must be signed in to change notification settings - Fork 0
Connect RAG search to agent SSE flow #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| # Demo Scope | ||
|
|
||
| Status: active for capstone demo | ||
| Last updated: 2026-05-31 | ||
|
|
||
| This document records the units OpenMath treats as demo-safe. The goal is not to | ||
| claim full curriculum coverage yet, but to make the capstone path predictable: | ||
| RAG references exist, strategy YAML exists, and the verification pipeline can | ||
| produce a result event. | ||
|
|
||
| ## Primary 5-minute demo | ||
|
|
||
| Use this flow for the main presentation: | ||
|
|
||
| 1. Grade: middle 1 | ||
| 2. Unit: `9수02-03` 일차방정식 | ||
| 3. Mode: `auto` | ||
| 4. Count: 2-5 | ||
| 5. Expected behavior: structural and conceptual attempts alternate, each runs | ||
| through RAG, intent, generation, SymPy verification, independent re-solve, | ||
| objective mapping, and final result streaming. | ||
|
|
||
| Why this unit: | ||
|
|
||
| - The corpus has multiple equation-style references. | ||
| - `math-engine /solve` handles the generated linear equations directly. | ||
| - The strategy is simple enough to explain in a short judge-facing demo. | ||
| - The output is easy to inspect visually on the result screen. | ||
|
|
||
| ## Secondary demo-safe units | ||
|
|
||
| These units have strategy YAML in `packages/agent/data/achievement-standards/` | ||
| and are suitable for fallback or comparison demos: | ||
|
|
||
| | Code | Unit | Grade | Demo note | | ||
| |---|---|---:|---| | ||
| | `9수01-05` | 제곱근과 실수 | 3 | Good for concept preservation examples | | ||
| | `9수02-07` | 연립일차방정식 | 2 | `/solve` now supports equation systems | | ||
| | `9수02-09` | 이차방정식 | 3 | Best for factoring and LaTeX rendering | | ||
|
|
||
| ## Broader strategy coverage | ||
|
|
||
| The current strategy set covers 12 units: | ||
|
|
||
| `9수01-01`, `9수01-05`, `9수02-01`, `9수02-03`, `9수02-06`, `9수02-07`, | ||
| `9수02-08`, `9수02-09`, `9수02-10`, `9수03-02`, `9수03-04`, `9수04-05`. | ||
|
|
||
| These are enough for UI exploration and workflow tests. Production-quality | ||
| coverage still depends on corpus density and LLM output quality for each unit. | ||
|
|
||
| ## Known limits | ||
|
|
||
| - Without `LLM_API_KEY` or `LLM_BASE_URL`, the agent uses the local fallback | ||
| generator. That verifies the pipeline shape, not final content quality. | ||
| - Some units can retrieve broader neighboring references when exact corpus | ||
| density is low. | ||
| - `mode: auto` is defined as alternating structural and conceptual attempts. | ||
| It does not rank the two modes yet. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -5,23 +5,43 @@ | |||||
| [비할당] 담당이 만지는 영역. agent 코드는 `RagClient` / `StrategyLoader` 인터페이스로만 접근하므로 | ||||||
| 이 디렉토리 안의 *형식*을 자유롭게 진화시킬 수 있다 (architecture.md D-7). | ||||||
|
|
||||||
| ## 구조 (계획) | ||||||
| ## 구조 | ||||||
|
|
||||||
| ``` | ||||||
| data/ | ||||||
| ├── corpus/ | ||||||
| │ └── math-sample-unified-v1.jsonl # math-sample-unified-v1 스키마 (2,400건) | ||||||
| │ # 출처: docs/PROGRESS.md §2.4 | ||||||
| │ └── openmath_rag_records.jsonl # 로컬/서버에만 배치하는 본 corpus | ||||||
| └── achievement-standards/ | ||||||
| ├── 9수04-12.yaml # 이차방정식의 풀이 | ||||||
| ├── 9수04-13.yaml | ||||||
| └── ... # 성취기준별 1:1 파일 (I-T3) | ||||||
| ├── 9수01-01.yaml # 소인수분해 | ||||||
| ├── 9수01-05.yaml # 제곱근과 실수 | ||||||
| ├── 9수02-01.yaml # 문자의 사용과 식의 값 | ||||||
| ├── 9수02-03.yaml # 일차방정식 | ||||||
| ├── 9수02-06.yaml # 일차부등식 | ||||||
| ├── 9수02-07.yaml # 연립일차방정식 | ||||||
| ├── 9수02-08.yaml # 다항식의 곱셈과 인수분해 | ||||||
| ├── 9수02-09.yaml # 이차방정식 | ||||||
| ├── 9수02-10.yaml # 이차방정식의 활용 | ||||||
| ├── 9수03-02.yaml # 일차함수와 그래프 | ||||||
| ├── 9수03-04.yaml # 이차함수와 그래프 | ||||||
| └── 9수04-05.yaml # 삼각비 | ||||||
| ``` | ||||||
|
|
||||||
| ## Corpus | ||||||
|
|
||||||
| `math-sample-unified-v1.jsonl` — `docs/PROGRESS.md` §2.3 정규화 결과. | ||||||
| 1차 MVP는 메모리 인덱스 (`createInMemoryRagClient`)로 로드. | ||||||
| 런타임 corpus는 `openmath-rag-record-v1` JSONL이다. 기본 로컬 경로: | ||||||
|
|
||||||
| ```text | ||||||
| /Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions. Prompt for AI agents
Suggested change
|
||||||
| ``` | ||||||
|
|
||||||
| 권장 실행 방식은 corpus를 repo에 커밋하지 않고 `.env`에 절대 경로를 지정하는 것이다. | ||||||
|
|
||||||
| ```env | ||||||
| CORPUS_JSONL=/absolute/path/to/openmath_rag_records.jsonl | ||||||
| ``` | ||||||
|
|
||||||
| 본 corpus는 크기가 커서 `.gitignore`에서 제외한다. 작은 fixture가 필요하면 파일명에 | ||||||
| `fixture`를 포함한 JSONL만 커밋한다. | ||||||
|
|
||||||
| 후속 swap (Postgres / Cube / pgvector)은 인터페이스 안정 후 결정 — Q-2 partial closure. | ||||||
|
|
||||||
|
|
@@ -31,17 +51,50 @@ data/ | |||||
|
|
||||||
| 파일명은 성취기준 코드와 1:1 (`9수04-12.yaml`). | ||||||
|
|
||||||
| 샘플 → `examples/9수04-12.yaml` 참조 (TODO: 작성). | ||||||
| 최소 템플릿: | ||||||
|
|
||||||
| ```yaml | ||||||
| code: 9수02-09 | ||||||
| title: 이차방정식 | ||||||
| school_level: middle | ||||||
| grade: 3 | ||||||
| techniques: | ||||||
| required_at_least_one_of: | ||||||
| - factoring | ||||||
| forbidden: [] | ||||||
| evaluation_dimensions: | ||||||
| - id: A | ||||||
| description: 이차식을 인수분해한다. | ||||||
| must_preserve: true | ||||||
| difficulty_range: | ||||||
| - easy | ||||||
| - medium | ||||||
| problem_types_supported: | ||||||
| - objective | ||||||
| structural_transforms: | ||||||
| - kind: coefficient_swap | ||||||
| range: [-10, 10] | ||||||
| exclude_zero: true | ||||||
| conceptual_transforms: | ||||||
| - kind: present_via_root_relations | ||||||
| ``` | ||||||
|
|
||||||
| 새 단원 추가 워크플로: | ||||||
|
|
||||||
| 1. `achievement-standards/<code>.yaml` 파일을 만든다. | ||||||
| 2. `code`, `title`, `school_level`, `grade`를 성취기준과 맞춘다. | ||||||
| 3. `evaluation_dimensions`에는 `must_preserve: true` 항목을 최소 1개 둔다. | ||||||
| 4. `problem_types_supported`와 `difficulty_range`를 시연 범위에 맞춘다. | ||||||
| 5. `pnpm -F @openmath/agent test`로 YAML loader와 workflow 회귀를 확인한다. | ||||||
|
|
||||||
| ## 변경 정책 | ||||||
|
|
||||||
| - 새 strategy 추가 시 schema 검증 통과해야 함 | ||||||
| - corpus 갱신은 [한진우]가 정규화 스크립트 (`scripts/normalize_sample_jsons.py` 등) 통과 후 commit | ||||||
| - `pnpm test`는 strategy YAML 파싱이 통과되는지 검증 (TODO: test 작성) | ||||||
| - corpus 갱신은 [한진우]가 정규화 스크립트 통과 후 외부 storage/로컬 경로로 배치 | ||||||
| - `pnpm -F @openmath/agent test`는 strategy YAML 파싱이 통과되는지 검증 | ||||||
|
|
||||||
| ## TODO | ||||||
|
|
||||||
| - [ ] 핵심 단원 12개에 대한 strategy YAML 작성 | ||||||
| - [ ] corpus JSONL을 이 디렉토리로 복사 (또는 symlink) | ||||||
| - [ ] strategy YAML 검증 테스트 | ||||||
| - [x] 핵심 단원 12개 strategy YAML 작성 | ||||||
| - [ ] 서버 배포 시 `CORPUS_JSONL` 위치 확정 | ||||||
| - [ ] Cube/pgvector 도입 결정 (Q-2) | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| code: 9수01-01 | ||
| title: 소인수분해 | ||
| school_level: middle | ||
| grade: 1 | ||
| techniques: | ||
| required_at_least_one_of: | ||
| - prime_factorization | ||
| - divisor_multiple_reasoning | ||
| forbidden: | ||
| - calculator_only | ||
| evaluation_dimensions: | ||
| - id: A | ||
| description: 자연수를 소수의 곱으로 분해한다. | ||
| must_preserve: true | ||
| - id: B | ||
| description: 소인수분해 결과를 약수와 배수 판단에 활용한다. | ||
| must_preserve: true | ||
| difficulty_range: | ||
| - easy | ||
| - medium | ||
| problem_types_supported: | ||
| - objective | ||
| - short_answer | ||
| structural_transforms: | ||
| - kind: coefficient_swap | ||
| range: [2, 120] | ||
| exclude_zero: true | ||
| conceptual_transforms: | ||
| - kind: inverse_question | ||
| hint: 소인수분해 결과가 주어졌을 때 원래 수나 약수 개수를 묻는다. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| code: 9수01-05 | ||
| title: 제곱근과 실수 | ||
| school_level: middle | ||
| grade: 3 | ||
| techniques: | ||
| required_at_least_one_of: | ||
| - square_root_definition | ||
| - real_number_classification | ||
| forbidden: | ||
| - decimal_approximation_only | ||
| evaluation_dimensions: | ||
| - id: A | ||
| description: 제곱근의 뜻을 이용해 양수의 두 제곱근을 구분한다. | ||
| must_preserve: true | ||
| - id: B | ||
| description: 유리수와 무리수를 실수 범위에서 분류한다. | ||
| must_preserve: true | ||
| difficulty_range: | ||
| - easy | ||
| - medium | ||
| problem_types_supported: | ||
| - objective | ||
| - short_answer | ||
| structural_transforms: | ||
| - kind: coefficient_swap | ||
| range: [2, 100] | ||
| exclude_zero: true | ||
| conceptual_transforms: | ||
| - kind: inverse_question | ||
| hint: 제곱근의 값이 주어졌을 때 원래 수를 찾는 형태로 바꾼다. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| code: 9수02-01 | ||
| title: 문자의 사용과 식의 값 | ||
| school_level: middle | ||
| grade: 1 | ||
| techniques: | ||
| required_at_least_one_of: | ||
| - algebraic_expression_modeling | ||
| - substitution | ||
| forbidden: | ||
| - guess_and_check_only | ||
| evaluation_dimensions: | ||
| - id: A | ||
| description: 수량 관계를 문자식으로 나타낸다. | ||
| must_preserve: true | ||
| - id: B | ||
| description: 주어진 문자 값으로 식의 값을 계산한다. | ||
| must_preserve: true | ||
| difficulty_range: | ||
| - easy | ||
| - medium | ||
| problem_types_supported: | ||
| - objective | ||
| - short_answer | ||
| structural_transforms: | ||
| - kind: variable_rename | ||
| allowed: | ||
| - x | ||
| - a | ||
| - n | ||
| - kind: coefficient_swap | ||
| range: [-9, 9] | ||
| exclude_zero: true | ||
| conceptual_transforms: | ||
| - kind: rephrase_as_word_problem | ||
| context: | ||
| - 물건 가격 | ||
| - 도형 둘레 | ||
| - 점수 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| code: 9수02-03 | ||
| title: 일차방정식 | ||
| school_level: middle | ||
| grade: 1 | ||
| techniques: | ||
| required_at_least_one_of: | ||
| - equation_balance | ||
| - linear_equation_solving | ||
| forbidden: | ||
| - calculator_only | ||
| evaluation_dimensions: | ||
| - id: A | ||
| description: 등식의 성질을 이용해 미지수 항을 분리한다. | ||
| must_preserve: true | ||
| - id: B | ||
| description: 양변에 같은 수를 더하거나 빼서 해를 구한다. | ||
| must_preserve: true | ||
| difficulty_range: | ||
| - easy | ||
| - medium | ||
| problem_types_supported: | ||
| - objective | ||
| - short_answer | ||
| structural_transforms: | ||
| - kind: coefficient_swap | ||
| range: [-9, 9] | ||
| exclude_zero: true | ||
| - kind: sign_flip | ||
| conceptual_transforms: | ||
| - kind: rephrase_as_word_problem | ||
| context: | ||
| - 가격 | ||
| - 거리 | ||
| - 나이 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| code: 9수02-06 | ||
| title: 일차부등식 | ||
| school_level: middle | ||
| grade: 2 | ||
| techniques: | ||
| required_at_least_one_of: | ||
| - inequality_balance | ||
| - interval_solution | ||
| forbidden: | ||
| - equation_only | ||
| evaluation_dimensions: | ||
| - id: A | ||
| description: 부등식의 성질을 이용해 미지수 항을 정리한다. | ||
| must_preserve: true | ||
| - id: B | ||
| description: 음수로 나누거나 곱할 때 부등호 방향을 바꾼다. | ||
| must_preserve: true | ||
| difficulty_range: | ||
| - easy | ||
| - medium | ||
| problem_types_supported: | ||
| - objective | ||
| - short_answer | ||
| structural_transforms: | ||
| - kind: coefficient_swap | ||
| range: [-8, 8] | ||
| exclude_zero: true | ||
| - kind: sign_flip | ||
| conceptual_transforms: | ||
| - kind: rephrase_as_word_problem | ||
| context: | ||
| - 최소 비용 | ||
| - 제한 조건 | ||
| - 점수 기준 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: The provider example now uses
openai-compatible, but the documented provider options in the same file still mentioncliproxy(an invalid value). This can cause.envsetup failures.Prompt for AI agents