Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,7 @@ __pycache__/
.env.local
*.env
!.env.example

# Large local RAG corpus
packages/agent/data/corpus/*.jsonl
!packages/agent/data/corpus/*fixture*.jsonl
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ LLMs produce plausible math problems with wrong solutions ([MathTrap300](https:/
| **Web** | Node 22 + Next.js 14 App Router + Tailwind v4 | Landing + 출제 워크플로우 UI. SSE consumer of `agent` |

검증 흐름은 결정론적 6단계 파이프라인 (RAG → Intent → Generate → SymPy → Re-solve → Objective map). LLM은 생성 단계와 독립 재풀이 단계에만 관여하고, 정답 판정은 결코 하지 않는다 — 자세한 결정 근거는 [`docs/specs/architecture.md`](docs/specs/architecture.md), 도메인 개념은 [`docs/specs/domain.md`](docs/specs/domain.md). 프론트 디자인 시스템은 [`packages/web/DESIGN.md`](packages/web/DESIGN.md) (editorial + productivity 듀얼-서피스).
캡스톤 시연 범위는 [`docs/product/DEMO_SCOPE.md`](docs/product/DEMO_SCOPE.md)에 고정한다.

LLM access is pluggable: direct OpenAI/Anthropic, or via [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) for unified Claude/GPT/Gemini routing.

Expand Down
58 changes: 58 additions & 0 deletions docs/product/DEMO_SCOPE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Demo Scope

Status: active for capstone demo
Last updated: 2026-05-31

This document records the units OpenMath treats as demo-safe. The goal is not to
claim full curriculum coverage yet, but to make the capstone path predictable:
RAG references exist, strategy YAML exists, and the verification pipeline can
produce a result event.

## Primary 5-minute demo

Use this flow for the main presentation:

1. Grade: middle 1
2. Unit: `9수02-03` 일차방정식
3. Mode: `auto`
4. Count: 2-5
5. Expected behavior: structural and conceptual attempts alternate, each runs
through RAG, intent, generation, SymPy verification, independent re-solve,
objective mapping, and final result streaming.

Why this unit:

- The corpus has multiple equation-style references.
- `math-engine /solve` handles the generated linear equations directly.
- The strategy is simple enough to explain in a short judge-facing demo.
- The output is easy to inspect visually on the result screen.

## Secondary demo-safe units

These units have strategy YAML in `packages/agent/data/achievement-standards/`
and are suitable for fallback or comparison demos:

| Code | Unit | Grade | Demo note |
|---|---|---:|---|
| `9수01-05` | 제곱근과 실수 | 3 | Good for concept preservation examples |
| `9수02-07` | 연립일차방정식 | 2 | `/solve` now supports equation systems |
| `9수02-09` | 이차방정식 | 3 | Best for factoring and LaTeX rendering |

## Broader strategy coverage

The current strategy set covers 12 units:

`9수01-01`, `9수01-05`, `9수02-01`, `9수02-03`, `9수02-06`, `9수02-07`,
`9수02-08`, `9수02-09`, `9수02-10`, `9수03-02`, `9수03-04`, `9수04-05`.

These are enough for UI exploration and workflow tests. Production-quality
coverage still depends on corpus density and LLM output quality for each unit.

## Known limits

- Without `LLM_API_KEY` or `LLM_BASE_URL`, the agent uses the local fallback
generator. That verifies the pipeline shape, not final content quality.
- Some units can retrieve broader neighboring references when exact corpus
density is low.
- `mode: auto` is defined as alternating structural and conceptual attempts.
It does not rank the two modes yet.
17 changes: 8 additions & 9 deletions packages/agent/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@ MATH_ENGINE_URL=http://localhost:8000
CORPUS_JSONL=/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl

# LLM Provider: "openai" or "cliproxy"
LLM_PROVIDER=openai
LLM_PROVIDER=openai-compatible

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The provider example now uses openai-compatible, but the documented provider options in the same file still mention cliproxy (an invalid value). This can cause .env setup failures.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/.env.example, line 6:

<comment>The provider example now uses `openai-compatible`, but the documented provider options in the same file still mention `cliproxy` (an invalid value). This can cause `.env` setup failures.</comment>

<file context>
@@ -3,13 +3,12 @@ MATH_ENGINE_URL=http://localhost:8000
 
 # LLM Provider: "openai" or "cliproxy"
-LLM_PROVIDER=openai
+LLM_PROVIDER=openai-compatible
+LLM_BASE_URL=http://localhost:8080/v1
+LLM_API_KEY=dummy-key
</file context>

LLM_BASE_URL=http://localhost:8080/v1
LLM_API_KEY=dummy-key
LLM_MODEL=gpt-4o

# OpenAI Direct
OPENAI_API_KEY=your-api-key-here
OPENAI_MODEL=gpt-4o

# CLIProxyAPI (alternative)
CLIPROXY_BASE_URL=http://localhost:8080/v1
CLIPROXY_API_KEY=dummy-key
CLIPROXY_MODEL=gpt-4o
# OpenAI Direct alternative
# LLM_PROVIDER=openai
# LLM_API_KEY=your-openai-api-key
# LLM_MODEL=gpt-4o
81 changes: 67 additions & 14 deletions packages/agent/data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,43 @@
[비할당] 담당이 만지는 영역. agent 코드는 `RagClient` / `StrategyLoader` 인터페이스로만 접근하므로
이 디렉토리 안의 *형식*을 자유롭게 진화시킬 수 있다 (architecture.md D-7).

## 구조 (계획)
## 구조

```
data/
├── corpus/
│ └── math-sample-unified-v1.jsonl # math-sample-unified-v1 스키마 (2,400건)
│ # 출처: docs/PROGRESS.md §2.4
│ └── openmath_rag_records.jsonl # 로컬/서버에만 배치하는 본 corpus
└── achievement-standards/
├── 9수04-12.yaml # 이차방정식의 풀이
├── 9수04-13.yaml
└── ... # 성취기준별 1:1 파일 (I-T3)
├── 9수01-01.yaml # 소인수분해
├── 9수01-05.yaml # 제곱근과 실수
├── 9수02-01.yaml # 문자의 사용과 식의 값
├── 9수02-03.yaml # 일차방정식
├── 9수02-06.yaml # 일차부등식
├── 9수02-07.yaml # 연립일차방정식
├── 9수02-08.yaml # 다항식의 곱셈과 인수분해
├── 9수02-09.yaml # 이차방정식
├── 9수02-10.yaml # 이차방정식의 활용
├── 9수03-02.yaml # 일차함수와 그래프
├── 9수03-04.yaml # 이차함수와 그래프
└── 9수04-05.yaml # 삼각비
```

## Corpus

`math-sample-unified-v1.jsonl` — `docs/PROGRESS.md` §2.3 정규화 결과.
1차 MVP는 메모리 인덱스 (`createInMemoryRagClient`)로 로드.
런타임 corpus는 `openmath-rag-record-v1` JSONL이다. 기본 로컬 경로:

```text
/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/data/README.md, line 34:

<comment>Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions.</comment>

<file context>
@@ -5,23 +5,43 @@
+런타임 corpus는 `openmath-rag-record-v1` JSONL이다. 기본 로컬 경로:
+
+```text
+/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl
+```
+
</file context>
Suggested change
/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl
/absolute/path/to/openmath_rag_records.jsonl

```

권장 실행 방식은 corpus를 repo에 커밋하지 않고 `.env`에 절대 경로를 지정하는 것이다.

```env
CORPUS_JSONL=/absolute/path/to/openmath_rag_records.jsonl
```

본 corpus는 크기가 커서 `.gitignore`에서 제외한다. 작은 fixture가 필요하면 파일명에
`fixture`를 포함한 JSONL만 커밋한다.

후속 swap (Postgres / Cube / pgvector)은 인터페이스 안정 후 결정 — Q-2 partial closure.

Expand All @@ -31,17 +51,50 @@ data/

파일명은 성취기준 코드와 1:1 (`9수04-12.yaml`).

샘플 → `examples/9수04-12.yaml` 참조 (TODO: 작성).
최소 템플릿:

```yaml
code: 9수02-09
title: 이차방정식
school_level: middle
grade: 3
techniques:
required_at_least_one_of:
- factoring
forbidden: []
evaluation_dimensions:
- id: A
description: 이차식을 인수분해한다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
structural_transforms:
- kind: coefficient_swap
range: [-10, 10]
exclude_zero: true
conceptual_transforms:
- kind: present_via_root_relations
```

새 단원 추가 워크플로:

1. `achievement-standards/<code>.yaml` 파일을 만든다.
2. `code`, `title`, `school_level`, `grade`를 성취기준과 맞춘다.
3. `evaluation_dimensions`에는 `must_preserve: true` 항목을 최소 1개 둔다.
4. `problem_types_supported`와 `difficulty_range`를 시연 범위에 맞춘다.
5. `pnpm -F @openmath/agent test`로 YAML loader와 workflow 회귀를 확인한다.

## 변경 정책

- 새 strategy 추가 시 schema 검증 통과해야 함
- corpus 갱신은 [한진우]가 정규화 스크립트 (`scripts/normalize_sample_jsons.py` 등) 통과 후 commit
- `pnpm test`는 strategy YAML 파싱이 통과되는지 검증 (TODO: test 작성)
- corpus 갱신은 [한진우]가 정규화 스크립트 통과 후 외부 storage/로컬 경로로 배치
- `pnpm -F @openmath/agent test`는 strategy YAML 파싱이 통과되는지 검증

## TODO

- [ ] 핵심 단원 12개에 대한 strategy YAML 작성
- [ ] corpus JSONL을 이 디렉토리로 복사 (또는 symlink)
- [ ] strategy YAML 검증 테스트
- [x] 핵심 단원 12개 strategy YAML 작성
- [ ] 서버 배포 시 `CORPUS_JSONL` 위치 확정
- [ ] Cube/pgvector 도입 결정 (Q-2)
30 changes: 30 additions & 0 deletions packages/agent/data/achievement-standards/9수01-01.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
code: 9수01-01
title: 소인수분해
school_level: middle
grade: 1
techniques:
required_at_least_one_of:
- prime_factorization
- divisor_multiple_reasoning
forbidden:
- calculator_only
evaluation_dimensions:
- id: A
description: 자연수를 소수의 곱으로 분해한다.
must_preserve: true
- id: B
description: 소인수분해 결과를 약수와 배수 판단에 활용한다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
- short_answer
structural_transforms:
- kind: coefficient_swap
range: [2, 120]
exclude_zero: true
conceptual_transforms:
- kind: inverse_question
hint: 소인수분해 결과가 주어졌을 때 원래 수나 약수 개수를 묻는다.
30 changes: 30 additions & 0 deletions packages/agent/data/achievement-standards/9수01-05.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
code: 9수01-05
title: 제곱근과 실수
school_level: middle
grade: 3
techniques:
required_at_least_one_of:
- square_root_definition
- real_number_classification
forbidden:
- decimal_approximation_only
evaluation_dimensions:
- id: A
description: 제곱근의 뜻을 이용해 양수의 두 제곱근을 구분한다.
must_preserve: true
- id: B
description: 유리수와 무리수를 실수 범위에서 분류한다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
- short_answer
structural_transforms:
- kind: coefficient_swap
range: [2, 100]
exclude_zero: true
conceptual_transforms:
- kind: inverse_question
hint: 제곱근의 값이 주어졌을 때 원래 수를 찾는 형태로 바꾼다.
38 changes: 38 additions & 0 deletions packages/agent/data/achievement-standards/9수02-01.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
code: 9수02-01
title: 문자의 사용과 식의 값
school_level: middle
grade: 1
techniques:
required_at_least_one_of:
- algebraic_expression_modeling
- substitution
forbidden:
- guess_and_check_only
evaluation_dimensions:
- id: A
description: 수량 관계를 문자식으로 나타낸다.
must_preserve: true
- id: B
description: 주어진 문자 값으로 식의 값을 계산한다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
- short_answer
structural_transforms:
- kind: variable_rename
allowed:
- x
- a
- n
- kind: coefficient_swap
range: [-9, 9]
exclude_zero: true
conceptual_transforms:
- kind: rephrase_as_word_problem
context:
- 물건 가격
- 도형 둘레
- 점수
34 changes: 34 additions & 0 deletions packages/agent/data/achievement-standards/9수02-03.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
code: 9수02-03
title: 일차방정식
school_level: middle
grade: 1
techniques:
required_at_least_one_of:
- equation_balance
- linear_equation_solving
forbidden:
- calculator_only
evaluation_dimensions:
- id: A
description: 등식의 성질을 이용해 미지수 항을 분리한다.
must_preserve: true
- id: B
description: 양변에 같은 수를 더하거나 빼서 해를 구한다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
- short_answer
structural_transforms:
- kind: coefficient_swap
range: [-9, 9]
exclude_zero: true
- kind: sign_flip
conceptual_transforms:
- kind: rephrase_as_word_problem
context:
- 가격
- 거리
- 나이
34 changes: 34 additions & 0 deletions packages/agent/data/achievement-standards/9수02-06.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
code: 9수02-06
title: 일차부등식
school_level: middle
grade: 2
techniques:
required_at_least_one_of:
- inequality_balance
- interval_solution
forbidden:
- equation_only
evaluation_dimensions:
- id: A
description: 부등식의 성질을 이용해 미지수 항을 정리한다.
must_preserve: true
- id: B
description: 음수로 나누거나 곱할 때 부등호 방향을 바꾼다.
must_preserve: true
difficulty_range:
- easy
- medium
problem_types_supported:
- objective
- short_answer
structural_transforms:
- kind: coefficient_swap
range: [-8, 8]
exclude_zero: true
- kind: sign_flip
conceptual_transforms:
- kind: rephrase_as_word_problem
context:
- 최소 비용
- 제한 조건
- 점수 기준
Loading
Loading