Korean documentation · English documentation
This is not an emotion classifier. CounselCue is a Korean one-to-one counselor-training simulation for practicing how counseling micro-skills and embodied cues such as facial movement, gaze, and posture may work together in relational delivery.
The same validating statement can be received differently depending on facial expression, gaze, silence, and response timing. Instead of inferring the counselor's internal emotional state, CounselCue treats the following relationship as the object of practice:
Counseling micro-skill + calibrated embodied delivery cues
↓ temporal alignment
aligned / possible mismatch / insufficient evidence
↓
client safety · guardedness · disclosure
↓
next response opens or withdraws
The research direction centers on professional affective performance, relational delivery, and cross-modal congruence between counseling micro-skills and embodied behavior. Current values are prototype rules for expert review and user research, not clinical thresholds or competency scores.
flowchart LR
A["Review the case and goals"] --> B["Choose a full session or micro-skill"]
B --> C["Practice with the virtual client"]
C --> D["Align language and delivery evidence"]
D --> E["Self-assess each scene"]
E --> F["Compare system evidence"]
F --> G["Replay a selected scene"]
G --> C
System evidence remains hidden until the learner records a self-assessment. A selected scene can then restore the earlier client state and utterance for another attempt.
| Choose a practice path | Conduct the session | Compare evidence after self-assessment |
|---|---|---|
 |
 |
 |
| Status | Scope |
|---|---|
| Implemented | Five selectable counseling cases with distinct Rocketbox avatars, FACS/viseme facial layers, five-state gaze behavior, Korean viseme planning, micro-blinks/breath/head motion, face-observation zoom and diagnostics, full and focused sessions, replay, relational trajectory, local JSONL logging, Korean/English UI, browser-native Korean input, microphone dictation, spotlight onboarding, and ElevenLabs v3 client speech through a server proxy |
| Experimental | Case-specific Korean client personas through OpenRouter, bounded relational-state prompting, four-state emotional voice direction, thirteen MediaPipe-derived counselor AU proxies, personal baseline calibration, and deterministic local fallback |
| Planned | Audio-aligned phoneme timing, expert case-authoring tools, consent and deletion flows, an educator dashboard, and multi-site user research |
| Requires validation | Agreement between AU proxies and human FACS coding, expert inter-rater reliability for feedback rules, culture-specific cue interpretation, learning transfer, and change in counseling competence |
The scene draws on a contemporary Korean private-practice context with warm ivory, sage, and walnut tones. Client observability takes priority over decoration: the face, upper body, and hands remain visible, while observation zoom moves between facial detail and posture without changing the counselor-client sightline.
The live room now uses the project's Blender-authored asset pack for the hanji floor lamp, counseling books, acoustic wall panel, woven rug, basket plants, round oak table, linen tissue box, and celadon tea cup.
The pilot catalog now includes workplace anxiety, adolescent academic pressure, career transition and burnout, older-adult bereavement, and international-student belonging. Selecting a case changes the client profile, facial avatar, nonverbal interpretation parameters, disclosure trajectory, and server persona key together.
| Five-case catalog | Face observation and gaze diagnostics |
|---|---|
 |
 |
The Rocketbox face audit detects 175 blendshapes on the reference avatar, including 15 visemes and the AU shapes used by the relational expression layer. The diagnostics panel exposes the current gaze state, LookAt weight, bound facial shapes, and active viseme for development and expert review.
| Blender-authored counseling room upgrade |
|---|
 |
| Korean interface | English interface |
|---|---|
 |
 |
| Spotlight onboarding | Browser-native voice and Korean input |
|---|---|
 |
 |
- Unity:
6000.4.9f1 - Start scene:
Assets/Scenes/KoreanCounselingRoom.unity - Rebuild the generated scene:
Tools → CounselCue → Build Korean Counseling Room - Windows build output:
Builds/CounselCue/CounselCue.exe - WebGL build output:
Builds/WebGL/index.html - Live browser demo: https://educatian.github.io/counselcue/
The hosted WebGL build uses a server-side persona endpoint when configured and falls back to the deterministic local case engine if the request fails. ElevenLabs and OpenAI keys remain on the edge worker; they are never embedded in Unity or JavaScript. UDP AU input remains desktop-only.
The local case-based counseling flow works without a webcam. AU input requires the separate Python/MediaPipe bridge, and GPT Realtime requires a developer-owned ephemeral-token broker. See the English build guide or Korean run guide for detailed setup and commands.
No packaged public demo release is available yet. The repository currently provides a source prototype for research and usability review.
- Raw webcam video is not saved; only derived signals are processed and logged locally.
- AU values are proxies derived from MediaPipe blendshapes, not certified FACS coding or emotion labels.
- Counselor input and derived signals are written to local JSONL, so educational deployment requires explicit consent, retention limits, deletion, and pseudonymization policies.
- Feedback is candidate evidence for reflection. It must not be used for diagnosis, clinical evaluation, counselor selection, or automated competency assessment.
- The LLM client cannot replace real counseling and requires safety controls, latency handling, deterministic fallback, and expert supervision.
Virtual-client research suggests potential value for repeatable, lower-pressure communication practice and reflection. However, much of the evidence relies on self-report, small samples, or adjacent medical and social-work contexts and therefore does not directly establish this project's effectiveness.
- Understanding empathy training with virtual patients
- Virtual simulations to train social workers for competency-based learning
- Virtual clients, real gains: GenAI-simulated counseling role-play
The complete construct model, cultural interpretation principles, and validation plan are documented in GAME_CONCEPT.md.
- Microsoft Rocketbox assets follow
Assets/ThirdParty/MicrosoftRocketbox/LICENSE.md. - UI sprites come from the CC0-licensed Kenney UI Pack 2.0.
- Noto Sans KR is distributed under the SIL Open Font License 1.1; the license is included at
Assets/Fonts/OFL.txt. - No root open-source license currently covers the entire repository. Do not assume redistribution rights for project code or generated assets until a project license is declared.
- Korean documentation: session flow, LXD loop, AU calibration, GPT Realtime architecture, and privacy boundaries
- English documentation: capabilities, architecture, build workflow, privacy, and validation boundaries
- GAME_CONCEPT.md: research framing, cultural profile, and validation plan
Research and training prototype. Not a diagnostic, emotion-classification, clinical-decision, or automated counselor-assessment tool.