feat(stt): WhatsApp-style voice recording overlay with waveform & timer - #2
Draft
npeham wants to merge 4 commits into
Draft
feat(stt): WhatsApp-style voice recording overlay with waveform & timer#2npeham wants to merge 4 commits into
npeham wants to merge 4 commits into
Conversation
Replaces the bare red-microphone toggle with a recording overlay that covers the chat input while recording, for both STT paths (browser and external): - New useAudioLevelMonitor hook captures a visualization-only mic stream and exposes a rolling buffer of normalized audio levels plus elapsed recording time. The timer keeps ticking even if the visualization stream cannot be acquired. - New RecordingControls overlay renders a live waveform, a pulsing recording indicator with mm:ss timer, a delete (discard) button and a send (stop & transcribe) button. - Adds clearRecording to the browser and external STT hooks and routes it through useSpeechToText so a recording can be cancelled without transcribing; the textarea is restored to its pre-recording value. - AudioRecorder renders the overlay while listening and wires up cancel/send; respects chat direction (RTL). - Adds en localization keys: com_ui_recording, com_ui_delete_recording, com_ui_send_recording. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WihkHtq2yYXkcmwu5WZ2fV
Addresses three issues with the voice recording overlay: - Mic button not visible until an unrelated re-render: AudioRecorder no longer early-returns null while textAreaRef.current is unset, so the button renders immediately on mount. - Waveform invisible: the bars used the bg-text-secondary/70 alpha modifier, but theme colors are raw CSS vars without an <alpha-value> placeholder, producing an invalid color. Use a solid color with the opacity utility and raise the minimum bar height. Also resume the AudioContext when it starts suspended so levels actually update. - Recording stopped on its own: removed silence-based auto-stop from the external STT path and force continuous mode for browser STT, so recording only ends when the user presses stop (or delete). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WihkHtq2yYXkcmwu5WZ2fV
… hint) iOS WebKit (Safari and Chrome on iPhone) only permits one active microphone capture at a time, so the visualization-only getUserMedia stream opened by useAudioLevelMonitor conflicted with the recorder and prevented recording from starting. - useAudioLevelMonitor now accepts an optional existing stream and reuses it for analysis instead of opening a second capture. The external STT hook exposes its recording stream, routed through useSpeechToText into the monitor. The browser path (no stream) still acquires its own visualization stream. - getUserMedia is now guarded against an undefined navigator.mediaDevices (non-secure context) and surfaces a clear "requires HTTPS" toast, which is the common reason recording silently fails on mobile when the app is served over plain HTTP. - Adds en key com_ui_speech_requires_secure_context. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WihkHtq2yYXkcmwu5WZ2fV
The speech.speechTab.speechToText.engineSTT schema only accepted
['openai', 'azureOpenAI'], but this value is a pure client-side UI
default that maps to the client STTEndpoints ('browser' | 'external').
The server only forwards it; provider selection is handled separately by
speech.stt. The mismatch meant admins could not set 'external' as the
default STT engine from librechat.yaml.
Widen the enum to ['browser', 'external'] so admins can default new
clients to external STT (which, unlike browser Web Speech API, works on
iOS). Per-device localStorage still takes precedence, so existing user
choices are preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WihkHtq2yYXkcmwu5WZ2fV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the bare red-microphone toggle with a WhatsApp-style recording overlay that takes over the chat input while recording. Works for both STT paths — browser (Web Speech API) and external (server-side transcription).
Before
Clicking the mic just turned the icon red. No waveform, no timer, no way to cancel — the only way to stop was clicking the mic again, which immediately submitted.
After
While recording, an overlay covers the input box with:
m:sstimer showing elapsed recording timeChanges
useAudioLevelMonitor— captures a visualization-only mic stream and exposes a rolling buffer of normalized audio levels (0..1) plus elapsed time. Path-independent so the same UI serves browser + external STT. The timer keeps ticking even if the visualization stream can't be acquired.RecordingControls— the overlay UI (waveform, timer, delete, send). Localized + RTL-aware.clearRecordingadded touseSpeechToTextBrowseranduseSpeechToTextExternal, routed throughuseSpeechToText. The external hook discards buffered audio chunks via a cancel flag in itsstophandler; the browser hook aborts the recognition session.AudioRecorderrenders the overlay while listening and wires up cancel/send; respects chat direction.com_ui_recording,com_ui_delete_recording,com_ui_send_recording.Notes / limitations
npm installis blocked by the network policy on a transitive dependency hosted off-registry (cdn.sheetjs.com/xlsx-*.tgz→403 Forbidden), sonode_modulescan't be installed here. The remainingtscerrors were allCannot find module 'react'-style (missing deps), not logic errors. Changes were verified by manual review. Please runnpm run smart-reinstall+cd client && npm run typechecklocally to confirm.getUserMediastream alongside the recorder's stream. This is intentional so both STT paths share one waveform implementation (the browser STT lib doesn't expose its stream). Concurrent input capture is supported by all major browsers.🤖 Generated with Claude Code
https://claude.ai/code/session_01WihkHtq2yYXkcmwu5WZ2fV
Generated by Claude Code