Add fast local English dictation and configurable triggers#3
Conversation
📝 WalkthroughWalkthroughThe PR adds Parakeet Unified support as the default Apple Silicon transcription engine, configurable keyboard and mouse triggers with hold/toggle modes, updated onboarding and settings UI, benchmark tooling, dependency pinning, and a local app build script. ChangesVoiceScribe transcription and trigger flow
Estimated code review effort: 5 (Critical) | ~90 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant KeyMouseMonitor
participant AppState
participant AudioRecorder
participant TranscriptionEngine
participant TextInserter
User->>KeyMouseMonitor: Press configured trigger
KeyMouseMonitor->>AppState: Report trigger state
AppState->>AudioRecorder: Start or stop recording
AppState->>TranscriptionEngine: Transcribe recorded audio
TranscriptionEngine-->>AppState: Return transcript
AppState->>TextInserter: Insert transcript
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
Benchmarks/australian-english-30/hypotheses.txt (1)
1-31: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winPopulate or remove this benchmark fixture.
hypotheses.txtcontains no hypotheses—only blank lines—so it provides no testable benchmark assumptions. Add the intended entries, ideally one per phrase, or omit the file until it has meaningful content.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Benchmarks/australian-english-30/hypotheses.txt` around lines 1 - 31, Populate hypotheses.txt with meaningful benchmark assumptions, preferably one hypothesis per phrase, or remove the empty fixture until those entries are available.
🧹 Nitpick comments (1)
Benchmarks/australian-english-30/score.py (1)
57-57: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMake the validated zip contract explicit.
The line-count check immediately above guarantees equal lengths, so use
strict=Trueto satisfy B905 and protect this invariant if the loop is later refactored. Confirm the benchmark tooling targets Python 3.10 or newer before applying this change.Proposed fix
- for index, (reference, hypothesis) in enumerate(zip(references, hypotheses), start=1): + for index, (reference, hypothesis) in enumerate( + zip(references, hypotheses, strict=True), start=1 + ):🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Benchmarks/australian-english-30/score.py` at line 57, Update the references/hypotheses zip call in the benchmark loop to use strict=True, preserving the existing start=1 enumeration. First confirm the benchmark tooling targets Python 3.10 or newer; apply the change only if that compatibility requirement is satisfied.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Sources/ParakeetSmoke/main.swift`:
- Around line 18-26: Move the “Loading model: complete” stderr write in the
startup flow to immediately after manager.loadModels() returns and before
starting transcription. Remove it from after manager.transcribe(samples) so the
loading message reflects only model loading completion.
In `@Sources/VoiceScribe/Core/KeyMouseMonitor.swift`:
- Around line 78-85: Update the Fn handling branch in KeyMouseMonitor to require
the incoming event’s keyCode to equal kVK_Function in addition to the existing
binding and flagsChanged checks. Preserve the current keyboard state update and
return behavior only for actual Fn events, allowing other modifier changes to
continue through normal processing.
In `@Sources/VoiceScribe/Core/TextInserter.swift`:
- Around line 16-32: Update the pasteboard setup in the text insertion method so
the transcription string is successfully set before clearing or replacing the
original clipboard. Preserve the existing backup and restoration flow, ensuring
a failed setString operation returns without wiping the prior clipboard.
In `@Sources/VoiceScribe/Views/OnboardingView.swift`:
- Around line 79-88: Update startDefaultModelDownloadIfNeeded to skip the
default-model prefetch on x86_64, gating the existing download flow to arm64
before accessing or downloading ModelMetadata.defaultModel; preserve the current
guards and Task behavior for supported arm64 systems.
In `@Sources/WhisperBenchmark/main.swift`:
- Around line 14-60: Remove the obsolete download: argument from the WhisperKit
initializer call in the benchmark setup, keeping the existing modelFolder,
computeOptions, verbose, logLevel, prewarm, and load arguments unchanged so it
matches the 0.18.0 public initializer.
---
Outside diff comments:
In `@Benchmarks/australian-english-30/hypotheses.txt`:
- Around line 1-31: Populate hypotheses.txt with meaningful benchmark
assumptions, preferably one hypothesis per phrase, or remove the empty fixture
until those entries are available.
---
Nitpick comments:
In `@Benchmarks/australian-english-30/score.py`:
- Line 57: Update the references/hypotheses zip call in the benchmark loop to
use strict=True, preserving the existing start=1 enumeration. First confirm the
benchmark tooling targets Python 3.10 or newer; apply the change only if that
compatibility requirement is satisfied.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 73980a2f-c9d2-4f13-b439-4fd2c19dc661
📒 Files selected for processing (33)
.gitignoreBenchmarks/australian-english-30/README.mdBenchmarks/australian-english-30/hypotheses.txtBenchmarks/australian-english-30/phrases.txtBenchmarks/australian-english-30/score.pyBenchmarks/performance/australian-english.aiffPackage.resolvedPackage.swiftREADME.mdResources/Info.plistSources/ParakeetSmoke/main.swiftSources/VoiceScribe/App/AppState.swiftSources/VoiceScribe/App/VoiceScribeApp.swiftSources/VoiceScribe/Core/AudioRecorder.swiftSources/VoiceScribe/Core/EscapeKeyMonitor.swiftSources/VoiceScribe/Core/FnKeyMonitor.swiftSources/VoiceScribe/Core/KeyMouseMonitor.swiftSources/VoiceScribe/Core/KeyboardBindingCaptureMonitor.swiftSources/VoiceScribe/Core/MouseButtonCaptureMonitor.swiftSources/VoiceScribe/Core/SpacebarMonitor.swiftSources/VoiceScribe/Core/TextInserter.swiftSources/VoiceScribe/Core/TranscriptionEngine.swiftSources/VoiceScribe/Views/MenuBarView.swiftSources/VoiceScribe/Views/OnboardingView.swiftSources/VoiceScribe/Views/SettingsView.swiftSources/VoiceScribeCore/KeyboardBinding.swiftSources/VoiceScribeCore/ModelMetadata.swiftSources/VoiceScribeCore/TriggerInteraction.swiftSources/VoiceScribeCore/TriggerKey.swiftSources/WhisperBenchmark/main.swiftTests/VoiceScribeTests/ModelMetadataTests.swiftTests/VoiceScribeTests/TriggerKeyTests.swiftbuild-local-app.sh
| FileHandle.standardError.write(Data("Loading model...\n".utf8)) | ||
| try await manager.loadModels() | ||
|
|
||
| let started = ContinuousClock.now | ||
| let transcript = try await manager.transcribe(samples) | ||
| let elapsed = started.duration(to: .now) | ||
| FileHandle.standardError.write(Data("Loading model: complete\n".utf8)) | ||
| print(transcript) | ||
| FileHandle.standardError.write(Data("Transcription time: \(elapsed)\n".utf8)) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
"Loading model: complete" is printed after transcription, not after model loading.
Line 24 writes the completion message to stderr after transcribe(samples) at line 22, making it appear that model loading includes transcription time. Move it to right after loadModels() returns.
🔧 Proposed fix: move the completion log to after loadModels()
FileHandle.standardError.write(Data("Loading model...\n".utf8))
try await manager.loadModels()
+ FileHandle.standardError.write(Data("Loading model: complete\n".utf8))
let started = ContinuousClock.now
let transcript = try await manager.transcribe(samples)
let elapsed = started.duration(to: .now)
- FileHandle.standardError.write(Data("Loading model: complete\n".utf8))
print(transcript)
FileHandle.standardError.write(Data("Transcription time: \(elapsed)\n".utf8))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| FileHandle.standardError.write(Data("Loading model...\n".utf8)) | |
| try await manager.loadModels() | |
| let started = ContinuousClock.now | |
| let transcript = try await manager.transcribe(samples) | |
| let elapsed = started.duration(to: .now) | |
| FileHandle.standardError.write(Data("Loading model: complete\n".utf8)) | |
| print(transcript) | |
| FileHandle.standardError.write(Data("Transcription time: \(elapsed)\n".utf8)) | |
| FileHandle.standardError.write(Data("Loading model...\n".utf8)) | |
| try await manager.loadModels() | |
| FileHandle.standardError.write(Data("Loading model: complete\n".utf8)) | |
| let started = ContinuousClock.now | |
| let transcript = try await manager.transcribe(samples) | |
| let elapsed = started.duration(to: .now) | |
| print(transcript) | |
| FileHandle.standardError.write(Data("Transcription time: \(elapsed)\n".utf8)) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/ParakeetSmoke/main.swift` around lines 18 - 26, Move the “Loading
model: complete” stderr write in the startup flow to immediately after
manager.loadModels() returns and before starting transcription. Remove it from
after manager.transcribe(samples) so the loading message reflects only model
loading completion.
| if keyboardBinding.keyCode == Int64(kVK_Function), type == .flagsChanged { | ||
| let pressed = event.flags.contains(.maskSecondaryFn) | ||
| if pressed != keyboardIsDown { | ||
| keyboardIsDown = pressed | ||
| onStateChanged(.keyboard, pressed) | ||
| } | ||
| return nil | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
CGEvent flagsChanged keyboardEventKeycode Fn key kVK_Function value
💡 Result:
The virtual key code constant for the Function (Fn) key on macOS is kVK_Function, which has a hexadecimal value of 0x3F [1][2][3]. When monitoring keyboard events using the Core Graphics framework, changes to the state of the Fn key (as well as other modifier keys like Shift, Control, Option, and Command) are captured via events of type CGEventType.flagsChanged [4][5]. Key implementation details include: 1. Identification: The constant kVK_Function is defined as 0x3F in the macOS SDK and is widely used across various programming environments (e.g., Swift, Rust, C++) to represent the physical Fn key [1][6][3][7][8]. 2. Detection and Monitoring: To intercept changes in modifier key states globally, you typically use an event tap created with CGEventTapCreate, specifying the event mask CGEventMaskBit(kCGEventFlagsChanged) [4][9]. 3. Modifier Flags: While kVK_Function represents the physical key code, the status of the Fn key as a modifier is often checked or set using the flag mask kCGEventFlagMaskSecondaryFn [10][11]. Note that simply setting flags on a CGEvent may not always trigger the desired behavior in target applications; if you are attempting to simulate key presses involving the Fn key, you may need to explicitly post the key-down and key-up events for the Fn key itself in sequence with the target key, rather than relying solely on setting event flags [10][11]. Ensure your application has the necessary Accessibility permissions in System Settings when attempting to monitor or inject global events [11].
Citations:
- 1: https://docs.rs/core-graphics/latest/src/core_graphics/event.rs.html
- 2: https://developer.mozilla.org/en-US/docs/Web/API/UI_Events/Keyboard_event_code_values
- 3: https://www.alanwsmith.com/en/2p20t0o0/
- 4: https://stackoverflow.com/questions/1603030/how-to-monitor-global-modifier-key-state-in-any-application
- 5: https://developer.apple.com/documentation/coregraphics/cgeventtype/flagschanged
- 6: https://chromium.googlesource.com/chromium/src/+/lkgr/ui/events/keycodes/keyboard_code_conversion_mac.mm
- 7: https://gist.github.com/dagronf/51a1ccf92f528ffab7183e3bdc457ac4
- 8: https://github.com/rustdesk/rustdesk/blob/master/libs/enigo/src/macos/keycodes.rs
- 9: https://stackoverflow.com/questions/35425272/emulate-send-modifier-key-cntrl-alt-fn-shift-in-osx
- 10: https://groups.google.com/g/cocoa-dev/c/XLuBQwZDqNc
- 11: https://stackoverflow.com/questions/79518299/posting-key-press-cgevent-fails-in-macos-15-sequoia
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="Sources/VoiceScribe/Core/KeyMouseMonitor.swift"
echo "== outline =="
ast-grep outline "$file" --view expanded || true
echo
echo "== relevant lines =="
sed -n '60,95p' "$file" | cat -n
echo
echo "== search for keyCode usage in file =="
rg -n "keyCode|flagsChanged|maskSecondaryFn|kVK_Function" "$file"Repository: jpthor/voicescribe
Length of output: 2587
🌐 Web query:
macOS CGEvent flagsChanged Fn keyboardEventKeycode kVK_Function keyboardEventKeycode
💡 Result:
In macOS development, the Fn (Function) key is treated as a modifier key, and its state changes are captured using the kCGEventFlagsChanged event type [1][2][3]. The virtual key code associated with the Fn key is kVK_Function, which has the hexadecimal value 0x3F (decimal 63) [4][5][6][7]. When using Core Graphics Event Taps to monitor or intercept keyboard events, you should use the kCGEventFlagsChanged event type to detect when modifier keys, including Fn, are pressed or released [2][3]. To specifically identify the Fn key in these events, you can check both the event's key code (kVK_Function) and its modifier flags [3]. Key technical details: - Virtual Key Code: kVK_Function = 0x3F [4][5][7]. - Modifier Flag Mask: In many macOS APIs, the Fn key state is represented by kCGEventFlagMaskSecondaryFn (or simply the.function modifier flag in Swift/AppKit) [8][4][9][3]. - Event Type: Monitor kCGEventFlagsChanged to handle changes in the Fn key state [1][3]. If you are using Swift, you can detect the Fn key state changes by monitoring.flagsChanged events and checking for the combination of kVK_Function (63) and the.function modifier flag [3]. Note that simulating the Fn key press sometimes requires specific permissions in System Settings (Accessibility) and, depending on the macOS version, may involve complex handling of modifier flags within the event object [8][10][9].
Citations:
- 1: https://developer.apple.com/documentation/coregraphics/cgeventtype/flagschanged
- 2: https://stackoverflow.com/questions/1603030/how-to-monitor-global-modifier-key-state-in-any-application
- 3: https://stackoverflow.com/questions/69889105/how-to-detect-fn-key-press-in-swift
- 4: https://docs.rs/core-graphics/latest/src/core_graphics/event.rs.html
- 5: https://stackoverflow.com/questions/3202629/where-can-i-find-a-list-of-mac-virtual-key-codes
- 6: https://www.alanwsmith.com/en/2p20t0o0/
- 7: https://github.com/rustdesk/rustdesk/blob/master/libs/enigo/src/macos/keycodes.rs
- 8: https://stackoverflow.com/questions/79518299/posting-key-press-cgevent-fails-in-macos-15-sequoia
- 9: https://github.com/rustdesk/rustdesk/blob/caadd72a/libs/enigo/src/macos/macos_impl.rs
- 10: https://stackoverflow.com/questions/35425272/emulate-send-modifier-key-cntrl-alt-fn-shift-in-osx
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="Sources/VoiceScribe/Core/KeyMouseMonitor.swift"
echo "== file map =="
ast-grep outline "$file" --view expanded || true
echo
echo "== relevant sections =="
sed -n '1,140p' "$file" | cat -n
echo
echo "== event/state callbacks in repo =="
rg -n "onStateChanged\(|keyboardIsDown|mouseIsDown|keyboardBinding|flagsChanged|kVK_Function|maskSecondaryFn" Sources/VoiceScribe -g '!**/*.xcassets'Repository: jpthor/voicescribe
Length of output: 10104
Fn binding should only handle Fn flagsChanged events.
flagsChanged fires for every modifier, so this branch currently swallows Shift/Control/Option/Command/CapsLock changes whenever the bound key is Fn. That can also emit a false .keyboard release when another modifier changes while Fn is held. Gate this on keyCode == kVK_Function here; the event key code is already available.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/VoiceScribe/Core/KeyMouseMonitor.swift` around lines 78 - 85, Update
the Fn handling branch in KeyMouseMonitor to require the incoming event’s
keyCode to equal kVK_Function in addition to the existing binding and
flagsChanged checks. Preserve the current keyboard state update and return
behavior only for actual Fn events, allowing other modifier changes to continue
through normal processing.
| let pasteboard = NSPasteboard.general | ||
|
|
||
| let backup = backupClipboard(pasteboard) | ||
|
|
||
| pasteboard.clearContents() | ||
| pasteboard.setString(text, forType: .string) | ||
| guard pasteboard.setString(text, forType: .string) else { return } | ||
| let transcriptionChangeCount = pasteboard.changeCount | ||
|
|
||
| simulatePaste() | ||
|
|
||
| DispatchQueue.main.asyncAfter(deadline: .now() + 0.15) { | ||
| DispatchQueue.main.asyncAfter(deadline: .now() + restoreDelay) { [weak self] in | ||
| guard let self else { return } | ||
| // If the user or another app copied something after transcription, | ||
| // preserve that newer clipboard rather than restoring stale data. | ||
| guard pasteboard.changeCount == transcriptionChangeCount else { return } | ||
| self.restoreClipboard(pasteboard, from: backup) | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Clipboard is wiped if setString fails.
clearContents() (Line 19) runs before the setString guard (Line 20). If setString returns false, the method returns early and the async restore never runs, so the previously backed-up clipboard is lost. Reorder so the original clipboard is only cleared once the transcription string is guaranteed to be set.
🛡️ Proposed fix
- pasteboard.clearContents()
- guard pasteboard.setString(text, forType: .string) else { return }
- let transcriptionChangeCount = pasteboard.changeCount
+ pasteboard.clearContents()
+ guard pasteboard.setString(text, forType: .string) else {
+ // Restore what we just cleared so the user's clipboard isn't lost.
+ restoreClipboard(pasteboard, from: backup)
+ return
+ }
+ let transcriptionChangeCount = pasteboard.changeCount📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| let pasteboard = NSPasteboard.general | |
| let backup = backupClipboard(pasteboard) | |
| pasteboard.clearContents() | |
| pasteboard.setString(text, forType: .string) | |
| guard pasteboard.setString(text, forType: .string) else { return } | |
| let transcriptionChangeCount = pasteboard.changeCount | |
| simulatePaste() | |
| DispatchQueue.main.asyncAfter(deadline: .now() + 0.15) { | |
| DispatchQueue.main.asyncAfter(deadline: .now() + restoreDelay) { [weak self] in | |
| guard let self else { return } | |
| // If the user or another app copied something after transcription, | |
| // preserve that newer clipboard rather than restoring stale data. | |
| guard pasteboard.changeCount == transcriptionChangeCount else { return } | |
| self.restoreClipboard(pasteboard, from: backup) | |
| } | |
| } | |
| let pasteboard = NSPasteboard.general | |
| let backup = backupClipboard(pasteboard) | |
| pasteboard.clearContents() | |
| guard pasteboard.setString(text, forType: .string) else { | |
| // Restore what we just cleared so the user's clipboard isn't lost. | |
| restoreClipboard(pasteboard, from: backup) | |
| return | |
| } | |
| let transcriptionChangeCount = pasteboard.changeCount | |
| simulatePaste() | |
| DispatchQueue.main.asyncAfter(deadline: .now() + restoreDelay) { [weak self] in | |
| guard let self else { return } | |
| // If the user or another app copied something after transcription, | |
| // preserve that newer clipboard rather than restoring stale data. | |
| guard pasteboard.changeCount == transcriptionChangeCount else { return } | |
| self.restoreClipboard(pasteboard, from: backup) | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/VoiceScribe/Core/TextInserter.swift` around lines 16 - 32, Update the
pasteboard setup in the text insertion method so the transcription string is
successfully set before clearing or replacing the original clipboard. Preserve
the existing backup and restoration flow, ensuring a failed setString operation
returns without wiping the prior clipboard.
| private func startDefaultModelDownloadIfNeeded() { | ||
| let model = ModelMetadata.defaultModel | ||
| guard !hasStartedDefaultModelDownload else { return } | ||
| guard !appState.transcriptionEngine.isModelDownloaded(model) else { return } | ||
|
|
||
| hasStartedTinyDownload = true | ||
| hasStartedDefaultModelDownload = true | ||
| Task { | ||
| try? await appState.transcriptionEngine.downloadModel("tiny") | ||
| try? await appState.transcriptionEngine.downloadModel(model) | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect defaultModel definition for arch conditionals.
rg -nP --type=swift -C4 'static\s+(let|var)\s+defaultModel'Repository: jpthor/voicescribe
Length of output: 156
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the relevant files and inspect the default model definition and onboarding logic.
git ls-files | rg 'Sources/.+VoiceScribe|ModelMetadata|TranscriptionEngine|OnboardingView\.swift$'
printf '\n--- defaultModel matches ---\n'
rg -n --type=swift 'defaultModel|ModelMetadata\.defaultModel|downloadModel\(' Sources
printf '\n--- arch checks ---\n'
rg -n --type=swift 'arm64|x86_64|`#if`|`#else`|`#endif`|targetEnvironment|arch' SourcesRepository: jpthor/voicescribe
Length of output: 5176
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the relevant files in small chunks once located.
for f in Sources/VoiceScribe/Models/ModelMetadata.swift Sources/VoiceScribe/Services/TranscriptionEngine.swift Sources/VoiceScribe/Views/OnboardingView.swift; do
if [ -f "$f" ]; then
echo "### $f"
wc -l "$f"
echo
sed -n '1,260p' "$f" | cat -n
echo
fi
doneRepository: jpthor/voicescribe
Length of output: 12281
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Focused probe for architecture-dependent default model and model URL lookup behavior.
python3 - <<'PY'
from pathlib import Path
import re
paths = [Path('Sources/VoiceScribe/Models/ModelMetadata.swift'),
Path('Sources/VoiceScribe/Services/TranscriptionEngine.swift'),
Path('Sources/VoiceScribe/Views/OnboardingView.swift')]
for p in paths:
if p.exists():
text = p.read_text()
print(f'## {p}')
for pat in [
r'defaultModel[^\n]*',
r'if .*arm64.*|`#if`.*arm64|`#else`|`#endif`',
r'modelURLs\[model\][^\n]*',
r'Unknown model[^\n]*',
]:
for m in re.finditer(pat, text):
start = max(0, m.start()-120)
end = min(len(text), m.end()+200)
print(text[start:end].replace('\n','\n'))
print('---')
PYRepository: jpthor/voicescribe
Length of output: 880
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Determine whether defaultModel differs by architecture using only static source inspection.
rg -n --type=swift -C6 'static\s+(let|var)\s+defaultModel|defaultModel\s*=' Sources/VoiceScribeRepository: jpthor/voicescribe
Length of output: 156
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search for architecture-specific defaults and the model lookup implementation.
rg -n --type=swift -C8 'defaultModel|modelURLs\[model\]|Unknown model|downloadModel\(' Sources/VoiceScribeRepository: jpthor/voicescribe
Length of output: 19486
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the non-arm64 model URL table and related model selection logic.
sed -n '430,490p' Sources/VoiceScribe/Core/TranscriptionEngine.swift | cat -n
printf '\n--- ModelMetadata ---\n'
sed -n '1,120p' Sources/VoiceScribeCore/ModelMetadata.swift | cat -n
printf '\n--- tests ---\n'
sed -n '1,220p' Tests/VoiceScribeTests/ModelMetadataTests.swift | cat -nRepository: jpthor/voicescribe
Length of output: 10802
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the x86_64 model URL map and the load/change paths around the default model.
sed -n '1,260p' Sources/VoiceScribe/Core/TranscriptionEngine.swift | cat -nRepository: jpthor/voicescribe
Length of output: 11449
Skip the Parakeet download on x86_64. ModelMetadata.defaultModel is always parakeet-unified-en, but the Intel download path only supports tiny/base/small/medium, so this try? makes the onboarding prefetch a silent no-op on Intel Macs. Gate it to arm64 or use the current selected model instead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/VoiceScribe/Views/OnboardingView.swift` around lines 79 - 88, Update
startDefaultModelDownloadIfNeeded to skip the default-model prefetch on x86_64,
gating the existing download flow to arm64 before accessing or downloading
ModelMetadata.defaultModel; preserve the current guards and Task behavior for
supported arm64 systems.
| if modelFolder.hasPrefix("download:") { | ||
| let variant = String(modelFolder.dropFirst("download:".count)) | ||
| let base = FileManager.default.homeDirectoryForCurrentUser | ||
| .appendingPathComponent("Library/Caches/WhisperKit") | ||
| let downloaded = try await WhisperKit.download( | ||
| variant: variant, | ||
| downloadBase: base, | ||
| from: "argmaxinc/whisperkit-coreml" | ||
| ) | ||
| modelFolder = downloaded.path | ||
| print("downloaded_model=\(modelFolder)") | ||
| } | ||
| let audioPath = CommandLine.arguments[2] | ||
| let mode = CommandLine.arguments.count > 3 ? CommandLine.arguments[3] : "default" | ||
| let runs = CommandLine.arguments.count > 4 ? Int(CommandLine.arguments[4]) ?? 3 : 3 | ||
|
|
||
| let options: DecodingOptions? | ||
| switch mode { | ||
| case "default": | ||
| options = nil | ||
| case "english": | ||
| options = DecodingOptions(language: "en", detectLanguage: false) | ||
| case "english-no-timestamps": | ||
| options = DecodingOptions( | ||
| language: "en", | ||
| detectLanguage: false, | ||
| withoutTimestamps: true | ||
| ) | ||
| default: | ||
| throw BenchmarkError.invalidMode(mode) | ||
| } | ||
|
|
||
| let loadStart = ContinuousClock.now | ||
| let whisper = try await WhisperKit( | ||
| modelFolder: modelFolder, | ||
| computeOptions: ModelComputeOptions(), | ||
| verbose: false, | ||
| logLevel: .error, | ||
| prewarm: false, | ||
| load: true, | ||
| download: false | ||
| ) | ||
| print("load_seconds=\(seconds(since: loadStart))") | ||
|
|
||
| for run in 1...runs { | ||
| let start = ContinuousClock.now | ||
| let results = try await whisper.transcribe(audioPath: audioPath, decodeOptions: options) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the whisper-benchmark product name in Package.swift
rg -n 'whisper-benchmark|WhisperBenchmark' Package.swift
# Check WhisperKit dependency version
rg -n 'WhisperKit' Package.swift Package.resolved
# Search for the API symbols in the WhisperKit package
fd -t d 'WhisperKit' .build 2>/dev/null | head -5
rg -rn 'public static func download|public init\(|struct DecodingOptions|func transcribe' .build/checkouts/WhisperKit 2>/dev/null | head -30Repository: jpthor/voicescribe
Length of output: 622
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate any local WhisperKit source or cached checkout
fd -a 'WhisperKit' . .build ~/Library/Caches 2>/dev/null | head -100
# Inspect the resolved dependency details
jq '.object.pins[] | select(.identity=="whisperkit")' Package.resolved 2>/dev/null || true
# Search for public API names in any locally available WhisperKit sources
for d in .build/checkouts/WhisperKit .build/repositories/WhisperKit.git ~/Library/Caches/org.swift.swiftpm/repositories/WhisperKit*; do
[ -e "$d" ] || continue
echo "== $d =="
rg -n 'public (static )?func download|public init\(|struct DecodingOptions|func transcribe|audioPath:decodeOptions|withoutTimestamps|detectLanguage' "$d" | head -200
doneRepository: jpthor/voicescribe
Length of output: 156
🌐 Web query:
WhisperKit 0.18.0 public API download variant downloadBase from init modelFolder computeOptions verbose logLevel prewarm load download DecodingOptions language detectLanguage withoutTimestamps transcribe audioPath decodeOptions
💡 Result:
In WhisperKit 0.18.0, the public API centers around the WhisperKit class and its configuration objects [1][2]. Below is a synthesis of the requested parameters and functionality. ### Initialization and Model Management WhisperKit is initialized via the WhisperKit class, which accepts a WhisperKitConfig object or a convenience initializer [2]. Key parameters include: - variant/model: Specifies the model to load (e.g., large-v3-v20240930_626MB) [3][2]. - modelFolder: A String representing the local path to the model directory [1][2]. - downloadBase: A URL specifying the base directory for model downloads; defaults to a system-appropriate Hugging Face cache directory [4][5]. - computeOptions: An object of type ModelComputeOptions used to configure inference compute settings (e.g., target devices, precision) [2]. - verbose and logLevel: Control the verbosity and logging level (e.g.,.info,.debug) for the library [2]. - prewarm: A boolean to trigger early initialization of model components [2][6]. - load: A boolean indicating whether to eager-load models during initialization; if false (default), models are loaded lazily upon the first request [1][2]. - download: A boolean that, when true, enables automatic downloading of the specified model [2][7]. ### Transcription Transcription is performed by invoking methods on an initialized WhisperKit instance, typically involving: - transcribe: The main function to initiate audio processing [8]. - audioPath: The path to the audio file to be transcribed [3]. - decodeOptions: A DecodingOptions object used to control decoding behavior [8][9]. This includes: - language: The source language code [3][9]. - detectLanguage: A boolean to enable automatic language detection [9]. - withoutTimestamps: A configuration within decoding options to suppress timestamp generation in the output [3][9]. In version 0.18.0, architectural changes improved model lifecycle management by unifying download, load, and unload processes through shared managers, with a focus on lazy loading by default to optimize resource usage [1][10].
Citations:
- 1: https://github.com/argmaxinc/WhisperKit/releases/tag/v0.18.0
- 2: https://github.com/argmaxinc/WhisperKit/blob/2870b46b/Sources/WhisperKit/Core/WhisperKit.swift
- 3: https://github.com/argmaxinc/WhisperKit/blob/main/README.md
- 4: Expose downloadBase in WhisperKit init argmaxinc/argmax-oss-swift#57
- 5: What is the default download directory for the whisper models? argmaxinc/argmax-oss-swift#345
- 6: https://github.com/argmaxinc/whisperkit/blob/main/Sources/WhisperKit/Core/Configurations.swift
- 7: useBackgroundDownloadSession in WhisperKitConfig is ignored during initialization argmaxinc/argmax-oss-swift#337
- 8: https://app.argmaxinc.com/docs/examples/file-transcription
- 9: https://github.com/argmaxinc/WhisperKit/blob/2870b46b/Sources/WhisperKit/Core/TranscribeTask.swift
- 10: https://github.com/argmaxinc/WhisperKit/releases
🌐 Web query:
site:github.com/argmaxinc/WhisperKit 0.18.0 "public static func download" "variant" "downloadBase" "from" "public init(" "modelFolder" "computeOptions" "verbose" "logLevel" "prewarm" "load" "download" "DecodingOptions" "withoutTimestamps" "transcribe(audioPath:decodeOptions:)"
💡 Result:
In WhisperKit, the components you referenced relate to model management and the primary transcription API [1][2][3]. The public static method for downloading models is defined within the WhisperKit core as follows: public static func download(variant: String, downloadBase: URL?, useBackgroundSession: Bool, from repo: String, token: String?) async throws -> String [2][4] This method allows you to specify the model variant and a custom downloadBase path, overriding the default behavior that typically saves models to the user's Documents or application support folders [5][6][4]. The initializer for the WhisperKit object is used to configure and load the model environment [7][8][9]. A typical structure for this initializer includes parameters for configuration and state: public init(modelFolder: String? = nil, computeOptions: ComputeOptions? = nil, verbose: Bool = false, logLevel: Logging.LogLevel =.info, prewarm: Bool = true, load: Bool = true) async throws [10][9] The transcribe function enables audio processing and is available in multiple overloads to support different input types [3]: public func transcribe(audioPath: String, decodeOptions: DecodingOptions? = nil) async throws -> TranscriptionResult? [11][3] In this context, DecodingOptions is used to configure parameters such as withoutTimestamps (a boolean property) [3]. Note that the library has evolved significantly; while earlier versions relied on direct initializer parameters, newer versions often utilize a WhisperKitConfig struct for cleaner management of these settings [7]. [1][7]
Citations:
- 1: https://github.com/argmaxinc/WhisperKit/releases/tag/v0.18.0
- 2: https://github.com/argmaxinc/WhisperKit/blob/2870b46b/Sources/WhisperKit/Core/WhisperKit.swift
- 3: v0.6.0 argmaxinc/argmax-oss-swift#119
- 4: useBackgroundDownloadSession in WhisperKitConfig is ignored during initialization argmaxinc/argmax-oss-swift#337
- 5: Allow setting a downloadBase so downloaded models are not forced into the user's Documents folder argmaxinc/argmax-oss-swift#34
- 6: Expose downloadBase in WhisperKit init argmaxinc/argmax-oss-swift#57
- 7: v0.9.0 argmaxinc/argmax-oss-swift#219
- 8: Using locally saved models argmaxinc/argmax-oss-swift#46
- 9: Unable to load models argmaxinc/argmax-oss-swift#7
- 10: https://github.com/argmaxinc/whisperkit/blob/main/Sources/WhisperKit/Core/Configurations.swift
- 11: https://github.com/argmaxinc/WhisperKit/blob/main/README.md
Remove the obsolete download: initializer argument WhisperKit(modelFolder:computeOptions:verbose:logLevel:prewarm:load:) is the 0.18.0 public initializer; download: isn’t part of that signature, so this call won’t match the pinned API.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/WhisperBenchmark/main.swift` around lines 14 - 60, Remove the
obsolete download: argument from the WhisperKit initializer call in the
benchmark setup, keeping the existing modelFolder, computeOptions, verbose,
logLevel, prewarm, and load arguments unchanged so it matches the 0.18.0 public
initializer.
1 participant
Summary
Performance
Measured locally on an M4 Max using the included 19-second Australian-English fixture:
The optimized model was approximately 3.5x faster and produced the same transcript for this fixture. No model was fine-tuned locally; this uses Argmax existing Core ML conversion from whisperkit-coreml.
Verification
Compatibility
This raises the minimum deployment target from macOS 13 to macOS 14 because the selected FluidAudio release requires Sonoma.
Summary by CodeRabbit
New Features
Documentation
Compatibility