feat: add keystroke captions and overlays for recorded videos

[pkazmier]

Introduces a caption system that burns keystroke overlays into video output using ASS subtitles via ffmpeg. Captions are enabled / disabled via CaptionOn and CaptionOff commands. Captions are configured via Set commands (CaptionFont, CaptionFontSize, CaptionKeyStyle, CaptionAlignment, etc.) and support "vim" and "icon" key rendering styles, customizable colors/positioning, and controls for max keys shown as well as an inactivity timeout.

Caption Settings

CaptionFont

Set CaptionFont "JetBrains Mono"
CaptionOn

Set CaptionFont "IBM Plex Mono"
CaptionOn

CaptionFontSize

Set CaptionFontSize 18
CaptionOn

Set CaptionFontSize 26
CaptionOn

CaptionMaxKeys

Set CaptionMaxKeys 5
CaptionOn

Set CaptionMaxKeys 15
CaptionOn

CaptionInactivityTimer

Set CaptionInactivityTimer 900ms
CaptionOn

Set CaptionInactivityTimer 1200ms
CaptionOn

CaptionFontColor

Set CaptionFontColor "#FF893E"
CaptionOn

CaptionHighlightColor

Set CaptionHighlightColor "#77FFEF"
CaptionOn

CaptionKeyStyle

Set CaptionKeyStyle "icon"
CaptionOn

Set CaptionKeyStyle "vim"
CaptionOn

CaptionBoxColor

Set CaptionBoxColor "#777777"
CaptionOn

CaptionBoxOpacity

Set CaptionBoxOpacity 0
CaptionOn

Set CaptionBoxOpacity 0.25
CaptionOn

Set CaptionBoxOpacity 0.75
CaptionOn

CaptionBoxPadding

Set CaptionBoxPadding 3
CaptionOn

Set CaptionBoxPadding 15
CaptionOn

CaptionAlignment

Set CaptionAlignment "bottom-right"
CaptionOn

I have demos for all of the positions as well: top-left, top-center, top-right, middle-left, middle-center, middle-right, bottom-left, bottom-center, and bottom-right.

CaptionMargin

Set CaptionAlignment "bottom-right"
Set CaptionPadding 10
Set CaptionMarginVertical 10
Set CaptionMarginRight 10
CaptionOn

[pkazmier]

I’m more than happy to update the README as well, but before doing so, I wanted to wait to see if there was interest. It took almost 4 hours to just write the demos and PR text itself.

[pkazmier]

I've also incorporated an Overlay command (similar to #716), but the implementation uses the same ASS subtitle features as the keystroke captioning, so it's a consistent look and feel to the captioning of keypresses. Likewise, the configuration of such has many of the same configuration options:

// OverlayOptions holds configuration for text overlays.
type OverlayOptions struct {
	Font           string
	FontSize       int
	Alignment      CaptionAlignment
	FontColor      string // #RRGGBB hex
	BoxColor       string // #RRGGBB hex
	BoxOpacity     float64
	BoxPadding     int
	MarginLeft     int
	MarginRight    int
	MarginVertical int
}

Demo

Overlay

Set OverlayAlignment "top-right"
CaptionOn

# Set up line editing to demo control keys
Hide
Type "bindkey -e"
Enter
Ctrl+l
Show

Type "echo Hello, Planet"
Sleep 0.5s
Enter
Sleep 2s
Overlay@10s "Demo of Command Line Editing"
Sleep 1s
Ctrl+p
Sleep 1.5s
Ctrl+w
Sleep 1.1s
Type "World"
Enter
Sleep 2s

Overlay with Multiline

Overlay@10s "Let's now edit\Nthe prior command\Nto show the world!"

Overlay Title Tucked into Corner

Set OverlayAlignment "top-right"
Set OverlayBoxColor "#ffffff"
Set OverlayBoxOpacity 1
Set OverlayFontColor "#000000"
Set OverlayMarginRight 10
Set OverlayMarginVertical 10
CaptionOn
...

Overlay without Captions

Of course you can use Overlays without captions:

Set OverlayAlignment "top-right"
Set OverlayBoxColor "#ffffff"
Set OverlayBoxOpacity 1
Set OverlayFontColor "#000000"
Set OverlayAlignment "bottom-right"
Set OverlayMarginRight 10
Set OverlayMarginVertical 10

[domenkozar]

Can Captions be turn off and on between other commands?

[pkazmier]

Can Captions be turn off and on between other commands?

Currently, they are disabled in Hide blocks and re-enabled in Show blocks. But I could create a CaptionShow and CaptionHide command that does the same if you think it would be useful.

[pkazmier]

Can Captions be turn off and on between other commands?

I've just updated my PR, which adds CaptionOn and CaptionOff commands. This allows users fine grained control over when to use and not use captions. As before, captioning is off by default and is is disabled in Hide / Show blocks. I've removed the old Set Caption true global setting in favor of the new commands.

Here is a demo showing selective captioning as well as an overlay title:

Set OverlayAlignment "top-right"
Set OverlayBoxColor "#ffffff"
Set OverlayBoxOpacity 1
Set OverlayFontColor "#000000"
Set OverlayMarginRight 10
Set OverlayMarginVertical 10

Overlay@100s "Selective Captioning"

CaptionOn
Type "echo captions are enabled"
Enter
Sleep 2s

CaptionOff
Type "echo captions are disabled"
Enter
Sleep 2s

CaptionOn
Type "echo captions are re-enabled"
Sleep 2s

[pkazmier]

Comments for the maintainers:

• Captions are enabled via CaptionOn. By default, they are disabled.
• Captions are disabled via CaptionOff.
• Captions are disabled during Hide / Show blocks.
• Overlay titles are added via Overlay[@5s] "place your text here". Without time specification, default is 3 seconds.
• The implementation of captions and overlay titles uses ASS v4+ subtitle specification, which requires ffmpeg to be compiled with libass library only if the tape file uses captions or overlays.
• I did not update the README in the PR (yet). I wanted to get feedback before doing so. Would you like me to include all of the demos I showcased in this PR?
• I used Claud Opus 4.6 to assist in the development of this feature.

Thanks for your consideration.

[pkazmier]

If this gets merged, I have another one almost ready to go that adds audio capabilities. Teaser:

Set CaptionAudio "thock"
Set CaptionAudioVolume 0.15
Set OverlayFontColor "#c6a0f6"
Set OverlayBoxOpacity 0

CaptionOn

Overlay@4s "Keypress Audio Demo"
Type "You should hear audio as this is captioned."
Sleep 2s
Ctrl+u

Overlay@4s "Audio Overlay Demo"
Set AudioVolume 1.0
Audio examples/settings/audio-overlay-demo.m4a
Type "You can play arbitrary audio files in the video."
Sleep 2s

set-caption-with-audio-overlay-title.mp4