Files

cottongin 8ad2f1c800 chore: stage all pending work — caption styling, media server, processing modal, docs, summaries

Includes:
- Extended caption styling (font, shadow, dimmed color, bg toggle)
- Media server, subtitle downloader, VTT parser, processing modal
- Waveform tiers, thumbnail/timeline improvements, transport controls
- Hybrid download model, dependency management, clip export enhancements
- 21 chat summaries, 2 implementation plans, 2 design specs

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-09-22 10:48:16 -04:00

3.6 KiB

Raw Permalink Blame History

Fix Caption Export and Add Burn-In Option

Task

Exported clips were not including captions despite the "include captions" option being checked. Additionally, the user requested a "burn-in" option to render subtitles directly into the video frames.

Root Causes

Precise mode burn-in (timestamp offset): -ss was placed before -i (input seeking), shifting output PTS to 0. The subtitles filter reads the original VTT with absolute timestamps (e.g., cues at 300s), but the output video starts at 0s — no cues matched.
Lossless mode muxing (VTT formatting): YouTube auto-generated VTT has karaoke-style <c> tags, inline timestamps (<00:00:00.599>), and positioning metadata (align:start position:0%) that confuse ffmpeg's VTT parser when converting to mov_text.
Silent fallback: When the ffmpeg subtitle export failed, the code silently fell back to exporting without subtitles, making the failure invisible to the user.

Changes Made

`src-tauri/src/services/clip_exporter.rs` (major rewrite)

Added sanitize_vtt_for_ffmpeg(): Strips karaoke <c> tags, inline timestamps, and positioning metadata from VTT files. Writes a cleaned temp file.
Added strip_vtt_tags(): Helper to remove all HTML-like tags from VTT cue text lines.
Replaced build_ffmpeg_args_with_subs() with two separate functions:
- build_ffmpeg_args_mux_subs(): Muxes subtitles as a track. Uses output seeking (-ss/-to after -i) for correct timestamp alignment. Works with both lossless and precise cut modes.
- build_ffmpeg_args_burnin_subs(): Burns subtitles into video using the subtitles filter. Uses output seeking so the filter reads correct VTT timestamps. Always re-encodes (overrides to H.264/AAC if lossless mode selected).
Updated export_single_clip_with_subs(): Now takes burn_in: bool, sanitizes VTT before use, dispatches to mux or burn-in builder. Removed the silent fallback — errors now surface to the user.
Updated export_merged_with_subs(): Takes burn_in: bool, passes through to per-segment export.
Updated tests: Replaced old tests for removed function, added tests for both new builders, added a VTT tag stripping test. All 40 tests pass.

`src-tauri/src/models.rs`

Added pub burn_in_captions: bool to ExportConfig.

`src-tauri/src/commands/export.rs`

Threads config.burn_in_captions through to export_single_clip_with_subs and export_merged_with_subs.

`src/lib/bindings/export.ts`

Added burnInCaptions: boolean to the TypeScript ExportConfig interface.

`src/lib/components/ExportDialog.svelte`

Added burnInCaptions state.
Added "Burn into video" sub-checkbox under the "Include captions" checkbox.
Contextual notes: "Captions will be muxed as a subtitle track" vs "Captions will be burned into the video" vs "Burn-in requires re-encoding (precise mode will be used)".
Threads burnInCaptions through to the export config.

Lessons Learned

ffmpeg's -ss before -i (input seeking) adjusts output PTS to start at 0, but the subtitles filter reads timestamps from the original VTT file — they must match. Output seeking (-ss after -i) preserves original timestamps.
YouTube auto-generated VTT files contain karaoke formatting that ffmpeg's VTT-to-mov_text converter can't handle cleanly. Sanitizing before use is essential.
Silent fallbacks that hide errors waste debugging time. Surface errors to the user.

Follow-up

The CORS fix from the prior session (media_server.rs adding CorsLayer::permissive()) is also needed for captions to display in the preview player.

3.6 KiB Raw Permalink Blame History