gui-video-clipper/chat-summaries/2026-09-21_19-31-progress-resize-captions-audio-upgrade-summary.md

Progress, Resize, Captions, Audio, Preview Upgrade

Date: 2026-09-21 19:31 Task: Implement 6 items from the "Progress Resize Captions Audio" plan

Changes Made

1. Waveform Speed Fix + Granular Progress (waveform-speed-fix)

Files: src-tauri/src/services/waveform_generator.rs, src-tauri/src/commands/media_analysis.rs, src/lib/bindings/mediaAnalysis.ts, src/lib/stores/videoSession.svelte.ts, src/lib/components/ProcessingModal.svelte

• Critical bug fix: aresample=N was setting the output sample rate to N Hz (e.g., 200,000 Hz) instead of producing N total samples. For a 54-min video, this generated ~650M samples. Fixed by computing actual rate as raw_count / duration and using -ar flag instead. ~3000x speedup for long videos.
• Changed waveform_generator.rs to use child process spawning with piped stdout/stderr, parse out_time_us= from ffmpeg progress output, and report progress via a callback.
• Added Channel<f64> progress parameter to extract_waveform_tiers Tauri command.
• Added processingProgress to session store; reset in setMetadata, clearMediaFields.
• Updated ProcessingModal to show progress bars for every step (not just download).

2. Fix Resize Handles (fix-resize)

Files: src/lib/components/VideoPlayer.svelte, src/App.svelte

• Changed VideoPlayer min-height from 200px to 0.
• Removed max-height: 60vh from timeline-clip-area.
• Lowered min-height constraints: timeline-clip-area 140→80, timeline-pane 80→40, cliplist-pane 40→20.
• Updated resize handler min/max constraints to match.

3. Custom Caption Rendering (fix-captions)

Files: src/lib/utils/vttParser.ts (new), src/lib/stores/preferences.svelte.ts, src/lib/components/VideoPlayer.svelte

• Created vttParser.ts with parseVtt() and getActiveCues() for custom VTT parsing.
• Replaced <track> element with custom HTML overlay positioned absolutely in the video player.
• Captions now rendered bottom-center (or top, configurable) with single background layer (no double-background).
• Added CaptionSettings interface to preferences store with font size, text color, background opacity, text outline, and position.

4. Caption Settings Panel (caption-settings-ui)

Files: src/lib/components/CaptionSettingsPanel.svelte (new), src/lib/components/VideoPlayer.svelte

• Created settings sub-menu with range sliders, color picker, checkbox, radio buttons.
• Accessible via ⚙ button next to CC toggle.
• Settings persist via preferences store.
• "Reset to defaults" button included.

5. Local HTTP Media Server for Audio (fix-audio)

Files: src-tauri/src/services/media_server.rs (new), src-tauri/src/services/mod.rs, src-tauri/src/lib.rs, src-tauri/Cargo.toml, src/lib/bindings/video.ts, src/lib/components/VideoPlayer.svelte

• Added axum + tower-http (with fs feature) dependencies.
• Created media server that starts on a random available port at app launch, serving files from filesystem root with full range-request support via ServeDir.
• Exposed port to frontend via get_media_server_port Tauri command.
• VideoPlayer now constructs video URLs as http://127.0.0.1:{port}/path/to/file.mp4 instead of using convertFileSrc().
• This fixes WKWebView's audio issues with the asset protocol.

6. Preview Upgrade Toast (preview-upgrade)

Files: src/lib/stores/videoSession.svelte.ts, src/lib/components/VideoPlayer.svelte, src/lib/components/StatusBar.svelte

• Added activeVideoPath (initially set to preview, upgradeable to export), showUpgradeToast, upgradePreview(), dismissUpgradeToast().
• Toast appears in top-right of video player when export download completes and processing is done.
• "Reload Preview (HQ)" button added to StatusBar.
• Video source now driven by activeVideoPath instead of previewFilePath.

Verification

• cargo build: ✅ (only pre-existing warnings)
• cargo test: ✅ 37 tests pass
• npm run check: ✅ (only pre-existing vite.config.ts errors)

Lessons Learned

• aresample=N in ffmpeg sets the output sample rate in Hz, not a total count. Must compute rate = count/duration.
• Tauri's asset protocol (https://asset.localhost/...) has issues with audio track streaming in WKWebView — a local HTTP server with range request support via axum/tower-http is a clean fix.
• TypeScript's control flow analysis can over-narrow $state values across async callback boundaries, requiring explicit cast (as ProcessingStep) to work around.
• Custom caption rendering (parsing VTT + HTML overlay) gives full styling control vs. browser ::cue pseudo-element limitations.

Follow-up Items

• Smoke test the full flow end-to-end with a real YouTube video.
• Verify audio playback works correctly via the local HTTP server.
• Test caption settings persistence across app restarts.
• Consider adding progress reporting to keyframe extraction and thumbnail generation (currently only waveform has it).