cottongin
8ad2f1c800
Includes: - Extended caption styling (font, shadow, dimmed color, bg toggle) - Media server, subtitle downloader, VTT parser, processing modal - Waveform tiers, thumbnail/timeline improvements, transport controls - Hybrid download model, dependency management, clip export enhancements - 21 chat summaries, 2 implementation plans, 2 design specs Co-authored-by: Cursor <cursoragent@cursor.com>
83 lines
4.9 KiB
Markdown
83 lines
4.9 KiB
Markdown
# Progress, Resize, Captions, Audio, Preview Upgrade
|
|
|
|
**Date:** 2026-09-21 19:31
|
|
**Task:** Implement 6 items from the "Progress Resize Captions Audio" plan
|
|
|
|
## Changes Made
|
|
|
|
### 1. Waveform Speed Fix + Granular Progress (waveform-speed-fix)
|
|
|
|
**Files:** `src-tauri/src/services/waveform_generator.rs`, `src-tauri/src/commands/media_analysis.rs`, `src/lib/bindings/mediaAnalysis.ts`, `src/lib/stores/videoSession.svelte.ts`, `src/lib/components/ProcessingModal.svelte`
|
|
|
|
- **Critical bug fix:** `aresample=N` was setting the output sample rate to N Hz (e.g., 200,000 Hz) instead of producing N total samples. For a 54-min video, this generated ~650M samples. Fixed by computing actual rate as `raw_count / duration` and using `-ar` flag instead. ~3000x speedup for long videos.
|
|
- Changed `waveform_generator.rs` to use child process spawning with piped stdout/stderr, parse `out_time_us=` from ffmpeg progress output, and report progress via a callback.
|
|
- Added `Channel<f64>` progress parameter to `extract_waveform_tiers` Tauri command.
|
|
- Added `processingProgress` to session store; reset in `setMetadata`, `clearMediaFields`.
|
|
- Updated `ProcessingModal` to show progress bars for every step (not just download).
|
|
|
|
### 2. Fix Resize Handles (fix-resize)
|
|
|
|
**Files:** `src/lib/components/VideoPlayer.svelte`, `src/App.svelte`
|
|
|
|
- Changed `VideoPlayer` min-height from 200px to 0.
|
|
- Removed `max-height: 60vh` from timeline-clip-area.
|
|
- Lowered min-height constraints: timeline-clip-area 140→80, timeline-pane 80→40, cliplist-pane 40→20.
|
|
- Updated resize handler min/max constraints to match.
|
|
|
|
### 3. Custom Caption Rendering (fix-captions)
|
|
|
|
**Files:** `src/lib/utils/vttParser.ts` (new), `src/lib/stores/preferences.svelte.ts`, `src/lib/components/VideoPlayer.svelte`
|
|
|
|
- Created `vttParser.ts` with `parseVtt()` and `getActiveCues()` for custom VTT parsing.
|
|
- Replaced `<track>` element with custom HTML overlay positioned absolutely in the video player.
|
|
- Captions now rendered bottom-center (or top, configurable) with single background layer (no double-background).
|
|
- Added `CaptionSettings` interface to preferences store with font size, text color, background opacity, text outline, and position.
|
|
|
|
### 4. Caption Settings Panel (caption-settings-ui)
|
|
|
|
**Files:** `src/lib/components/CaptionSettingsPanel.svelte` (new), `src/lib/components/VideoPlayer.svelte`
|
|
|
|
- Created settings sub-menu with range sliders, color picker, checkbox, radio buttons.
|
|
- Accessible via ⚙ button next to CC toggle.
|
|
- Settings persist via preferences store.
|
|
- "Reset to defaults" button included.
|
|
|
|
### 5. Local HTTP Media Server for Audio (fix-audio)
|
|
|
|
**Files:** `src-tauri/src/services/media_server.rs` (new), `src-tauri/src/services/mod.rs`, `src-tauri/src/lib.rs`, `src-tauri/Cargo.toml`, `src/lib/bindings/video.ts`, `src/lib/components/VideoPlayer.svelte`
|
|
|
|
- Added `axum` + `tower-http` (with fs feature) dependencies.
|
|
- Created media server that starts on a random available port at app launch, serving files from filesystem root with full range-request support via `ServeDir`.
|
|
- Exposed port to frontend via `get_media_server_port` Tauri command.
|
|
- VideoPlayer now constructs video URLs as `http://127.0.0.1:{port}/path/to/file.mp4` instead of using `convertFileSrc()`.
|
|
- This fixes WKWebView's audio issues with the asset protocol.
|
|
|
|
### 6. Preview Upgrade Toast (preview-upgrade)
|
|
|
|
**Files:** `src/lib/stores/videoSession.svelte.ts`, `src/lib/components/VideoPlayer.svelte`, `src/lib/components/StatusBar.svelte`
|
|
|
|
- Added `activeVideoPath` (initially set to preview, upgradeable to export), `showUpgradeToast`, `upgradePreview()`, `dismissUpgradeToast()`.
|
|
- Toast appears in top-right of video player when export download completes and processing is done.
|
|
- "Reload Preview (HQ)" button added to StatusBar.
|
|
- Video source now driven by `activeVideoPath` instead of `previewFilePath`.
|
|
|
|
## Verification
|
|
|
|
- `cargo build`: ✅ (only pre-existing warnings)
|
|
- `cargo test`: ✅ 37 tests pass
|
|
- `npm run check`: ✅ (only pre-existing vite.config.ts errors)
|
|
|
|
## Lessons Learned
|
|
|
|
- `aresample=N` in ffmpeg sets the **output sample rate in Hz**, not a total count. Must compute rate = count/duration.
|
|
- Tauri's asset protocol (`https://asset.localhost/...`) has issues with audio track streaming in WKWebView — a local HTTP server with range request support via axum/tower-http is a clean fix.
|
|
- TypeScript's control flow analysis can over-narrow `$state` values across async callback boundaries, requiring explicit cast (`as ProcessingStep`) to work around.
|
|
- Custom caption rendering (parsing VTT + HTML overlay) gives full styling control vs. browser `::cue` pseudo-element limitations.
|
|
|
|
## Follow-up Items
|
|
|
|
- Smoke test the full flow end-to-end with a real YouTube video.
|
|
- Verify audio playback works correctly via the local HTTP server.
|
|
- Test caption settings persistence across app restarts.
|
|
- Consider adding progress reporting to keyframe extraction and thumbnail generation (currently only waveform has it).
|