48 lines
4.3 KiB
Markdown
48 lines
4.3 KiB
Markdown
|
# Pre-Process Waveform Tiers + Processing Modal
|
|||
|
|
|
|||
|
|
**Date:** 2026-09-21 15:10
|
|||
|
|
**Task:** Replace on-demand waveform range extraction with one-time multi-tier pre-processing pipeline, add a processing modal, eliminate the reactive `$effect` loop bug.
|
|||
|
|
|
|||
|
|
## Problem
|
|||
|
|
|
|||
|
|
The `$effect` in `Timeline.svelte` that fetched waveform ranges on demand created an infinite reactive loop: writing to `session.waveformDetailPeaks` re-triggered the same `$effect`, which re-evaluated `needsDetail`, which fired again. The terminal showed `waveform-range` calls repeating 40+ times for the same range, killing performance.
|
|||
|
|
|
|||
|
|
## Changes Made
|
|||
|
|
|
|||
|
|
### 1. Rust Backend — Waveform Tiers (`waveform-tiers-rust`)
|
|||
|
|
|
|||
|
|
- **`src-tauri/src/models.rs`**: Added `WaveformTiers` struct with `tier0` (~2K peaks), `tier1` (~10K peaks), `tier2` (all raw peaks, 50K–200K depending on duration).
|
|||
|
|
- **`src-tauri/src/services/waveform_generator.rs`**: Rewrote entirely. New `extract_waveform_tiers(file_path, duration)` function runs ffmpeg once at a sample rate yielding ~100 peaks/sec (capped 50K–200K), then downsamples into 3 tiers in Rust. Added `downsample()` helper (max-of-chunk). Removed `extract_waveform_range` function. Added 6 unit tests for downsample and compute_peaks.
|
|||
|
|
- **`src-tauri/src/commands/media_analysis.rs`**: Replaced `extract_waveform` and `extract_waveform_range` commands with single `extract_waveform_tiers` command.
|
|||
|
|
- **`src-tauri/src/lib.rs`**: Updated handler registration — `extract_waveform_tiers` replaces `extract_waveform` + `extract_waveform_range`.
|
|||
|
|
|
|||
|
|
### 2. Frontend — Tier-Based Rendering (`waveform-tiers-frontend`)
|
|||
|
|
|
|||
|
|
- **`src/lib/bindings/mediaAnalysis.ts`**: Added `WaveformTiers` interface and `extractWaveformTiers` binding. Removed `extractWaveformRange` binding.
|
|||
|
|
- **`src/lib/timeline/waveformRenderer.ts`**: Rewrote. `WaveformData` is now `{ tiers: WaveformTiers | null }`. `pickTier()` selects the lowest-resolution tier with ≥1 peak per pixel for the visible range — pure arithmetic, zero IPC.
|
|||
|
|
- **`src/lib/timeline/renderer.ts`**: Updated `drawTimeline` signature and waveform condition to use `waveform.tiers`.
|
|||
|
|
- **`src/lib/stores/videoSession.svelte.ts`**: Replaced `waveformPeaks`, `waveformDetailPeaks`, `waveformDetailStart`, `waveformDetailEnd` with `waveformTiers: WaveformTiers | null`. Added `processingStep` state. Processing pipeline now calls `extractWaveformTiers` instead of `extractWaveform`.
|
|||
|
|
- **`src/lib/components/Timeline.svelte`**: **Deleted the entire `$effect` for waveform range fetching** — the infinite loop bug is eliminated. Removed `detailFetchTimer`, `detailFetchInFlight`, `needsDetail`, `extractWaveformRange` import. `waveformData` is now simply `{ tiers: session.waveformTiers }`.
|
|||
|
|
|
|||
|
|
### 3. Processing Modal (`processing-modal`)
|
|||
|
|
|
|||
|
|
- **`src/lib/components/ProcessingModal.svelte`** (new): Modal overlay showing processing steps with icons (✓ done, ⏳ current, ○ pending). Steps: downloading → waveform → keyframes → thumbnails → captions → done. Includes download progress bar.
|
|||
|
|
- **`src/App.svelte`**: Imported and renders `<ProcessingModal />` when `session.processingStep` is neither `'idle'` nor `'done'`.
|
|||
|
|
- **`src/lib/stores/videoSession.svelte.ts`**: `processingStep` is set at each stage of `triggerPostDownloadProcessing` and `beginDownload`.
|
|||
|
|
|
|||
|
|
## Verification
|
|||
|
|
|
|||
|
|
- **Rust**: `cargo build` succeeds (3 pre-existing warnings). `cargo test` passes all 37 tests (including 6 new waveform generator tests).
|
|||
|
|
- **Frontend**: `svelte-check` reports only pre-existing vite.config.ts errors and a11y warnings — no new issues.
|
|||
|
|
|
|||
|
|
## Lessons Learned
|
|||
|
|
|
|||
|
|
- The on-demand `$effect` → async IPC → write reactive state → re-trigger `$effect` pattern is fundamentally broken in Svelte 5. The correct approach is pre-computation: do all CPU work up front, store the results, and let the renderer pick the right data with pure arithmetic.
|
|||
|
|
- For waveform at typical zoom levels (even 100x on an 800px-wide canvas), 50K peaks gives ~6 peaks/pixel, which is plenty. No need for on-demand range extraction.
|
|||
|
|
- Downsampling by max-of-chunk preserves peak amplitude fidelity while drastically reducing array size.
|
|||
|
|
|
|||
|
|
## Follow-up Items
|
|||
|
|
|
|||
|
|
- Audio playback may need verification after the session store changes (the `previewFilePath` → `convertFileSrc` pipeline is unchanged, but should be tested).
|
|||
|
|
- Consider persisting waveform tiers to disk cache for instant reload on re-open.
|