gui-video-clipper/docs/superpowers/specs/2026-09-21-video-clipper-design.md

# Video Clipper — Design Spec

**Date:** 2026-09-21
**Status:** Draft
**Stack:** Tauri v2 + Svelte (frontend) + Rust (backend) + ffmpeg + yt-dlp

## Overview

A macOS GUI app for creating clips from online videos (primarily YouTube) without requiring the user to manually download the video first. The app provides a timeline-based interface for scrubbing through video, marking clip boundaries, and exporting clips via ffmpeg.

Think "LosslessCut, but paste a URL instead of opening a file."

## Core User Flow

1. User pastes a YouTube (or other supported) URL.
2. App resolves the URL via yt-dlp → retrieves metadata and a direct stream URL.
3. Video begins playing immediately via the stream URL (HTML5 `<video>` element).
4. A background download of the full video starts simultaneously.
5. User scrubs the timeline, marks clip in/out points.
6. User exports clips → ffmpeg operates on the local (downloaded) file.

## Architecture

Three layers:

### 1. Tauri/Rust Backend

Manages all subprocess orchestration, file I/O, and state persistence. Communicates with the frontend via Tauri commands (request/response) and Tauri events (push notifications like download progress).

**Components:**

| Component | Responsibility |
|---|---|
| `DependencyManager` | Check the full dependency chain on `$PATH`: ffmpeg/ffprobe, yt-dlp, a JavaScript runtime (Deno preferred, Node.js fallback), and the `bgutil-ytdlp-pot-provider` yt-dlp plugin. On first launch or when anything is missing, shows a step-by-step **Setup Wizard** that walks the user through installing each dependency one at a time. See the Dependency Chain section. |
| `VideoResolver` | Takes a URL, calls `yt-dlp --dump-json <url>` for metadata and `yt-dlp -g <url>` for the direct stream URL. Returns a `VideoMetadata` struct. |
| `DownloadManager` | Runs `yt-dlp -o <temp_path> <url>` in the background. Emits progress events to the frontend. Manages temp file lifecycle (cleanup on cancel/error). |
| `KeyframeIndex` | After download completes, runs `ffprobe -select_streams v -show_entries frame=pts_time,flags -of csv <file>` to extract keyframe positions. Returns a sorted list of keyframe timestamps. |
| `ClipExporter` | Runs ffmpeg for clip extraction. Supports two modes: lossless (stream copy, keyframe-aligned) and precise (re-encode, exact frames). Handles both individual and merged export. |
| `WaveformGenerator` | After download completes, extracts audio peak data from the video file via ffmpeg. Computes ~8000 peak samples for the full video duration and sends them to the frontend for Canvas rendering. |
| `ThumbnailExtractor` | After download completes, extracts frames at computed intervals via ffmpeg (1/sec for <10 min, 1/2s for 10–60 min, 1/5s for >60 min). Outputs small JPEGs (160×90px) packed into sprite sheets for efficient rendering. |
| `PreferencesStore` | Persists user preferences (output directory, window geometry, cookie source) via `tauri-plugin-store`. |

### 2. Svelte Frontend

Renders the video player, custom timeline, clip management panel, and export dialog.

**Components:**

| Component | Responsibility |
|---|---|
| `UrlInput` | Text field for URL entry. Detects paste, shows loading spinner during resolution, displays errors. |
| `VideoPlayer` | Wraps the HTML5 `<video>` element. Exposes play/pause/seek API. Stays synced with the Timeline component via a shared Svelte store. |
| `Timeline` | Canvas-based timeline with four visual layers (top to bottom): thumbnail strip, waveform, clip markers, controls. Renders: playhead (current position), clip regions (colored rectangles with draggable start/end handles), keyframe markers (after index is built), thumbnail frames (after extraction), audio waveform (after extraction). Supports click-to-seek, drag-to-scrub, zoom via scroll wheel or slider. |
| `TransportControls` | Play/pause, frame step (±1 frame), keyframe jump, seek ±1s/±5s buttons. Wired to keyboard shortcuts. |
| `ClipList` | Scrollable panel listing all clips. Each row shows: color swatch, label (editable), start time (editable), end time (editable), delete button. Clicking a row selects the clip and jumps the playhead to its start. |
| `ExportDialog` | Modal dialog for export configuration. See Export Flow section. |
| `StatusBar` | Bottom bar showing: download progress (percentage + ETA), active export progress, tool availability status. |
| `SetupWizard` | Step-by-step modal shown on first launch or when dependencies are missing. Walks the user through installing each dependency one at a time with green checkmarks for what's already present. See the Dependency Chain section. |

### 3. External CLI Tools

- **yt-dlp:** URL resolution, metadata extraction, stream URL retrieval, video downloading. All yt-dlp invocations include the user's configured cookie source (see Cookie/Auth section).
- **ffmpeg:** Clip extraction (both lossless and re-encode modes), clip merging, waveform peak extraction, thumbnail frame extraction.
- **ffprobe:** Keyframe position extraction (part of the ffmpeg package).
- **deno** (or **node**): JavaScript runtime required by the PO Token provider plugin for YouTube playback.
- **bgutil-ytdlp-pot-provider:** yt-dlp plugin that automatically generates per-video Proof-of-Origin tokens required by YouTube.

## Data Model

```typescript
interface Clip {
  id: string;
  startTime: number;  // seconds (float, e.g. 83.450)
  endTime: number;    // seconds (float)
  label: string;      // auto-generated "Clip 1", "Clip 2", editable by user
  color: string;      // auto-assigned from a palette for visual distinction
}

interface VideoSession {
  url: string;
  title: string;
  duration: number;          // seconds
  fps: number;               // frames per second, from yt-dlp metadata
  thumbnailUrl: string | null;
  streamUrl: string;          // direct stream URL for preview playback
  localFilePath: string | null; // set once background download completes
  downloadProgress: number;   // 0.0 - 1.0
  keyframePositions: number[]; // populated after download + ffprobe
  waveformPeaks: number[];    // normalized audio peaks (0.0–1.0), ~8000 samples, populated after download + ffmpeg
  thumbnailSpritesheets: ThumbnailSpritesheet[]; // populated after download + ffmpeg
  clips: Clip[];
}

interface ThumbnailSpritesheet {
  filePath: string;            // path to the spritesheet image
  startIndex: number;          // index of first thumbnail in this sheet
  count: number;               // number of thumbnails in this sheet
  thumbWidth: number;          // pixel width of each thumbnail (e.g. 160)
  thumbHeight: number;         // pixel height of each thumbnail (e.g. 90)
  columns: number;             // thumbnails per row in the spritesheet
  intervalSeconds: number;     // time interval between thumbnails
}

type CookieSource =
  | { type: "browser"; browser: "firefox" }
  | { type: "file"; path: string }
  | { type: "none" };

interface UserPreferences {
  outputDirectory: string;    // default: home directory (~/)
  windowBounds: { x: number; y: number; width: number; height: number } | null;
  cookieSource: CookieSource; // default: { type: "browser", browser: "firefox" }
}
```

## Timeline & Interaction Design

### Layout

```
┌─────────────────────────────────────────────────────────┐
│  [Paste URL...]                              [⚙ Prefs]  │
├─────────────────────────────────────────────────────────┤
│                                                         │
│                    VIDEO PLAYER                         │
│                  (16:9 aspect ratio)                    │
│                                                         │
├─────────────────────────────────────────────────────────┤
│  [◄K] [◄|] [▶/❚❚] [|►] [K►]  00:03:42 / 00:12:35     │
│                                          [I] [O]        │
├─────────────────────────────────────────────────────────┤
│  ▼ playhead                                             │
│  ┃                                                      │
│  ┃ [thumb] [thumb] [thumb] [thumb] [thumb] [thumb]      │ ← Thumbnail strip
│  ┃ ∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿ │ ← Waveform
│  ┃     ▓▓▓▓▓▓▓clip 1▓▓▓▓▓▓▓▓▓     ▓▓▓clip 2▓▓▓       │ ← Clip markers
│  ┃                                                      │
│  [🔍−] ═══════════●══════════ [🔍+]     zoom slider    │ ← Controls
├─────────────────────────────────────────────────────────┤
│  CLIPS                                                  │
│  ┌──────────────────────────────────────────┐           │
│  │ Clip 1  │ 01:23.450 → 02:45.120 │  [✕]  │          │
│  │ Clip 2  │ 05:10.000 → 05:55.800 │  [✕]  │          │
│  └──────────────────────────────────────────┘           │
│                                                         │
│  [Export Selected]  [Export All]                         │
└─────────────────────────────────────────────────────────┘
```

The timeline area has four visual layers (top to bottom):
1. **Thumbnail strip** — Video frame thumbnails at adaptive density based on zoom level.
2. **Waveform** — Audio amplitude visualization.
3. **Clip markers** — Colored regions with draggable handles, overlaying the above.
4. **Controls** — Zoom slider and playhead position.

Before the background download completes, the thumbnail and waveform layers show a static placeholder (subtle gradient bar with "Generating…" label). They fade in with real data once extraction finishes.

### Timeline Interactions

- **Scrub:** Click anywhere on the timeline to jump. Click-and-drag to scrub continuously.
- **Mark in-point:** Press `I` or click `[I]` → if a clip is currently selected, updates that clip's start time. Otherwise, begins a new clip by recording a pending in-point at the current playhead position.
- **Mark out-point:** Press `O` or click `[O]` → if a clip is selected, updates that clip's end time. If there is a pending in-point (from a previous `I` press), creates a new clip spanning from the pending in-point to the current playhead. If neither, does nothing.
- **Adjust boundaries:** Drag the left/right edges of any clip region. Cursor changes to a resize handle near edges.
- **Fine-tune in clip list:** Click timestamp values in the clip list to edit them numerically.
- **Select clip:** Click a clip region on the timeline or its row in the clip list. Playhead jumps to its start.
- **Delete clip:** Click `[✕]`, or select and press `Delete`.
- **Zoom:** Scroll wheel on the timeline, or use the zoom slider. Zooming centers on the playhead.

### Keyboard Shortcuts

| Key | Action |
|---|---|
| `Space` | Play / Pause |
| `,` (comma) | Step back 1 frame |
| `.` (period) | Step forward 1 frame |
| `Shift+,` | Jump to previous keyframe |
| `Shift+.` | Jump to next keyframe |
| `←` / `→` | Seek ±5 seconds |
| `Shift+←` / `Shift+→` | Seek ±1 second |
| `I` | Set in-point (clip start) |
| `O` | Set out-point (clip end) |
| `J` / `K` / `L` | Shuttle: reverse / pause / forward |
| `Delete` | Remove selected clip |
| `Cmd+E` | Open export dialog |
| `Cmd+A` | Select all clips |

### Frame Stepping Implementation

- Frame stepping uses the video's FPS from yt-dlp metadata. Step = seek by `1/fps` seconds.
- Keyframe jumping requires the keyframe index, which is built after the background download completes via ffprobe.
- Until the keyframe index is ready, keyframe jump buttons are disabled with a tooltip: "Available after download completes."

## Export Flow

### Export Dialog

```
┌─────────── Export Clips ───────────┐
│                                     │
│  Clips to export:                   │
│  ○ Selected clip (Clip 2)           │
│  ● All clips (3)                    │
│                                     │
│  Output mode:                       │
│  ● Individual files                 │
│  ○ Merged into one video            │
│                                     │
│  Cut mode:                          │
│  ● Lossless (fast, keyframe-aligned)│
│  ○ Precise (re-encode, exact frames)│
│                                     │
│  Format: [MP4 (H.264)  ▾]          │
│                                     │
│  Save to: [~/  📁]                  │
│                                     │
│  Naming: {title} - Clip {n}         │
│                                     │
│  [Cancel]              [Export]      │
└─────────────────────────────────────┘
```

### Export Behavior

- **Lossless mode:** Runs `ffmpeg -ss <start> -to <end> -i <file> -c copy <output>`. Fast, no quality loss, but cuts align to nearest keyframe.
- **Precise mode:** Runs `ffmpeg -ss <start> -to <end> -i <file> -c:v libx264 -c:a aac <output>`. Slower, re-encodes, but frame-accurate.
- **Merged export:** Concatenates clips using ffmpeg's concat demuxer. Clips are ordered by their start time.
- **Format:** For lossless, determined by source container (remux). For precise, user picks MP4/MKV/WebM.
- **Save location:** Defaults to `~/` on first run. User picks a directory via native macOS folder picker. The choice persists across sessions. Always overridable per-export.
- **Naming:** Auto-generated: `{video title} - Clip 1.mp4`, `{video title} - Clip 2.mp4`. Merged: `{video title} - Merged.mp4`. Existing files are not silently overwritten — append ` (2)`, ` (3)`, etc.

### Export Prerequisites

Export requires the background download to be complete (ffmpeg operates on the local file, not the stream URL). If the download is still in progress when the user clicks Export:

- Show: "Download at 72% — export will start automatically when ready."
- User can wait or cancel.
- Export begins automatically once the download finishes.

## Dependency Chain & Setup Wizard

YouTube's 2026 anti-bot measures require more than just ffmpeg and yt-dlp. The full dependency chain:

| Dependency | Purpose | Install Method | Check |
|---|---|---|---|
| `ffmpeg` (+ `ffprobe`) | Clip extraction, waveform generation, thumbnail extraction, keyframe indexing | `brew install ffmpeg` | `which ffmpeg` |
| `yt-dlp` | URL resolution, metadata, streaming, downloading | `brew install yt-dlp` | `which yt-dlp` |
| `deno` (preferred) or `node` | JavaScript runtime for PO Token generation | `brew install deno` | `which deno \|\| which node` |
| `bgutil-ytdlp-pot-provider` | yt-dlp plugin: automatic per-video Proof-of-Origin token generation for YouTube | `python3 -m pip install -U bgutil-ytdlp-pot-provider` | Run `yt-dlp -v --simulate "https://www.youtube.com/watch?v=dQw4w9WgXcQ"` and check for `GetPOT` in `[debug] Extractor Plugins:` output |

### Setup Wizard UX

On first launch (or when any dependency is missing), the app shows a step-by-step Setup Wizard modal:

1. Each dependency is listed as a row with a status indicator: ✅ (installed), ❌ (missing), or ⏳ (installing).
2. The wizard checks each dependency in order. For the first missing item, it shows a description of what it does and an **[Install]** button.
3. Clicking **[Install]** runs the appropriate install command. Output is shown in a scrollable log area within the wizard.
4. After successful install, the status updates to ✅ and the wizard advances to the next item.
5. A **[Skip]** button is available for each step (the app will work for non-YouTube sources without the PO token chain, and without cookies for public content).
6. Once all critical dependencies are present (at minimum: ffmpeg + yt-dlp), the wizard closes.

The wizard also runs a quick self-check on subsequent launches. If everything is present, it's invisible. If something was uninstalled or broken, it resurfaces.

## Cookie / Authentication

YouTube requires session cookies for private, members-only, and age-restricted content. Even for public content, cookies can help avoid rate limiting and bot detection.

### Cookie Source Configuration

In the Preferences panel, the user configures their cookie source:

- **Default: Firefox** (`--cookies-from-browser firefox`). Firefox is the only browser where `--cookies-from-browser` reliably works — Chromium browsers (Chrome 127+) encrypt cookies with a process-bound key that yt-dlp can't decrypt.
- **Fallback: cookies.txt file** — User points to a Netscape-format cookies.txt file (exported via a browser extension). Useful if Firefox isn't available.
- **None** — No cookies. Works for public, non-restricted content on non-YouTube sites.

If the user selects Firefox but Firefox isn't installed, the app shows a note: "Firefox is recommended for reliable YouTube authentication. Alternatively, import a cookies.txt file."

### Integration with yt-dlp

Every yt-dlp invocation (metadata, stream URL, download) includes the configured cookie flag:
- Firefox: `--cookies-from-browser firefox`
- cookies.txt: `--cookies /path/to/cookies.txt`
- None: no cookie flag

## Waveform Visualization

### Generation

After the background download completes, the `WaveformGenerator` backend component extracts audio peak data:

1. Run ffmpeg to decode the audio track and compute amplitude peaks at a fixed sample count (~8000 samples for the full video duration).
2. Normalize peaks to a 0.0–1.0 range.
3. Send the peak array to the frontend via a Tauri event.

The exact ffmpeg pipeline: decode audio → resample to a low rate matching the desired sample count → extract absolute amplitude values → normalize.

### Display

- **Before download completes:** Static placeholder — a subtle striped gradient bar with a centered "Generating waveform…" label. Clearly communicates that data is coming.
- **After extraction:** Replace the placeholder with the real waveform, rendered on the Canvas as a filled amplitude graph. Smooth fade-in transition.
- The waveform responds to zoom: at higher zoom levels, more detail is visible. At lower zoom levels, peaks are aggregated (max value per visible pixel).

## Thumbnail Strip

### Generation

After the background download completes, the `ThumbnailExtractor` backend component extracts video frames:

1. Compute the extraction interval based on video duration:
   - Short videos (<10 min): 1 frame per second
   - Medium videos (10–60 min): 1 frame per 2 seconds
   - Long videos (>60 min): 1 frame per 5 seconds
2. Run ffmpeg to extract frames at the computed interval, scaled to 160×90px (16:9).
3. Pack extracted frames into sprite sheets (e.g. 10×10 grids = 100 thumbnails per sheet) for efficient loading and rendering.
4. Send sprite sheet metadata (file paths, dimensions, interval) to the frontend via a Tauri event.

### Display

- **Before download completes:** Static placeholder — a subtle gradient bar with "Generating thumbnails…" label. Matches the waveform placeholder style.
- **After extraction:** Replace the placeholder with real thumbnails, drawn from sprite sheets onto the Canvas.
- **Adaptive density:** The timeline shows ~1 thumbnail per 100px of visible width. Zooming in reveals more granular frames (showing frames closer together in time); zooming out shows fewer (skipping intermediate frames). The extraction provides enough frames to support all reasonable zoom levels; the Canvas renderer picks which frames to display based on the current visible time range and pixel width.

## Edge Cases

| Scenario | Behavior |
|---|---|
| Stream URL expires during preview | Detect playback error on `<video>`, re-resolve via yt-dlp, resume at same position. |
| Age-restricted / auth-required video | If no cookie source is configured, show error: "This video requires authentication. Configure cookies in Preferences." If cookies are configured but still fail, show the yt-dlp error with guidance. |
| Overlapping clips | Allowed. Each clip is independent. In merged export, clips are sorted by start time. If clip B starts before clip A ends, clip B's start is trimmed to clip A's end to avoid duplicate frames. |
| Very long video (>2 hours) | No special limits. Timeline zoom handles precision. |
| Any dependency missing on startup | Setup Wizard appears, walking user through installing each missing dependency step by step. See Dependency Chain section. |
| PO Token provider not working | yt-dlp may still work for some content without the PO token, but YouTube will likely block playback. Show a warning: "YouTube access may be limited. Check the PO Token provider in Preferences." |
| Waveform/thumbnail generation fails | Non-fatal. Log the ffmpeg error, keep the placeholder visible with an unobtrusive "Failed to generate" message. All other features continue working. |
| Export fails mid-way | Show ffmpeg error output in a dismissable panel. Partial files cleaned up. |
| Non-YouTube URL | yt-dlp supports 1000+ sites. App tries any URL. Shows "Unsupported URL" if yt-dlp fails. |
| Network disconnection during download | Download pauses. App shows "Network unavailable" status. Resumes automatically when connection returns (yt-dlp handles resume via `-c`). |
| App closed with active download | Clean up temp files. No persistence of partial downloads between sessions. |

## Future Features (Not in MVP)

These are explicitly out of scope for the initial version but should be kept in mind during architecture so they don't require rewrites:

- **Local file support** — Drag & drop a local `.mp4`/`.mkv`/`.webm` file. Skip the yt-dlp step, go straight to playback + timeline.
- **Video quality/resolution selection** — After URL resolution, show available formats and let the user pick (720p, 1080p, 4K, etc.) before downloading.
- **Save/load clip projects** — Persist clip data (URL, timestamps, labels) to a project file. Reopen later to continue editing or re-export.
- **Batch processing** — Queue multiple URLs, each with their own clips.
- **Custom ffmpeg flags** — Power-user option to pass arbitrary ffmpeg arguments during export.
- **Cross-platform** — Tauri supports Windows/Linux. The architecture doesn't preclude this, but macOS is the target for MVP.
- **Additional cookie browsers** — Support `--cookies-from-browser` for browsers beyond Firefox (if Chromium cookie encryption issues are resolved upstream).
- **Progressive waveform** — Build waveform data in real-time via Web Audio API as the stream plays, rather than waiting for download.