Add video clipper design spec
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
235
docs/superpowers/specs/2026-09-21-video-clipper-design.md
Normal file
235
docs/superpowers/specs/2026-09-21-video-clipper-design.md
Normal file
@@ -0,0 +1,235 @@
|
||||
# Video Clipper — Design Spec
|
||||
|
||||
**Date:** 2026-09-21
|
||||
**Status:** Draft
|
||||
**Stack:** Tauri v2 + Svelte (frontend) + Rust (backend) + ffmpeg + yt-dlp
|
||||
|
||||
## Overview
|
||||
|
||||
A macOS GUI app for creating clips from online videos (primarily YouTube) without requiring the user to manually download the video first. The app provides a timeline-based interface for scrubbing through video, marking clip boundaries, and exporting clips via ffmpeg.
|
||||
|
||||
Think "LosslessCut, but paste a URL instead of opening a file."
|
||||
|
||||
## Core User Flow
|
||||
|
||||
1. User pastes a YouTube (or other supported) URL.
|
||||
2. App resolves the URL via yt-dlp → retrieves metadata and a direct stream URL.
|
||||
3. Video begins playing immediately via the stream URL (HTML5 `<video>` element).
|
||||
4. A background download of the full video starts simultaneously.
|
||||
5. User scrubs the timeline, marks clip in/out points.
|
||||
6. User exports clips → ffmpeg operates on the local (downloaded) file.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three layers:
|
||||
|
||||
### 1. Tauri/Rust Backend
|
||||
|
||||
Manages all subprocess orchestration, file I/O, and state persistence. Communicates with the frontend via Tauri commands (request/response) and Tauri events (push notifications like download progress).
|
||||
|
||||
**Components:**
|
||||
|
||||
| Component | Responsibility |
|
||||
|---|---|
|
||||
| `DependencyManager` | Check for ffmpeg/yt-dlp on `$PATH`. If missing, offer to install via `brew install`. Shows a modal on startup if dependencies are absent. |
|
||||
| `VideoResolver` | Takes a URL, calls `yt-dlp --dump-json <url>` for metadata and `yt-dlp -g <url>` for the direct stream URL. Returns a `VideoMetadata` struct. |
|
||||
| `DownloadManager` | Runs `yt-dlp -o <temp_path> <url>` in the background. Emits progress events to the frontend. Manages temp file lifecycle (cleanup on cancel/error). |
|
||||
| `KeyframeIndex` | After download completes, runs `ffprobe -select_streams v -show_entries frame=pts_time,flags -of csv <file>` to extract keyframe positions. Returns a sorted list of keyframe timestamps. |
|
||||
| `ClipExporter` | Runs ffmpeg for clip extraction. Supports two modes: lossless (stream copy, keyframe-aligned) and precise (re-encode, exact frames). Handles both individual and merged export. |
|
||||
| `PreferencesStore` | Persists user preferences (output directory, window geometry) via `tauri-plugin-store`. |
|
||||
|
||||
### 2. Svelte Frontend
|
||||
|
||||
Renders the video player, custom timeline, clip management panel, and export dialog.
|
||||
|
||||
**Components:**
|
||||
|
||||
| Component | Responsibility |
|
||||
|---|---|
|
||||
| `UrlInput` | Text field for URL entry. Detects paste, shows loading spinner during resolution, displays errors. |
|
||||
| `VideoPlayer` | Wraps the HTML5 `<video>` element. Exposes play/pause/seek API. Stays synced with the Timeline component via a shared Svelte store. |
|
||||
| `Timeline` | Canvas-based timeline. Renders: playhead (current position), clip regions (colored rectangles with draggable start/end handles), keyframe markers (after index is built). Supports click-to-seek, drag-to-scrub, zoom via scroll wheel or slider. |
|
||||
| `TransportControls` | Play/pause, frame step (±1 frame), keyframe jump, seek ±1s/±5s buttons. Wired to keyboard shortcuts. |
|
||||
| `ClipList` | Scrollable panel listing all clips. Each row shows: color swatch, label (editable), start time (editable), end time (editable), delete button. Clicking a row selects the clip and jumps the playhead to its start. |
|
||||
| `ExportDialog` | Modal dialog for export configuration. See Export Flow section. |
|
||||
| `StatusBar` | Bottom bar showing: download progress (percentage + ETA), active export progress, tool availability status. |
|
||||
| `DependencyPrompt` | Modal shown on startup if ffmpeg/yt-dlp are missing. Offers to install via Homebrew. |
|
||||
|
||||
### 3. External CLI Tools
|
||||
|
||||
- **yt-dlp:** URL resolution, metadata extraction, stream URL retrieval, video downloading.
|
||||
- **ffmpeg:** Clip extraction (both lossless and re-encode modes), clip merging.
|
||||
- **ffprobe:** Keyframe position extraction (part of the ffmpeg package).
|
||||
|
||||
## Data Model
|
||||
|
||||
```typescript
|
||||
interface Clip {
|
||||
id: string;
|
||||
startTime: number; // seconds (float, e.g. 83.450)
|
||||
endTime: number; // seconds (float)
|
||||
label: string; // auto-generated "Clip 1", "Clip 2", editable by user
|
||||
color: string; // auto-assigned from a palette for visual distinction
|
||||
}
|
||||
|
||||
interface VideoSession {
|
||||
url: string;
|
||||
title: string;
|
||||
duration: number; // seconds
|
||||
fps: number; // frames per second, from yt-dlp metadata
|
||||
thumbnailUrl: string | null;
|
||||
streamUrl: string; // direct stream URL for preview playback
|
||||
localFilePath: string | null; // set once background download completes
|
||||
downloadProgress: number; // 0.0 - 1.0
|
||||
keyframePositions: number[]; // populated after download + ffprobe
|
||||
clips: Clip[];
|
||||
}
|
||||
|
||||
interface UserPreferences {
|
||||
outputDirectory: string; // default: home directory (~/)
|
||||
windowBounds: { x: number; y: number; width: number; height: number } | null;
|
||||
}
|
||||
```
|
||||
|
||||
## Timeline & Interaction Design
|
||||
|
||||
### Layout
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ [Paste URL...] [⚙ Prefs] │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ VIDEO PLAYER │
|
||||
│ (16:9 aspect ratio) │
|
||||
│ │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ [◄K] [◄|] [▶/❚❚] [|►] [K►] 00:03:42 / 00:12:35 │
|
||||
│ [I] [O] │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ ▼ playhead │
|
||||
│ ┃ │
|
||||
│ ╠══════════════════════════════════════════════════════╣
|
||||
│ ║ ┃ ▓▓▓▓▓▓▓▓▓▓▓ ┃ ┃ ▓▓▓▓▓ ┃ ║
|
||||
│ ╠══════════════════════════════════════════════════════╣
|
||||
│ ┃ │
|
||||
│ [🔍−] ═══════════●══════════ [🔍+] zoom slider │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ CLIPS │
|
||||
│ ┌──────────────────────────────────────────┐ │
|
||||
│ │ Clip 1 │ 01:23.450 → 02:45.120 │ [✕] │ │
|
||||
│ │ Clip 2 │ 05:10.000 → 05:55.800 │ [✕] │ │
|
||||
│ └──────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Export Selected] [Export All] │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Timeline Interactions
|
||||
|
||||
- **Scrub:** Click anywhere on the timeline to jump. Click-and-drag to scrub continuously.
|
||||
- **Mark in-point:** Press `I` or click `[I]` → if a clip is currently selected, updates that clip's start time. Otherwise, begins a new clip by recording a pending in-point at the current playhead position.
|
||||
- **Mark out-point:** Press `O` or click `[O]` → if a clip is selected, updates that clip's end time. If there is a pending in-point (from a previous `I` press), creates a new clip spanning from the pending in-point to the current playhead. If neither, does nothing.
|
||||
- **Adjust boundaries:** Drag the left/right edges of any clip region. Cursor changes to a resize handle near edges.
|
||||
- **Fine-tune in clip list:** Click timestamp values in the clip list to edit them numerically.
|
||||
- **Select clip:** Click a clip region on the timeline or its row in the clip list. Playhead jumps to its start.
|
||||
- **Delete clip:** Click `[✕]`, or select and press `Delete`.
|
||||
- **Zoom:** Scroll wheel on the timeline, or use the zoom slider. Zooming centers on the playhead.
|
||||
|
||||
### Keyboard Shortcuts
|
||||
|
||||
| Key | Action |
|
||||
|---|---|
|
||||
| `Space` | Play / Pause |
|
||||
| `,` (comma) | Step back 1 frame |
|
||||
| `.` (period) | Step forward 1 frame |
|
||||
| `Shift+,` | Jump to previous keyframe |
|
||||
| `Shift+.` | Jump to next keyframe |
|
||||
| `←` / `→` | Seek ±5 seconds |
|
||||
| `Shift+←` / `Shift+→` | Seek ±1 second |
|
||||
| `I` | Set in-point (clip start) |
|
||||
| `O` | Set out-point (clip end) |
|
||||
| `J` / `K` / `L` | Shuttle: reverse / pause / forward |
|
||||
| `Delete` | Remove selected clip |
|
||||
| `Cmd+E` | Open export dialog |
|
||||
| `Cmd+A` | Select all clips |
|
||||
|
||||
### Frame Stepping Implementation
|
||||
|
||||
- Frame stepping uses the video's FPS from yt-dlp metadata. Step = seek by `1/fps` seconds.
|
||||
- Keyframe jumping requires the keyframe index, which is built after the background download completes via ffprobe.
|
||||
- Until the keyframe index is ready, keyframe jump buttons are disabled with a tooltip: "Available after download completes."
|
||||
|
||||
## Export Flow
|
||||
|
||||
### Export Dialog
|
||||
|
||||
```
|
||||
┌─────────── Export Clips ───────────┐
|
||||
│ │
|
||||
│ Clips to export: │
|
||||
│ ○ Selected clip (Clip 2) │
|
||||
│ ● All clips (3) │
|
||||
│ │
|
||||
│ Output mode: │
|
||||
│ ● Individual files │
|
||||
│ ○ Merged into one video │
|
||||
│ │
|
||||
│ Cut mode: │
|
||||
│ ● Lossless (fast, keyframe-aligned)│
|
||||
│ ○ Precise (re-encode, exact frames)│
|
||||
│ │
|
||||
│ Format: [MP4 (H.264) ▾] │
|
||||
│ │
|
||||
│ Save to: [~/ 📁] │
|
||||
│ │
|
||||
│ Naming: {title} - Clip {n} │
|
||||
│ │
|
||||
│ [Cancel] [Export] │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Export Behavior
|
||||
|
||||
- **Lossless mode:** Runs `ffmpeg -ss <start> -to <end> -i <file> -c copy <output>`. Fast, no quality loss, but cuts align to nearest keyframe.
|
||||
- **Precise mode:** Runs `ffmpeg -ss <start> -to <end> -i <file> -c:v libx264 -c:a aac <output>`. Slower, re-encodes, but frame-accurate.
|
||||
- **Merged export:** Concatenates clips using ffmpeg's concat demuxer. Clips are ordered by their start time.
|
||||
- **Format:** For lossless, determined by source container (remux). For precise, user picks MP4/MKV/WebM.
|
||||
- **Save location:** Defaults to `~/` on first run. User picks a directory via native macOS folder picker. The choice persists across sessions. Always overridable per-export.
|
||||
- **Naming:** Auto-generated: `{video title} - Clip 1.mp4`, `{video title} - Clip 2.mp4`. Merged: `{video title} - Merged.mp4`. Existing files are not silently overwritten — append ` (2)`, ` (3)`, etc.
|
||||
|
||||
### Export Prerequisites
|
||||
|
||||
Export requires the background download to be complete (ffmpeg operates on the local file, not the stream URL). If the download is still in progress when the user clicks Export:
|
||||
|
||||
- Show: "Download at 72% — export will start automatically when ready."
|
||||
- User can wait or cancel.
|
||||
- Export begins automatically once the download finishes.
|
||||
|
||||
## Edge Cases
|
||||
|
||||
| Scenario | Behavior |
|
||||
|---|---|
|
||||
| Stream URL expires during preview | Detect playback error on `<video>`, re-resolve via yt-dlp, resume at same position. |
|
||||
| Age-restricted / auth-required video | Show error: "This video requires authentication." Guidance points to future cookie support. |
|
||||
| Overlapping clips | Allowed. Each clip is independent. In merged export, clips are sorted by start time. If clip B starts before clip A ends, clip B's start is trimmed to clip A's end to avoid duplicate frames. |
|
||||
| Very long video (>2 hours) | No special limits. Timeline zoom handles precision. |
|
||||
| ffmpeg/yt-dlp missing on startup | Modal: "Required tools not found. Install via Homebrew?" → [Install] runs `brew install ffmpeg yt-dlp` with visible output. [Manual] dismisses. |
|
||||
| Export fails mid-way | Show ffmpeg error output in a dismissable panel. Partial files cleaned up. |
|
||||
| Non-YouTube URL | yt-dlp supports 1000+ sites. App tries any URL. Shows "Unsupported URL" if yt-dlp fails. |
|
||||
| Network disconnection during download | Download pauses. App shows "Network unavailable" status. Resumes automatically when connection returns (yt-dlp handles resume via `-c`). |
|
||||
| App closed with active download | Clean up temp files. No persistence of partial downloads between sessions. |
|
||||
|
||||
## Future Features (Not in MVP)
|
||||
|
||||
These are explicitly out of scope for the initial version but should be kept in mind during architecture so they don't require rewrites:
|
||||
|
||||
- **Local file support** — Drag & drop a local `.mp4`/`.mkv`/`.webm` file. Skip the yt-dlp step, go straight to playback + timeline.
|
||||
- **Video quality/resolution selection** — After URL resolution, show available formats and let the user pick (720p, 1080p, 4K, etc.) before downloading.
|
||||
- **Save/load clip projects** — Persist clip data (URL, timestamps, labels) to a project file. Reopen later to continue editing or re-export.
|
||||
- **Waveform/audio visualization** — Show audio waveform on the timeline for easier cut-point identification.
|
||||
- **Thumbnail strip** — Show video frame thumbnails along the timeline for visual scrubbing.
|
||||
- **Batch processing** — Queue multiple URLs, each with their own clips.
|
||||
- **Cookie/auth support** — Allow yt-dlp `--cookies-from-browser` or manual cookie file for age-restricted/private videos.
|
||||
- **Custom ffmpeg flags** — Power-user option to pass arbitrary ffmpeg arguments during export.
|
||||
- **Cross-platform** — Tauri supports Windows/Linux. The architecture doesn't preclude this, but macOS is the target for MVP.
|
||||
Reference in New Issue
Block a user