gui-video-clipper/chat-summaries/2026-09-21_14-09-fix-multiclip-caption-support-summary.md

# Fix Multi-Clip Workflow + Caption Support

**Date:** 2026-09-21 14:09  
**Task:** Fix multi-clip creation and add time-synced caption support

## Task Description

Two main issues addressed:
1. **Multi-clip bug**: Pressing I/O keys always edited the selected clip instead of creating new clips when the playhead was outside the clip region.
2. **Caption support**: Full pipeline for downloading, displaying, and exporting time-synced captions (embedded > external, English preferred, auto-generated deprioritized).

## Changes Made

### 1. Fix Multi-Clip Workflow

**`src/lib/stores/clips.svelte.ts`**:
- Added `isTimeInsideClip(time, clipId)` helper with 0.5s tolerance
- `markInPoint(time)`: Now checks if playhead is inside the selected clip's range. If outside, deselects and sets `pendingInPoint` (starts new clip). If inside, edits the existing clip.
- `markOutPoint(time)`: Same logic — edits selected clip if inside, creates new clip from pending in-point if outside.

**`src/lib/components/Timeline.svelte`**:
- Added `selectClip(null)` call when clicking empty timeline space (no `hitTestClip` result), so the next I/O presses create a new clip.

### 2. Caption Metadata Parsing

**`src-tauri/src/services/video_resolver.rs`**:
- Extended `YtDlpJson` struct with `subtitles` and `automatic_captions` fields (both `HashMap<String, Vec<SubtitleFormat>>`)
- Added `detect_captions()` function: checks for English subtitles (en, en-US, en-GB variants), prioritizes manual over auto-generated
- Added `SubtitleFormat` struct for deserialization
- Added 4 new unit tests for caption detection

**`src-tauri/src/models.rs`**:
- Added `has_captions: bool` and `captions_are_auto: bool` to `VideoMetadata`

**`src/lib/bindings/video.ts`**:
- Added `hasCaptions` and `captionsAreAuto` to `VideoMetadata` interface

### 3. Caption Download

**`src-tauri/src/services/subtitle_downloader.rs`** (new file):
- Downloads English VTT subtitles via `yt-dlp --write-subs` (manual) or `--write-auto-subs` (auto-generated)
- Uses `--sub-langs en.*,en --sub-format vtt --convert-subs vtt`
- Searches output directory for `.en.vtt` files, falls back to any `.vtt`

**`src-tauri/src/commands/video.rs`**:
- Added `download_subtitles` Tauri command

**`src/lib/bindings/video.ts`**:
- Added `downloadSubtitles()` binding

### 4. Embedded Subtitle Check

**`src-tauri/src/commands/media_analysis.rs`**:
- Added `check_embedded_subtitles` command that uses `ffprobe -show_streams -select_streams s` to detect subtitle streams, then extracts as VTT via `ffmpeg -map 0:s:0 -f webvtt`

**`src/lib/bindings/mediaAnalysis.ts`**:
- Added `checkEmbeddedSubtitles()` binding

### 5. Caption Display in Player

**`src/lib/stores/videoSession.svelte.ts`**:
- Added `hasCaptions`, `captionsAreAuto`, `captionFilePath` to session state
- Added `loadCaptions()` function that checks embedded subs first, then downloads external
- Integrated into `triggerPostDownloadProcessing()`

**`src/lib/components/VideoPlayer.svelte`**:
- Loads VTT file as blob URL via `captionBlobUrl`
- Renders `<track>` element with proper `srclang`, `label` (with "(auto)" suffix), and `default` attribute
- Added CC toggle button (bottom-right overlay) with active/inactive styling
- Syncs `textTracks[].mode` with `captionsEnabled` state

### 6. Caption Export

**`src-tauri/src/models.rs`**:
- Added `include_captions: bool` and `caption_file_path: Option<String>` to `ExportConfig`

**`src-tauri/src/services/clip_exporter.rs`**:
- Added `build_ffmpeg_args_with_subs()`: lossless → mux as `mov_text`/`srt`; precise → burn-in via `subtitles=` filter
- Added `export_single_clip_with_subs()` with fallback to no-subs on failure
- Added `export_merged_with_subs()`
- Added 2 new unit tests for subtitle-aware arg building

**`src-tauri/src/commands/export.rs`**:
- Updated to conditionally use `_with_subs` variants based on `include_captions`

**`src/lib/bindings/export.ts`**:
- Added `includeCaptions` and `captionFilePath` to `ExportConfig` interface

**`src/lib/components/ExportDialog.svelte`**:
- Added "Captions" section with checkbox toggle (only shown when captions are available)
- Shows "(auto-generated)" label when applicable
- Shows mux vs. burn-in note based on cut mode

### Infrastructure

**`src-tauri/src/services/mod.rs`**: Registered `subtitle_downloader` module  
**`src-tauri/src/lib.rs`**: Registered `download_subtitles` and `check_embedded_subtitles` commands

## Build & Test Results

- **Rust**: 34 tests pass (including 4 new caption tests + 2 new export tests)
- **Frontend**: Builds cleanly (0 errors)
- **Rust build**: Compiles with only pre-existing unused variant warnings

## Lessons Learned

- `yt-dlp --dump-json` includes `subtitles` and `automatic_captions` maps — both map language codes to arrays of `{ext, url}` objects
- WKWebView `<track>` elements work with blob URLs but need `textTracks[].mode` managed manually to sync with a toggle
- For ffmpeg subtitle burn-in, paths with colons need escaping (`\:`) in the `subtitles=` filter
- Subtitle muxing in lossless mode requires separate `-i` for the subtitle file and explicit stream mapping (`-map 0:v -map 0:a -map 1:s`)

## Follow-up Items

- The `SubtitleFormat.ext` and `SubtitleFormat.url` fields generate "never read" warnings — they exist for serde deserialization but could be suppressed with `#[allow(dead_code)]`
- Caption time offset accuracy: when using `-ss` before `-i`, subtitle timestamps may need adjustment for precise alignment in edge cases
- Sprite sheet optimization (noted from prior session) is still pending