chore: stage all pending work — caption styling, media server, processing modal, docs, summaries
Includes: - Extended caption styling (font, shadow, dimmed color, bg toggle) - Media server, subtitle downloader, VTT parser, processing modal - Waveform tiers, thumbnail/timeline improvements, transport controls - Hybrid download model, dependency management, clip export enhancements - 21 chat summaries, 2 implementation plans, 2 design specs Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
858
docs/superpowers/plans/2026-09-22-extended-caption-styling.md
Normal file
858
docs/superpowers/plans/2026-09-22-extended-caption-styling.md
Normal file
@@ -0,0 +1,858 @@
|
||||
# Extended Caption Styling Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add system font selection, drop shadow controls, configurable dimmed text color, and background toggle to the shared caption settings, wired through to both preview and burn-in export.
|
||||
|
||||
**Architecture:** Extend `CaptionSettings` (frontend) and `CaptionStyle` (backend) with new fields. Add a `list_system_fonts` Tauri command using `fc-list`. Update the settings panel UI, the ASS generation code, and the preview caption rendering.
|
||||
|
||||
**Tech Stack:** Svelte 5, Rust/Tauri v2, ASS subtitle format, fontconfig (`fc-list`)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Background and shadow are mutually exclusive: enabling one disables the other
|
||||
- `fc-list` fallback: if unavailable, use preset list: Arial, Helvetica, Verdana, Georgia, Times New Roman, Courier New, Impact
|
||||
- Dimmed color auto mode derives from `textColor` + `dimmedOpacity`; custom mode uses `dimmedColor` directly
|
||||
- ASS `BackColour` serves as background color (when background ON) or shadow color (when shadow ON)
|
||||
- All new fields must have backwards-compatible defaults (existing saved preferences without new fields should merge cleanly via `{ ...DEFAULT_CAPTION_SETTINGS, ...savedCaptions }`)
|
||||
- No new crate dependencies
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Data model — add new fields to frontend and backend
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/stores/preferences.svelte.ts:6-27`
|
||||
- Modify: `src-tauri/src/models.rs:91-108`
|
||||
- Modify: `src/lib/bindings/export.ts:3-10`
|
||||
- Modify: `src/lib/components/ExportDialog.svelte:86-94`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Existing `CaptionSettings`, `CaptionStyle`, `CaptionStyle` (TS binding)
|
||||
- Produces: Extended versions of all three with new fields. All later tasks depend on these types.
|
||||
|
||||
- [ ] **Step 1: Update `CaptionSettings` interface and defaults**
|
||||
|
||||
In `src/lib/stores/preferences.svelte.ts`, replace the interface and defaults:
|
||||
|
||||
```typescript
|
||||
export interface CaptionSettings {
|
||||
fontFamily: string;
|
||||
fontSize: number;
|
||||
textColor: string;
|
||||
bold: boolean;
|
||||
outlineColor: string;
|
||||
backgroundEnabled: boolean;
|
||||
backgroundOpacity: number;
|
||||
backgroundColor: string;
|
||||
textOutline: boolean;
|
||||
shadowEnabled: boolean;
|
||||
shadowDepth: number;
|
||||
shadowColor: string;
|
||||
dimmedColorMode: 'auto' | 'custom';
|
||||
dimmedOpacity: number;
|
||||
dimmedColor: string;
|
||||
position: 'bottom' | 'top';
|
||||
wordHighlight: boolean;
|
||||
}
|
||||
|
||||
export const DEFAULT_CAPTION_SETTINGS: CaptionSettings = {
|
||||
fontFamily: 'Arial',
|
||||
fontSize: 18,
|
||||
textColor: '#ffffff',
|
||||
bold: true,
|
||||
outlineColor: '#000000',
|
||||
backgroundEnabled: true,
|
||||
backgroundOpacity: 0.75,
|
||||
backgroundColor: '#000000',
|
||||
textOutline: true,
|
||||
shadowEnabled: false,
|
||||
shadowDepth: 2,
|
||||
shadowColor: '#000000',
|
||||
dimmedColorMode: 'auto',
|
||||
dimmedOpacity: 0.4,
|
||||
dimmedColor: '#999999',
|
||||
position: 'bottom',
|
||||
wordHighlight: true,
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `CaptionStyle` struct in Rust**
|
||||
|
||||
In `src-tauri/src/models.rs`, replace the struct:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct CaptionStyle {
|
||||
pub font_family: String,
|
||||
pub font_size: u32,
|
||||
pub text_color: String,
|
||||
pub bold: bool,
|
||||
pub outline_color: String,
|
||||
pub background_enabled: bool,
|
||||
pub background_opacity: f64,
|
||||
pub background_color: String,
|
||||
pub text_outline: bool,
|
||||
pub shadow_enabled: bool,
|
||||
pub shadow_depth: u32,
|
||||
pub shadow_color: String,
|
||||
pub dimmed_color_mode: String,
|
||||
pub dimmed_opacity: f64,
|
||||
pub dimmed_color: String,
|
||||
pub position: String,
|
||||
pub word_highlight: bool,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update TypeScript `CaptionStyle` binding**
|
||||
|
||||
In `src/lib/bindings/export.ts`, replace the interface:
|
||||
|
||||
```typescript
|
||||
export interface CaptionStyle {
|
||||
fontFamily: string;
|
||||
fontSize: number;
|
||||
textColor: string;
|
||||
bold: boolean;
|
||||
outlineColor: string;
|
||||
backgroundEnabled: boolean;
|
||||
backgroundOpacity: number;
|
||||
backgroundColor: string;
|
||||
textOutline: boolean;
|
||||
shadowEnabled: boolean;
|
||||
shadowDepth: number;
|
||||
shadowColor: string;
|
||||
dimmedColorMode: string;
|
||||
dimmedOpacity: number;
|
||||
dimmedColor: string;
|
||||
position: string;
|
||||
wordHighlight: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update ExportDialog passthrough**
|
||||
|
||||
In `src/lib/components/ExportDialog.svelte`, replace the `captionStyle` construction:
|
||||
|
||||
```typescript
|
||||
captionStyle: (burnInCaptions && includeCaptions && hasCaptions)
|
||||
? {
|
||||
fontFamily: preferences.captionSettings.fontFamily,
|
||||
fontSize: preferences.captionSettings.fontSize,
|
||||
textColor: preferences.captionSettings.textColor,
|
||||
bold: preferences.captionSettings.bold,
|
||||
outlineColor: preferences.captionSettings.outlineColor,
|
||||
backgroundEnabled: preferences.captionSettings.backgroundEnabled,
|
||||
backgroundOpacity: preferences.captionSettings.backgroundOpacity,
|
||||
backgroundColor: preferences.captionSettings.backgroundColor,
|
||||
textOutline: preferences.captionSettings.textOutline,
|
||||
shadowEnabled: preferences.captionSettings.shadowEnabled,
|
||||
shadowDepth: preferences.captionSettings.shadowDepth,
|
||||
shadowColor: preferences.captionSettings.shadowColor,
|
||||
dimmedColorMode: preferences.captionSettings.dimmedColorMode,
|
||||
dimmedOpacity: preferences.captionSettings.dimmedOpacity,
|
||||
dimmedColor: preferences.captionSettings.dimmedColor,
|
||||
position: preferences.captionSettings.position,
|
||||
wordHighlight: preferences.captionSettings.wordHighlight,
|
||||
}
|
||||
: null,
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Build and test**
|
||||
|
||||
Run: `cd src-tauri && cargo build 2>&1 | tail -5`
|
||||
Expected: Clean build (warnings OK if pre-existing).
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | tail -5`
|
||||
Expected: All 23 tests pass (they don't construct CaptionStyle directly).
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/lib/stores/preferences.svelte.ts src-tauri/src/models.rs src/lib/bindings/export.ts src/lib/components/ExportDialog.svelte
|
||||
git commit -m "feat(captions): extend data model with font, shadow, dimmed color, bg toggle fields"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: System font detection — add `list_system_fonts` command
|
||||
|
||||
**Files:**
|
||||
- Modify: `src-tauri/src/commands/media_analysis.rs` (add new command)
|
||||
- Modify: `src-tauri/src/lib.rs:31-46` (register command)
|
||||
- Modify: `src/lib/bindings/mediaAnalysis.ts` (add TS binding)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `fc-list` CLI tool (from fontconfig)
|
||||
- Produces: `list_system_fonts() -> Result<Vec<String>, String>` (Rust), `listSystemFonts(): Promise<string[]>` (TS)
|
||||
|
||||
- [ ] **Step 1: Add the Rust command**
|
||||
|
||||
Append to `src-tauri/src/commands/media_analysis.rs`:
|
||||
|
||||
```rust
|
||||
#[tauri::command]
|
||||
pub async fn list_system_fonts() -> Result<Vec<String>, String> {
|
||||
tokio::task::spawn_blocking(|| {
|
||||
// Try fc-list first (available if fontconfig is installed, which ffmpeg depends on)
|
||||
let output = std::process::Command::new("fc-list")
|
||||
.args([":lang=en", "family"])
|
||||
.output();
|
||||
|
||||
match output {
|
||||
Ok(out) if out.status.success() => {
|
||||
let text = String::from_utf8_lossy(&out.stdout);
|
||||
let mut families: Vec<String> = text
|
||||
.lines()
|
||||
.flat_map(|line| line.split(','))
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty())
|
||||
.collect();
|
||||
families.sort_unstable();
|
||||
families.dedup();
|
||||
Ok(families)
|
||||
}
|
||||
_ => {
|
||||
// Fallback: common cross-platform fonts
|
||||
Ok(vec![
|
||||
"Arial".to_string(),
|
||||
"Comic Sans MS".to_string(),
|
||||
"Courier New".to_string(),
|
||||
"Georgia".to_string(),
|
||||
"Helvetica".to_string(),
|
||||
"Impact".to_string(),
|
||||
"Times New Roman".to_string(),
|
||||
"Verdana".to_string(),
|
||||
])
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Font detection failed: {e}"))?
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register the command in `lib.rs`**
|
||||
|
||||
In `src-tauri/src/lib.rs`, add `media_analysis::list_system_fonts` to the `generate_handler!` macro, after `media_analysis::check_embedded_subtitles`:
|
||||
|
||||
```rust
|
||||
media_analysis::check_embedded_subtitles,
|
||||
media_analysis::list_system_fonts,
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add the TS binding**
|
||||
|
||||
Append to `src/lib/bindings/mediaAnalysis.ts`:
|
||||
|
||||
```typescript
|
||||
export async function listSystemFonts(): Promise<string[]> {
|
||||
return invoke<string[]>('list_system_fonts');
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Build and test**
|
||||
|
||||
Run: `cd src-tauri && cargo build 2>&1 | tail -5`
|
||||
Expected: Clean build.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/src/commands/media_analysis.rs src-tauri/src/lib.rs src/lib/bindings/mediaAnalysis.ts
|
||||
git commit -m "feat(captions): add list_system_fonts command with fc-list + fallback"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: ASS generation — wire all new fields into burn-in subtitle output
|
||||
|
||||
**Files:**
|
||||
- Modify: `src-tauri/src/services/clip_exporter.rs:549-584`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `CaptionStyle` struct from Task 1 (with all new fields)
|
||||
- Produces: Updated `vtt_to_ass_with_karaoke()` that uses font family, background/shadow mutual exclusivity, and dimmed color settings
|
||||
|
||||
- [ ] **Step 1: Replace the style extraction block in `vtt_to_ass_with_karaoke`**
|
||||
|
||||
In `src-tauri/src/services/clip_exporter.rs`, replace lines 550-566 (the style extraction block) with:
|
||||
|
||||
```rust
|
||||
// Font
|
||||
let font_name = style.map_or("Arial".to_string(), |s| s.font_family.clone());
|
||||
let font_size = style.map_or(50, |s| (s.font_size as f64 * 2.75).round() as u32);
|
||||
let bold_flag: i32 = if style.map_or(true, |s| s.bold) { -1 } else { 0 };
|
||||
|
||||
// Colors
|
||||
let primary_colour = style.map_or("&H00FFFFFF".to_string(), |s| hex_to_ass_color(&s.text_color));
|
||||
let outline_colour = style.map_or("&H00000000".to_string(), |s| hex_to_ass_color(&s.outline_color));
|
||||
|
||||
// Dimmed color (SecondaryColour) — auto derives from text color, custom uses direct color
|
||||
let secondary_colour = style.map_or("&H73CCCCCC".to_string(), |s| {
|
||||
if s.dimmed_color_mode == "custom" {
|
||||
hex_to_ass_color(&s.dimmed_color)
|
||||
} else {
|
||||
let alpha = ((1.0 - s.dimmed_opacity.clamp(0.0, 1.0)) * 255.0).round() as u8;
|
||||
hex_to_ass_color_with_alpha(&s.text_color, alpha)
|
||||
}
|
||||
});
|
||||
|
||||
// Background/Shadow mutual exclusivity
|
||||
let bg_enabled = style.map_or(true, |s| s.background_enabled);
|
||||
let shadow_enabled = style.map_or(false, |s| s.shadow_enabled);
|
||||
let text_outline = style.map_or(true, |s| s.text_outline);
|
||||
|
||||
let (border_style, outline_val, shadow_val, back_colour) = if bg_enabled {
|
||||
// Background mode: box + optional outline, no shadow
|
||||
let bc = style.map_or("&H80000000".to_string(), |s| {
|
||||
let alpha = ((1.0 - s.background_opacity.clamp(0.0, 1.0)) * 255.0).round() as u8;
|
||||
hex_to_ass_color_with_alpha(&s.background_color, alpha)
|
||||
});
|
||||
let bs = if text_outline { 4 } else { 3 };
|
||||
let ol = if text_outline { 2 } else { 0 };
|
||||
(bs, ol, 0, bc)
|
||||
} else if shadow_enabled {
|
||||
// Shadow mode: outline + shadow, no box
|
||||
let sc = style.map_or("&H00000000".to_string(), |s| hex_to_ass_color(&s.shadow_color));
|
||||
let sd = style.map_or(2, |s| s.shadow_depth) as i32;
|
||||
let ol = if text_outline { 2 } else { 0 };
|
||||
(1, ol, sd, sc)
|
||||
} else {
|
||||
// Neither: just outline if enabled
|
||||
let ol = if text_outline { 2 } else { 0 };
|
||||
(1, ol, 0, "&HFF000000".to_string()) // fully transparent back
|
||||
};
|
||||
|
||||
let word_highlight = style.map_or(true, |s| s.word_highlight);
|
||||
let margin_v: i32 = 40;
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update the Style format string to use `font_name`**
|
||||
|
||||
Replace the Style line (currently around line 580-584):
|
||||
|
||||
```rust
|
||||
output.push_str(&format!(
|
||||
"Style: Default,{},{},{},{},{},{},{},0,0,0,100,100,0,0,{},{},{},2,20,20,{},1\n",
|
||||
font_name, font_size, primary_colour, secondary_colour, outline_colour, back_colour,
|
||||
bold_flag, border_style, outline_val, shadow_val, margin_v
|
||||
));
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run tests**
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | tail -10`
|
||||
Expected: All 23 tests pass.
|
||||
|
||||
Run: `cd src-tauri && cargo build 2>&1 | tail -5`
|
||||
Expected: Clean build.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/src/services/clip_exporter.rs
|
||||
git commit -m "feat(captions): wire font, shadow, dimmed color, bg toggle into ASS generation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Preview captions — use new settings in VideoPlayer
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/components/VideoPlayer.svelte:29-40, 184-189`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `CaptionSettings` from Task 1 (via `preferences.captionSettings`)
|
||||
- Produces: Updated preview caption rendering using font, background toggle, shadow, and dimmed color settings
|
||||
|
||||
- [ ] **Step 1: Update the `captionStyle` derived**
|
||||
|
||||
In `src/lib/components/VideoPlayer.svelte`, replace the `captionStyle` derived block (lines 29-36):
|
||||
|
||||
```typescript
|
||||
let captionStyle = $derived.by(() => {
|
||||
const s = preferences.captionSettings;
|
||||
const bg = s.backgroundEnabled
|
||||
? `rgba(${hexToRgb(s.backgroundColor)}, ${s.backgroundOpacity})`
|
||||
: 'transparent';
|
||||
const shadow = s.shadowEnabled
|
||||
? `${s.shadowDepth}px ${s.shadowDepth}px ${s.shadowDepth * 2}px ${s.shadowColor}`
|
||||
: s.textOutline
|
||||
? '1px 1px 2px rgba(0,0,0,0.9), -1px -1px 2px rgba(0,0,0,0.9)'
|
||||
: 'none';
|
||||
return {
|
||||
fontFamily: s.fontFamily,
|
||||
fontSize: `${s.fontSize}px`,
|
||||
fontWeight: s.bold ? 'bold' : 'normal',
|
||||
color: s.textColor,
|
||||
background: bg,
|
||||
textShadow: shadow,
|
||||
};
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the `hexToRgb` helper**
|
||||
|
||||
Add this helper function at the top of the `<script>` block (after imports):
|
||||
|
||||
```typescript
|
||||
function hexToRgb(hex: string): string {
|
||||
const h = hex.replace('#', '');
|
||||
const r = parseInt(h.substring(0, 2), 16);
|
||||
const g = parseInt(h.substring(2, 4), 16);
|
||||
const b = parseInt(h.substring(4, 6), 16);
|
||||
return `${r}, ${g}, ${b}`;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Update the dimmed word color in the template**
|
||||
|
||||
In the caption rendering section (around line 184-189), update the unspoken word styling:
|
||||
|
||||
Replace:
|
||||
```svelte
|
||||
style="color: {word.spoken ? captionStyle.color : captionStyle.color}; opacity: {word.spoken ? 1 : 0.4};"
|
||||
```
|
||||
|
||||
With:
|
||||
```svelte
|
||||
style="color: {word.spoken
|
||||
? captionStyle.color
|
||||
: preferences.captionSettings.dimmedColorMode === 'custom'
|
||||
? preferences.captionSettings.dimmedColor
|
||||
: captionStyle.color}; opacity: {word.spoken ? 1 : (preferences.captionSettings.dimmedColorMode === 'custom' ? 1 : preferences.captionSettings.dimmedOpacity)};"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update template references to use new properties**
|
||||
|
||||
`$derived.by` still produces a reactive value, so all template references remain `captionStyle.X` (unchanged from before). Add `font-family` and `font-weight` to the inline styles on `.caption-text`:
|
||||
|
||||
Replace the `style` attribute on `.caption-text`:
|
||||
```svelte
|
||||
style="font-family: {captionStyle.fontFamily}; font-size: {captionStyle.fontSize}; font-weight: {captionStyle.fontWeight}; background: {captionStyle.background}; text-shadow: {captionStyle.textShadow};"
|
||||
```
|
||||
|
||||
And the non-karaoke span:
|
||||
```svelte
|
||||
<span style="color: {captionStyle.color};">{cue.text}</span>
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/lib/components/VideoPlayer.svelte
|
||||
git commit -m "feat(captions): use font, shadow, dimmed color, bg toggle in preview captions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Settings panel UI — add all new controls
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/lib/components/CaptionSettingsPanel.svelte`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `CaptionSettings` from Task 1, `listSystemFonts()` from Task 2
|
||||
- Produces: Full settings panel with all controls
|
||||
|
||||
- [ ] **Step 1: Rewrite the panel component**
|
||||
|
||||
Replace the entire contents of `src/lib/components/CaptionSettingsPanel.svelte` with the expanded panel. The component should:
|
||||
|
||||
1. Import `listSystemFonts` from `$lib/bindings/mediaAnalysis`
|
||||
2. Call `listSystemFonts()` on mount, store result in `let systemFonts = $state<string[]>(['Arial'])`
|
||||
3. Use `onMount` to trigger the font loading
|
||||
4. Layout the controls in this order:
|
||||
- **Font**: `<select>` dropdown with system fonts, each `<option>` styled with `font-family: {font}`. Font size slider. Bold checkbox.
|
||||
- **Text Color**: Color picker
|
||||
- **Dimmed Text**: Auto/Custom toggle (radio). When auto: opacity slider (0.1-0.9). When custom: color picker.
|
||||
- **Outline**: Checkbox + color picker (picker disabled when unchecked)
|
||||
- **Background**: Checkbox + color picker + opacity slider. When enabled, sets `shadowEnabled: false`.
|
||||
- **Shadow**: Checkbox + depth slider (1-5) + color picker. When enabled, sets `backgroundEnabled: false`.
|
||||
- **Position**: Bottom/Top radio
|
||||
- **Word Highlight**: Checkbox
|
||||
- **Reset to defaults** button
|
||||
|
||||
The panel width stays at 320px. Max-height with overflow-y scroll for smaller screens.
|
||||
|
||||
Full component code:
|
||||
|
||||
```svelte
|
||||
<script lang="ts">
|
||||
import { onMount } from 'svelte';
|
||||
import {
|
||||
preferences,
|
||||
setCaptionSettings,
|
||||
DEFAULT_CAPTION_SETTINGS,
|
||||
type CaptionSettings,
|
||||
} from '$lib/stores/preferences.svelte';
|
||||
import { listSystemFonts } from '$lib/bindings/mediaAnalysis';
|
||||
|
||||
let { onClose }: { onClose: () => void } = $props();
|
||||
|
||||
let settings = $derived(preferences.captionSettings);
|
||||
let systemFonts = $state<string[]>(['Arial']);
|
||||
|
||||
onMount(async () => {
|
||||
try {
|
||||
systemFonts = await listSystemFonts();
|
||||
} catch {
|
||||
systemFonts = ['Arial', 'Helvetica', 'Verdana', 'Georgia', 'Times New Roman', 'Courier New', 'Impact'];
|
||||
}
|
||||
});
|
||||
|
||||
function update(partial: Partial<CaptionSettings>) {
|
||||
// Enforce mutual exclusivity
|
||||
if (partial.backgroundEnabled) {
|
||||
partial.shadowEnabled = false;
|
||||
} else if (partial.shadowEnabled) {
|
||||
partial.backgroundEnabled = false;
|
||||
}
|
||||
setCaptionSettings(partial);
|
||||
}
|
||||
|
||||
function resetToDefaults() {
|
||||
setCaptionSettings({ ...DEFAULT_CAPTION_SETTINGS });
|
||||
}
|
||||
</script>
|
||||
|
||||
<div class="panel">
|
||||
<div class="panel-header">
|
||||
<h3>Caption Appearance</h3>
|
||||
<button class="close-btn" onclick={onClose} title="Close">x</button>
|
||||
</div>
|
||||
|
||||
<div class="scroll-area">
|
||||
<!-- Font Family -->
|
||||
<div class="control">
|
||||
<label>Font</label>
|
||||
<select
|
||||
value={settings.fontFamily}
|
||||
onchange={(e) => update({ fontFamily: (e.target as HTMLSelectElement).value })}
|
||||
>
|
||||
{#each systemFonts as font}
|
||||
<option value={font} style="font-family: {font}">{font}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- Font Size -->
|
||||
<div class="control">
|
||||
<label>
|
||||
Font Size
|
||||
<span class="value">{settings.fontSize}px</span>
|
||||
</label>
|
||||
<input
|
||||
type="range" min="12" max="36" step="1"
|
||||
value={settings.fontSize}
|
||||
oninput={(e) => update({ fontSize: parseInt((e.target as HTMLInputElement).value, 10) })}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<!-- Text Color + Bold -->
|
||||
<div class="control row">
|
||||
<label>Text Color</label>
|
||||
<div class="row-controls">
|
||||
<input
|
||||
type="color" value={settings.textColor}
|
||||
oninput={(e) => update({ textColor: (e.target as HTMLInputElement).value })}
|
||||
/>
|
||||
<label class="inline-check">
|
||||
<input type="checkbox" checked={settings.bold}
|
||||
onchange={(e) => update({ bold: (e.target as HTMLInputElement).checked })} />
|
||||
Bold
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Dimmed Text Color -->
|
||||
<div class="control">
|
||||
<label>Dimmed Text</label>
|
||||
<div class="radio-row">
|
||||
<label>
|
||||
<input type="radio" name="dimmed-mode" value="auto"
|
||||
checked={settings.dimmedColorMode === 'auto'}
|
||||
onchange={() => update({ dimmedColorMode: 'auto' })} />
|
||||
Auto
|
||||
</label>
|
||||
<label>
|
||||
<input type="radio" name="dimmed-mode" value="custom"
|
||||
checked={settings.dimmedColorMode === 'custom'}
|
||||
onchange={() => update({ dimmedColorMode: 'custom' })} />
|
||||
Custom
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
{#if settings.dimmedColorMode === 'auto'}
|
||||
<div class="control">
|
||||
<label>
|
||||
Dim Opacity
|
||||
<span class="value">{Math.round(settings.dimmedOpacity * 100)}%</span>
|
||||
</label>
|
||||
<input
|
||||
type="range" min="10" max="90" step="5"
|
||||
value={Math.round(settings.dimmedOpacity * 100)}
|
||||
oninput={(e) => update({ dimmedOpacity: parseInt((e.target as HTMLInputElement).value, 10) / 100 })}
|
||||
/>
|
||||
</div>
|
||||
{:else}
|
||||
<div class="control row">
|
||||
<label>Dimmed Color</label>
|
||||
<div class="row-controls">
|
||||
<input
|
||||
type="color" value={settings.dimmedColor}
|
||||
oninput={(e) => update({ dimmedColor: (e.target as HTMLInputElement).value })}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<!-- Outline -->
|
||||
<div class="control row">
|
||||
<label>
|
||||
<input type="checkbox" checked={settings.textOutline}
|
||||
onchange={(e) => update({ textOutline: (e.target as HTMLInputElement).checked })} />
|
||||
Outline
|
||||
</label>
|
||||
<div class="row-controls">
|
||||
<input
|
||||
type="color" value={settings.outlineColor}
|
||||
disabled={!settings.textOutline}
|
||||
class:disabled={!settings.textOutline}
|
||||
oninput={(e) => update({ outlineColor: (e.target as HTMLInputElement).value })}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Background (mutually exclusive with Shadow) -->
|
||||
<div class="control row">
|
||||
<label>
|
||||
<input type="checkbox" checked={settings.backgroundEnabled}
|
||||
onchange={(e) => update({ backgroundEnabled: (e.target as HTMLInputElement).checked })} />
|
||||
Background
|
||||
</label>
|
||||
<div class="row-controls">
|
||||
<input
|
||||
type="color" value={settings.backgroundColor}
|
||||
disabled={!settings.backgroundEnabled}
|
||||
class:disabled={!settings.backgroundEnabled}
|
||||
oninput={(e) => update({ backgroundColor: (e.target as HTMLInputElement).value })}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{#if settings.backgroundEnabled}
|
||||
<div class="control">
|
||||
<label>
|
||||
BG Opacity
|
||||
<span class="value">{Math.round(settings.backgroundOpacity * 100)}%</span>
|
||||
</label>
|
||||
<input
|
||||
type="range" min="0" max="100" step="5"
|
||||
value={Math.round(settings.backgroundOpacity * 100)}
|
||||
oninput={(e) => update({ backgroundOpacity: parseInt((e.target as HTMLInputElement).value, 10) / 100 })}
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<!-- Shadow (mutually exclusive with Background) -->
|
||||
<div class="control row">
|
||||
<label>
|
||||
<input type="checkbox" checked={settings.shadowEnabled}
|
||||
onchange={(e) => update({ shadowEnabled: (e.target as HTMLInputElement).checked })} />
|
||||
Drop Shadow
|
||||
</label>
|
||||
<div class="row-controls">
|
||||
<input
|
||||
type="color" value={settings.shadowColor}
|
||||
disabled={!settings.shadowEnabled}
|
||||
class:disabled={!settings.shadowEnabled}
|
||||
oninput={(e) => update({ shadowColor: (e.target as HTMLInputElement).value })}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{#if settings.shadowEnabled}
|
||||
<div class="control">
|
||||
<label>
|
||||
Shadow Depth
|
||||
<span class="value">{settings.shadowDepth}px</span>
|
||||
</label>
|
||||
<input
|
||||
type="range" min="1" max="5" step="1"
|
||||
value={settings.shadowDepth}
|
||||
oninput={(e) => update({ shadowDepth: parseInt((e.target as HTMLInputElement).value, 10) })}
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<!-- Position -->
|
||||
<div class="control">
|
||||
<label>Position</label>
|
||||
<div class="radio-row">
|
||||
<label>
|
||||
<input type="radio" name="caption-position" value="bottom"
|
||||
checked={settings.position === 'bottom'}
|
||||
onchange={() => update({ position: 'bottom' })} />
|
||||
Bottom
|
||||
</label>
|
||||
<label>
|
||||
<input type="radio" name="caption-position" value="top"
|
||||
checked={settings.position === 'top'}
|
||||
onchange={() => update({ position: 'top' })} />
|
||||
Top
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Word Highlight -->
|
||||
<div class="control">
|
||||
<label>
|
||||
<input type="checkbox" checked={settings.wordHighlight}
|
||||
onchange={(e) => update({ wordHighlight: (e.target as HTMLInputElement).checked })} />
|
||||
Word-by-word highlight
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="panel-footer">
|
||||
<button class="reset-btn" onclick={resetToDefaults}>Reset to defaults</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<style>
|
||||
.panel {
|
||||
position: absolute;
|
||||
bottom: 44px;
|
||||
right: 12px;
|
||||
background: var(--bg-primary, #1e1e2e);
|
||||
border: 1px solid var(--border, #353550);
|
||||
border-radius: 8px;
|
||||
padding: 12px 16px;
|
||||
width: 320px;
|
||||
max-height: 480px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
z-index: 20;
|
||||
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.5);
|
||||
}
|
||||
|
||||
.panel-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 8px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
h3 { margin: 0; font-size: 13px; font-weight: 600; color: var(--text-primary, #cdd6f4); }
|
||||
|
||||
.close-btn {
|
||||
background: none; border: none; color: var(--text-secondary, #6c7086);
|
||||
cursor: pointer; font-size: 14px; padding: 2px 6px; border-radius: 3px;
|
||||
}
|
||||
.close-btn:hover { background: var(--bg-tertiary, #2a2a3e); color: var(--text-primary, #cdd6f4); }
|
||||
|
||||
.scroll-area {
|
||||
overflow-y: auto;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
padding-right: 4px;
|
||||
}
|
||||
|
||||
.control { margin-bottom: 10px; }
|
||||
.control label {
|
||||
display: flex; align-items: center; gap: 6px;
|
||||
font-size: 12px; color: var(--text-secondary, #6c7086); margin-bottom: 4px;
|
||||
}
|
||||
.control.row { display: flex; align-items: center; justify-content: space-between; }
|
||||
.control.row > label { margin-bottom: 0; }
|
||||
|
||||
.row-controls { display: flex; align-items: center; gap: 8px; }
|
||||
.inline-check {
|
||||
display: flex; align-items: center; gap: 4px;
|
||||
font-size: 12px; color: var(--text-secondary, #6c7086);
|
||||
margin-bottom: 0 !important; cursor: pointer;
|
||||
}
|
||||
|
||||
.value {
|
||||
margin-left: auto; font-size: 11px;
|
||||
color: var(--text-primary, #cdd6f4); font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
select {
|
||||
width: 100%; padding: 4px 6px; font-size: 12px;
|
||||
background: var(--bg-secondary, #252535); color: var(--text-primary, #cdd6f4);
|
||||
border: 1px solid var(--border, #353550); border-radius: 4px;
|
||||
}
|
||||
|
||||
input[type='range'] { width: 100%; accent-color: #89b4fa; }
|
||||
input[type='color'] {
|
||||
width: 32px; height: 24px; padding: 0;
|
||||
border: 1px solid var(--border, #353550); border-radius: 3px;
|
||||
cursor: pointer; background: none;
|
||||
}
|
||||
input[type='color'].disabled { opacity: 0.3; cursor: not-allowed; }
|
||||
input[type='checkbox'] { accent-color: #89b4fa; }
|
||||
input[type='radio'] { accent-color: #89b4fa; }
|
||||
|
||||
.radio-row { display: flex; gap: 16px; }
|
||||
.radio-row label { font-size: 12px; color: var(--text-secondary, #6c7086); cursor: pointer; }
|
||||
|
||||
.panel-footer {
|
||||
margin-top: 8px; padding-top: 8px; flex-shrink: 0;
|
||||
border-top: 1px solid var(--border, #353550);
|
||||
}
|
||||
.reset-btn {
|
||||
font-size: 11px; color: var(--text-secondary, #6c7086);
|
||||
background: none; border: none; cursor: pointer; padding: 2px 0;
|
||||
}
|
||||
.reset-btn:hover { color: var(--text-primary, #cdd6f4); }
|
||||
</style>
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Build check**
|
||||
|
||||
Run: `npm run check 2>&1 | grep "Error:" | head -10`
|
||||
Expected: Only pre-existing errors (node:path, node:process, node:url, overload). No new errors.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/lib/components/CaptionSettingsPanel.svelte
|
||||
git commit -m "feat(captions): expand settings panel with font, shadow, dimmed color, bg toggle controls"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Build, test, and verify
|
||||
|
||||
**Files:**
|
||||
- No new changes (verification only)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: All changes from Tasks 1-5
|
||||
|
||||
- [ ] **Step 1: Full Rust build and test**
|
||||
|
||||
Run: `cd src-tauri && cargo build 2>&1 | tail -5`
|
||||
Expected: Clean build.
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | tail -10`
|
||||
Expected: All tests pass.
|
||||
|
||||
- [ ] **Step 2: Frontend type check**
|
||||
|
||||
Run: `npm run check 2>&1 | grep "Error:" | head -10`
|
||||
Expected: Only pre-existing errors. No new errors from caption changes.
|
||||
|
||||
- [ ] **Step 3: Manual smoke test**
|
||||
|
||||
Run: `cargo tauri dev`
|
||||
|
||||
Verify:
|
||||
1. Caption settings panel opens, shows font dropdown (populated with system fonts)
|
||||
2. Selecting a different font updates the preview captions
|
||||
3. Enabling shadow disables background, and vice versa
|
||||
4. Dimmed color auto mode shows opacity slider; custom mode shows color picker
|
||||
5. Export a short clip with burn-in and verify the ASS uses the selected font and style settings
|
||||
@@ -0,0 +1,604 @@
|
||||
# Paged Teleprompter Burn-In Subtitles Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the rolling per-line ASS subtitle generation with a paged teleprompter model where 2-line blocks display with continuous karaoke highlighting and cross-fade transitions.
|
||||
|
||||
**Architecture:** Rewrite `vtt_to_ass_with_karaoke()` in `clip_exporter.rs` to group `SpokenLine`s into 2-line pages, emit one ASS Dialogue per page with stitched `\k` tags across `\N` line breaks, and use `\fad` for cross-fade transitions. No `\move` or `\clip` tags. Single-file change.
|
||||
|
||||
**Tech Stack:** Rust, ASS subtitle format (libass), FFmpeg `ass=` filter
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- All changes are in `src-tauri/src/services/clip_exporter.rs` only
|
||||
- No new dependencies
|
||||
- Existing tests must continue to pass (they test FFmpeg arg construction, not ASS content)
|
||||
- ASS header/style generation stays unchanged
|
||||
- `extract_spoken_lines()` stays unchanged
|
||||
- `parse_vtt_word_timings()` stays unchanged
|
||||
- The `ass=` filter usage in `build_ffmpeg_args_burnin_subs()` is unchanged
|
||||
- Cross-fade duration: 300ms
|
||||
- Gap threshold for page splitting: 2.0 seconds
|
||||
- PlayRes: 1920×1080
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add `build_page_karaoke_text` and page-grouping logic
|
||||
|
||||
**Files:**
|
||||
- Modify: `src-tauri/src/services/clip_exporter.rs:432-460` (replace `RollingLayout`, add new function)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `SpokenLine` struct (unchanged, line 349), `parse_vtt_word_timings()` (unchanged, line 259), `strip_vtt_tags()` (unchanged, line 170)
|
||||
- Produces: `fn build_page_karaoke_text(lines: &[&SpokenLine]) -> String` — returns ASS text with `\k` tags and `\N` separator for 1-2 line pages. `fn group_into_pages(spoken_lines: &[SpokenLine], gap_threshold: f64) -> Vec<Vec<usize>>` — returns groups of indices into spoken_lines.
|
||||
|
||||
- [ ] **Step 1: Write failing tests for `build_page_karaoke_text`**
|
||||
|
||||
Add these tests to the existing `#[cfg(test)] mod tests` block at line 1155:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_build_page_karaoke_single_line_with_karaoke() {
|
||||
let line = SpokenLine {
|
||||
plain_text: "hello world".to_string(),
|
||||
raw_text: "hello<00:00:01.000><c> world</c>".to_string(),
|
||||
start_time: 0.5,
|
||||
end_time: 1.5,
|
||||
has_karaoke: true,
|
||||
is_non_speech: false,
|
||||
};
|
||||
let result = build_page_karaoke_text(&[&line]);
|
||||
// "hello" starts at 0.5, "world" starts at 1.0, ends at 1.5
|
||||
// hello duration = 1.0 - 0.5 = 0.5s = 50cs
|
||||
// world duration = 1.5 - 1.0 = 0.5s = 50cs
|
||||
assert!(result.contains("{\\k50}hello"));
|
||||
assert!(result.contains("{\\k50}world"));
|
||||
assert!(!result.contains("\\N"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_build_page_karaoke_two_lines_stitched() {
|
||||
let line1 = SpokenLine {
|
||||
plain_text: "hello world".to_string(),
|
||||
raw_text: "hello<00:00:01.000><c> world</c>".to_string(),
|
||||
start_time: 0.5,
|
||||
end_time: 1.5,
|
||||
has_karaoke: true,
|
||||
is_non_speech: false,
|
||||
};
|
||||
let line2 = SpokenLine {
|
||||
plain_text: "foo bar".to_string(),
|
||||
raw_text: "foo<00:00:02.500><c> bar</c>".to_string(),
|
||||
start_time: 2.0,
|
||||
end_time: 3.0,
|
||||
has_karaoke: true,
|
||||
is_non_speech: false,
|
||||
};
|
||||
let result = build_page_karaoke_text(&[&line1, &line2]);
|
||||
// Last word of line1 ("world") should span from 1.0 to line2 first word (2.0) = 100cs
|
||||
assert!(result.contains("\\N"));
|
||||
assert!(result.contains("{\\k100}world"));
|
||||
// line2: "foo" at 2.0, "bar" at 2.5, end at 3.0
|
||||
assert!(result.contains("{\\k50}foo"));
|
||||
assert!(result.contains("{\\k50}bar"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_build_page_karaoke_no_karaoke_tags() {
|
||||
let line = SpokenLine {
|
||||
plain_text: "just plain text".to_string(),
|
||||
raw_text: "just plain text".to_string(),
|
||||
start_time: 1.0,
|
||||
end_time: 3.0,
|
||||
has_karaoke: false,
|
||||
is_non_speech: false,
|
||||
};
|
||||
let result = build_page_karaoke_text(&[&line]);
|
||||
// Single \k covering the full 2.0s = 200cs
|
||||
assert!(result.contains("{\\k200}just plain text"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_group_into_pages_even() {
|
||||
let lines = vec![
|
||||
SpokenLine { plain_text: "a".into(), raw_text: "a".into(), start_time: 0.0, end_time: 1.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "b".into(), raw_text: "b".into(), start_time: 1.0, end_time: 2.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "c".into(), raw_text: "c".into(), start_time: 2.0, end_time: 3.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "d".into(), raw_text: "d".into(), start_time: 3.0, end_time: 4.0, has_karaoke: false, is_non_speech: false },
|
||||
];
|
||||
let pages = group_into_pages(&lines, 2.0);
|
||||
assert_eq!(pages.len(), 2);
|
||||
assert_eq!(pages[0], vec![0, 1]);
|
||||
assert_eq!(pages[1], vec![2, 3]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_group_into_pages_gap_splits() {
|
||||
let lines = vec![
|
||||
SpokenLine { plain_text: "a".into(), raw_text: "a".into(), start_time: 0.0, end_time: 1.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "b".into(), raw_text: "b".into(), start_time: 5.0, end_time: 6.0, has_karaoke: false, is_non_speech: false },
|
||||
];
|
||||
// Gap between a (end=1.0) and b (start=5.0) is 4.0s > 2.0s threshold
|
||||
let pages = group_into_pages(&lines, 2.0);
|
||||
assert_eq!(pages.len(), 2);
|
||||
assert_eq!(pages[0], vec![0]);
|
||||
assert_eq!(pages[1], vec![1]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_group_into_pages_odd_count() {
|
||||
let lines = vec![
|
||||
SpokenLine { plain_text: "a".into(), raw_text: "a".into(), start_time: 0.0, end_time: 1.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "b".into(), raw_text: "b".into(), start_time: 1.0, end_time: 2.0, has_karaoke: false, is_non_speech: false },
|
||||
SpokenLine { plain_text: "c".into(), raw_text: "c".into(), start_time: 2.0, end_time: 3.0, has_karaoke: false, is_non_speech: false },
|
||||
];
|
||||
let pages = group_into_pages(&lines, 2.0);
|
||||
assert_eq!(pages.len(), 2);
|
||||
assert_eq!(pages[0], vec![0, 1]);
|
||||
assert_eq!(pages[1], vec![2]);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | head -40`
|
||||
Expected: Compilation errors — `build_page_karaoke_text` and `group_into_pages` not found.
|
||||
|
||||
- [ ] **Step 3: Implement `group_into_pages`**
|
||||
|
||||
Add this function after `build_karaoke_text` (around line 450), replacing the `RollingLayout` struct (lines 452-458):
|
||||
|
||||
```rust
|
||||
/// Group spoken lines into 2-line pages for the teleprompter display.
|
||||
/// If the gap between two consecutive lines exceeds `gap_threshold` seconds,
|
||||
/// the pair is split into separate single-line pages.
|
||||
/// Non-speech lines (e.g., [Music]) always get their own page.
|
||||
fn group_into_pages(spoken_lines: &[SpokenLine], gap_threshold: f64) -> Vec<Vec<usize>> {
|
||||
let mut pages: Vec<Vec<usize>> = Vec::new();
|
||||
let mut i = 0;
|
||||
|
||||
while i < spoken_lines.len() {
|
||||
let line = &spoken_lines[i];
|
||||
|
||||
// Non-speech cues always get their own page
|
||||
if line.is_non_speech {
|
||||
pages.push(vec![i]);
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Try to pair with the next line
|
||||
if i + 1 < spoken_lines.len() {
|
||||
let next = &spoken_lines[i + 1];
|
||||
let gap = next.start_time - line.end_time;
|
||||
|
||||
// Pair them if gap is small enough and next isn't non-speech
|
||||
if gap <= gap_threshold && !next.is_non_speech {
|
||||
pages.push(vec![i, i + 1]);
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// Solo page (last line, or gap too large, or next is non-speech)
|
||||
pages.push(vec![i]);
|
||||
i += 1;
|
||||
}
|
||||
|
||||
pages
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Implement `build_page_karaoke_text`**
|
||||
|
||||
Add this function right after `group_into_pages`:
|
||||
|
||||
```rust
|
||||
/// Build karaoke text for a 1-or-2-line page, stitching word timings
|
||||
/// across lines with \N as the visual line break. The \k durations flow
|
||||
/// continuously so karaoke highlighting progresses top-to-bottom.
|
||||
fn build_page_karaoke_text(lines: &[&SpokenLine]) -> String {
|
||||
// Collect all (word, absolute_start_time) across all lines in page order.
|
||||
// For the cross-line stitch, the last word of line N extends to the first
|
||||
// word of line N+1.
|
||||
struct WordEntry {
|
||||
text: String,
|
||||
start: f64,
|
||||
is_line_break_before: bool, // insert \N before this word
|
||||
}
|
||||
|
||||
let mut entries: Vec<WordEntry> = Vec::new();
|
||||
|
||||
for (line_idx, line) in lines.iter().enumerate() {
|
||||
let is_new_line = line_idx > 0;
|
||||
|
||||
if line.has_karaoke {
|
||||
let words = parse_vtt_word_timings(&line.raw_text, line.start_time);
|
||||
if words.len() >= 2 {
|
||||
for (w_idx, (word, start)) in words.iter().enumerate() {
|
||||
entries.push(WordEntry {
|
||||
text: word.clone(),
|
||||
start: *start,
|
||||
is_line_break_before: is_new_line && w_idx == 0,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
// Fallback: treat entire line as one word
|
||||
entries.push(WordEntry {
|
||||
text: strip_vtt_tags(&line.raw_text),
|
||||
start: line.start_time,
|
||||
is_line_break_before: is_new_line,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
// No karaoke data — one entry for the whole line
|
||||
entries.push(WordEntry {
|
||||
text: line.plain_text.clone(),
|
||||
start: line.start_time,
|
||||
is_line_break_before: is_new_line,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if entries.is_empty() {
|
||||
return String::new();
|
||||
}
|
||||
|
||||
// The page's end time is the last line's end_time
|
||||
let page_end = lines.last().unwrap().end_time;
|
||||
|
||||
// Build the output with \k tags
|
||||
let mut parts: Vec<String> = Vec::new();
|
||||
for (i, entry) in entries.iter().enumerate() {
|
||||
let next_start = if i + 1 < entries.len() {
|
||||
entries[i + 1].start
|
||||
} else {
|
||||
page_end
|
||||
};
|
||||
let duration_cs = ((next_start - entry.start) * 100.0).round().max(1.0) as u64;
|
||||
|
||||
let prefix = if entry.is_line_break_before { "\\N" } else if i > 0 { " " } else { "" };
|
||||
parts.push(format!("{}{{\\k{}}}{}", prefix, duration_cs, entry.text));
|
||||
}
|
||||
|
||||
parts.join("")
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | tail -20`
|
||||
Expected: All new tests pass. All existing tests still pass.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/src/services/clip_exporter.rs
|
||||
git commit -m "feat(subtitles): add page grouping and cross-line karaoke stitching"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Rewrite `vtt_to_ass_with_karaoke` to use the paged model
|
||||
|
||||
**Files:**
|
||||
- Modify: `src-tauri/src/services/clip_exporter.rs:465-643` (the `vtt_to_ass_with_karaoke` function body)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `build_page_karaoke_text()` and `group_into_pages()` from Task 1, `extract_spoken_lines()` (unchanged, line 362), `format_ass_timestamp()` (unchanged, line 244), style helper functions (unchanged)
|
||||
- Produces: Same public signature `pub fn vtt_to_ass_with_karaoke(caption_path, temp_dir, style) -> Result<String, String>` — writes ASS file to temp_dir and returns its path. Existing callers are unchanged.
|
||||
|
||||
- [ ] **Step 1: Write a test for the full ASS generation**
|
||||
|
||||
Add to the test module:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_vtt_to_ass_paged_output() {
|
||||
let vtt_content = "\
|
||||
WEBVTT
|
||||
|
||||
00:00:00.500 --> 00:00:01.500
|
||||
hello<00:00:01.000><c> world</c>
|
||||
|
||||
00:00:02.000 --> 00:00:03.000
|
||||
foo<00:00:02.500><c> bar</c>
|
||||
|
||||
00:00:03.000 --> 00:00:04.000
|
||||
baz<00:00:03.500><c> qux</c>
|
||||
|
||||
00:00:04.000 --> 00:00:05.000
|
||||
last<00:00:04.500><c> line</c>
|
||||
";
|
||||
let temp = std::env::temp_dir().join("test-paged-ass");
|
||||
let _ = std::fs::remove_dir_all(&temp);
|
||||
std::fs::create_dir_all(&temp).unwrap();
|
||||
|
||||
let vtt_path = temp.join("test.vtt");
|
||||
std::fs::write(&vtt_path, vtt_content).unwrap();
|
||||
|
||||
let result = vtt_to_ass_with_karaoke(
|
||||
vtt_path.to_str().unwrap(),
|
||||
temp.to_str().unwrap(),
|
||||
None,
|
||||
);
|
||||
assert!(result.is_ok());
|
||||
|
||||
let ass_path = result.unwrap();
|
||||
let ass_content = std::fs::read_to_string(&ass_path).unwrap();
|
||||
|
||||
// Should have ASS header
|
||||
assert!(ass_content.contains("[Script Info]"));
|
||||
assert!(ass_content.contains("PlayResX: 1920"));
|
||||
assert!(ass_content.contains("[Events]"));
|
||||
|
||||
// Should use \an2\pos (static position), NOT \move
|
||||
assert!(ass_content.contains("\\an2\\pos("));
|
||||
assert!(!ass_content.contains("\\move("));
|
||||
|
||||
// Should have \N line breaks (paged 2-line blocks)
|
||||
assert!(ass_content.contains("\\N"));
|
||||
|
||||
// Should have \fad for cross-fade transitions
|
||||
assert!(ass_content.contains("\\fad("));
|
||||
|
||||
// Should have karaoke \k tags
|
||||
assert!(ass_content.contains("\\k"));
|
||||
|
||||
// 4 lines → 2 pages → 2 Dialogue events (plus possible non-speech)
|
||||
let dialogue_count = ass_content.matches("Dialogue:").count();
|
||||
assert_eq!(dialogue_count, 2, "Expected 2 pages (4 lines / 2). Got {dialogue_count}.\nASS:\n{ass_content}");
|
||||
|
||||
let _ = std::fs::remove_dir_all(&temp);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_vtt_to_ass_paged_gap_splits_page() {
|
||||
let vtt_content = "\
|
||||
WEBVTT
|
||||
|
||||
00:00:00.500 --> 00:00:01.500
|
||||
hello<00:00:01.000><c> world</c>
|
||||
|
||||
00:00:05.000 --> 00:00:06.000
|
||||
far<00:00:05.500><c> away</c>
|
||||
";
|
||||
let temp = std::env::temp_dir().join("test-paged-gap-ass");
|
||||
let _ = std::fs::remove_dir_all(&temp);
|
||||
std::fs::create_dir_all(&temp).unwrap();
|
||||
|
||||
let vtt_path = temp.join("test.vtt");
|
||||
std::fs::write(&vtt_path, vtt_content).unwrap();
|
||||
|
||||
let result = vtt_to_ass_with_karaoke(
|
||||
vtt_path.to_str().unwrap(),
|
||||
temp.to_str().unwrap(),
|
||||
None,
|
||||
);
|
||||
assert!(result.is_ok());
|
||||
|
||||
let ass_content = std::fs::read_to_string(result.unwrap()).unwrap();
|
||||
|
||||
// Gap between lines is 3.5s > 2.0s threshold → should split into 2 single-line pages
|
||||
let dialogue_count = ass_content.matches("Dialogue:").count();
|
||||
assert_eq!(dialogue_count, 2, "Gap >2s should split into separate pages. Got {dialogue_count}.\nASS:\n{ass_content}");
|
||||
|
||||
// Single-line pages should NOT have \N
|
||||
// (Each Dialogue has its own text without \N)
|
||||
for line in ass_content.lines() {
|
||||
if line.starts_with("Dialogue:") {
|
||||
assert!(!line.contains("\\N"), "Single-line page should not contain \\N: {line}");
|
||||
}
|
||||
}
|
||||
|
||||
let _ = std::fs::remove_dir_all(&temp);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify the new ones fail**
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests::test_vtt_to_ass_paged -- --nocapture 2>&1`
|
||||
Expected: Failures — the current implementation uses `\move` and generates more Dialogue events per line.
|
||||
|
||||
- [ ] **Step 3: Rewrite `vtt_to_ass_with_karaoke`**
|
||||
|
||||
Replace the entire function body (lines 465-643) with the paged implementation. The function signature stays the same:
|
||||
|
||||
```rust
|
||||
/// Convert a word-timed VTT file to an ASS file with paged teleprompter display.
|
||||
/// Lines are grouped into 2-line pages. Karaoke \k tags flow continuously from
|
||||
/// line 1 through line 2 within each page. Pages cross-fade with \fad transitions.
|
||||
pub fn vtt_to_ass_with_karaoke(
|
||||
caption_path: &str,
|
||||
temp_dir: &str,
|
||||
style: Option<&CaptionStyle>,
|
||||
) -> Result<String, String> {
|
||||
let content = std::fs::read_to_string(caption_path)
|
||||
.map_err(|e| format!("Failed to read VTT file '{}': {e}", caption_path))?;
|
||||
|
||||
std::fs::create_dir_all(temp_dir)
|
||||
.map_err(|e| format!("Failed to create temp dir for ASS: {e}"))?;
|
||||
|
||||
let out_path = Path::new(temp_dir).join("karaoke.ass");
|
||||
|
||||
// Build style values from CaptionStyle or use defaults
|
||||
let font_size = style.map_or(45, |s| (s.font_size as f64 * 2.5).round() as u32);
|
||||
let primary_colour = style.map_or("&H00FFFFFF".to_string(), |s| hex_to_ass_color(&s.text_color));
|
||||
let secondary_colour = "&H73CCCCCC".to_string();
|
||||
let outline_colour = "&H00000000".to_string();
|
||||
let back_colour = style.map_or("&H80000000".to_string(), |s| opacity_to_ass_back_colour(s.background_opacity));
|
||||
let text_outline = style.map_or(true, |s| s.text_outline);
|
||||
let (border_style, outline_val, shadow_val) = if text_outline {
|
||||
(4, 2, 0)
|
||||
} else {
|
||||
(3, 0, 0)
|
||||
};
|
||||
let margin_v: i32 = 40;
|
||||
|
||||
let mut output = String::new();
|
||||
|
||||
// ASS Script Info header
|
||||
output.push_str("[Script Info]\n");
|
||||
output.push_str("ScriptType: v4.00+\n");
|
||||
output.push_str("PlayResX: 1920\n");
|
||||
output.push_str("PlayResY: 1080\n");
|
||||
output.push_str("WrapStyle: 0\n");
|
||||
output.push_str("ScaledBorderAndShadow: yes\n");
|
||||
output.push('\n');
|
||||
|
||||
// V4+ Styles
|
||||
output.push_str("[V4+ Styles]\n");
|
||||
output.push_str("Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding\n");
|
||||
output.push_str(&format!(
|
||||
"Style: Default,Arial,{},{},{},{},{},0,0,0,0,100,100,0,0,{},{},{},2,20,20,{},1\n",
|
||||
font_size, primary_colour, secondary_colour, outline_colour, back_colour,
|
||||
border_style, outline_val, shadow_val, margin_v
|
||||
));
|
||||
output.push('\n');
|
||||
|
||||
// Events
|
||||
output.push_str("[Events]\n");
|
||||
output.push_str("Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n");
|
||||
|
||||
let spoken_lines = extract_spoken_lines(&content);
|
||||
let gap_threshold = 2.0;
|
||||
let crossfade_ms = 300;
|
||||
let crossfade_s = crossfade_ms as f64 / 1000.0;
|
||||
|
||||
let pages = group_into_pages(&spoken_lines, gap_threshold);
|
||||
let page_count = pages.len();
|
||||
|
||||
let cx = 960;
|
||||
let y_bottom = 1080 - margin_v;
|
||||
|
||||
for (page_idx, page_indices) in pages.iter().enumerate() {
|
||||
let page_lines: Vec<&SpokenLine> = page_indices.iter().map(|&i| &spoken_lines[i]).collect();
|
||||
let first_line = page_lines[0];
|
||||
let last_line = *page_lines.last().unwrap();
|
||||
|
||||
// Non-speech cue: simple static display, no karaoke, no cross-fade
|
||||
if page_lines.len() == 1 && first_line.is_non_speech {
|
||||
output.push_str(&format!(
|
||||
"Dialogue: 0,{},{},Default,,0,0,0,,{{\\an2\\pos({},{})}}{}\n",
|
||||
format_ass_timestamp(first_line.start_time),
|
||||
format_ass_timestamp(first_line.end_time),
|
||||
cx, y_bottom,
|
||||
first_line.plain_text
|
||||
));
|
||||
continue;
|
||||
}
|
||||
|
||||
// Compute display timing with cross-fade overlap
|
||||
let natural_start = first_line.start_time;
|
||||
let natural_end = last_line.end_time;
|
||||
|
||||
let is_first = page_idx == 0;
|
||||
let is_last = page_idx == page_count - 1;
|
||||
|
||||
let display_start = if is_first {
|
||||
natural_start
|
||||
} else {
|
||||
(natural_start - crossfade_s).max(0.0)
|
||||
};
|
||||
let display_end = if is_last {
|
||||
natural_end
|
||||
} else {
|
||||
natural_end + crossfade_s
|
||||
};
|
||||
|
||||
let fade_in = if is_first { 0 } else { crossfade_ms };
|
||||
let fade_out = if is_last { 0 } else { crossfade_ms };
|
||||
|
||||
let karaoke_text = build_page_karaoke_text(&page_lines);
|
||||
|
||||
output.push_str(&format!(
|
||||
"Dialogue: 0,{},{},Default,,0,0,0,,{{\\an2\\pos({},{})\\fad({},{})}}{}\n",
|
||||
format_ass_timestamp(display_start),
|
||||
format_ass_timestamp(display_end),
|
||||
cx, y_bottom,
|
||||
fade_in, fade_out,
|
||||
karaoke_text
|
||||
));
|
||||
}
|
||||
|
||||
std::fs::write(&out_path, &output)
|
||||
.map_err(|e| format!("Failed to write ASS file: {e}"))?;
|
||||
|
||||
Ok(out_path.to_string_lossy().to_string())
|
||||
}
|
||||
```
|
||||
|
||||
Also remove the now-unused `RollingLayout` struct (around line 452-458). The old `build_karaoke_text` function (line 432-449) can remain — it's not called by the new code but doesn't hurt and could be useful for future single-line scenarios.
|
||||
|
||||
- [ ] **Step 4: Run all tests**
|
||||
|
||||
Run: `cd src-tauri && cargo test --lib services::clip_exporter::tests -- --nocapture 2>&1 | tail -30`
|
||||
Expected: All tests pass — both new paged tests and all existing tests.
|
||||
|
||||
- [ ] **Step 5: Build the full project**
|
||||
|
||||
Run: `cd src-tauri && cargo build 2>&1 | tail -10`
|
||||
Expected: Clean build, no warnings about unused code (other than pre-existing ones).
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src-tauri/src/services/clip_exporter.rs
|
||||
git commit -m "feat(subtitles): rewrite ASS generation to paged teleprompter model
|
||||
|
||||
Replace per-line 3-phase rolling animation with 2-line paged blocks.
|
||||
Karaoke \\k tags flow continuously across \\N line breaks within each page.
|
||||
Pages cross-fade with \\fad transitions (300ms). No \\move or \\clip tags.
|
||||
Reading flow is top-to-bottom within each page, matching teleprompter style."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Manual smoke test and cleanup
|
||||
|
||||
**Files:**
|
||||
- Modify: `src-tauri/src/services/clip_exporter.rs` (only if issues found)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: The full paged ASS generation pipeline from Tasks 1-2
|
||||
- Produces: Verified working burn-in subtitle export
|
||||
|
||||
- [ ] **Step 1: Run the app and test export**
|
||||
|
||||
Run: `cargo tauri dev`
|
||||
|
||||
Test flow:
|
||||
1. Paste a YouTube URL with captions (e.g., `https://youtu.be/NruccMk0Jls`)
|
||||
2. Wait for processing to complete
|
||||
3. Mark a short clip (~10-15 seconds) containing speech
|
||||
4. Open export dialog, enable captions with burn-in
|
||||
5. Export the clip
|
||||
6. Open the exported file and verify:
|
||||
- Subtitles show as 2-line blocks
|
||||
- Karaoke highlighting progresses top line → bottom line
|
||||
- Page transitions cross-fade smoothly (no hard cuts or jumps)
|
||||
- No position jumping or "2-line block scroll"
|
||||
|
||||
- [ ] **Step 2: Inspect the generated ASS file**
|
||||
|
||||
Check the temp ASS file to verify structure:
|
||||
|
||||
Run: `cat /tmp/video-clipper*/karaoke.ass | head -40`
|
||||
|
||||
Verify:
|
||||
- Each `Dialogue:` line contains `\an2\pos(` (static position)
|
||||
- No `\move(` tags anywhere
|
||||
- Multi-line pages contain `\N` separator
|
||||
- `\fad(` tags present on each Dialogue
|
||||
- `\k` tags flow through both lines of each page
|
||||
|
||||
- [ ] **Step 3: Remove dead code if any**
|
||||
|
||||
If `RollingLayout` struct is still present, remove it. If `build_karaoke_text` is unused and triggers a warning, add `#[allow(dead_code)]` or remove it. Run `cargo build` to confirm clean.
|
||||
|
||||
- [ ] **Step 4: Final commit if cleanup was needed**
|
||||
|
||||
```bash
|
||||
git add src-tauri/src/services/clip_exporter.rs
|
||||
git commit -m "chore: remove dead rolling layout code"
|
||||
```
|
||||
Reference in New Issue
Block a user