Add design spec: Immich photo frame for M5Stack PaperColor

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-03 10:27:08 -04:00
commit ac5ab960b1
1 changed files with 354 additions and 0 deletions
--- a/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md
+++ b/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md
@@ -0,0 +1,354 @@
+# Immich Photo Frame — M5Stack PaperColor Firmware Design
+
+**Date**: 2026-08-03  
+**Status**: Approved  
+**Platform**: M5Stack PaperColor (ESP32-S3R8, 4" E Ink Spectra 6)
+
+## Overview
+
+Custom firmware that turns the M5Stack PaperColor into a self-contained photo frame connected to a self-hosted Immich instance. The device fetches photos over WiFi, dithers them to the 6-color e-ink palette on-device, and displays them in a configurable slideshow. A web-based management UI served from the device allows album selection, slideshow settings, and OTA firmware updates.
+
+## Hardware Summary
+
+| Component | Detail |
+|-----------|--------|
+| SoC | ESP32-S3R8 (dual-core LX7 @ 240MHz) |
+| Flash | 16MB |
+| PSRAM | 8MB |
+| Display | 4" E Ink Spectra 6, 600×400 (landscape orientation), 6 colors |
+| Battery | 1250mAh |
+| Power IC | M5PM1 (I2C 0x6E) |
+| Buttons | BTN_TOP (G1), BTN_UP (G9), BTN_DOWN (G10), Power (side) |
+| LEDs | 2× RGB NeoPixel (G21) |
+| WiFi | 2.4GHz |
+| Storage | microSD slot (unused in v1), NVS, LittleFS |
+| Audio | ES8311 + ES7210 + speaker — **permanently disabled** |
+
+## Architecture
+
+### FreeRTOS Task Layout
+
+```
+┌─────────────────────────────────────────────────────┐
+│                   FreeRTOS Tasks                      │
+├──────────────┬──────────────┬───────────────────────┤
+│ Display Task │ Web Server   │ Battery Monitor Task  │
+│ (Core 1)     │ Task (Core 0)│ (Core 0, low priority)│
+├──────────────┴──────────────┴───────────────────────┤
+│              Shared State (mutex-protected)           │
+│  - Settings (NVS)                                    │
+│  - Photo queue / cursor                              │
+│  - Battery level                                     │
+│  - Slideshow state (playing/paused)                  │
+├──────────────────────────────────────────────────────┤
+│              Services Layer                           │
+│  - ImmichClient (HTTPS + API key)                    │
+│  - ImagePipeline (decode → dither → framebuffer)     │
+│  - PowerManager (M5PM1 interface, sleep control)     │
+│  - WiFiManager (AP mode ↔ Station mode)              │
+├──────────────────────────────────────────────────────┤
+│              Hardware Abstraction                     │
+│  - M5Unified / M5GFX (display, buttons, LEDs)       │
+│  - M5PM1 (power management IC)                      │
+│  - NVS / LittleFS (persistent storage)              │
+└─────────────────────────────────────────────────────┘
+```
+
+- **Core 1 (app core)**: Display task handles JPEG decode + dithering (CPU-intensive).
+- **Core 0 (protocol core)**: Web server, WiFi stack, battery monitor (I/O-bound).
+- **Automatic light sleep**: `esp_pm_configure()` with `light_sleep_enable = true`. CPU sleeps at ~5-15mA between task activity. WiFi stays associated via DTIM wake intervals.
+
+### Power States
+
+| State | What's Active | Current | When |
+|-------|---------------|---------|------|
+| Active (refresh) | CPU full, WiFi, PSRAM, e-ink | ~200mA | 15-25s per photo |
+| Idle (light sleep) | WiFi associated, web server listening | ~5-15mA | Between refreshes |
+| Deep sleep | RTC + power button wake only | ~93µA | User-triggered off mode |
+
+## Connectivity
+
+### WiFi
+
+- Station mode for normal operation
+- AP mode (`PaperColor-Setup`) for initial provisioning or when station fails 3× on boot
+- mDNS: device advertises as `papercolor.local`
+- WiFi power save: `WIFI_PS_MIN_MODEM`
+
+### Immich Integration
+
+**Base URL**: Configurable (default `https://photos.example.com`)  
+**Auth**: API key in `x-api-key` header, stored in NVS
+
+**API Endpoints Used:**
+
+| Purpose | Endpoint |
+|---------|----------|
+| List albums | `GET /api/albums` |
+| Album assets | `GET /api/albums/{id}` |
+| Timeline (all photos) | `GET /api/timeline/buckets` + `GET /api/timeline/bucket` |
+| Asset metadata | `GET /api/assets/{id}` |
+| Download photo | `GET /api/assets/{id}/thumbnail?size=preview` or `GET /api/assets/{id}/original` |
+| Favorites | `GET /api/assets?isFavorite=true` |
+
+**Photo Queue:**
+1. Build shuffled/sorted list of asset IDs from selected albums
+2. Maintain cursor position (persisted in NVS)
+3. Re-sync queue every 24 hours (or on manual refresh from web UI) to pick up new photos added to Immich
+4. Portrait detection via EXIF orientation or aspect ratio check (height > width = portrait)
+
+**Cycling Modes:**
+- Random: shuffled queue, no repeats until all shown
+- Chronological: sorted by `dateTimeOriginal` ascending
+- Reverse-chronological: descending
+- Favorites-weighted: favorites appear 3× more often in shuffle
+
+## Image Pipeline
+
+```
+JPEG Download → JPEG Decode → Resize/Crop → 6-Color Dither → Framebuffer → E-ink Refresh
+```
+
+### Stages
+
+1. **Download**: HTTP(S) stream into PSRAM buffer. Preview ~200-400KB, original up to ~15MB.
+2. **Decode**: TJPGD (ESP32 built-in) or JPEGDecoder library → RGB888. For originals larger than needed, decode at reduced scale (1/2 or 1/4).
+3. **Resize/Crop**:
+   - Landscape photos: scale to fill 600×400, center-crop overflow
+   - Portrait photos: look ahead up to 5 items in the queue for another portrait to pair with. If found, scale each to fill ~267×400, place side-by-side with 8px gap. If no pair found within lookahead, display single portrait centered with equal margins on each side.
+4. **Dither**: Floyd-Steinberg error-diffusion against the Spectra 6 palette. Initial RGB values based on typical Spectra 6 measurements (will be fine-tuned with test images on the actual panel):
+   - Black (0, 0, 0)
+   - White (255, 255, 255)
+   - Red (200, 30, 30)
+   - Green (30, 160, 30)
+   - Blue (30, 30, 200)
+   - Yellow (220, 200, 30)
+5. **Display**: Write dithered buffer via M5GFX, trigger EPD refresh (10-20s).
+
+### Memory Budget (PSRAM)
+
+| Buffer | Size |
+|--------|------|
+| JPEG download | ~400KB (preview) / 15MB (original) |
+| Decoded RGB888 | 720KB (600×400×3) |
+| Dither working | In-place (reuses decode buffer) |
+| Framebuffer | ~120KB |
+| **Total typical** | **~1.5-2MB** |
+
+### Image Quality Setting
+
+Configurable via web UI:
+- **Preview** (default): Immich's preview thumbnail (~1080p). Fast download, adequate for 600×400.
+- **Original**: Full-resolution source. Better dithering input but larger download and decode time.
+
+## Display Behavior
+
+- **Orientation**: Landscape (600×400) — device physically rotated
+- **Landscape photos**: Fill-crop to 600×400
+- **Portrait photos**: Paired side-by-side (two ~267×400 panels within the 600×400 frame)
+- **Metadata overlay** (off by default, each toggleable):
+  - Date taken
+  - Location / city
+  - People names (Immich face detection)
+  - Album name
+  - Camera / lens info
+- **Metadata position**: Bottom overlay or top overlay (configurable)
+
+## Slideshow Settings
+
+| Setting | Options | Default |
+|---------|---------|---------|
+| Interval | 1m, 5m, 15m, 30m, 60m | 5m |
+| Cycling mode | Random, Chrono, Reverse-chrono, Favorites-weighted | Random |
+| Image quality | Preview, Original | Preview |
+| Metadata | Bitmask of fields | All off (0x00) |
+
+## Button Mapping
+
+| Button | GPIO | Action |
+|--------|------|--------|
+| BTN_TOP | G1 | Random photo (jump to random position in queue) |
+| BTN_UP | G9 | Next photo (advance queue) |
+| BTN_DOWN | G10 | Play/Pause toggle |
+
+**Combos:**
+- BTN_TOP + BTN_DOWN held 3s: Enter deep sleep
+- BTN_UP held 5s: Factory reset (clear WiFi credentials, reboot to AP mode)
+
+**Feedback:**
+- Brief white RGB LED flash (100ms) on any button press to confirm input
+- Buttons configured as GPIO interrupts with 50ms debounce
+- Interrupts wake CPU from light sleep
+- Events queued via FreeRTOS queue, consumed by display task
+
+## Power Management
+
+### Audio (Permanently Disabled)
+
+- GPIO 45 (AUDIO_PWR_EN): held LOW at boot — audio codecs never powered
+- GPIO 46 (SPK_EN): held LOW at boot — speaker amp never enabled
+
+### M5PM1 Power Rail Control
+
+| Rail | Control | State |
+|------|---------|-------|
+| E-paper (PY_EPD_EN) | Enable during refresh only | Off between refreshes |
+| RGB LED power | Enable for battery warnings | Off otherwise |
+| Grove port (BOOST5V_EN_PP) | Disabled | Not used |
+| Audio codec (CODEC_3V3_L3B) | Disabled | Never powered |
+
+### Battery Monitoring
+
+- Read battery level from M5PM1 via I2C every 60 seconds
+- LED behavior (2× RGB NeoPixel on G21):
+  - \> 25%: LEDs off
+  - 10-25%: Orange pulse every 5 minutes
+  - ≤ 10%: Red pulse every 2 minutes
+  - Charging detected: Green pulse every 5 minutes
+  - < 5%: Auto-enter deep sleep to protect battery
+
+### Deep Sleep
+
+- **Trigger**: Web UI toggle, or BTN_TOP + BTN_DOWN held 3s
+- **Wake source**: Power button only
+- **On wake**: Full reboot, restore state from NVS
+
+## Web Management UI
+
+### Provisioning (AP Mode)
+
+1. First boot (no WiFi creds) → create AP `PaperColor-Setup`
+2. Captive portal serves setup wizard:
+   - Scan and select WiFi network + enter password
+   - Immich URL (pre-filled `https://photos.example.com`)
+   - Immich API key
+3. Save to NVS → reboot to station mode
+4. Station connection fails 3× on boot → fallback to AP mode
+
+### LAN Web UI
+
+Served by ESPAsyncWebServer on port 80 at `http://papercolor.local`
+
+**Pages:**
+
+| Page | Content |
+|------|---------|
+| Dashboard | Current photo (low-res preview), battery %, WiFi RSSI, uptime, next refresh timer |
+| Albums | Immich album list with checkboxes, "All photos" toggle, asset counts |
+| Slideshow | Interval presets, cycling mode, play/pause |
+| Display | Image quality, metadata toggles, metadata position |
+| Device | WiFi settings, Immich URL/key, deep sleep toggle, LED brightness, reboot |
+| Firmware | Current version display, OTA binary upload form |
+
+**Tech stack:**
+- Vanilla JS or Preact (tiny bundle), compiled/minified at build time
+- Static files served from LittleFS partition
+- Device exposes REST API for all settings and actions
+
+### REST API
+
+```
+GET  /api/status          → { battery, wifi_rssi, uptime, current_photo, state }
+GET  /api/albums          → cached Immich album list
+POST /api/albums/select   → { album_ids: [...] }
+GET  /api/settings        → all device settings
+POST /api/settings        → update settings
+POST /api/action/next     → trigger next photo
+POST /api/action/random   → trigger random photo
+POST /api/action/pause    → pause slideshow
+POST /api/action/play     → resume slideshow
+POST /api/action/sleep    → enter deep sleep
+POST /api/firmware        → OTA binary upload
+```
+
+## OTA Updates
+
+- Dual app partitions (app0 / app1) for safe rollback
+- Upload via web UI (`POST /api/firmware`)
+- Write to inactive partition → verify checksum → switch boot partition → reboot
+- If new firmware fails to boot (watchdog), ESP32 rolls back automatically
+
+## Storage
+
+### Flash Partition Table (16MB)
+
+| Partition | Type | Size |
+|-----------|------|------|
+| nvs | data/nvs | 24KB |
+| otadata | data/ota | 8KB |
+| app0 | app/ota_0 | 6.5MB |
+| app1 | app/ota_1 | 6.5MB |
+| littlefs | data/littlefs | 512KB |
+| coredump | data/coredump | 64KB |
+
+### NVS Keys
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `wifi_ssid` | string | — | WiFi SSID |
+| `wifi_pass` | string | — | WiFi password |
+| `immich_url` | string | `https://photos.example.com` | Immich base URL |
+| `immich_key` | string | — | API key |
+| `img_quality` | u8 | 0 | 0=preview, 1=original |
+| `interval_m` | u8 | 5 | Slideshow interval (minutes) |
+| `cycle_mode` | u8 | 0 | 0=random, 1=chrono, 2=reverse, 3=favorites |
+| `meta_flags` | u8 | 0x00 | Bitmask: date/location/people/album/camera |
+| `meta_pos` | u8 | 0 | 0=bottom, 1=top |
+| `queue_cursor` | u32 | 0 | Position in photo queue |
+| `albums_json` | string | `[]` | Selected album IDs (JSON array) |
+| `led_bright` | u8 | 50 | LED brightness (0-255) |
+
+### LittleFS
+
+Web UI static assets (HTML/JS/CSS), built and uploaded as part of firmware build process.
+
+## Development Setup
+
+### PlatformIO Configuration
+
+```ini
+[env:m5stack-papercolor]
+platform = espressif32 @ 6.12.0
+board = esp32s3box
+framework = arduino
+board_build.partitions = partitions_custom.csv
+board_upload.flash_size = 16MB
+board_upload.maximum_size = 16777216
+board_build.arduino.memory_type = qio_opi
+board_build.filesystem = littlefs
+monitor_speed = 115200
+build_flags =
+    -DESP32S3
+    -DBOARD_HAS_PSRAM
+    -DCORE_DEBUG_LEVEL=3
+    -DARDUINO_USB_CDC_ON_BOOT=1
+    -DARDUINO_USB_MODE=1
+lib_deps =
+    m5stack/M5Unified
+    m5stack/M5GFX
+    m5stack/M5PM1
+    me-no-dev/ESPAsyncWebServer
+    bblanchon/ArduinoJson
+```
+
+### Dependencies
+
+| Library | Purpose |
+|---------|---------|
+| M5Unified | Hardware abstraction (buttons, display init) |
+| M5GFX | Graphics/display driver for E Ink Spectra 6 |
+| M5PM1 | Power management IC control |
+| ESPAsyncWebServer | Non-blocking web server |
+| ArduinoJson | JSON parsing for Immich API responses |
+| LittleFS | Filesystem for web UI assets |
+| WiFi / HTTPClient | Networking (built into ESP32 Arduino core) |
+
+## Out of Scope (v1)
+
+- Tailscale/WireGuard networking
+- Audio playback/voice interaction
+- microSD card usage (photos fetched over WiFi only)
+- Temperature/humidity sensor integration
+- RTC alarm-based scheduling
+- IR remote control
+- Multi-device sync