From ac5ab960b1a217f9b191b94cefc7af1b291de353 Mon Sep 17 00:00:00 2001 From: cottongin Date: Mon, 3 Aug 2026 10:27:08 -0400 Subject: [PATCH] Add design spec: Immich photo frame for M5Stack PaperColor Co-authored-by: Cursor --- .../2026-08-03-immich-photo-frame-design.md | 354 ++++++++++++++++++ 1 file changed, 354 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md diff --git a/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md b/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md new file mode 100644 index 0000000..bfa8440 --- /dev/null +++ b/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md @@ -0,0 +1,354 @@ +# Immich Photo Frame — M5Stack PaperColor Firmware Design + +**Date**: 2026-08-03 +**Status**: Approved +**Platform**: M5Stack PaperColor (ESP32-S3R8, 4" E Ink Spectra 6) + +## Overview + +Custom firmware that turns the M5Stack PaperColor into a self-contained photo frame connected to a self-hosted Immich instance. The device fetches photos over WiFi, dithers them to the 6-color e-ink palette on-device, and displays them in a configurable slideshow. A web-based management UI served from the device allows album selection, slideshow settings, and OTA firmware updates. + +## Hardware Summary + +| Component | Detail | +|-----------|--------| +| SoC | ESP32-S3R8 (dual-core LX7 @ 240MHz) | +| Flash | 16MB | +| PSRAM | 8MB | +| Display | 4" E Ink Spectra 6, 600×400 (landscape orientation), 6 colors | +| Battery | 1250mAh | +| Power IC | M5PM1 (I2C 0x6E) | +| Buttons | BTN_TOP (G1), BTN_UP (G9), BTN_DOWN (G10), Power (side) | +| LEDs | 2× RGB NeoPixel (G21) | +| WiFi | 2.4GHz | +| Storage | microSD slot (unused in v1), NVS, LittleFS | +| Audio | ES8311 + ES7210 + speaker — **permanently disabled** | + +## Architecture + +### FreeRTOS Task Layout + +``` +┌─────────────────────────────────────────────────────┐ +│ FreeRTOS Tasks │ +├──────────────┬──────────────┬───────────────────────┤ +│ Display Task │ Web Server │ Battery Monitor Task │ +│ (Core 1) │ Task (Core 0)│ (Core 0, low priority)│ +├──────────────┴──────────────┴───────────────────────┤ +│ Shared State (mutex-protected) │ +│ - Settings (NVS) │ +│ - Photo queue / cursor │ +│ - Battery level │ +│ - Slideshow state (playing/paused) │ +├──────────────────────────────────────────────────────┤ +│ Services Layer │ +│ - ImmichClient (HTTPS + API key) │ +│ - ImagePipeline (decode → dither → framebuffer) │ +│ - PowerManager (M5PM1 interface, sleep control) │ +│ - WiFiManager (AP mode ↔ Station mode) │ +├──────────────────────────────────────────────────────┤ +│ Hardware Abstraction │ +│ - M5Unified / M5GFX (display, buttons, LEDs) │ +│ - M5PM1 (power management IC) │ +│ - NVS / LittleFS (persistent storage) │ +└─────────────────────────────────────────────────────┘ +``` + +- **Core 1 (app core)**: Display task handles JPEG decode + dithering (CPU-intensive). +- **Core 0 (protocol core)**: Web server, WiFi stack, battery monitor (I/O-bound). +- **Automatic light sleep**: `esp_pm_configure()` with `light_sleep_enable = true`. CPU sleeps at ~5-15mA between task activity. WiFi stays associated via DTIM wake intervals. + +### Power States + +| State | What's Active | Current | When | +|-------|---------------|---------|------| +| Active (refresh) | CPU full, WiFi, PSRAM, e-ink | ~200mA | 15-25s per photo | +| Idle (light sleep) | WiFi associated, web server listening | ~5-15mA | Between refreshes | +| Deep sleep | RTC + power button wake only | ~93µA | User-triggered off mode | + +## Connectivity + +### WiFi + +- Station mode for normal operation +- AP mode (`PaperColor-Setup`) for initial provisioning or when station fails 3× on boot +- mDNS: device advertises as `papercolor.local` +- WiFi power save: `WIFI_PS_MIN_MODEM` + +### Immich Integration + +**Base URL**: Configurable (default `https://photos.example.com`) +**Auth**: API key in `x-api-key` header, stored in NVS + +**API Endpoints Used:** + +| Purpose | Endpoint | +|---------|----------| +| List albums | `GET /api/albums` | +| Album assets | `GET /api/albums/{id}` | +| Timeline (all photos) | `GET /api/timeline/buckets` + `GET /api/timeline/bucket` | +| Asset metadata | `GET /api/assets/{id}` | +| Download photo | `GET /api/assets/{id}/thumbnail?size=preview` or `GET /api/assets/{id}/original` | +| Favorites | `GET /api/assets?isFavorite=true` | + +**Photo Queue:** +1. Build shuffled/sorted list of asset IDs from selected albums +2. Maintain cursor position (persisted in NVS) +3. Re-sync queue every 24 hours (or on manual refresh from web UI) to pick up new photos added to Immich +4. Portrait detection via EXIF orientation or aspect ratio check (height > width = portrait) + +**Cycling Modes:** +- Random: shuffled queue, no repeats until all shown +- Chronological: sorted by `dateTimeOriginal` ascending +- Reverse-chronological: descending +- Favorites-weighted: favorites appear 3× more often in shuffle + +## Image Pipeline + +``` +JPEG Download → JPEG Decode → Resize/Crop → 6-Color Dither → Framebuffer → E-ink Refresh +``` + +### Stages + +1. **Download**: HTTP(S) stream into PSRAM buffer. Preview ~200-400KB, original up to ~15MB. +2. **Decode**: TJPGD (ESP32 built-in) or JPEGDecoder library → RGB888. For originals larger than needed, decode at reduced scale (1/2 or 1/4). +3. **Resize/Crop**: + - Landscape photos: scale to fill 600×400, center-crop overflow + - Portrait photos: look ahead up to 5 items in the queue for another portrait to pair with. If found, scale each to fill ~267×400, place side-by-side with 8px gap. If no pair found within lookahead, display single portrait centered with equal margins on each side. +4. **Dither**: Floyd-Steinberg error-diffusion against the Spectra 6 palette. Initial RGB values based on typical Spectra 6 measurements (will be fine-tuned with test images on the actual panel): + - Black (0, 0, 0) + - White (255, 255, 255) + - Red (200, 30, 30) + - Green (30, 160, 30) + - Blue (30, 30, 200) + - Yellow (220, 200, 30) +5. **Display**: Write dithered buffer via M5GFX, trigger EPD refresh (10-20s). + +### Memory Budget (PSRAM) + +| Buffer | Size | +|--------|------| +| JPEG download | ~400KB (preview) / 15MB (original) | +| Decoded RGB888 | 720KB (600×400×3) | +| Dither working | In-place (reuses decode buffer) | +| Framebuffer | ~120KB | +| **Total typical** | **~1.5-2MB** | + +### Image Quality Setting + +Configurable via web UI: +- **Preview** (default): Immich's preview thumbnail (~1080p). Fast download, adequate for 600×400. +- **Original**: Full-resolution source. Better dithering input but larger download and decode time. + +## Display Behavior + +- **Orientation**: Landscape (600×400) — device physically rotated +- **Landscape photos**: Fill-crop to 600×400 +- **Portrait photos**: Paired side-by-side (two ~267×400 panels within the 600×400 frame) +- **Metadata overlay** (off by default, each toggleable): + - Date taken + - Location / city + - People names (Immich face detection) + - Album name + - Camera / lens info +- **Metadata position**: Bottom overlay or top overlay (configurable) + +## Slideshow Settings + +| Setting | Options | Default | +|---------|---------|---------| +| Interval | 1m, 5m, 15m, 30m, 60m | 5m | +| Cycling mode | Random, Chrono, Reverse-chrono, Favorites-weighted | Random | +| Image quality | Preview, Original | Preview | +| Metadata | Bitmask of fields | All off (0x00) | + +## Button Mapping + +| Button | GPIO | Action | +|--------|------|--------| +| BTN_TOP | G1 | Random photo (jump to random position in queue) | +| BTN_UP | G9 | Next photo (advance queue) | +| BTN_DOWN | G10 | Play/Pause toggle | + +**Combos:** +- BTN_TOP + BTN_DOWN held 3s: Enter deep sleep +- BTN_UP held 5s: Factory reset (clear WiFi credentials, reboot to AP mode) + +**Feedback:** +- Brief white RGB LED flash (100ms) on any button press to confirm input +- Buttons configured as GPIO interrupts with 50ms debounce +- Interrupts wake CPU from light sleep +- Events queued via FreeRTOS queue, consumed by display task + +## Power Management + +### Audio (Permanently Disabled) + +- GPIO 45 (AUDIO_PWR_EN): held LOW at boot — audio codecs never powered +- GPIO 46 (SPK_EN): held LOW at boot — speaker amp never enabled + +### M5PM1 Power Rail Control + +| Rail | Control | State | +|------|---------|-------| +| E-paper (PY_EPD_EN) | Enable during refresh only | Off between refreshes | +| RGB LED power | Enable for battery warnings | Off otherwise | +| Grove port (BOOST5V_EN_PP) | Disabled | Not used | +| Audio codec (CODEC_3V3_L3B) | Disabled | Never powered | + +### Battery Monitoring + +- Read battery level from M5PM1 via I2C every 60 seconds +- LED behavior (2× RGB NeoPixel on G21): + - \> 25%: LEDs off + - 10-25%: Orange pulse every 5 minutes + - ≤ 10%: Red pulse every 2 minutes + - Charging detected: Green pulse every 5 minutes + - < 5%: Auto-enter deep sleep to protect battery + +### Deep Sleep + +- **Trigger**: Web UI toggle, or BTN_TOP + BTN_DOWN held 3s +- **Wake source**: Power button only +- **On wake**: Full reboot, restore state from NVS + +## Web Management UI + +### Provisioning (AP Mode) + +1. First boot (no WiFi creds) → create AP `PaperColor-Setup` +2. Captive portal serves setup wizard: + - Scan and select WiFi network + enter password + - Immich URL (pre-filled `https://photos.example.com`) + - Immich API key +3. Save to NVS → reboot to station mode +4. Station connection fails 3× on boot → fallback to AP mode + +### LAN Web UI + +Served by ESPAsyncWebServer on port 80 at `http://papercolor.local` + +**Pages:** + +| Page | Content | +|------|---------| +| Dashboard | Current photo (low-res preview), battery %, WiFi RSSI, uptime, next refresh timer | +| Albums | Immich album list with checkboxes, "All photos" toggle, asset counts | +| Slideshow | Interval presets, cycling mode, play/pause | +| Display | Image quality, metadata toggles, metadata position | +| Device | WiFi settings, Immich URL/key, deep sleep toggle, LED brightness, reboot | +| Firmware | Current version display, OTA binary upload form | + +**Tech stack:** +- Vanilla JS or Preact (tiny bundle), compiled/minified at build time +- Static files served from LittleFS partition +- Device exposes REST API for all settings and actions + +### REST API + +``` +GET /api/status → { battery, wifi_rssi, uptime, current_photo, state } +GET /api/albums → cached Immich album list +POST /api/albums/select → { album_ids: [...] } +GET /api/settings → all device settings +POST /api/settings → update settings +POST /api/action/next → trigger next photo +POST /api/action/random → trigger random photo +POST /api/action/pause → pause slideshow +POST /api/action/play → resume slideshow +POST /api/action/sleep → enter deep sleep +POST /api/firmware → OTA binary upload +``` + +## OTA Updates + +- Dual app partitions (app0 / app1) for safe rollback +- Upload via web UI (`POST /api/firmware`) +- Write to inactive partition → verify checksum → switch boot partition → reboot +- If new firmware fails to boot (watchdog), ESP32 rolls back automatically + +## Storage + +### Flash Partition Table (16MB) + +| Partition | Type | Size | +|-----------|------|------| +| nvs | data/nvs | 24KB | +| otadata | data/ota | 8KB | +| app0 | app/ota_0 | 6.5MB | +| app1 | app/ota_1 | 6.5MB | +| littlefs | data/littlefs | 512KB | +| coredump | data/coredump | 64KB | + +### NVS Keys + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `wifi_ssid` | string | — | WiFi SSID | +| `wifi_pass` | string | — | WiFi password | +| `immich_url` | string | `https://photos.example.com` | Immich base URL | +| `immich_key` | string | — | API key | +| `img_quality` | u8 | 0 | 0=preview, 1=original | +| `interval_m` | u8 | 5 | Slideshow interval (minutes) | +| `cycle_mode` | u8 | 0 | 0=random, 1=chrono, 2=reverse, 3=favorites | +| `meta_flags` | u8 | 0x00 | Bitmask: date/location/people/album/camera | +| `meta_pos` | u8 | 0 | 0=bottom, 1=top | +| `queue_cursor` | u32 | 0 | Position in photo queue | +| `albums_json` | string | `[]` | Selected album IDs (JSON array) | +| `led_bright` | u8 | 50 | LED brightness (0-255) | + +### LittleFS + +Web UI static assets (HTML/JS/CSS), built and uploaded as part of firmware build process. + +## Development Setup + +### PlatformIO Configuration + +```ini +[env:m5stack-papercolor] +platform = espressif32 @ 6.12.0 +board = esp32s3box +framework = arduino +board_build.partitions = partitions_custom.csv +board_upload.flash_size = 16MB +board_upload.maximum_size = 16777216 +board_build.arduino.memory_type = qio_opi +board_build.filesystem = littlefs +monitor_speed = 115200 +build_flags = + -DESP32S3 + -DBOARD_HAS_PSRAM + -DCORE_DEBUG_LEVEL=3 + -DARDUINO_USB_CDC_ON_BOOT=1 + -DARDUINO_USB_MODE=1 +lib_deps = + m5stack/M5Unified + m5stack/M5GFX + m5stack/M5PM1 + me-no-dev/ESPAsyncWebServer + bblanchon/ArduinoJson +``` + +### Dependencies + +| Library | Purpose | +|---------|---------| +| M5Unified | Hardware abstraction (buttons, display init) | +| M5GFX | Graphics/display driver for E Ink Spectra 6 | +| M5PM1 | Power management IC control | +| ESPAsyncWebServer | Non-blocking web server | +| ArduinoJson | JSON parsing for Immich API responses | +| LittleFS | Filesystem for web UI assets | +| WiFi / HTTPClient | Networking (built into ESP32 Arduino core) | + +## Out of Scope (v1) + +- Tailscale/WireGuard networking +- Audio playback/voice interaction +- microSD card usage (photos fetched over WiFi only) +- Temperature/humidity sensor integration +- RTC alarm-based scheduling +- IR remote control +- Multi-device sync