cottongin/immich-frame
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
14e67a179313858e8eced3d65c31862f695f20f4
immich-frame/docs/superpowers/specs/2026-08-03-immich-photo-frame-design.md

14 KiB
Raw Blame History

Immich Photo Frame — M5Stack PaperColor Firmware Design

Date: 2026-08-03
Status: Approved
Platform: M5Stack PaperColor (ESP32-S3R8, 4" E Ink Spectra 6)

Overview

Custom firmware that turns the M5Stack PaperColor into a self-contained photo frame connected to a self-hosted Immich instance. The device fetches photos over WiFi, dithers them to the 6-color e-ink palette on-device, and displays them in a configurable slideshow. A web-based management UI served from the device allows album selection, slideshow settings, and OTA firmware updates.

Hardware Summary

Component Detail
SoC ESP32-S3R8 (dual-core LX7 @ 240MHz)
Flash 16MB
PSRAM 8MB
Display 4" E Ink Spectra 6, 600×400 (landscape orientation), 6 colors
Battery 1250mAh
Power IC M5PM1 (I2C 0x6E)
Buttons BTN_TOP (G1), BTN_UP (G9), BTN_DOWN (G10), Power (side)
LEDs 2× RGB NeoPixel (G21)
WiFi 2.4GHz
Storage microSD slot (unused in v1), NVS, LittleFS
Audio ES8311 + ES7210 + speaker — permanently disabled

Architecture

FreeRTOS Task Layout

┌─────────────────────────────────────────────────────┐
│                   FreeRTOS Tasks                      │
├──────────────┬──────────────┬───────────────────────┤
│ Display Task │ Web Server   │ Battery Monitor Task  │
│ (Core 1)     │ Task (Core 0)│ (Core 0, low priority)│
├──────────────┴──────────────┴───────────────────────┤
│              Shared State (mutex-protected)           │
│  - Settings (NVS)                                    │
│  - Photo queue / cursor                              │
│  - Battery level                                     │
│  - Slideshow state (playing/paused)                  │
├──────────────────────────────────────────────────────┤
│              Services Layer                           │
│  - ImmichClient (HTTPS + API key)                    │
│  - ImagePipeline (decode → dither → framebuffer)     │
│  - PowerManager (M5PM1 interface, sleep control)     │
│  - WiFiManager (AP mode ↔ Station mode)              │
├──────────────────────────────────────────────────────┤
│              Hardware Abstraction                     │
│  - M5Unified / M5GFX (display, buttons, LEDs)       │
│  - M5PM1 (power management IC)                      │
│  - NVS / LittleFS (persistent storage)              │
└─────────────────────────────────────────────────────┘
  • Core 1 (app core): Display task handles JPEG decode + dithering (CPU-intensive).
  • Core 0 (protocol core): Web server, WiFi stack, battery monitor (I/O-bound).
  • Automatic light sleep: esp_pm_configure() with light_sleep_enable = true. CPU sleeps at ~5-15mA between task activity. WiFi stays associated via DTIM wake intervals.

Power States

State What's Active Current When
Active (refresh) CPU full, WiFi, PSRAM, e-ink ~200mA 15-25s per photo
Idle (light sleep) WiFi associated, web server listening ~5-15mA Between refreshes
Deep sleep RTC + power button wake only ~93µA User-triggered off mode

Connectivity

WiFi

  • Station mode for normal operation
  • AP mode (PaperColor-Setup) for initial provisioning or when station fails 3× on boot
  • mDNS: device advertises as papercolor.local
  • WiFi power save: WIFI_PS_MIN_MODEM

Immich Integration

Base URL: Configurable (default https://photos.example.com)
Auth: API key in x-api-key header, stored in NVS

API Endpoints Used:

Purpose Endpoint
List albums GET /api/albums
Album assets GET /api/albums/{id}
Timeline (all photos) GET /api/timeline/buckets + GET /api/timeline/bucket
Asset metadata GET /api/assets/{id}
Download photo GET /api/assets/{id}/thumbnail?size=preview or GET /api/assets/{id}/original
Favorites GET /api/assets?isFavorite=true

Photo Queue:

  1. Build shuffled/sorted list of asset IDs from selected albums
  2. Maintain cursor position (persisted in NVS)
  3. Re-sync queue every 24 hours (or on manual refresh from web UI) to pick up new photos added to Immich
  4. Portrait detection via EXIF orientation or aspect ratio check (height > width = portrait)

Cycling Modes:

  • Random: shuffled queue, no repeats until all shown
  • Chronological: sorted by dateTimeOriginal ascending
  • Reverse-chronological: descending
  • Favorites-weighted: favorites appear 3× more often in shuffle

Image Pipeline

JPEG Download → JPEG Decode → Resize/Crop → 6-Color Dither → Framebuffer → E-ink Refresh

Stages

  1. Download: HTTP(S) stream into PSRAM buffer. Preview ~200-400KB, original up to ~15MB.
  2. Decode: TJPGD (ESP32 built-in) or JPEGDecoder library → RGB888. For originals larger than needed, decode at reduced scale (1/2 or 1/4).
  3. Resize/Crop:
    • Landscape photos: scale to fill 600×400, center-crop overflow
    • Portrait photos: look ahead up to 5 items in the queue for another portrait to pair with. If found, scale each to fill ~267×400, place side-by-side with 8px gap. If no pair found within lookahead, display single portrait centered with equal margins on each side.
  4. Dither: Floyd-Steinberg error-diffusion against the Spectra 6 palette. Initial RGB values based on typical Spectra 6 measurements (will be fine-tuned with test images on the actual panel):
    • Black (0, 0, 0)
    • White (255, 255, 255)
    • Red (200, 30, 30)
    • Green (30, 160, 30)
    • Blue (30, 30, 200)
    • Yellow (220, 200, 30)
  5. Display: Write dithered buffer via M5GFX, trigger EPD refresh (10-20s).

Memory Budget (PSRAM)

Buffer Size
JPEG download ~400KB (preview) / 15MB (original)
Decoded RGB888 720KB (600×400×3)
Dither working In-place (reuses decode buffer)
Framebuffer ~120KB
Total typical ~1.5-2MB

Image Quality Setting

Configurable via web UI:

  • Preview (default): Immich's preview thumbnail (~1080p). Fast download, adequate for 600×400.
  • Original: Full-resolution source. Better dithering input but larger download and decode time.

Display Behavior

  • Orientation: Landscape (600×400) — device physically rotated
  • Landscape photos: Fill-crop to 600×400
  • Portrait photos: Paired side-by-side (two ~267×400 panels within the 600×400 frame)
  • Metadata overlay (off by default, each toggleable):
    • Date taken
    • Location / city
    • People names (Immich face detection)
    • Album name
    • Camera / lens info
  • Metadata position: Bottom overlay or top overlay (configurable)

Slideshow Settings

Setting Options Default
Interval 1m, 5m, 15m, 30m, 60m 5m
Cycling mode Random, Chrono, Reverse-chrono, Favorites-weighted Random
Image quality Preview, Original Preview
Metadata Bitmask of fields All off (0x00)

Button Mapping

Button GPIO Action
BTN_TOP G1 Random photo (jump to random position in queue)
BTN_UP G9 Next photo (advance queue)
BTN_DOWN G10 Play/Pause toggle

Combos:

  • BTN_TOP + BTN_DOWN held 3s: Enter deep sleep
  • BTN_UP held 5s: Factory reset (clear WiFi credentials, reboot to AP mode)

Feedback:

  • Brief white RGB LED flash (100ms) on any button press to confirm input
  • Buttons configured as GPIO interrupts with 50ms debounce
  • Interrupts wake CPU from light sleep
  • Events queued via FreeRTOS queue, consumed by display task

Power Management

Audio (Permanently Disabled)

  • GPIO 45 (AUDIO_PWR_EN): held LOW at boot — audio codecs never powered
  • GPIO 46 (SPK_EN): held LOW at boot — speaker amp never enabled

M5PM1 Power Rail Control

Rail Control State
E-paper (PY_EPD_EN) Enable during refresh only Off between refreshes
RGB LED power Enable for battery warnings Off otherwise
Grove port (BOOST5V_EN_PP) Disabled Not used
Audio codec (CODEC_3V3_L3B) Disabled Never powered

Battery Monitoring

  • Read battery level from M5PM1 via I2C every 60 seconds
  • LED behavior (2× RGB NeoPixel on G21):
    • > 25%: LEDs off
    • 10-25%: Orange pulse every 5 minutes
    • ≤ 10%: Red pulse every 2 minutes
    • Charging detected: Green pulse every 5 minutes
    • < 5%: Auto-enter deep sleep to protect battery

Deep Sleep

  • Trigger: Web UI toggle, or BTN_TOP + BTN_DOWN held 3s
  • Wake source: Power button only
  • On wake: Full reboot, restore state from NVS

Web Management UI

Provisioning (AP Mode)

  1. First boot (no WiFi creds) → create AP PaperColor-Setup
  2. Captive portal serves setup wizard:
    • Scan and select WiFi network + enter password
    • Immich URL (pre-filled https://photos.example.com)
    • Immich API key
  3. Save to NVS → reboot to station mode
  4. Station connection fails 3× on boot → fallback to AP mode

LAN Web UI

Served by ESPAsyncWebServer on port 80 at http://papercolor.local

Pages:

Page Content
Dashboard Current photo (low-res preview), battery %, WiFi RSSI, uptime, next refresh timer
Albums Immich album list with checkboxes, "All photos" toggle, asset counts
Slideshow Interval presets, cycling mode, play/pause
Display Image quality, metadata toggles, metadata position
Device WiFi settings, Immich URL/key, deep sleep toggle, LED brightness, reboot
Firmware Current version display, OTA binary upload form

Tech stack:

  • Vanilla JS or Preact (tiny bundle), compiled/minified at build time
  • Static files served from LittleFS partition
  • Device exposes REST API for all settings and actions

REST API

GET  /api/status          → { battery, wifi_rssi, uptime, current_photo, state }
GET  /api/albums          → cached Immich album list
POST /api/albums/select   → { album_ids: [...] }
GET  /api/settings        → all device settings
POST /api/settings        → update settings
POST /api/action/next     → trigger next photo
POST /api/action/random   → trigger random photo
POST /api/action/pause    → pause slideshow
POST /api/action/play     → resume slideshow
POST /api/action/sleep    → enter deep sleep
POST /api/firmware        → OTA binary upload

OTA Updates

  • Dual app partitions (app0 / app1) for safe rollback
  • Upload via web UI (POST /api/firmware)
  • Write to inactive partition → verify checksum → switch boot partition → reboot
  • If new firmware fails to boot (watchdog), ESP32 rolls back automatically

Storage

Flash Partition Table (16MB)

Partition Type Size
nvs data/nvs 24KB
otadata data/ota 8KB
app0 app/ota_0 6.5MB
app1 app/ota_1 6.5MB
littlefs data/littlefs 512KB
coredump data/coredump 64KB

NVS Keys

Key Type Default Description
wifi_ssid string — WiFi SSID
wifi_pass string — WiFi password
immich_url string https://photos.example.com Immich base URL
immich_key string — API key
img_quality u8 0 0=preview, 1=original
interval_m u8 5 Slideshow interval (minutes)
cycle_mode u8 0 0=random, 1=chrono, 2=reverse, 3=favorites
meta_flags u8 0x00 Bitmask: date/location/people/album/camera
meta_pos u8 0 0=bottom, 1=top
queue_cursor u32 0 Position in photo queue
albums_json string [] Selected album IDs (JSON array)
led_bright u8 50 LED brightness (0-255)

LittleFS

Web UI static assets (HTML/JS/CSS), built and uploaded as part of firmware build process.

Development Setup

PlatformIO Configuration

[env:m5stack-papercolor]
platform = espressif32 @ 6.12.0
board = esp32s3box
framework = arduino
board_build.partitions = partitions_custom.csv
board_upload.flash_size = 16MB
board_upload.maximum_size = 16777216
board_build.arduino.memory_type = qio_opi
board_build.filesystem = littlefs
monitor_speed = 115200
build_flags =
    -DESP32S3
    -DBOARD_HAS_PSRAM
    -DCORE_DEBUG_LEVEL=3
    -DARDUINO_USB_CDC_ON_BOOT=1
    -DARDUINO_USB_MODE=1
lib_deps =
    m5stack/M5Unified
    m5stack/M5GFX
    m5stack/M5PM1
    me-no-dev/ESPAsyncWebServer
    bblanchon/ArduinoJson

Dependencies

Library Purpose
M5Unified Hardware abstraction (buttons, display init)
M5GFX Graphics/display driver for E Ink Spectra 6
M5PM1 Power management IC control
ESPAsyncWebServer Non-blocking web server
ArduinoJson JSON parsing for Immich API responses
LittleFS Filesystem for web UI assets
WiFi / HTTPClient Networking (built into ESP32 Arduino core)

Out of Scope (v1)

  • Tailscale/WireGuard networking
  • Audio playback/voice interaction
  • microSD card usage (photos fetched over WiFi only)
  • Temperature/humidity sensor integration
  • RTC alarm-based scheduling
  • IR remote control
  • Multi-device sync
Reference in New Issue View Git Blame Copy Permalink