# Immich Photo Frame — M5Stack PaperColor Firmware Design

**Date**: 2026-08-03  
**Status**: Approved  
**Platform**: M5Stack PaperColor (ESP32-S3R8, 4" E Ink Spectra 6)

## Overview

Custom firmware that turns the M5Stack PaperColor into a self-contained photo frame connected to a self-hosted Immich instance. The device fetches photos over WiFi, dithers them to the 6-color e-ink palette on-device, and displays them in a configurable slideshow. A web-based management UI served from the device allows album selection, slideshow settings, and OTA firmware updates.

## Hardware Summary

| Component | Detail |
|-----------|--------|
| SoC | ESP32-S3R8 (dual-core LX7 @ 240MHz) |
| Flash | 16MB |
| PSRAM | 8MB |
| Display | 4" E Ink Spectra 6, 600×400 (landscape orientation), 6 colors |
| Battery | 1250mAh |
| Power IC | M5PM1 (I2C 0x6E) |
| Buttons | BTN_TOP (G1), BTN_UP (G9), BTN_DOWN (G10), Power (side) |
| LEDs | 2× RGB NeoPixel (G21) |
| WiFi | 2.4GHz |
| Storage | microSD slot (unused in v1), NVS, LittleFS |
| Audio | ES8311 + ES7210 + speaker — **permanently disabled** |

## Architecture

### FreeRTOS Task Layout

```
┌─────────────────────────────────────────────────────┐
│                   FreeRTOS Tasks                      │
├──────────────┬──────────────┬───────────────────────┤
│ Display Task │ Web Server   │ Battery Monitor Task  │
│ (Core 1)     │ Task (Core 0)│ (Core 0, low priority)│
├──────────────┴──────────────┴───────────────────────┤
│              Shared State (mutex-protected)           │
│  - Settings (NVS)                                    │
│  - Photo queue / cursor                              │
│  - Battery level                                     │
│  - Slideshow state (playing/paused)                  │
├──────────────────────────────────────────────────────┤
│              Services Layer                           │
│  - ImmichClient (HTTPS + API key)                    │
│  - ImagePipeline (decode → dither → framebuffer)     │
│  - PowerManager (M5PM1 interface, sleep control)     │
│  - WiFiManager (AP mode ↔ Station mode)              │
├──────────────────────────────────────────────────────┤
│              Hardware Abstraction                     │
│  - M5Unified / M5GFX (display, buttons, LEDs)       │
│  - M5PM1 (power management IC)                      │
│  - NVS / LittleFS (persistent storage)              │
└─────────────────────────────────────────────────────┘
```

- **Core 1 (app core)**: Display task handles JPEG decode + dithering (CPU-intensive).
- **Core 0 (protocol core)**: Web server, WiFi stack, battery monitor (I/O-bound).
- **Automatic light sleep**: `esp_pm_configure()` with `light_sleep_enable = true`. CPU sleeps at ~5-15mA between task activity. WiFi stays associated via DTIM wake intervals.

### Power States

| State | What's Active | Current | When |
|-------|---------------|---------|------|
| Active (refresh) | CPU full, WiFi, PSRAM, e-ink | ~200mA | 15-25s per photo |
| Idle (light sleep) | WiFi associated, web server listening | ~5-15mA | Between refreshes |
| Deep sleep | RTC + power button wake only | ~93µA | User-triggered off mode |

## Connectivity

### WiFi

- Station mode for normal operation
- AP mode (`PaperColor-Setup`) for initial provisioning or when station fails 3× on boot
- mDNS: device advertises as `papercolor.local`
- WiFi power save: `WIFI_PS_MIN_MODEM`

### Immich Integration

**Base URL**: Configurable (default `https://photos.example.com`)  
**Auth**: API key in `x-api-key` header, stored in NVS

**API Endpoints Used:**

| Purpose | Endpoint |
|---------|----------|
| List albums | `GET /api/albums` |
| Album assets | `GET /api/albums/{id}` |
| Timeline (all photos) | `GET /api/timeline/buckets` + `GET /api/timeline/bucket` |
| Asset metadata | `GET /api/assets/{id}` |
| Download photo | `GET /api/assets/{id}/thumbnail?size=preview` or `GET /api/assets/{id}/original` |
| Favorites | `GET /api/assets?isFavorite=true` |

**Photo Queue:**
1. Build shuffled/sorted list of asset IDs from selected albums
2. Maintain cursor position (persisted in NVS)
3. Re-sync queue every 24 hours (or on manual refresh from web UI) to pick up new photos added to Immich
4. Portrait detection via EXIF orientation or aspect ratio check (height > width = portrait)

**Cycling Modes:**
- Random: shuffled queue, no repeats until all shown
- Chronological: sorted by `dateTimeOriginal` ascending
- Reverse-chronological: descending
- Favorites-weighted: favorites appear 3× more often in shuffle

## Image Pipeline

```
JPEG Download → JPEG Decode → Resize/Crop → 6-Color Dither → Framebuffer → E-ink Refresh
```

### Stages

1. **Download**: HTTP(S) stream into PSRAM buffer. Preview ~200-400KB, original up to ~15MB.
2. **Decode**: TJPGD (ESP32 built-in) or JPEGDecoder library → RGB888. For originals larger than needed, decode at reduced scale (1/2 or 1/4).
3. **Resize/Crop**:
   - Landscape photos: scale to fill 600×400, center-crop overflow
   - Portrait photos: look ahead up to 5 items in the queue for another portrait to pair with. If found, scale each to fill ~267×400, place side-by-side with 8px gap. If no pair found within lookahead, display single portrait centered with equal margins on each side.
4. **Dither**: Floyd-Steinberg error-diffusion against the Spectra 6 palette. Initial RGB values based on typical Spectra 6 measurements (will be fine-tuned with test images on the actual panel):
   - Black (0, 0, 0)
   - White (255, 255, 255)
   - Red (200, 30, 30)
   - Green (30, 160, 30)
   - Blue (30, 30, 200)
   - Yellow (220, 200, 30)
5. **Display**: Write dithered buffer via M5GFX, trigger EPD refresh (10-20s).

### Memory Budget (PSRAM)

| Buffer | Size |
|--------|------|
| JPEG download | ~400KB (preview) / 15MB (original) |
| Decoded RGB888 | 720KB (600×400×3) |
| Dither working | In-place (reuses decode buffer) |
| Framebuffer | ~120KB |
| **Total typical** | **~1.5-2MB** |

### Image Quality Setting

Configurable via web UI:
- **Preview** (default): Immich's preview thumbnail (~1080p). Fast download, adequate for 600×400.
- **Original**: Full-resolution source. Better dithering input but larger download and decode time.

## Display Behavior

- **Orientation**: Landscape (600×400) — device physically rotated
- **Landscape photos**: Fill-crop to 600×400
- **Portrait photos**: Paired side-by-side (two ~267×400 panels within the 600×400 frame)
- **Metadata overlay** (off by default, each toggleable):
  - Date taken
  - Location / city
  - People names (Immich face detection)
  - Album name
  - Camera / lens info
- **Metadata position**: Bottom overlay or top overlay (configurable)

## Slideshow Settings

| Setting | Options | Default |
|---------|---------|---------|
| Interval | 1m, 5m, 15m, 30m, 60m | 5m |
| Cycling mode | Random, Chrono, Reverse-chrono, Favorites-weighted | Random |
| Image quality | Preview, Original | Preview |
| Metadata | Bitmask of fields | All off (0x00) |

## Button Mapping

| Button | GPIO | Action |
|--------|------|--------|
| BTN_TOP | G1 | Random photo (jump to random position in queue) |
| BTN_UP | G9 | Next photo (advance queue) |
| BTN_DOWN | G10 | Play/Pause toggle |

**Combos:**
- BTN_TOP + BTN_DOWN held 3s: Enter deep sleep
- BTN_UP held 5s: Factory reset (clear WiFi credentials, reboot to AP mode)

**Feedback:**
- Brief white RGB LED flash (100ms) on any button press to confirm input
- Buttons configured as GPIO interrupts with 50ms debounce
- Interrupts wake CPU from light sleep
- Events queued via FreeRTOS queue, consumed by display task

## Power Management

### Audio (Permanently Disabled)

- GPIO 45 (AUDIO_PWR_EN): held LOW at boot — audio codecs never powered
- GPIO 46 (SPK_EN): held LOW at boot — speaker amp never enabled

### M5PM1 Power Rail Control

| Rail | Control | State |
|------|---------|-------|
| E-paper (PY_EPD_EN) | Enable during refresh only | Off between refreshes |
| RGB LED power | Enable for battery warnings | Off otherwise |
| Grove port (BOOST5V_EN_PP) | Disabled | Not used |
| Audio codec (CODEC_3V3_L3B) | Disabled | Never powered |

### Battery Monitoring

- Read battery level from M5PM1 via I2C every 60 seconds
- LED behavior (2× RGB NeoPixel on G21):
  - \> 25%: LEDs off
  - 10-25%: Orange pulse every 5 minutes
  - ≤ 10%: Red pulse every 2 minutes
  - Charging detected: Green pulse every 5 minutes
  - < 5%: Auto-enter deep sleep to protect battery

### Deep Sleep

- **Trigger**: Web UI toggle, or BTN_TOP + BTN_DOWN held 3s
- **Wake source**: Power button only
- **On wake**: Full reboot, restore state from NVS

## Web Management UI

### Provisioning (AP Mode)

1. First boot (no WiFi creds) → create AP `PaperColor-Setup`
2. Captive portal serves setup wizard:
   - Scan and select WiFi network + enter password
   - Immich URL (pre-filled `https://photos.example.com`)
   - Immich API key
3. Save to NVS → reboot to station mode
4. Station connection fails 3× on boot → fallback to AP mode

### LAN Web UI

Served by ESPAsyncWebServer on port 80 at `http://papercolor.local`

**Pages:**

| Page | Content |
|------|---------|
| Dashboard | Current photo (low-res preview), battery %, WiFi RSSI, uptime, next refresh timer |
| Albums | Immich album list with checkboxes, "All photos" toggle, asset counts |
| Slideshow | Interval presets, cycling mode, play/pause |
| Display | Image quality, metadata toggles, metadata position |
| Device | WiFi settings, Immich URL/key, deep sleep toggle, LED brightness, reboot |
| Firmware | Current version display, OTA binary upload form |

**Tech stack:**
- Vanilla JS or Preact (tiny bundle), compiled/minified at build time
- Static files served from LittleFS partition
- Device exposes REST API for all settings and actions

### REST API

```
GET  /api/status          → { battery, wifi_rssi, uptime, current_photo, state }
GET  /api/albums          → cached Immich album list
POST /api/albums/select   → { album_ids: [...] }
GET  /api/settings        → all device settings
POST /api/settings        → update settings
POST /api/action/next     → trigger next photo
POST /api/action/random   → trigger random photo
POST /api/action/pause    → pause slideshow
POST /api/action/play     → resume slideshow
POST /api/action/sleep    → enter deep sleep
POST /api/firmware        → OTA binary upload
```

## OTA Updates

- Dual app partitions (app0 / app1) for safe rollback
- Upload via web UI (`POST /api/firmware`)
- Write to inactive partition → verify checksum → switch boot partition → reboot
- If new firmware fails to boot (watchdog), ESP32 rolls back automatically

## Storage

### Flash Partition Table (16MB)

| Partition | Type | Size |
|-----------|------|------|
| nvs | data/nvs | 24KB |
| otadata | data/ota | 8KB |
| app0 | app/ota_0 | 6.5MB |
| app1 | app/ota_1 | 6.5MB |
| littlefs | data/littlefs | 512KB |
| coredump | data/coredump | 64KB |

### NVS Keys

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `wifi_ssid` | string | — | WiFi SSID |
| `wifi_pass` | string | — | WiFi password |
| `immich_url` | string | `https://photos.example.com` | Immich base URL |
| `immich_key` | string | — | API key |
| `img_quality` | u8 | 0 | 0=preview, 1=original |
| `interval_m` | u8 | 5 | Slideshow interval (minutes) |
| `cycle_mode` | u8 | 0 | 0=random, 1=chrono, 2=reverse, 3=favorites |
| `meta_flags` | u8 | 0x00 | Bitmask: date/location/people/album/camera |
| `meta_pos` | u8 | 0 | 0=bottom, 1=top |
| `queue_cursor` | u32 | 0 | Position in photo queue |
| `albums_json` | string | `[]` | Selected album IDs (JSON array) |
| `led_bright` | u8 | 50 | LED brightness (0-255) |

### LittleFS

Web UI static assets (HTML/JS/CSS), built and uploaded as part of firmware build process.

## Development Setup

### PlatformIO Configuration

```ini
[env:m5stack-papercolor]
platform = espressif32 @ 6.12.0
board = esp32s3box
framework = arduino
board_build.partitions = partitions_custom.csv
board_upload.flash_size = 16MB
board_upload.maximum_size = 16777216
board_build.arduino.memory_type = qio_opi
board_build.filesystem = littlefs
monitor_speed = 115200
build_flags =
    -DESP32S3
    -DBOARD_HAS_PSRAM
    -DCORE_DEBUG_LEVEL=3
    -DARDUINO_USB_CDC_ON_BOOT=1
    -DARDUINO_USB_MODE=1
lib_deps =
    m5stack/M5Unified
    m5stack/M5GFX
    m5stack/M5PM1
    me-no-dev/ESPAsyncWebServer
    bblanchon/ArduinoJson
```

### Dependencies

| Library | Purpose |
|---------|---------|
| M5Unified | Hardware abstraction (buttons, display init) |
| M5GFX | Graphics/display driver for E Ink Spectra 6 |
| M5PM1 | Power management IC control |
| ESPAsyncWebServer | Non-blocking web server |
| ArduinoJson | JSON parsing for Immich API responses |
| LittleFS | Filesystem for web UI assets |
| WiFi / HTTPClient | Networking (built into ESP32 Arduino core) |

## Out of Scope (v1)

- Tailscale/WireGuard networking
- Audio playback/voice interaction
- microSD card usage (photos fetched over WiFi only)
- Temperature/humidity sensor integration
- RTC alarm-based scheduling
- IR remote control
- Multi-device sync