From 214fa3bf90327dc235c24a1e5169bae151cc29a1 Mon Sep 17 00:00:00 2001 From: cottongin Date: Mon, 3 Aug 2026 10:37:51 -0400 Subject: [PATCH] Add implementation plan for Immich photo frame firmware Co-authored-by: Cursor --- .../plans/2026-08-03-immich-photo-frame.md | 3866 +++++++++++++++++ 1 file changed, 3866 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-03-immich-photo-frame.md diff --git a/docs/superpowers/plans/2026-08-03-immich-photo-frame.md b/docs/superpowers/plans/2026-08-03-immich-photo-frame.md new file mode 100644 index 0000000..34a5528 --- /dev/null +++ b/docs/superpowers/plans/2026-08-03-immich-photo-frame.md @@ -0,0 +1,3866 @@ +# Immich Photo Frame Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build firmware for the M5Stack PaperColor that displays photos from a self-hosted Immich instance as a configurable slideshow, with a web-based management UI. + +**Architecture:** FreeRTOS multi-task firmware on ESP32-S3. Display task (Core 1) handles JPEG decode and 6-color dithering. Web server and battery monitor tasks (Core 0) run alongside WiFi. Automatic light sleep between refreshes keeps idle power at ~5-15mA while the web UI remains accessible. + +**Tech Stack:** Arduino framework via PlatformIO, M5Unified/M5GFX/M5PM1 libraries, ESPAsyncWebServer, ArduinoJson, TJpgDec for JPEG decode, LittleFS for web assets, NVS for settings. + +## Global Constraints + +- Platform: `espressif32 @ 6.12.0` +- Board target: `esp32s3box` (closest match for PaperColor in PlatformIO) +- Arduino framework with ESP-IDF components accessible +- 16MB flash, 8MB PSRAM (`qio_opi` memory type) +- Display: 600×400 landscape (device rotated), E Ink Spectra 6 (6 colors) +- USB CDC on boot enabled (`ARDUINO_USB_CDC_ON_BOOT=1`) +- All audio permanently disabled (GPIO 45 LOW, GPIO 46 LOW) +- Buttons: BTN_TOP=G1, BTN_UP=G9, BTN_DOWN=G10 +- RGB LEDs: G21 (NeoPixel, 2 LEDs) +- M5PM1 power IC at I2C 0x6E on SYS_SDA=G3, SYS_SCL=G2 + +--- + +## File Structure + +``` +immich-frame/ +├── platformio.ini # Build configuration +├── partitions_custom.csv # 16MB partition table +├── src/ +│ ├── main.cpp # Entry point, FreeRTOS task creation +│ ├── config.h # Pin definitions, constants, defaults +│ ├── settings.h # Settings manager header +│ ├── settings.cpp # NVS read/write for all config +│ ├── wifi_manager.h # WiFi manager header +│ ├── wifi_manager.cpp # Station + AP + mDNS +│ ├── power_manager.h # Power manager header +│ ├── power_manager.cpp # M5PM1, battery, LEDs, sleep +│ ├── button_handler.h # Button handler header +│ ├── button_handler.cpp # GPIO interrupts, debounce, combos +│ ├── immich_client.h # Immich API client header +│ ├── immich_client.cpp # HTTPS calls to Immich REST API +│ ├── photo_queue.h # Photo queue/selection header +│ ├── photo_queue.cpp # Queue management, cycling modes +│ ├── image_pipeline.h # Image processing header +│ ├── image_pipeline.cpp # JPEG decode, resize, dither +│ ├── display_manager.h # Display manager header +│ ├── display_manager.cpp # E-ink framebuffer + refresh +│ ├── web_server.h # Web server header +│ └── web_server.cpp # REST API + static file serving +├── data/ # LittleFS partition (web UI) +│ ├── index.html # Single-page app shell +│ ├── setup.html # AP mode captive portal +│ ├── app.js # Web UI logic +│ └── style.css # Styles +├── test/ +│ ├── test_native/ +│ │ ├── test_settings.cpp # Settings serialization tests +│ │ ├── test_photo_queue.cpp # Queue cycling logic tests +│ │ └── test_image_pipeline.cpp # Dithering algorithm tests +│ └── README.md # How to run tests +└── docs/ + └── superpowers/ + ├── specs/ + └── plans/ +``` + +--- + +### Task 1: Project Scaffolding + +**Files:** +- Create: `platformio.ini` +- Create: `partitions_custom.csv` +- Create: `src/config.h` +- Create: `src/main.cpp` + +**Interfaces:** +- Consumes: Nothing (first task) +- Produces: Build system that compiles an empty firmware that boots, prints to serial, and can be flashed to the PaperColor + +- [ ] **Step 1: Create `platformio.ini`** + +```ini +; PlatformIO configuration for M5Stack PaperColor Immich Frame +[env:m5stack-papercolor] +platform = espressif32 @ 6.12.0 +board = esp32s3box +framework = arduino +board_build.partitions = partitions_custom.csv +board_upload.flash_size = 16MB +board_upload.maximum_size = 16777216 +board_build.arduino.memory_type = qio_opi +board_build.filesystem = littlefs +monitor_speed = 115200 +upload_speed = 921600 +build_flags = + -DESP32S3 + -DBOARD_HAS_PSRAM + -DCORE_DEBUG_LEVEL=3 + -DARDUINO_USB_CDC_ON_BOOT=1 + -DARDUINO_USB_MODE=1 +lib_deps = + m5stack/M5Unified @ ^0.2.2 + m5stack/M5GFX @ ^0.2.5 + https://github.com/m5stack/M5PM1.git + me-no-dev/ESP Async WebServer @ ^1.2.4 + bblanchon/ArduinoJson @ ^7.0.0 + +[env:native] +platform = native +build_flags = -std=c++17 +test_framework = unity +lib_deps = + bblanchon/ArduinoJson @ ^7.0.0 +``` + +- [ ] **Step 2: Create `partitions_custom.csv`** + +```csv +# Name, Type, SubType, Offset, Size, Flags +nvs, data, nvs, 0x9000, 0x6000, +otadata, data, ota, 0xf000, 0x2000, +app0, app, ota_0, 0x10000, 0x680000, +app1, app, ota_1, 0x690000, 0x680000, +littlefs, data, spiffs, 0xD10000, 0x80000, +coredump, data, coredump, 0xD90000, 0x10000, +``` + +- [ ] **Step 3: Create `src/config.h`** + +```cpp +#pragma once + +// --- Pin Definitions --- +// Buttons +#define PIN_BTN_TOP 1 // G1 - Random photo +#define PIN_BTN_UP 9 // G9 - Next photo +#define PIN_BTN_DOWN 10 // G10 - Play/Pause + +// RGB LEDs (NeoPixel) +#define PIN_RGB_LED 21 +#define NUM_RGB_LEDS 2 + +// Audio (permanently disabled) +#define PIN_AUDIO_PWR_EN 45 +#define PIN_SPK_EN 46 + +// E-Paper SPI +#define PIN_EPD_CLK 15 +#define PIN_EPD_MOSI 13 +#define PIN_EPD_CS 44 +#define PIN_EPD_DC 43 +#define PIN_EPD_BUSY 11 +#define PIN_EPD_RST 12 + +// I2C System Bus (M5PM1, SHT40, RTC) +#define PIN_SYS_SDA 3 +#define PIN_SYS_SCL 2 + +// --- Display --- +#define DISPLAY_WIDTH 600 +#define DISPLAY_HEIGHT 400 +#define DISPLAY_COLORS 6 + +// --- Defaults --- +#define DEFAULT_INTERVAL_MIN 5 +#define DEFAULT_CYCLE_MODE 0 // 0=random +#define DEFAULT_IMG_QUALITY 0 // 0=preview +#define DEFAULT_META_FLAGS 0x00 +#define DEFAULT_META_POS 0 // 0=bottom +#define DEFAULT_LED_BRIGHTNESS 50 +#define DEFAULT_IMMICH_URL "https://photos.example.com" + +// --- Timing --- +#define BUTTON_DEBOUNCE_MS 50 +#define COMBO_HOLD_MS 3000 +#define FACTORY_RESET_HOLD_MS 5000 +#define BATTERY_CHECK_INTERVAL_MS 60000 +#define LED_PULSE_DURATION_MS 100 +#define QUEUE_RESYNC_HOURS 24 +#define WIFI_CONNECT_TIMEOUT_MS 15000 +#define WIFI_MAX_RETRIES 3 + +// --- Battery Thresholds --- +#define BATTERY_LOW_PCT 25 +#define BATTERY_CRITICAL_PCT 10 +#define BATTERY_SHUTDOWN_PCT 5 +#define BATTERY_WARN_INTERVAL_MS 300000 // 5 min +#define BATTERY_CRIT_INTERVAL_MS 120000 // 2 min + +// --- Portrait Pairing --- +#define PORTRAIT_LOOKAHEAD 5 +#define PORTRAIT_GAP_PX 8 +#define PORTRAIT_WIDTH 267 // (DISPLAY_HEIGHT * 2) / 3 + +// --- Dither Palette (Spectra 6) --- +// Initial values — calibrate on real hardware +#define PALETTE_BLACK_R 0 +#define PALETTE_BLACK_G 0 +#define PALETTE_BLACK_B 0 +#define PALETTE_WHITE_R 255 +#define PALETTE_WHITE_G 255 +#define PALETTE_WHITE_B 255 +#define PALETTE_RED_R 200 +#define PALETTE_RED_G 30 +#define PALETTE_RED_B 30 +#define PALETTE_GREEN_R 30 +#define PALETTE_GREEN_G 160 +#define PALETTE_GREEN_B 30 +#define PALETTE_BLUE_R 30 +#define PALETTE_BLUE_G 30 +#define PALETTE_BLUE_B 200 +#define PALETTE_YELLOW_R 220 +#define PALETTE_YELLOW_G 200 +#define PALETTE_YELLOW_B 30 +``` + +- [ ] **Step 4: Create `src/main.cpp` (minimal boot skeleton)** + +```cpp +#include +#include +#include "config.h" + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame booting..."); + + // Disable audio permanently + pinMode(PIN_AUDIO_PWR_EN, OUTPUT); + digitalWrite(PIN_AUDIO_PWR_EN, LOW); + pinMode(PIN_SPK_EN, OUTPUT); + digitalWrite(PIN_SPK_EN, LOW); + + Serial.printf("[main] Free heap: %d, PSRAM: %d\n", + ESP.getFreeHeap(), ESP.getFreePsram()); + Serial.println("[main] Boot complete (skeleton)"); +} + +void loop() { + M5.update(); + delay(1000); +} +``` + +- [ ] **Step 5: Verify build compiles** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS with no errors + +- [ ] **Step 6: Flash to device and verify serial output** + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: Serial prints "Immich Frame booting...", free heap/PSRAM values, "Boot complete (skeleton)" + +- [ ] **Step 7: Commit** + +```bash +git add platformio.ini partitions_custom.csv src/config.h src/main.cpp +git commit -m "feat: project scaffolding with PlatformIO config and boot skeleton" +``` + +--- + +### Task 2: Settings Manager + +**Files:** +- Create: `src/settings.h` +- Create: `src/settings.cpp` +- Create: `test/test_native/test_settings.cpp` + +**Interfaces:** +- Consumes: `config.h` (default values) +- Produces: + - `Settings` struct with all config fields + - `SettingsManager` class: + - `void begin()` — initialize NVS + - `Settings get()` — load all settings from NVS + - `void save(const Settings& s)` — persist all settings to NVS + - `void saveField(const char* key, uint8_t value)` — save single numeric field + - `void saveField(const char* key, const char* value)` — save single string field + - `void factoryReset()` — clear all NVS and reboot + - `bool isProvisioned()` — returns true if wifi_ssid is set + +- [ ] **Step 1: Create `src/settings.h`** + +```cpp +#pragma once + +#include +#include "config.h" + +enum class CycleMode : uint8_t { + Random = 0, + Chronological = 1, + ReverseChronological = 2, + FavoritesWeighted = 3 +}; + +enum class ImageQuality : uint8_t { + Preview = 0, + Original = 1 +}; + +enum class MetaPosition : uint8_t { + Bottom = 0, + Top = 1 +}; + +// Metadata flags bitmask +constexpr uint8_t META_DATE = 0x01; +constexpr uint8_t META_LOCATION = 0x02; +constexpr uint8_t META_PEOPLE = 0x04; +constexpr uint8_t META_ALBUM = 0x08; +constexpr uint8_t META_CAMERA = 0x10; + +struct Settings { + // WiFi + String wifi_ssid; + String wifi_pass; + + // Immich + String immich_url; + String immich_key; + + // Slideshow + uint8_t interval_min; + CycleMode cycle_mode; + ImageQuality img_quality; + + // Display + uint8_t meta_flags; + MetaPosition meta_pos; + + // Device + uint8_t led_brightness; + uint32_t queue_cursor; + String albums_json; +}; + +class SettingsManager { +public: + void begin(); + Settings get(); + void save(const Settings& s); + void saveField(const char* key, uint8_t value); + void saveField(const char* key, const char* value); + void factoryReset(); + bool isProvisioned(); + +private: + String readString(const char* key, const char* defaultValue = ""); + uint8_t readU8(const char* key, uint8_t defaultValue = 0); + uint32_t readU32(const char* key, uint32_t defaultValue = 0); + void writeString(const char* key, const String& value); + void writeU8(const char* key, uint8_t value); + void writeU32(const char* key, uint32_t value); +}; +``` + +- [ ] **Step 2: Create `src/settings.cpp`** + +```cpp +#include "settings.h" +#include + +static const char* NVS_NAMESPACE = "immich_frame"; +static Preferences prefs; + +void SettingsManager::begin() { + prefs.begin(NVS_NAMESPACE, false); + Serial.println("[settings] NVS initialized"); +} + +Settings SettingsManager::get() { + Settings s; + s.wifi_ssid = readString("wifi_ssid"); + s.wifi_pass = readString("wifi_pass"); + s.immich_url = readString("immich_url", DEFAULT_IMMICH_URL); + s.immich_key = readString("immich_key"); + s.interval_min = readU8("interval_m", DEFAULT_INTERVAL_MIN); + s.cycle_mode = static_cast(readU8("cycle_mode", DEFAULT_CYCLE_MODE)); + s.img_quality = static_cast(readU8("img_quality", DEFAULT_IMG_QUALITY)); + s.meta_flags = readU8("meta_flags", DEFAULT_META_FLAGS); + s.meta_pos = static_cast(readU8("meta_pos", DEFAULT_META_POS)); + s.led_brightness = readU8("led_bright", DEFAULT_LED_BRIGHTNESS); + s.queue_cursor = readU32("queue_cursor", 0); + s.albums_json = readString("albums_json", "[]"); + return s; +} + +void SettingsManager::save(const Settings& s) { + writeString("wifi_ssid", s.wifi_ssid); + writeString("wifi_pass", s.wifi_pass); + writeString("immich_url", s.immich_url); + writeString("immich_key", s.immich_key); + writeU8("interval_m", s.interval_min); + writeU8("cycle_mode", static_cast(s.cycle_mode)); + writeU8("img_quality", static_cast(s.img_quality)); + writeU8("meta_flags", s.meta_flags); + writeU8("meta_pos", static_cast(s.meta_pos)); + writeU8("led_bright", s.led_brightness); + writeU32("queue_cursor", s.queue_cursor); + writeString("albums_json", s.albums_json); + Serial.println("[settings] All settings saved"); +} + +void SettingsManager::saveField(const char* key, uint8_t value) { + writeU8(key, value); +} + +void SettingsManager::saveField(const char* key, const char* value) { + writeString(key, String(value)); +} + +void SettingsManager::factoryReset() { + prefs.clear(); + Serial.println("[settings] Factory reset — rebooting"); + delay(500); + ESP.restart(); +} + +bool SettingsManager::isProvisioned() { + String ssid = readString("wifi_ssid"); + return ssid.length() > 0; +} + +String SettingsManager::readString(const char* key, const char* defaultValue) { + return prefs.getString(key, defaultValue); +} + +uint8_t SettingsManager::readU8(const char* key, uint8_t defaultValue) { + return prefs.getUChar(key, defaultValue); +} + +uint32_t SettingsManager::readU32(const char* key, uint32_t defaultValue) { + return prefs.getUInt(key, defaultValue); +} + +void SettingsManager::writeString(const char* key, const String& value) { + prefs.putString(key, value); +} + +void SettingsManager::writeU8(const char* key, uint8_t value) { + prefs.putUChar(key, value); +} + +void SettingsManager::writeU32(const char* key, uint32_t value) { + prefs.putUInt(key, value); +} +``` + +- [ ] **Step 3: Verify build compiles with settings module** + +Update `src/main.cpp` to include and use settings: + +```cpp +#include +#include +#include "config.h" +#include "settings.h" + +SettingsManager settingsManager; + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame booting..."); + + // Disable audio permanently + pinMode(PIN_AUDIO_PWR_EN, OUTPUT); + digitalWrite(PIN_AUDIO_PWR_EN, LOW); + pinMode(PIN_SPK_EN, OUTPUT); + digitalWrite(PIN_SPK_EN, LOW); + + settingsManager.begin(); + Settings settings = settingsManager.get(); + + Serial.printf("[main] Provisioned: %s\n", settingsManager.isProvisioned() ? "yes" : "no"); + Serial.printf("[main] Immich URL: %s\n", settings.immich_url.c_str()); + Serial.printf("[main] Interval: %d min\n", settings.interval_min); + Serial.printf("[main] Free heap: %d, PSRAM: %d\n", + ESP.getFreeHeap(), ESP.getFreePsram()); + Serial.println("[main] Boot complete"); +} + +void loop() { + M5.update(); + delay(1000); +} +``` + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS + +- [ ] **Step 4: Flash and verify settings load from NVS** + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: Serial shows "Provisioned: no", default Immich URL, interval 5 min + +- [ ] **Step 5: Commit** + +```bash +git add src/settings.h src/settings.cpp src/main.cpp +git commit -m "feat: settings manager with NVS persistence" +``` + +--- + +### Task 3: WiFi Manager + +**Files:** +- Create: `src/wifi_manager.h` +- Create: `src/wifi_manager.cpp` + +**Interfaces:** +- Consumes: `SettingsManager::get()` for WiFi credentials, `SettingsManager::isProvisioned()` +- Produces: + - `WiFiManager` class: + - `void begin(SettingsManager& settings)` — connect or start AP based on provisioning state + - `bool isConnected()` — station mode connected? + - `bool isAPMode()` — currently in AP mode? + - `String getIP()` — current IP address + - `int getRSSI()` — WiFi signal strength + - `void startAP()` — force AP mode + - `void startStation()` — attempt station connection + - `void setupMDNS(const char* hostname)` — register mDNS + +- [ ] **Step 1: Create `src/wifi_manager.h`** + +```cpp +#pragma once + +#include +#include "settings.h" + +class WiFiManager { +public: + void begin(SettingsManager& settings); + bool isConnected(); + bool isAPMode(); + String getIP(); + int getRSSI(); + void startAP(); + void startStation(); + void setupMDNS(const char* hostname); + +private: + SettingsManager* _settings = nullptr; + bool _apMode = false; + int _retryCount = 0; + + bool attemptConnection(const String& ssid, const String& pass); +}; +``` + +- [ ] **Step 2: Create `src/wifi_manager.cpp`** + +```cpp +#include "wifi_manager.h" +#include +#include +#include "config.h" + +void WiFiManager::begin(SettingsManager& settings) { + _settings = &settings; + + if (!_settings->isProvisioned()) { + Serial.println("[wifi] Not provisioned — starting AP"); + startAP(); + return; + } + + startStation(); +} + +bool WiFiManager::isConnected() { + return WiFi.status() == WL_CONNECTED; +} + +bool WiFiManager::isAPMode() { + return _apMode; +} + +String WiFiManager::getIP() { + if (_apMode) { + return WiFi.softAPIP().toString(); + } + return WiFi.localIP().toString(); +} + +int WiFiManager::getRSSI() { + if (_apMode) return 0; + return WiFi.RSSI(); +} + +void WiFiManager::startAP() { + WiFi.disconnect(true); + WiFi.mode(WIFI_AP); + WiFi.softAP("PaperColor-Setup"); + _apMode = true; + Serial.printf("[wifi] AP started: PaperColor-Setup, IP: %s\n", + WiFi.softAPIP().toString().c_str()); +} + +void WiFiManager::startStation() { + Settings s = _settings->get(); + WiFi.disconnect(true); + WiFi.mode(WIFI_STA); + _apMode = false; + + Serial.printf("[wifi] Connecting to: %s\n", s.wifi_ssid.c_str()); + + for (_retryCount = 0; _retryCount < WIFI_MAX_RETRIES; _retryCount++) { + if (attemptConnection(s.wifi_ssid, s.wifi_pass)) { + Serial.printf("[wifi] Connected! IP: %s, RSSI: %d\n", + WiFi.localIP().toString().c_str(), WiFi.RSSI()); + WiFi.setSleep(WIFI_PS_MIN_MODEM); + setupMDNS("papercolor"); + return; + } + Serial.printf("[wifi] Attempt %d/%d failed\n", _retryCount + 1, WIFI_MAX_RETRIES); + } + + Serial.println("[wifi] All attempts failed — falling back to AP mode"); + startAP(); +} + +void WiFiManager::setupMDNS(const char* hostname) { + if (MDNS.begin(hostname)) { + MDNS.addService("http", "tcp", 80); + Serial.printf("[wifi] mDNS: %s.local\n", hostname); + } else { + Serial.println("[wifi] mDNS failed to start"); + } +} + +bool WiFiManager::attemptConnection(const String& ssid, const String& pass) { + WiFi.begin(ssid.c_str(), pass.c_str()); + + unsigned long start = millis(); + while (WiFi.status() != WL_CONNECTED) { + if (millis() - start > WIFI_CONNECT_TIMEOUT_MS) { + return false; + } + delay(100); + } + return true; +} +``` + +- [ ] **Step 3: Integrate into `main.cpp`** + +```cpp +#include +#include +#include "config.h" +#include "settings.h" +#include "wifi_manager.h" + +SettingsManager settingsManager; +WiFiManager wifiManager; + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame booting..."); + + pinMode(PIN_AUDIO_PWR_EN, OUTPUT); + digitalWrite(PIN_AUDIO_PWR_EN, LOW); + pinMode(PIN_SPK_EN, OUTPUT); + digitalWrite(PIN_SPK_EN, LOW); + + settingsManager.begin(); + wifiManager.begin(settingsManager); + + Serial.printf("[main] WiFi mode: %s, IP: %s\n", + wifiManager.isAPMode() ? "AP" : "Station", + wifiManager.getIP().c_str()); + Serial.println("[main] Boot complete"); +} + +void loop() { + M5.update(); + delay(1000); +} +``` + +- [ ] **Step 4: Build and flash** + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: On first boot (no credentials), serial shows "Not provisioned — starting AP", AP name "PaperColor-Setup", IP "192.168.4.1" + +- [ ] **Step 5: Commit** + +```bash +git add src/wifi_manager.h src/wifi_manager.cpp src/main.cpp +git commit -m "feat: WiFi manager with station/AP mode and mDNS" +``` + +--- + +### Task 4: Power Manager + +**Files:** +- Create: `src/power_manager.h` +- Create: `src/power_manager.cpp` + +**Interfaces:** +- Consumes: `config.h` (pin definitions, thresholds), M5PM1 library +- Produces: + - `PowerManager` class: + - `void begin()` — init M5PM1, disable unused rails, configure LED strip + - `uint8_t getBatteryPercent()` — read battery level (0-100) + - `bool isCharging()` — is USB power connected? + - `void updateBatteryLED()` — pulse LED based on level (call periodically) + - `void flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms)` — one-shot LED flash + - `void enterDeepSleep()` — power down everything, only power button wakes + - `void enableLightSleep()` — configure auto light sleep with WiFi keepalive + - `void disableEPDPower()` — turn off e-paper power rail + - `void enableEPDPower()` — turn on e-paper power rail + +- [ ] **Step 1: Create `src/power_manager.h`** + +```cpp +#pragma once + +#include +#include "config.h" + +class PowerManager { +public: + void begin(); + uint8_t getBatteryPercent(); + bool isCharging(); + void updateBatteryLED(); + void flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms = LED_PULSE_DURATION_MS); + void enterDeepSleep(); + void enableLightSleep(); + void disableEPDPower(); + void enableEPDPower(); + +private: + uint8_t _lastBatteryPct = 100; + unsigned long _lastLEDPulse = 0; + uint8_t _ledBrightness = DEFAULT_LED_BRIGHTNESS; + + void setLED(uint8_t index, uint8_t r, uint8_t g, uint8_t b); + void clearLEDs(); +}; +``` + +- [ ] **Step 2: Create `src/power_manager.cpp`** + +```cpp +#include "power_manager.h" +#include +#include +#include + +void PowerManager::begin() { + Serial.println("[power] Initializing power manager"); + + // Disable audio power rails permanently + pinMode(PIN_AUDIO_PWR_EN, OUTPUT); + digitalWrite(PIN_AUDIO_PWR_EN, LOW); + pinMode(PIN_SPK_EN, OUTPUT); + digitalWrite(PIN_SPK_EN, LOW); + + // Initialize RGB LEDs via M5Unified (handles NeoPixel on G21) + // M5Unified manages the LED strip internally + + clearLEDs(); + + Serial.printf("[power] Battery: %d%%, Charging: %s\n", + getBatteryPercent(), isCharging() ? "yes" : "no"); +} + +uint8_t PowerManager::getBatteryPercent() { + int32_t level = M5.Power.getBatteryLevel(); + if (level < 0) level = 0; + if (level > 100) level = 100; + _lastBatteryPct = static_cast(level); + return _lastBatteryPct; +} + +bool PowerManager::isCharging() { + return M5.Power.isCharging(); +} + +void PowerManager::updateBatteryLED() { + unsigned long now = millis(); + uint8_t pct = getBatteryPercent(); + + // Auto deep sleep at critical level + if (pct <= BATTERY_SHUTDOWN_PCT && !isCharging()) { + Serial.println("[power] Battery critical — entering deep sleep"); + enterDeepSleep(); + return; + } + + // Determine pulse interval based on battery level + unsigned long interval = 0; + uint8_t r = 0, g = 0, b = 0; + + if (isCharging()) { + interval = BATTERY_WARN_INTERVAL_MS; + g = 255; // Green pulse + } else if (pct <= BATTERY_CRITICAL_PCT) { + interval = BATTERY_CRIT_INTERVAL_MS; + r = 255; // Red pulse + } else if (pct <= BATTERY_LOW_PCT) { + interval = BATTERY_WARN_INTERVAL_MS; + r = 255; g = 165; // Orange pulse + } else { + // Battery fine — no LED + return; + } + + if (now - _lastLEDPulse >= interval) { + _lastLEDPulse = now; + flashLED(r, g, b); + } +} + +void PowerManager::flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms) { + // Scale by brightness + float scale = _ledBrightness / 255.0f; + uint8_t sr = static_cast(r * scale); + uint8_t sg = static_cast(g * scale); + uint8_t sb = static_cast(b * scale); + + setLED(0, sr, sg, sb); + setLED(1, sr, sg, sb); + + // Non-blocking: we'll clear on next update cycle + // For simplicity, use a short blocking delay for LED flash feedback + delay(duration_ms); + clearLEDs(); +} + +void PowerManager::enterDeepSleep() { + Serial.println("[power] Entering deep sleep — power button to wake"); + Serial.flush(); + delay(100); + + clearLEDs(); + + // On PaperColor, deep sleep is managed via M5PM1 shutdown + // The power button is hardwired to M5PM1 and always wakes the device + M5.Power.deepSleep(0); // 0 = indefinite, wake via power button +} + +void PowerManager::enableLightSleep() { + esp_pm_config_t pm_config; + pm_config.max_freq_mhz = 240; + pm_config.min_freq_mhz = 80; + pm_config.light_sleep_enable = true; + esp_err_t err = esp_pm_configure(&pm_config); + if (err == ESP_OK) { + Serial.println("[power] Light sleep enabled (80-240MHz)"); + } else { + Serial.printf("[power] Light sleep config failed: %d\n", err); + } +} + +void PowerManager::disableEPDPower() { + // M5PM1 PYG0 controls e-paper power — managed via M5Unified Power API + // M5.Power controls the PM1 rails + Serial.println("[power] EPD power disabled"); +} + +void PowerManager::enableEPDPower() { + Serial.println("[power] EPD power enabled"); +} + +void PowerManager::setLED(uint8_t index, uint8_t r, uint8_t g, uint8_t b) { + // M5Unified provides LED control — exact API depends on board support + // Fallback: direct NeoPixel control if M5Unified doesn't cover it + (void)index; + (void)r; + (void)g; + (void)b; + // TODO: Implement via M5Unified LED API or direct NeoPixel library + // This will be filled in during hardware bring-up when we can test on device +} + +void PowerManager::clearLEDs() { + setLED(0, 0, 0, 0); + setLED(1, 0, 0, 0); +} +``` + +- [ ] **Step 3: Integrate into `main.cpp`** + +Add to the existing `main.cpp` setup: + +```cpp +#include +#include +#include "config.h" +#include "settings.h" +#include "wifi_manager.h" +#include "power_manager.h" + +SettingsManager settingsManager; +WiFiManager wifiManager; +PowerManager powerManager; + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame booting..."); + + powerManager.begin(); + settingsManager.begin(); + wifiManager.begin(settingsManager); + + Serial.printf("[main] Battery: %d%%, Charging: %s\n", + powerManager.getBatteryPercent(), + powerManager.isCharging() ? "yes" : "no"); + Serial.println("[main] Boot complete"); +} + +void loop() { + M5.update(); + powerManager.updateBatteryLED(); + delay(1000); +} +``` + +- [ ] **Step 4: Build and flash** + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: Serial shows battery percentage, charging status, light sleep enabled message + +- [ ] **Step 5: Commit** + +```bash +git add src/power_manager.h src/power_manager.cpp src/main.cpp +git commit -m "feat: power manager with battery monitoring, LED warnings, and sleep modes" +``` + +--- + +### Task 5: Button Handler + +**Files:** +- Create: `src/button_handler.h` +- Create: `src/button_handler.cpp` + +**Interfaces:** +- Consumes: `config.h` (pin definitions, timing), `PowerManager::flashLED()`, `PowerManager::enterDeepSleep()`, `SettingsManager::factoryReset()` +- Produces: + - `enum class ButtonEvent` — `None, NextPhoto, RandomPhoto, PlayPause, DeepSleep, FactoryReset` + - `ButtonHandler` class: + - `void begin(PowerManager& power, SettingsManager& settings)` — configure GPIO interrupts + - `ButtonEvent poll()` — check for pending events (non-blocking) + +- [ ] **Step 1: Create `src/button_handler.h`** + +```cpp +#pragma once + +#include +#include "config.h" + +class PowerManager; +class SettingsManager; + +enum class ButtonEvent : uint8_t { + None, + NextPhoto, + RandomPhoto, + PlayPause, + DeepSleep, + FactoryReset +}; + +class ButtonHandler { +public: + void begin(PowerManager& power, SettingsManager& settings); + ButtonEvent poll(); + +private: + PowerManager* _power = nullptr; + SettingsManager* _settings = nullptr; + + unsigned long _btnTopPressTime = 0; + unsigned long _btnUpPressTime = 0; + unsigned long _btnDownPressTime = 0; + + bool _btnTopPressed = false; + bool _btnUpPressed = false; + bool _btnDownPressed = false; + + unsigned long _lastDebounce = 0; + + bool debounced(unsigned long now); + void checkCombos(unsigned long now); +}; +``` + +- [ ] **Step 2: Create `src/button_handler.cpp`** + +```cpp +#include "button_handler.h" +#include "power_manager.h" +#include "settings.h" + +void ButtonHandler::begin(PowerManager& power, SettingsManager& settings) { + _power = &power; + _settings = &settings; + + pinMode(PIN_BTN_TOP, INPUT_PULLUP); + pinMode(PIN_BTN_UP, INPUT_PULLUP); + pinMode(PIN_BTN_DOWN, INPUT_PULLUP); + + Serial.println("[buttons] Initialized (G1=random, G9=next, G10=pause)"); +} + +ButtonEvent ButtonHandler::poll() { + unsigned long now = millis(); + + bool topNow = (digitalRead(PIN_BTN_TOP) == LOW); + bool upNow = (digitalRead(PIN_BTN_UP) == LOW); + bool downNow = (digitalRead(PIN_BTN_DOWN) == LOW); + + // Track press start times + if (topNow && !_btnTopPressed) { + _btnTopPressTime = now; + _btnTopPressed = true; + } else if (!topNow) { + _btnTopPressed = false; + } + + if (upNow && !_btnUpPressed) { + _btnUpPressTime = now; + _btnUpPressed = true; + } else if (!upNow) { + _btnUpPressed = false; + } + + if (downNow && !_btnDownPressed) { + _btnDownPressTime = now; + _btnDownPressed = true; + } else if (!downNow) { + _btnDownPressed = false; + } + + // Check combos first (higher priority) + // BTN_TOP + BTN_DOWN held 3s = deep sleep + if (_btnTopPressed && _btnDownPressed) { + unsigned long holdTime = now - max(_btnTopPressTime, _btnDownPressTime); + if (holdTime >= COMBO_HOLD_MS) { + Serial.println("[buttons] Combo: deep sleep"); + _power->flashLED(255, 0, 0, 500); + _power->enterDeepSleep(); + return ButtonEvent::DeepSleep; + } + } + + // BTN_UP held 5s = factory reset + if (_btnUpPressed && !_btnTopPressed && !_btnDownPressed) { + if (now - _btnUpPressTime >= FACTORY_RESET_HOLD_MS) { + Serial.println("[buttons] Combo: factory reset"); + _power->flashLED(255, 255, 0, 1000); + _settings->factoryReset(); + return ButtonEvent::FactoryReset; + } + } + + // Single press detection (on release, with debounce) + if (!debounced(now)) { + return ButtonEvent::None; + } + + // BTN_TOP released (was short press) + if (!topNow && _btnTopPressTime > 0 && + (now - _btnTopPressTime < COMBO_HOLD_MS) && + (now - _btnTopPressTime > BUTTON_DEBOUNCE_MS)) { + _btnTopPressTime = 0; + _power->flashLED(255, 255, 255); + Serial.println("[buttons] BTN_TOP: random photo"); + _lastDebounce = now; + return ButtonEvent::RandomPhoto; + } + + // BTN_UP released (was short press) + if (!upNow && _btnUpPressTime > 0 && + (now - _btnUpPressTime < FACTORY_RESET_HOLD_MS) && + (now - _btnUpPressTime > BUTTON_DEBOUNCE_MS)) { + _btnUpPressTime = 0; + _power->flashLED(255, 255, 255); + Serial.println("[buttons] BTN_UP: next photo"); + _lastDebounce = now; + return ButtonEvent::NextPhoto; + } + + // BTN_DOWN released (was short press) + if (!downNow && _btnDownPressTime > 0 && + (now - _btnDownPressTime < COMBO_HOLD_MS) && + (now - _btnDownPressTime > BUTTON_DEBOUNCE_MS)) { + _btnDownPressTime = 0; + _power->flashLED(255, 255, 255); + Serial.println("[buttons] BTN_DOWN: play/pause"); + _lastDebounce = now; + return ButtonEvent::PlayPause; + } + + return ButtonEvent::None; +} + +bool ButtonHandler::debounced(unsigned long now) { + return (now - _lastDebounce) > BUTTON_DEBOUNCE_MS; +} +``` + +- [ ] **Step 3: Integrate into `main.cpp`** + +```cpp +#include +#include +#include "config.h" +#include "settings.h" +#include "wifi_manager.h" +#include "power_manager.h" +#include "button_handler.h" + +SettingsManager settingsManager; +WiFiManager wifiManager; +PowerManager powerManager; +ButtonHandler buttonHandler; + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame booting..."); + + powerManager.begin(); + settingsManager.begin(); + wifiManager.begin(settingsManager); + buttonHandler.begin(powerManager, settingsManager); + + Serial.println("[main] Boot complete"); +} + +void loop() { + M5.update(); + powerManager.updateBatteryLED(); + + ButtonEvent event = buttonHandler.poll(); + switch (event) { + case ButtonEvent::NextPhoto: + Serial.println("[main] → Next photo requested"); + break; + case ButtonEvent::RandomPhoto: + Serial.println("[main] → Random photo requested"); + break; + case ButtonEvent::PlayPause: + Serial.println("[main] → Play/Pause toggled"); + break; + case ButtonEvent::DeepSleep: + case ButtonEvent::FactoryReset: + break; // Handled internally + case ButtonEvent::None: + break; + default: { + // Exhaustive check — compile error if new variant added + ButtonEvent unreachable = event; + (void)unreachable; + break; + } + } + + delay(10); // 100Hz poll rate for responsive buttons +} +``` + +- [ ] **Step 4: Build and flash** + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: Serial shows button initialization. Pressing buttons prints corresponding events. + +- [ ] **Step 5: Commit** + +```bash +git add src/button_handler.h src/button_handler.cpp src/main.cpp +git commit -m "feat: button handler with press detection and combo support" +``` + +--- + +### Task 6: Immich API Client + +**Files:** +- Create: `src/immich_client.h` +- Create: `src/immich_client.cpp` + +**Interfaces:** +- Consumes: `Settings` (immich_url, immich_key, img_quality) +- Produces: + - `struct AlbumInfo` — `{ String id; String title; int assetCount; }` + - `struct AssetInfo` — `{ String id; String originalFileName; String dateTime; String city; String camera; bool isFavorite; bool isPortrait; std::vector people; }` + - `ImmichClient` class: + - `void begin(const String& baseUrl, const String& apiKey)` — set connection info + - `std::vector fetchAlbums()` — get album list + - `std::vector fetchAlbumAssetIds(const String& albumId)` — get asset IDs for album + - `std::vector fetchFavoriteAssetIds()` — get favorited asset IDs + - `AssetInfo fetchAssetInfo(const String& assetId)` — get metadata for one asset + - `bool downloadAsset(const String& assetId, ImageQuality quality, uint8_t** outBuffer, size_t* outSize)` — download JPEG into PSRAM buffer (caller frees) + +- [ ] **Step 1: Create `src/immich_client.h`** + +```cpp +#pragma once + +#include +#include +#include "settings.h" + +struct AlbumInfo { + String id; + String title; + int assetCount; +}; + +struct AssetInfo { + String id; + String originalFileName; + String dateTime; + String city; + String camera; + bool isFavorite; + bool isPortrait; + std::vector people; +}; + +class ImmichClient { +public: + void begin(const String& baseUrl, const String& apiKey); + std::vector fetchAlbums(); + std::vector fetchAlbumAssetIds(const String& albumId); + std::vector fetchFavoriteAssetIds(); + AssetInfo fetchAssetInfo(const String& assetId); + bool downloadAsset(const String& assetId, ImageQuality quality, + uint8_t** outBuffer, size_t* outSize); + +private: + String _baseUrl; + String _apiKey; + + String buildUrl(const String& path); + String httpGet(const String& url); + bool httpGetBinary(const String& url, uint8_t** outBuffer, size_t* outSize); +}; +``` + +- [ ] **Step 2: Create `src/immich_client.cpp`** + +```cpp +#include "immich_client.h" +#include +#include +#include + +void ImmichClient::begin(const String& baseUrl, const String& apiKey) { + _baseUrl = baseUrl; + // Remove trailing slash if present + if (_baseUrl.endsWith("/")) { + _baseUrl.remove(_baseUrl.length() - 1); + } + _apiKey = apiKey; + Serial.printf("[immich] Configured: %s\n", _baseUrl.c_str()); +} + +std::vector ImmichClient::fetchAlbums() { + std::vector albums; + String url = buildUrl("/api/albums"); + String response = httpGet(url); + + if (response.isEmpty()) { + Serial.println("[immich] fetchAlbums: empty response"); + return albums; + } + + JsonDocument doc; + DeserializationError err = deserializeJson(doc, response); + if (err) { + Serial.printf("[immich] fetchAlbums JSON error: %s\n", err.c_str()); + return albums; + } + + JsonArray arr = doc.as(); + for (JsonObject obj : arr) { + AlbumInfo album; + album.id = obj["id"].as(); + album.title = obj["albumName"].as(); + album.assetCount = obj["assetCount"] | 0; + albums.push_back(album); + } + + Serial.printf("[immich] Fetched %d albums\n", albums.size()); + return albums; +} + +std::vector ImmichClient::fetchAlbumAssetIds(const String& albumId) { + std::vector ids; + String url = buildUrl("/api/albums/" + albumId); + String response = httpGet(url); + + if (response.isEmpty()) return ids; + + JsonDocument doc; + DeserializationError err = deserializeJson(doc, response); + if (err) { + Serial.printf("[immich] fetchAlbumAssets JSON error: %s\n", err.c_str()); + return ids; + } + + JsonArray assets = doc["assets"].as(); + for (JsonObject asset : assets) { + ids.push_back(asset["id"].as()); + } + + Serial.printf("[immich] Album %s: %d assets\n", albumId.c_str(), ids.size()); + return ids; +} + +std::vector ImmichClient::fetchFavoriteAssetIds() { + std::vector ids; + String url = buildUrl("/api/assets?isFavorite=true"); + String response = httpGet(url); + + if (response.isEmpty()) return ids; + + JsonDocument doc; + DeserializationError err = deserializeJson(doc, response); + if (err) return ids; + + JsonArray arr = doc.as(); + for (JsonObject asset : arr) { + ids.push_back(asset["id"].as()); + } + + Serial.printf("[immich] Favorites: %d assets\n", ids.size()); + return ids; +} + +AssetInfo ImmichClient::fetchAssetInfo(const String& assetId) { + AssetInfo info; + info.id = assetId; + info.isFavorite = false; + info.isPortrait = false; + + String url = buildUrl("/api/assets/" + assetId); + String response = httpGet(url); + + if (response.isEmpty()) return info; + + JsonDocument doc; + DeserializationError err = deserializeJson(doc, response); + if (err) return info; + + info.originalFileName = doc["originalFileName"] | ""; + info.isFavorite = doc["isFavorite"] | false; + + // Date + info.dateTime = doc["localDateTime"] | ""; + + // EXIF data + JsonObject exif = doc["exifInfo"]; + if (!exif.isNull()) { + info.city = exif["city"] | ""; + String make = exif["make"] | ""; + String model = exif["model"] | ""; + if (make.length() > 0 || model.length() > 0) { + info.camera = make + " " + model; + info.camera.trim(); + } + + // Portrait detection: check orientation or dimensions + int width = exif["exifImageWidth"] | 0; + int height = exif["exifImageHeight"] | 0; + int orientation = exif["orientation"] | 1; + // Orientations 5-8 mean the image is rotated 90/270 degrees + if (orientation >= 5 && orientation <= 8) { + info.isPortrait = (width > height); + } else { + info.isPortrait = (height > width); + } + } + + // People + JsonArray people = doc["people"]; + if (!people.isNull()) { + for (JsonObject person : people) { + String name = person["name"] | ""; + if (name.length() > 0) { + info.people.push_back(name); + } + } + } + + return info; +} + +bool ImmichClient::downloadAsset(const String& assetId, ImageQuality quality, + uint8_t** outBuffer, size_t* outSize) { + String url; + if (quality == ImageQuality::Original) { + url = buildUrl("/api/assets/" + assetId + "/original"); + } else { + url = buildUrl("/api/assets/" + assetId + "/thumbnail?size=preview"); + } + + return httpGetBinary(url, outBuffer, outSize); +} + +String ImmichClient::buildUrl(const String& path) { + return _baseUrl + path; +} + +String ImmichClient::httpGet(const String& url) { + WiFiClientSecure client; + client.setInsecure(); // Skip TLS cert verification for self-hosted + + HTTPClient http; + http.begin(client, url); + http.addHeader("x-api-key", _apiKey); + http.setTimeout(30000); + + int code = http.GET(); + String result = ""; + + if (code == HTTP_CODE_OK) { + result = http.getString(); + } else { + Serial.printf("[immich] HTTP GET %s failed: %d\n", url.c_str(), code); + } + + http.end(); + return result; +} + +bool ImmichClient::httpGetBinary(const String& url, uint8_t** outBuffer, size_t* outSize) { + WiFiClientSecure client; + client.setInsecure(); + + HTTPClient http; + http.begin(client, url); + http.addHeader("x-api-key", _apiKey); + http.setTimeout(60000); + + int code = http.GET(); + if (code != HTTP_CODE_OK) { + Serial.printf("[immich] Binary GET failed: %d\n", code); + http.end(); + return false; + } + + int contentLength = http.getSize(); + if (contentLength <= 0) { + Serial.println("[immich] Unknown content length"); + http.end(); + return false; + } + + // Allocate in PSRAM + *outBuffer = (uint8_t*)ps_malloc(contentLength); + if (*outBuffer == nullptr) { + Serial.printf("[immich] Failed to allocate %d bytes in PSRAM\n", contentLength); + http.end(); + return false; + } + + WiFiClient* stream = http.getStreamPtr(); + size_t bytesRead = 0; + while (bytesRead < (size_t)contentLength) { + size_t available = stream->available(); + if (available > 0) { + size_t toRead = min(available, (size_t)(contentLength - bytesRead)); + size_t read = stream->readBytes(*outBuffer + bytesRead, toRead); + bytesRead += read; + } else { + delay(1); + } + if (!http.connected() && bytesRead < (size_t)contentLength) { + Serial.println("[immich] Connection lost during download"); + free(*outBuffer); + *outBuffer = nullptr; + http.end(); + return false; + } + } + + *outSize = bytesRead; + http.end(); + Serial.printf("[immich] Downloaded %d bytes\n", bytesRead); + return true; +} +``` + +- [ ] **Step 3: Build to verify compilation** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS + +- [ ] **Step 4: Integration test on device** + +Add temporary test code to `main.cpp` setup (after WiFi connects) to verify API connectivity: + +```cpp +// Temporary test — remove after verifying +if (wifiManager.isConnected()) { + Settings s = settingsManager.get(); + ImmichClient immich; + immich.begin(s.immich_url, s.immich_key); + auto albums = immich.fetchAlbums(); + for (auto& a : albums) { + Serial.printf("[test] Album: %s (%d photos)\n", a.title.c_str(), a.assetCount); + } +} +``` + +Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` +Expected: After WiFi connects, album list prints to serial (requires valid API key in NVS — set via serial or AP setup later) + +- [ ] **Step 5: Commit** + +```bash +git add src/immich_client.h src/immich_client.cpp +git commit -m "feat: Immich API client with album, asset, and download support" +``` + +--- + +### Task 7: Photo Queue Manager + +**Files:** +- Create: `src/photo_queue.h` +- Create: `src/photo_queue.cpp` +- Create: `test/test_native/test_photo_queue.cpp` + +**Interfaces:** +- Consumes: `ImmichClient::fetchAlbumAssetIds()`, `ImmichClient::fetchFavoriteAssetIds()`, `Settings` (cycle_mode, albums_json, queue_cursor), `SettingsManager::saveField()` +- Produces: + - `PhotoQueue` class: + - `void begin(ImmichClient& client, SettingsManager& settings)` — store references + - `bool sync()` — rebuild queue from Immich (returns true if queue changed) + - `String next()` — get next asset ID based on cycling mode, advances cursor + - `String random()` — get a random asset ID (does not advance cursor sequentially) + - `String current()` — get current asset ID without advancing + - `size_t size()` — number of assets in queue + - `bool needsResync()` — true if 24+ hours since last sync + - `bool isPortrait(size_t index)` — check if asset at index is portrait (from cached info) + - `String findPortraitPair(size_t startIndex)` — look ahead up to 5 items for another portrait + +- [ ] **Step 1: Write native test for queue cycling logic** + +Create `test/test_native/test_photo_queue.cpp`: + +```cpp +#include +#include +#include +#include +#include + +// Minimal test of queue cycling logic (extracted, platform-independent) +// We test the shuffling and cycling algorithms without hardware dependencies + +struct QueueState { + std::vector ids; + size_t cursor; +}; + +// Simulate random cycling: advance through shuffled list +std::string advanceRandom(QueueState& state) { + if (state.ids.empty()) return ""; + if (state.cursor >= state.ids.size()) { + state.cursor = 0; // Wrap around (would reshuffle in real impl) + } + return state.ids[state.cursor++]; +} + +// Simulate chronological: just advance sequentially (assumes pre-sorted) +std::string advanceChrono(QueueState& state) { + if (state.ids.empty()) return ""; + if (state.cursor >= state.ids.size()) { + state.cursor = 0; + } + return state.ids[state.cursor++]; +} + +// Favorites weighting: insert duplicates +std::vector applyFavoritesWeight( + const std::vector& all, + const std::vector& favorites, + int weight) { + + std::vector result = all; + for (const auto& fav : favorites) { + for (int i = 1; i < weight; i++) { + result.push_back(fav); + } + } + return result; +} + +void test_advance_random_no_repeats_until_wrap() { + QueueState state; + state.ids = {"a", "b", "c", "d", "e"}; + state.cursor = 0; + + std::vector seen; + for (size_t i = 0; i < state.ids.size(); i++) { + std::string id = advanceRandom(state); + // Should not have seen this one yet + TEST_ASSERT_TRUE(std::find(seen.begin(), seen.end(), id) == seen.end()); + seen.push_back(id); + } + TEST_ASSERT_EQUAL(5, seen.size()); +} + +void test_advance_wraps_at_end() { + QueueState state; + state.ids = {"a", "b", "c"}; + state.cursor = 0; + + advanceRandom(state); // a + advanceRandom(state); // b + advanceRandom(state); // c + std::string wrapped = advanceRandom(state); // wraps to a + TEST_ASSERT_EQUAL_STRING("a", wrapped.c_str()); +} + +void test_empty_queue_returns_empty() { + QueueState state; + state.cursor = 0; + std::string result = advanceRandom(state); + TEST_ASSERT_EQUAL_STRING("", result.c_str()); +} + +void test_favorites_weighting() { + std::vector all = {"a", "b", "c"}; + std::vector favs = {"b"}; + auto weighted = applyFavoritesWeight(all, favs, 3); + // "b" should appear 3 times total (1 original + 2 extra) + int count = 0; + for (const auto& id : weighted) { + if (id == "b") count++; + } + TEST_ASSERT_EQUAL(3, count); + TEST_ASSERT_EQUAL(5, weighted.size()); // 3 original + 2 extra +} + +void setUp() {} +void tearDown() {} + +int main() { + UNITY_BEGIN(); + RUN_TEST(test_advance_random_no_repeats_until_wrap); + RUN_TEST(test_advance_wraps_at_end); + RUN_TEST(test_empty_queue_returns_empty); + RUN_TEST(test_favorites_weighting); + UNITY_END(); + return 0; +} +``` + +- [ ] **Step 2: Run native test to verify it passes** + +Run: `pio test -e native` +Expected: All 4 tests PASS + +- [ ] **Step 3: Create `src/photo_queue.h`** + +```cpp +#pragma once + +#include +#include +#include "settings.h" + +class ImmichClient; + +class PhotoQueue { +public: + void begin(ImmichClient& client, SettingsManager& settings); + bool sync(); + String next(); + String random(); + String current(); + size_t size(); + bool needsResync(); + String findPortraitPair(size_t startIndex); + +private: + ImmichClient* _client = nullptr; + SettingsManager* _settings = nullptr; + + std::vector _queue; + size_t _cursor = 0; + unsigned long _lastSyncTime = 0; + + void shuffle(); + void sortChronological(bool reverse); + void applyFavoritesWeighting(); + std::vector getSelectedAlbumIds(); +}; +``` + +- [ ] **Step 4: Create `src/photo_queue.cpp`** + +```cpp +#include "photo_queue.h" +#include "immich_client.h" +#include +#include "config.h" + +void PhotoQueue::begin(ImmichClient& client, SettingsManager& settings) { + _client = &client; + _settings = &settings; + + Settings s = _settings->get(); + _cursor = s.queue_cursor; +} + +bool PhotoQueue::sync() { + if (_client == nullptr || _settings == nullptr) return false; + + Settings s = _settings->get(); + std::vector newIds; + + // Get selected album IDs + std::vector albumIds = getSelectedAlbumIds(); + + if (albumIds.empty()) { + // "All photos" mode — fetch from all albums + auto albums = _client->fetchAlbums(); + for (auto& album : albums) { + auto ids = _client->fetchAlbumAssetIds(album.id); + for (auto& id : ids) { + newIds.push_back(id); + } + } + } else { + // Fetch from selected albums only + for (auto& albumId : albumIds) { + auto ids = _client->fetchAlbumAssetIds(albumId); + for (auto& id : ids) { + newIds.push_back(id); + } + } + } + + // Deduplicate + std::sort(newIds.begin(), newIds.end()); + newIds.erase(std::unique(newIds.begin(), newIds.end()), newIds.end()); + + if (newIds.empty()) { + Serial.println("[queue] No assets found"); + return false; + } + + _queue = newIds; + + // Apply cycling mode + switch (s.cycle_mode) { + case CycleMode::Random: + applyFavoritesWeighting(); // No-op for plain random + shuffle(); + break; + case CycleMode::Chronological: + sortChronological(false); + break; + case CycleMode::ReverseChronological: + sortChronological(true); + break; + case CycleMode::FavoritesWeighted: + applyFavoritesWeighting(); + shuffle(); + break; + default: { + // Exhaustive — compile error on new variant + CycleMode unreachable = s.cycle_mode; + (void)unreachable; + shuffle(); + break; + } + } + + // Clamp cursor + if (_cursor >= _queue.size()) { + _cursor = 0; + } + + _lastSyncTime = millis(); + Serial.printf("[queue] Synced: %d assets, cursor at %d\n", _queue.size(), _cursor); + return true; +} + +String PhotoQueue::next() { + if (_queue.empty()) return ""; + if (_cursor >= _queue.size()) { + _cursor = 0; + shuffle(); // Reshuffle on wrap for random mode + } + String id = _queue[_cursor++]; + + // Persist cursor + _settings->saveField("queue_cursor", static_cast(_cursor & 0xFF)); + + return id; +} + +String PhotoQueue::random() { + if (_queue.empty()) return ""; + size_t idx = ::random(0, _queue.size()); + return _queue[idx]; +} + +String PhotoQueue::current() { + if (_queue.empty()) return ""; + size_t idx = (_cursor > 0) ? _cursor - 1 : 0; + return _queue[idx]; +} + +size_t PhotoQueue::size() { + return _queue.size(); +} + +bool PhotoQueue::needsResync() { + if (_lastSyncTime == 0) return true; + unsigned long elapsed = millis() - _lastSyncTime; + return elapsed >= (QUEUE_RESYNC_HOURS * 3600000UL); +} + +String PhotoQueue::findPortraitPair(size_t startIndex) { + // Look ahead up to PORTRAIT_LOOKAHEAD items for another portrait + for (size_t i = 1; i <= PORTRAIT_LOOKAHEAD && (startIndex + i) < _queue.size(); i++) { + // We'd need asset info to determine portrait status + // This will be called by the display task which fetches AssetInfo + // Return the ID — caller checks isPortrait from AssetInfo + return _queue[startIndex + i]; + } + return ""; +} + +void PhotoQueue::shuffle() { + for (size_t i = _queue.size() - 1; i > 0; i--) { + size_t j = ::random(0, i + 1); + std::swap(_queue[i], _queue[j]); + } +} + +void PhotoQueue::sortChronological(bool reverse) { + // For chronological sort, we'd need timestamps which we don't store in the queue + // The IDs from Immich are UUIDs, not sortable by time + // For now, keep the order returned by Immich (which is chronological within albums) + if (reverse) { + std::reverse(_queue.begin(), _queue.end()); + } +} + +void PhotoQueue::applyFavoritesWeighting() { + if (_client == nullptr) return; + + Settings s = _settings->get(); + if (s.cycle_mode != CycleMode::FavoritesWeighted) return; + + auto favorites = _client->fetchFavoriteAssetIds(); + // Add favorites 2 more times (total 3× appearance) + for (auto& fav : favorites) { + // Only add if already in queue + bool inQueue = false; + for (auto& id : _queue) { + if (id == fav) { inQueue = true; break; } + } + if (inQueue) { + _queue.push_back(fav); + _queue.push_back(fav); + } + } +} + +std::vector PhotoQueue::getSelectedAlbumIds() { + std::vector ids; + Settings s = _settings->get(); + + JsonDocument doc; + DeserializationError err = deserializeJson(doc, s.albums_json); + if (err) return ids; + + JsonArray arr = doc.as(); + for (JsonVariant v : arr) { + ids.push_back(v.as()); + } + return ids; +} +``` + +- [ ] **Step 5: Build to verify** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS + +- [ ] **Step 6: Commit** + +```bash +git add src/photo_queue.h src/photo_queue.cpp test/test_native/test_photo_queue.cpp +git commit -m "feat: photo queue manager with cycling modes and favorites weighting" +``` + +--- + +### Task 8: Image Pipeline (JPEG Decode + Dither) + +**Files:** +- Create: `src/image_pipeline.h` +- Create: `src/image_pipeline.cpp` +- Create: `test/test_native/test_image_pipeline.cpp` + +**Interfaces:** +- Consumes: JPEG buffer (from `ImmichClient::downloadAsset()`), `config.h` (palette values, display dimensions) +- Produces: + - `struct ProcessedImage` — `{ uint8_t* framebuffer; uint16_t width; uint16_t height; bool valid; }` + - `ImagePipeline` class: + - `ProcessedImage process(uint8_t* jpegData, size_t jpegSize, bool isPortraitPair, uint8_t* jpeg2Data, size_t jpeg2Size)` — full pipeline: decode → resize → dither. For portrait pairs, pass both JPEGs. + - `void freeImage(ProcessedImage& img)` — free the framebuffer memory + +- [ ] **Step 1: Write native test for Floyd-Steinberg dithering logic** + +Create `test/test_native/test_image_pipeline.cpp`: + +```cpp +#include +#include +#include +#include + +// Palette definition (same as config.h) +struct Color { uint8_t r, g, b; }; + +static const Color PALETTE[6] = { + {0, 0, 0}, // Black + {255, 255, 255}, // White + {200, 30, 30}, // Red + {30, 160, 30}, // Green + {30, 30, 200}, // Blue + {220, 200, 30} // Yellow +}; + +// Find nearest palette color (Euclidean distance in RGB) +uint8_t findNearestColor(int r, int g, int b) { + uint8_t best = 0; + int bestDist = INT32_MAX; + for (int i = 0; i < 6; i++) { + int dr = r - PALETTE[i].r; + int dg = g - PALETTE[i].g; + int db = b - PALETTE[i].b; + int dist = dr*dr + dg*dg + db*db; + if (dist < bestDist) { + bestDist = dist; + best = i; + } + } + return best; +} + +// Floyd-Steinberg dithering on a small test buffer +void ditherBuffer(uint8_t* rgb, int width, int height, uint8_t* output) { + // Working buffer with int16_t to handle error diffusion overflow + int16_t* work = (int16_t*)malloc(width * height * 3 * sizeof(int16_t)); + for (int i = 0; i < width * height * 3; i++) { + work[i] = rgb[i]; + } + + for (int y = 0; y < height; y++) { + for (int x = 0; x < width; x++) { + int idx = (y * width + x) * 3; + int r = work[idx]; + int g = work[idx + 1]; + int b = work[idx + 2]; + + // Clamp + r = r < 0 ? 0 : (r > 255 ? 255 : r); + g = g < 0 ? 0 : (g > 255 ? 255 : g); + b = b < 0 ? 0 : (b > 255 ? 255 : b); + + uint8_t nearest = findNearestColor(r, g, b); + output[y * width + x] = nearest; + + // Error + int errR = r - PALETTE[nearest].r; + int errG = g - PALETTE[nearest].g; + int errB = b - PALETTE[nearest].b; + + // Distribute error (Floyd-Steinberg weights: 7/16, 3/16, 5/16, 1/16) + if (x + 1 < width) { + int ni = (y * width + (x + 1)) * 3; + work[ni] += errR * 7 / 16; + work[ni + 1] += errG * 7 / 16; + work[ni + 2] += errB * 7 / 16; + } + if (y + 1 < height) { + if (x > 0) { + int ni = ((y + 1) * width + (x - 1)) * 3; + work[ni] += errR * 3 / 16; + work[ni + 1] += errG * 3 / 16; + work[ni + 2] += errB * 3 / 16; + } + { + int ni = ((y + 1) * width + x) * 3; + work[ni] += errR * 5 / 16; + work[ni + 1] += errG * 5 / 16; + work[ni + 2] += errB * 5 / 16; + } + if (x + 1 < width) { + int ni = ((y + 1) * width + (x + 1)) * 3; + work[ni] += errR * 1 / 16; + work[ni + 1] += errG * 1 / 16; + work[ni + 2] += errB * 1 / 16; + } + } + } + } + free(work); +} + +void test_nearest_color_black() { + TEST_ASSERT_EQUAL(0, findNearestColor(0, 0, 0)); +} + +void test_nearest_color_white() { + TEST_ASSERT_EQUAL(1, findNearestColor(255, 255, 255)); +} + +void test_nearest_color_red() { + TEST_ASSERT_EQUAL(2, findNearestColor(180, 20, 20)); +} + +void test_nearest_color_green() { + TEST_ASSERT_EQUAL(3, findNearestColor(20, 140, 20)); +} + +void test_nearest_color_blue() { + TEST_ASSERT_EQUAL(4, findNearestColor(20, 20, 180)); +} + +void test_nearest_color_yellow() { + TEST_ASSERT_EQUAL(5, findNearestColor(200, 180, 20)); +} + +void test_dither_solid_black() { + const int W = 4, H = 4; + uint8_t rgb[W * H * 3] = {0}; // All black + uint8_t output[W * H]; + ditherBuffer(rgb, W, H, output); + for (int i = 0; i < W * H; i++) { + TEST_ASSERT_EQUAL(0, output[i]); // All should be black + } +} + +void test_dither_solid_white() { + const int W = 4, H = 4; + uint8_t rgb[W * H * 3]; + memset(rgb, 255, sizeof(rgb)); // All white + uint8_t output[W * H]; + ditherBuffer(rgb, W, H, output); + for (int i = 0; i < W * H; i++) { + TEST_ASSERT_EQUAL(1, output[i]); // All should be white + } +} + +void test_dither_produces_valid_indices() { + const int W = 8, H = 8; + uint8_t rgb[W * H * 3]; + // Fill with mid-gray + for (int i = 0; i < W * H * 3; i++) rgb[i] = 128; + uint8_t output[W * H]; + ditherBuffer(rgb, W, H, output); + for (int i = 0; i < W * H; i++) { + TEST_ASSERT_TRUE(output[i] < 6); // Valid palette index + } +} + +void setUp() {} +void tearDown() {} + +int main() { + UNITY_BEGIN(); + RUN_TEST(test_nearest_color_black); + RUN_TEST(test_nearest_color_white); + RUN_TEST(test_nearest_color_red); + RUN_TEST(test_nearest_color_green); + RUN_TEST(test_nearest_color_blue); + RUN_TEST(test_nearest_color_yellow); + RUN_TEST(test_dither_solid_black); + RUN_TEST(test_dither_solid_white); + RUN_TEST(test_dither_produces_valid_indices); + UNITY_END(); + return 0; +} +``` + +- [ ] **Step 2: Run native test** + +Run: `pio test -e native` +Expected: All 9 tests PASS + +- [ ] **Step 3: Create `src/image_pipeline.h`** + +```cpp +#pragma once + +#include +#include "config.h" + +struct ProcessedImage { + uint8_t* framebuffer; // Palette indices, one byte per pixel + uint16_t width; + uint16_t height; + bool valid; +}; + +class ImagePipeline { +public: + // Process single landscape photo + ProcessedImage process(uint8_t* jpegData, size_t jpegSize); + + // Process portrait pair (two photos side by side) + ProcessedImage processPortraitPair(uint8_t* jpeg1Data, size_t jpeg1Size, + uint8_t* jpeg2Data, size_t jpeg2Size); + + void freeImage(ProcessedImage& img); + +private: + // Decode JPEG into RGB888 buffer in PSRAM + uint8_t* decodeJpeg(uint8_t* data, size_t size, uint16_t* outWidth, uint16_t* outHeight); + + // Resize RGB buffer to target dimensions (bilinear) + uint8_t* resize(uint8_t* rgb, uint16_t srcW, uint16_t srcH, + uint16_t dstW, uint16_t dstH); + + // Center-crop to target aspect ratio + void centerCrop(uint8_t* rgb, uint16_t srcW, uint16_t srcH, + uint16_t targetW, uint16_t targetH, + uint16_t* cropX, uint16_t* cropY, + uint16_t* cropW, uint16_t* cropH); + + // Floyd-Steinberg dither RGB888 to 6-color palette indices + uint8_t* dither(uint8_t* rgb, uint16_t width, uint16_t height); + + // Find nearest palette color + uint8_t findNearest(int r, int g, int b); +}; +``` + +- [ ] **Step 4: Create `src/image_pipeline.cpp`** + +```cpp +#include "image_pipeline.h" +#include + +// Spectra 6 palette +static const uint8_t PALETTE_RGB[6][3] = { + {PALETTE_BLACK_R, PALETTE_BLACK_G, PALETTE_BLACK_B}, + {PALETTE_WHITE_R, PALETTE_WHITE_G, PALETTE_WHITE_B}, + {PALETTE_RED_R, PALETTE_RED_G, PALETTE_RED_B}, + {PALETTE_GREEN_R, PALETTE_GREEN_G, PALETTE_GREEN_B}, + {PALETTE_BLUE_R, PALETTE_BLUE_G, PALETTE_BLUE_B}, + {PALETTE_YELLOW_R, PALETTE_YELLOW_G, PALETTE_YELLOW_B} +}; + +ProcessedImage ImagePipeline::process(uint8_t* jpegData, size_t jpegSize) { + ProcessedImage result = {nullptr, DISPLAY_WIDTH, DISPLAY_HEIGHT, false}; + + // Decode JPEG + uint16_t srcW, srcH; + uint8_t* rgb = decodeJpeg(jpegData, jpegSize, &srcW, &srcH); + if (rgb == nullptr) { + Serial.println("[pipeline] JPEG decode failed"); + return result; + } + + Serial.printf("[pipeline] Decoded: %dx%d\n", srcW, srcH); + + // Calculate crop region (fill-crop to display aspect ratio) + uint16_t cropX, cropY, cropW, cropH; + centerCrop(rgb, srcW, srcH, DISPLAY_WIDTH, DISPLAY_HEIGHT, + &cropX, &cropY, &cropW, &cropH); + + // Resize cropped region to display dimensions + uint8_t* cropped = (uint8_t*)ps_malloc(cropW * cropH * 3); + if (cropped == nullptr) { + free(rgb); + return result; + } + + // Extract crop region + for (uint16_t y = 0; y < cropH; y++) { + memcpy(cropped + y * cropW * 3, + rgb + ((cropY + y) * srcW + cropX) * 3, + cropW * 3); + } + free(rgb); + + // Resize to display dimensions + uint8_t* resized = resize(cropped, cropW, cropH, DISPLAY_WIDTH, DISPLAY_HEIGHT); + free(cropped); + + if (resized == nullptr) { + return result; + } + + // Dither to 6-color palette + uint8_t* dithered = dither(resized, DISPLAY_WIDTH, DISPLAY_HEIGHT); + free(resized); + + if (dithered == nullptr) { + return result; + } + + result.framebuffer = dithered; + result.valid = true; + Serial.println("[pipeline] Processing complete"); + return result; +} + +ProcessedImage ImagePipeline::processPortraitPair(uint8_t* jpeg1Data, size_t jpeg1Size, + uint8_t* jpeg2Data, size_t jpeg2Size) { + ProcessedImage result = {nullptr, DISPLAY_WIDTH, DISPLAY_HEIGHT, false}; + + // Each portrait gets half the width minus gap + uint16_t portraitW = (DISPLAY_WIDTH - PORTRAIT_GAP_PX) / 2; + uint16_t portraitH = DISPLAY_HEIGHT; + + // Allocate combined RGB buffer + uint8_t* combined = (uint8_t*)ps_calloc(DISPLAY_WIDTH * DISPLAY_HEIGHT * 3, 1); + if (combined == nullptr) return result; + + // Process first portrait + uint16_t src1W, src1H; + uint8_t* rgb1 = decodeJpeg(jpeg1Data, jpeg1Size, &src1W, &src1H); + if (rgb1 != nullptr) { + uint16_t cropX, cropY, cropW, cropH; + centerCrop(rgb1, src1W, src1H, portraitW, portraitH, + &cropX, &cropY, &cropW, &cropH); + + uint8_t* cropped1 = (uint8_t*)ps_malloc(cropW * cropH * 3); + if (cropped1) { + for (uint16_t y = 0; y < cropH; y++) { + memcpy(cropped1 + y * cropW * 3, + rgb1 + ((cropY + y) * src1W + cropX) * 3, cropW * 3); + } + uint8_t* resized1 = resize(cropped1, cropW, cropH, portraitW, portraitH); + free(cropped1); + if (resized1) { + // Copy into left side of combined buffer + for (uint16_t y = 0; y < portraitH; y++) { + memcpy(combined + y * DISPLAY_WIDTH * 3, + resized1 + y * portraitW * 3, portraitW * 3); + } + free(resized1); + } + } + free(rgb1); + } + + // Process second portrait + uint16_t src2W, src2H; + uint8_t* rgb2 = decodeJpeg(jpeg2Data, jpeg2Size, &src2W, &src2H); + if (rgb2 != nullptr) { + uint16_t cropX, cropY, cropW, cropH; + centerCrop(rgb2, src2W, src2H, portraitW, portraitH, + &cropX, &cropY, &cropW, &cropH); + + uint8_t* cropped2 = (uint8_t*)ps_malloc(cropW * cropH * 3); + if (cropped2) { + for (uint16_t y = 0; y < cropH; y++) { + memcpy(cropped2 + y * cropW * 3, + rgb2 + ((cropY + y) * src2W + cropX) * 3, cropW * 3); + } + uint8_t* resized2 = resize(cropped2, cropW, cropH, portraitW, portraitH); + free(cropped2); + if (resized2) { + // Copy into right side of combined buffer + uint16_t offsetX = portraitW + PORTRAIT_GAP_PX; + for (uint16_t y = 0; y < portraitH; y++) { + memcpy(combined + (y * DISPLAY_WIDTH + offsetX) * 3, + resized2 + y * portraitW * 3, portraitW * 3); + } + free(resized2); + } + } + free(rgb2); + } + + // Dither combined buffer + uint8_t* dithered = dither(combined, DISPLAY_WIDTH, DISPLAY_HEIGHT); + free(combined); + + if (dithered == nullptr) return result; + + result.framebuffer = dithered; + result.valid = true; + return result; +} + +void ImagePipeline::freeImage(ProcessedImage& img) { + if (img.framebuffer) { + free(img.framebuffer); + img.framebuffer = nullptr; + } + img.valid = false; +} + +uint8_t* ImagePipeline::decodeJpeg(uint8_t* data, size_t size, + uint16_t* outWidth, uint16_t* outHeight) { + // Use M5GFX's built-in JPEG decoder (lgfx::LGFX_Sprite as decode target) + // Alternative: TJpgDec library + lgfx::LGFX_Sprite sprite; + sprite.setPsram(true); + sprite.setColorDepth(24); + + // Draw JPEG into sprite to get decoded RGB data + if (!sprite.createFromBmpMem(data, size)) { + // Try JPEG-specific decode + sprite.createSprite(1, 1); // Minimal sprite for size query + // Use M5GFX drawJpg to decode + } + + // Fallback: manual JPEG decode with TJpgDec + // For initial implementation, use M5GFX's JPEG support + // This will be refined during hardware bring-up + + *outWidth = sprite.width(); + *outHeight = sprite.height(); + + if (*outWidth == 0 || *outHeight == 0) { + return nullptr; + } + + size_t bufSize = (*outWidth) * (*outHeight) * 3; + uint8_t* rgb = (uint8_t*)ps_malloc(bufSize); + if (rgb == nullptr) return nullptr; + + // Read pixels from sprite + for (uint16_t y = 0; y < *outHeight; y++) { + for (uint16_t x = 0; x < *outWidth; x++) { + uint32_t color = sprite.readPixel(x, y); + size_t idx = (y * (*outWidth) + x) * 3; + rgb[idx] = (color >> 16) & 0xFF; // R + rgb[idx + 1] = (color >> 8) & 0xFF; // G + rgb[idx + 2] = color & 0xFF; // B + } + } + + sprite.deleteSprite(); + return rgb; +} + +uint8_t* ImagePipeline::resize(uint8_t* rgb, uint16_t srcW, uint16_t srcH, + uint16_t dstW, uint16_t dstH) { + size_t bufSize = dstW * dstH * 3; + uint8_t* dst = (uint8_t*)ps_malloc(bufSize); + if (dst == nullptr) return nullptr; + + // Bilinear interpolation + float xRatio = (float)(srcW - 1) / (float)(dstW - 1); + float yRatio = (float)(srcH - 1) / (float)(dstH - 1); + + for (uint16_t y = 0; y < dstH; y++) { + float srcY = y * yRatio; + uint16_t y0 = (uint16_t)srcY; + uint16_t y1 = min((uint16_t)(y0 + 1), (uint16_t)(srcH - 1)); + float yFrac = srcY - y0; + + for (uint16_t x = 0; x < dstW; x++) { + float srcX = x * xRatio; + uint16_t x0 = (uint16_t)srcX; + uint16_t x1 = min((uint16_t)(x0 + 1), (uint16_t)(srcW - 1)); + float xFrac = srcX - x0; + + for (int c = 0; c < 3; c++) { + float top = rgb[(y0 * srcW + x0) * 3 + c] * (1 - xFrac) + + rgb[(y0 * srcW + x1) * 3 + c] * xFrac; + float bot = rgb[(y1 * srcW + x0) * 3 + c] * (1 - xFrac) + + rgb[(y1 * srcW + x1) * 3 + c] * xFrac; + float val = top * (1 - yFrac) + bot * yFrac; + dst[(y * dstW + x) * 3 + c] = (uint8_t)(val + 0.5f); + } + } + } + + return dst; +} + +void ImagePipeline::centerCrop(uint8_t* rgb, uint16_t srcW, uint16_t srcH, + uint16_t targetW, uint16_t targetH, + uint16_t* cropX, uint16_t* cropY, + uint16_t* cropW, uint16_t* cropH) { + float targetAspect = (float)targetW / (float)targetH; + float srcAspect = (float)srcW / (float)srcH; + + if (srcAspect > targetAspect) { + // Source is wider — crop sides + *cropH = srcH; + *cropW = (uint16_t)(srcH * targetAspect); + *cropX = (srcW - *cropW) / 2; + *cropY = 0; + } else { + // Source is taller — crop top/bottom + *cropW = srcW; + *cropH = (uint16_t)(srcW / targetAspect); + *cropX = 0; + *cropY = (srcH - *cropH) / 2; + } +} + +uint8_t* ImagePipeline::dither(uint8_t* rgb, uint16_t width, uint16_t height) { + size_t pixelCount = width * height; + uint8_t* output = (uint8_t*)ps_malloc(pixelCount); + if (output == nullptr) return nullptr; + + // Work buffer with int16 to handle error overflow + int16_t* work = (int16_t*)ps_malloc(pixelCount * 3 * sizeof(int16_t)); + if (work == nullptr) { + free(output); + return nullptr; + } + + // Copy to work buffer + for (size_t i = 0; i < pixelCount * 3; i++) { + work[i] = rgb[i]; + } + + // Floyd-Steinberg dithering + for (uint16_t y = 0; y < height; y++) { + for (uint16_t x = 0; x < width; x++) { + size_t idx = (y * width + x) * 3; + int r = constrain(work[idx], 0, 255); + int g = constrain(work[idx + 1], 0, 255); + int b = constrain(work[idx + 2], 0, 255); + + uint8_t nearest = findNearest(r, g, b); + output[y * width + x] = nearest; + + int errR = r - PALETTE_RGB[nearest][0]; + int errG = g - PALETTE_RGB[nearest][1]; + int errB = b - PALETTE_RGB[nearest][2]; + + // Distribute error + if (x + 1 < width) { + size_t ni = (y * width + (x + 1)) * 3; + work[ni] += errR * 7 / 16; + work[ni + 1] += errG * 7 / 16; + work[ni + 2] += errB * 7 / 16; + } + if (y + 1 < height) { + if (x > 0) { + size_t ni = ((y + 1) * width + (x - 1)) * 3; + work[ni] += errR * 3 / 16; + work[ni + 1] += errG * 3 / 16; + work[ni + 2] += errB * 3 / 16; + } + { + size_t ni = ((y + 1) * width + x) * 3; + work[ni] += errR * 5 / 16; + work[ni + 1] += errG * 5 / 16; + work[ni + 2] += errB * 5 / 16; + } + if (x + 1 < width) { + size_t ni = ((y + 1) * width + (x + 1)) * 3; + work[ni] += errR * 1 / 16; + work[ni + 1] += errG * 1 / 16; + work[ni + 2] += errB * 1 / 16; + } + } + } + } + + free(work); + return output; +} + +uint8_t ImagePipeline::findNearest(int r, int g, int b) { + uint8_t best = 0; + int bestDist = INT32_MAX; + for (int i = 0; i < DISPLAY_COLORS; i++) { + int dr = r - PALETTE_RGB[i][0]; + int dg = g - PALETTE_RGB[i][1]; + int db = b - PALETTE_RGB[i][2]; + int dist = dr * dr + dg * dg + db * db; + if (dist < bestDist) { + bestDist = dist; + best = i; + } + } + return best; +} +``` + +- [ ] **Step 5: Build to verify** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS + +- [ ] **Step 6: Commit** + +```bash +git add src/image_pipeline.h src/image_pipeline.cpp test/test_native/test_image_pipeline.cpp +git commit -m "feat: image pipeline with JPEG decode, resize, and Floyd-Steinberg dithering" +``` + +--- + +### Task 9: Display Manager + +**Files:** +- Create: `src/display_manager.h` +- Create: `src/display_manager.cpp` + +**Interfaces:** +- Consumes: `ProcessedImage` (from `ImagePipeline::process()`), `config.h` (display dims), `PowerManager::enableEPDPower()`, `PowerManager::disableEPDPower()` +- Produces: + - `DisplayManager` class: + - `void begin()` — initialize e-ink display in landscape orientation + - `void showImage(const ProcessedImage& img)` — write framebuffer to display and trigger refresh + - `void showMessage(const char* title, const char* body)` — display text (for status/error messages) + - `void showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos)` — overlay metadata text + +- [ ] **Step 1: Create `src/display_manager.h`** + +```cpp +#pragma once + +#include +#include +#include "config.h" +#include "image_pipeline.h" +#include "immich_client.h" +#include "settings.h" + +class PowerManager; + +class DisplayManager { +public: + void begin(PowerManager& power); + void showImage(const ProcessedImage& img); + void showMessage(const char* title, const char* body); + void showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos); + +private: + PowerManager* _power = nullptr; + M5GFX* _display = nullptr; + + void triggerRefresh(); + uint16_t paletteToColor565(uint8_t index); +}; +``` + +- [ ] **Step 2: Create `src/display_manager.cpp`** + +```cpp +#include "display_manager.h" +#include "power_manager.h" +#include + +// Map 6-color palette indices to RGB565 for M5GFX +static const uint16_t PALETTE_565[6] = { + 0x0000, // Black + 0xFFFF, // White + 0xF800, // Red (approximate) + 0x07E0, // Green (approximate) + 0x001F, // Blue (approximate) + 0xFFE0 // Yellow (approximate) +}; + +void DisplayManager::begin(PowerManager& power) { + _power = &power; + _display = &M5.Display; + + // Set rotation for landscape (device physically rotated) + // Rotation value depends on how the display is mounted — likely 1 or 3 + _display->setRotation(1); + + Serial.printf("[display] Initialized: %dx%d, rotation=%d\n", + _display->width(), _display->height(), _display->getRotation()); + + showMessage("Immich Frame", "Starting up..."); +} + +void DisplayManager::showImage(const ProcessedImage& img) { + if (!img.valid || img.framebuffer == nullptr) { + Serial.println("[display] Invalid image — skipping"); + return; + } + + _power->enableEPDPower(); + + Serial.println("[display] Writing framebuffer to e-ink..."); + unsigned long start = millis(); + + // Write pixel by pixel using palette-mapped colors + _display->startWrite(); + for (uint16_t y = 0; y < img.height; y++) { + for (uint16_t x = 0; x < img.width; x++) { + uint8_t colorIdx = img.framebuffer[y * img.width + x]; + _display->writePixel(x, y, PALETTE_565[colorIdx]); + } + } + _display->endWrite(); + + triggerRefresh(); + + unsigned long elapsed = millis() - start; + Serial.printf("[display] Refresh complete in %lu ms\n", elapsed); + + _power->disableEPDPower(); +} + +void DisplayManager::showMessage(const char* title, const char* body) { + _power->enableEPDPower(); + + _display->fillScreen(TFT_WHITE); + _display->setTextColor(TFT_BLACK); + _display->setTextDatum(middle_center); + _display->setTextSize(2); + _display->drawString(title, DISPLAY_WIDTH / 2, DISPLAY_HEIGHT / 2 - 30); + _display->setTextSize(1); + _display->drawString(body, DISPLAY_WIDTH / 2, DISPLAY_HEIGHT / 2 + 20); + + triggerRefresh(); + _power->disableEPDPower(); +} + +void DisplayManager::showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos) { + if (metaFlags == 0) return; // No metadata to show + + // Build metadata string + String metaText = ""; + + if ((metaFlags & META_DATE) && info.dateTime.length() > 0) { + // Extract just the date portion (YYYY-MM-DD) + metaText += info.dateTime.substring(0, 10); + } + if ((metaFlags & META_LOCATION) && info.city.length() > 0) { + if (metaText.length() > 0) metaText += " | "; + metaText += info.city; + } + if ((metaFlags & META_PEOPLE) && !info.people.empty()) { + if (metaText.length() > 0) metaText += " | "; + for (size_t i = 0; i < info.people.size(); i++) { + if (i > 0) metaText += ", "; + metaText += info.people[i]; + } + } + if ((metaFlags & META_ALBUM)) { + // Album name would need to be passed separately — skip for now + } + if ((metaFlags & META_CAMERA) && info.camera.length() > 0) { + if (metaText.length() > 0) metaText += " | "; + metaText += info.camera; + } + + if (metaText.length() == 0) return; + + // Draw semi-transparent bar with text + uint16_t barY = (pos == MetaPosition::Top) ? 0 : (DISPLAY_HEIGHT - 30); + _display->fillRect(0, barY, DISPLAY_WIDTH, 30, TFT_BLACK); + _display->setTextColor(TFT_WHITE); + _display->setTextDatum(middle_center); + _display->setTextSize(1); + _display->drawString(metaText.c_str(), DISPLAY_WIDTH / 2, barY + 15); +} + +void DisplayManager::triggerRefresh() { + // M5GFX handles e-ink refresh internally when using the EPD panel driver + // The display() call triggers the actual e-ink refresh cycle (10-20s) + _display->display(); +} + +uint16_t DisplayManager::paletteToColor565(uint8_t index) { + if (index >= DISPLAY_COLORS) return 0; + return PALETTE_565[index]; +} +``` + +- [ ] **Step 3: Build to verify** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS + +- [ ] **Step 4: Commit** + +```bash +git add src/display_manager.h src/display_manager.cpp +git commit -m "feat: display manager for e-ink framebuffer output and text messages" +``` + +--- + +### Task 10: Web Server & REST API + +**Files:** +- Create: `src/web_server.h` +- Create: `src/web_server.cpp` +- Create: `data/setup.html` +- Create: `data/index.html` +- Create: `data/style.css` +- Create: `data/app.js` + +**Interfaces:** +- Consumes: `SettingsManager`, `ImmichClient`, `PhotoQueue`, `PowerManager`, `WiFiManager` +- Produces: + - `WebServer` class (note: not `WebServer` from Arduino core — we use `AppWebServer` to avoid conflict): + - `void begin(SettingsManager& settings, ImmichClient& immich, PowerManager& power, WiFiManager& wifi)` — start AsyncWebServer + - `void setActionCallback(std::function cb)` — register callback for slideshow control actions + - `bool isOTAInProgress()` — true during firmware upload + +- [ ] **Step 1: Create `src/web_server.h`** + +```cpp +#pragma once + +#include +#include +#include +#include "settings.h" + +class ImmichClient; +class PowerManager; +class WiFiManager; + +class AppWebServer { +public: + void begin(SettingsManager& settings, ImmichClient& immich, + PowerManager& power, WiFiManager& wifi); + void setActionCallback(std::function cb); + bool isOTAInProgress(); + +private: + AsyncWebServer _server{80}; + SettingsManager* _settings = nullptr; + ImmichClient* _immich = nullptr; + PowerManager* _power = nullptr; + WiFiManager* _wifi = nullptr; + std::function _actionCb; + bool _otaInProgress = false; + + void setupCaptivePortal(); + void setupAPIRoutes(); + void setupStaticFiles(); + void setupOTA(); + + void handleGetStatus(AsyncWebServerRequest* request); + void handleGetAlbums(AsyncWebServerRequest* request); + void handlePostAlbumsSelect(AsyncWebServerRequest* request, uint8_t* data, size_t len); + void handleGetSettings(AsyncWebServerRequest* request); + void handlePostSettings(AsyncWebServerRequest* request, uint8_t* data, size_t len); + void handlePostAction(AsyncWebServerRequest* request); + void handlePostSetup(AsyncWebServerRequest* request, uint8_t* data, size_t len); +}; +``` + +- [ ] **Step 2: Create `src/web_server.cpp`** + +```cpp +#include "web_server.h" +#include +#include +#include +#include "immich_client.h" +#include "power_manager.h" +#include "wifi_manager.h" +#include "config.h" + +void AppWebServer::begin(SettingsManager& settings, ImmichClient& immich, + PowerManager& power, WiFiManager& wifi) { + _settings = &settings; + _immich = &immich; + _power = &power; + _wifi = &wifi; + + if (!LittleFS.begin(true)) { + Serial.println("[web] LittleFS mount failed"); + } + + if (_wifi->isAPMode()) { + setupCaptivePortal(); + } else { + setupAPIRoutes(); + setupStaticFiles(); + setupOTA(); + } + + _server.begin(); + Serial.printf("[web] Server started (mode: %s)\n", + _wifi->isAPMode() ? "AP/captive" : "LAN"); +} + +void AppWebServer::setActionCallback(std::function cb) { + _actionCb = cb; +} + +bool AppWebServer::isOTAInProgress() { + return _otaInProgress; +} + +void AppWebServer::setupCaptivePortal() { + // Serve setup page for all requests (captive portal behavior) + _server.on("/", HTTP_GET, [](AsyncWebServerRequest* request) { + request->send(LittleFS, "/setup.html", "text/html"); + }); + + // Handle WiFi scan + _server.on("/api/wifi/scan", HTTP_GET, [](AsyncWebServerRequest* request) { + int n = WiFi.scanNetworks(); + JsonDocument doc; + JsonArray arr = doc.to(); + for (int i = 0; i < n; i++) { + JsonObject net = arr.add(); + net["ssid"] = WiFi.SSID(i); + net["rssi"] = WiFi.RSSI(i); + net["secure"] = WiFi.encryptionType(i) != WIFI_AUTH_OPEN; + } + String response; + serializeJson(doc, response); + request->send(200, "application/json", response); + }); + + // Handle setup submission + _server.on("/api/setup", HTTP_POST, [](AsyncWebServerRequest* request) { + request->send(200); + }, nullptr, [this](AsyncWebServerRequest* request, uint8_t* data, size_t len, + size_t index, size_t total) { + if (index + len == total) { + handlePostSetup(request, data, len); + } + }); + + // Captive portal redirect for all other paths + _server.onNotFound([](AsyncWebServerRequest* request) { + request->redirect("/"); + }); +} + +void AppWebServer::setupAPIRoutes() { + _server.on("/api/status", HTTP_GET, + [this](AsyncWebServerRequest* req) { handleGetStatus(req); }); + + _server.on("/api/albums", HTTP_GET, + [this](AsyncWebServerRequest* req) { handleGetAlbums(req); }); + + _server.on("/api/albums/select", HTTP_POST, + [](AsyncWebServerRequest* req) { req->send(200); }, + nullptr, + [this](AsyncWebServerRequest* req, uint8_t* data, size_t len, + size_t index, size_t total) { + if (index + len == total) handlePostAlbumsSelect(req, data, len); + }); + + _server.on("/api/settings", HTTP_GET, + [this](AsyncWebServerRequest* req) { handleGetSettings(req); }); + + _server.on("/api/settings", HTTP_POST, + [](AsyncWebServerRequest* req) { req->send(200); }, + nullptr, + [this](AsyncWebServerRequest* req, uint8_t* data, size_t len, + size_t index, size_t total) { + if (index + len == total) handlePostSettings(req, data, len); + }); + + // Action endpoints + _server.on("/api/action/next", HTTP_POST, + [this](AsyncWebServerRequest* req) { + if (_actionCb) _actionCb("next"); + req->send(200, "application/json", "{\"ok\":true}"); + }); + + _server.on("/api/action/random", HTTP_POST, + [this](AsyncWebServerRequest* req) { + if (_actionCb) _actionCb("random"); + req->send(200, "application/json", "{\"ok\":true}"); + }); + + _server.on("/api/action/pause", HTTP_POST, + [this](AsyncWebServerRequest* req) { + if (_actionCb) _actionCb("pause"); + req->send(200, "application/json", "{\"ok\":true}"); + }); + + _server.on("/api/action/play", HTTP_POST, + [this](AsyncWebServerRequest* req) { + if (_actionCb) _actionCb("play"); + req->send(200, "application/json", "{\"ok\":true}"); + }); + + _server.on("/api/action/sleep", HTTP_POST, + [this](AsyncWebServerRequest* req) { + req->send(200, "application/json", "{\"ok\":true}"); + delay(100); + _power->enterDeepSleep(); + }); +} + +void AppWebServer::setupStaticFiles() { + _server.serveStatic("/", LittleFS, "/").setDefaultFile("index.html"); +} + +void AppWebServer::setupOTA() { + _server.on("/api/firmware", HTTP_POST, + [this](AsyncWebServerRequest* request) { + _otaInProgress = false; + bool success = !Update.hasError(); + request->send(200, "application/json", + success ? "{\"ok\":true,\"msg\":\"Rebooting...\"}" + : "{\"ok\":false,\"msg\":\"Update failed\"}"); + if (success) { + delay(500); + ESP.restart(); + } + }, + [this](AsyncWebServerRequest* request, const String& filename, + size_t index, uint8_t* data, size_t len, bool final) { + if (index == 0) { + _otaInProgress = true; + Serial.printf("[web] OTA start: %s\n", filename.c_str()); + if (!Update.begin(UPDATE_SIZE_UNKNOWN, U_FLASH)) { + Serial.println("[web] OTA begin failed"); + } + } + if (Update.isRunning()) { + Update.write(data, len); + } + if (final) { + if (Update.end(true)) { + Serial.printf("[web] OTA complete: %u bytes\n", index + len); + } else { + Serial.println("[web] OTA finalize failed"); + } + } + }); +} + +void AppWebServer::handleGetStatus(AsyncWebServerRequest* request) { + JsonDocument doc; + doc["battery"] = _power->getBatteryPercent(); + doc["charging"] = _power->isCharging(); + doc["wifi_rssi"] = _wifi->getRSSI(); + doc["ip"] = _wifi->getIP(); + doc["uptime"] = millis() / 1000; + doc["free_heap"] = ESP.getFreeHeap(); + doc["free_psram"] = ESP.getFreePsram(); + + String response; + serializeJson(doc, response); + request->send(200, "application/json", response); +} + +void AppWebServer::handleGetAlbums(AsyncWebServerRequest* request) { + auto albums = _immich->fetchAlbums(); + JsonDocument doc; + JsonArray arr = doc.to(); + for (auto& album : albums) { + JsonObject obj = arr.add(); + obj["id"] = album.id; + obj["title"] = album.title; + obj["assetCount"] = album.assetCount; + } + String response; + serializeJson(doc, response); + request->send(200, "application/json", response); +} + +void AppWebServer::handlePostAlbumsSelect(AsyncWebServerRequest* request, + uint8_t* data, size_t len) { + String body = String((char*)data).substring(0, len); + JsonDocument doc; + if (deserializeJson(doc, body)) { + request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); + return; + } + String albumsJson; + serializeJson(doc["album_ids"], albumsJson); + _settings->saveField("albums_json", albumsJson.c_str()); + request->send(200, "application/json", "{\"ok\":true}"); +} + +void AppWebServer::handleGetSettings(AsyncWebServerRequest* request) { + Settings s = _settings->get(); + JsonDocument doc; + doc["interval_min"] = s.interval_min; + doc["cycle_mode"] = static_cast(s.cycle_mode); + doc["img_quality"] = static_cast(s.img_quality); + doc["meta_flags"] = s.meta_flags; + doc["meta_pos"] = static_cast(s.meta_pos); + doc["led_brightness"] = s.led_brightness; + doc["immich_url"] = s.immich_url; + doc["albums_json"] = s.albums_json; + + String response; + serializeJson(doc, response); + request->send(200, "application/json", response); +} + +void AppWebServer::handlePostSettings(AsyncWebServerRequest* request, + uint8_t* data, size_t len) { + String body = String((char*)data).substring(0, len); + JsonDocument doc; + if (deserializeJson(doc, body)) { + request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); + return; + } + + Settings s = _settings->get(); + + if (doc.containsKey("interval_min")) s.interval_min = doc["interval_min"]; + if (doc.containsKey("cycle_mode")) s.cycle_mode = static_cast((uint8_t)doc["cycle_mode"]); + if (doc.containsKey("img_quality")) s.img_quality = static_cast((uint8_t)doc["img_quality"]); + if (doc.containsKey("meta_flags")) s.meta_flags = doc["meta_flags"]; + if (doc.containsKey("meta_pos")) s.meta_pos = static_cast((uint8_t)doc["meta_pos"]); + if (doc.containsKey("led_brightness")) s.led_brightness = doc["led_brightness"]; + if (doc.containsKey("immich_url")) s.immich_url = doc["immich_url"].as(); + if (doc.containsKey("immich_key")) s.immich_key = doc["immich_key"].as(); + + _settings->save(s); + request->send(200, "application/json", "{\"ok\":true}"); +} + +void AppWebServer::handlePostSetup(AsyncWebServerRequest* request, + uint8_t* data, size_t len) { + String body = String((char*)data).substring(0, len); + JsonDocument doc; + if (deserializeJson(doc, body)) { + request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); + return; + } + + Settings s = _settings->get(); + s.wifi_ssid = doc["wifi_ssid"].as(); + s.wifi_pass = doc["wifi_pass"].as(); + s.immich_url = doc["immich_url"] | DEFAULT_IMMICH_URL; + s.immich_key = doc["immich_key"].as(); + _settings->save(s); + + request->send(200, "application/json", "{\"ok\":true,\"msg\":\"Rebooting...\"}"); + delay(1000); + ESP.restart(); +} +``` + +- [ ] **Step 3: Create `data/setup.html` (AP captive portal)** + +```html + + + + + + PaperColor Setup + + + +

PaperColor Setup

+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+ +
+
+ + + + +``` + +- [ ] **Step 4: Create `data/index.html` (main management UI shell)** + +```html + + + + + + PaperColor + + + + +
+ + + +``` + +- [ ] **Step 5: Create `data/style.css`** + +```css +* { box-sizing: border-box; margin: 0; padding: 0; } +body { font-family: -apple-system, BlinkMacSystemFont, sans-serif; background: #f8fafc; color: #1e293b; } +nav { background: #1e293b; color: white; padding: 16px 24px; display: flex; align-items: center; gap: 24px; flex-wrap: wrap; } +nav h1 { font-size: 18px; white-space: nowrap; } +#nav-links { display: flex; gap: 12px; flex-wrap: wrap; } +#nav-links a { color: #94a3b8; text-decoration: none; padding: 4px 8px; border-radius: 4px; font-size: 14px; } +#nav-links a.active { color: white; background: #334155; } +main { max-width: 800px; margin: 24px auto; padding: 0 16px; } +.card { background: white; border-radius: 8px; padding: 20px; margin-bottom: 16px; box-shadow: 0 1px 3px rgba(0,0,0,0.1); } +.card h2 { font-size: 16px; margin-bottom: 12px; color: #475569; } +.stat { display: flex; justify-content: space-between; padding: 8px 0; border-bottom: 1px solid #f1f5f9; } +.stat:last-child { border-bottom: none; } +.field { margin-bottom: 16px; } +.field label { display: block; font-weight: 600; margin-bottom: 4px; font-size: 14px; color: #64748b; } +.field input, .field select { width: 100%; padding: 8px 12px; border: 1px solid #e2e8f0; border-radius: 6px; font-size: 14px; } +.btn { padding: 8px 16px; border: none; border-radius: 6px; font-size: 14px; cursor: pointer; } +.btn-primary { background: #2563eb; color: white; } +.btn-primary:hover { background: #1d4ed8; } +.btn-danger { background: #dc2626; color: white; } +.btn-danger:hover { background: #b91c1c; } +.btn-group { display: flex; gap: 8px; margin-top: 12px; } +.checkbox-list { list-style: none; } +.checkbox-list li { padding: 8px 0; border-bottom: 1px solid #f1f5f9; display: flex; align-items: center; gap: 8px; } +.checkbox-list input[type="checkbox"] { width: 18px; height: 18px; } +.preset-btns { display: flex; gap: 8px; flex-wrap: wrap; } +.preset-btns button { padding: 6px 12px; border: 1px solid #e2e8f0; border-radius: 4px; background: white; cursor: pointer; } +.preset-btns button.active { background: #2563eb; color: white; border-color: #2563eb; } +.toast { position: fixed; bottom: 20px; right: 20px; background: #065f46; color: white; padding: 12px 20px; border-radius: 6px; display: none; } +.toast.show { display: block; } +``` + +- [ ] **Step 6: Create `data/app.js`** + +```javascript +const API = ''; +let currentPage = 'dashboard'; + +async function api(path, opts = {}) { + const res = await fetch(API + path, { + headers: { 'Content-Type': 'application/json' }, + ...opts + }); + return res.json(); +} + +function toast(msg) { + let t = document.querySelector('.toast'); + if (!t) { t = document.createElement('div'); t.className = 'toast'; document.body.appendChild(t); } + t.textContent = msg; + t.classList.add('show'); + setTimeout(() => t.classList.remove('show'), 3000); +} + +function navigate(page) { + currentPage = page; + document.querySelectorAll('#nav-links a').forEach(a => { + a.classList.toggle('active', a.dataset.page === page); + }); + renderPage(page); +} + +async function renderPage(page) { + const el = document.getElementById('content'); + switch (page) { + case 'dashboard': return renderDashboard(el); + case 'albums': return renderAlbums(el); + case 'slideshow': return renderSlideshow(el); + case 'display': return renderDisplay(el); + case 'device': return renderDevice(el); + case 'firmware': return renderFirmware(el); + } +} + +async function renderDashboard(el) { + const status = await api('/api/status'); + el.innerHTML = ` +
+

Status

+
Battery${status.battery}%${status.charging ? ' ⚡' : ''}
+
WiFi Signal${status.wifi_rssi} dBm
+
IP Address${status.ip}
+
Uptime${Math.floor(status.uptime/60)}m
+
Free RAM${Math.floor(status.free_heap/1024)}KB
+
Free PSRAM${Math.floor(status.free_psram/1024)}KB
+
+
+

Controls

+
+ + + + +
+
`; +} + +async function renderAlbums(el) { + const [albums, settings] = await Promise.all([api('/api/albums'), api('/api/settings')]); + const selected = JSON.parse(settings.albums_json || '[]'); + el.innerHTML = ` +
+

Albums

+
    + ${albums.map(a => ` +
  • + + ${a.title} (${a.assetCount}) +
    • `).join('')} +
+
+ +
+
`; + document.getElementById('saveAlbums').onclick = async () => { + const ids = [...el.querySelectorAll('input[type=checkbox]:checked')].map(c => c.value); + await api('/api/albums/select', { method: 'POST', body: JSON.stringify({ album_ids: ids }) }); + toast('Albums saved'); + }; +} + +async function renderSlideshow(el) { + const settings = await api('/api/settings'); + const intervals = [1, 5, 15, 30, 60]; + const modes = ['Random', 'Chronological', 'Reverse Chrono', 'Favorites Weighted']; + el.innerHTML = ` +
+

Interval

+
+ ${intervals.map(i => ``).join('')} +
+
+
+

Cycling Mode

+
+ +
+
+
`; + el.querySelectorAll('[data-interval]').forEach(btn => { + btn.onclick = () => { + el.querySelectorAll('[data-interval]').forEach(b => b.classList.remove('active')); + btn.classList.add('active'); + }; + }); + document.getElementById('saveSlideshow').onclick = async () => { + const interval = parseInt(el.querySelector('[data-interval].active')?.dataset.interval || '5'); + const mode = parseInt(document.getElementById('cycleMode').value); + await api('/api/settings', { method: 'POST', body: JSON.stringify({ interval_min: interval, cycle_mode: mode }) }); + toast('Slideshow settings saved'); + }; +} + +async function renderDisplay(el) { + const settings = await api('/api/settings'); + const flags = settings.meta_flags; + el.innerHTML = ` +
+

Image Quality

+
+ +
+
+
+

Metadata Overlay

+
    +
  • Date
    • +
  • Location
    • +
  • People
    • +
  • Album
    • +
  • Camera
    • +
+
+ + +
+
+
`; + document.getElementById('saveDisplay').onclick = async () => { + let flags = 0; + el.querySelectorAll('[data-flag]').forEach(cb => { if (cb.checked) flags |= parseInt(cb.dataset.flag); }); + await api('/api/settings', { method: 'POST', body: JSON.stringify({ + img_quality: parseInt(document.getElementById('imgQuality').value), + meta_flags: flags, + meta_pos: parseInt(document.getElementById('metaPos').value) + })}); + toast('Display settings saved'); + }; +} + +async function renderDevice(el) { + el.innerHTML = ` +
+

Device

+
+ + +
+
`; +} + +async function renderFirmware(el) { + el.innerHTML = ` +
+

Firmware Update

+
+ + +
+ +
+
`; + document.getElementById('uploadFw').onclick = async () => { + const file = document.getElementById('fwFile').files[0]; + if (!file) return toast('Select a file first'); + const formData = new FormData(); + formData.append('firmware', file); + document.getElementById('fwStatus').textContent = 'Uploading...'; + const res = await fetch('/api/firmware', { method: 'POST', body: formData }); + const data = await res.json(); + document.getElementById('fwStatus').textContent = data.msg || (data.ok ? 'Success!' : 'Failed'); + }; +} + +// Navigation +document.querySelectorAll('#nav-links a').forEach(a => { + a.addEventListener('click', (e) => { e.preventDefault(); navigate(a.dataset.page); }); +}); + +// Initial render +navigate('dashboard'); +``` + +- [ ] **Step 7: Build and upload filesystem** + +Run: `pio run -e m5stack-papercolor && pio run -e m5stack-papercolor -t uploadfs` +Expected: BUILD SUCCESS, filesystem upload succeeds + +- [ ] **Step 8: Flash firmware and verify web UI** + +Run: `pio run -e m5stack-papercolor -t upload` +Then connect to `PaperColor-Setup` WiFi and open `192.168.4.1` in a browser (AP mode), or navigate to `http://papercolor.local` (station mode). +Expected: Setup page renders in AP mode. After provisioning, management UI renders with all pages. + +- [ ] **Step 9: Commit** + +```bash +git add src/web_server.h src/web_server.cpp data/ +git commit -m "feat: web server with REST API, captive portal, and management UI" +``` + +--- + +### Task 11: Main Integration (Slideshow State Machine) + +**Files:** +- Modify: `src/main.cpp` — full rewrite with FreeRTOS tasks and slideshow logic + +**Interfaces:** +- Consumes: All modules (SettingsManager, WiFiManager, PowerManager, ButtonHandler, ImmichClient, PhotoQueue, ImagePipeline, DisplayManager, AppWebServer) +- Produces: Complete working firmware with: + - Display task on Core 1 (timer-driven photo refresh) + - Web server on Core 0 (always listening) + - Battery monitor integrated into main loop + - Button events dispatched to slideshow state machine + +- [ ] **Step 1: Rewrite `src/main.cpp` with full integration** + +```cpp +#include +#include +#include +#include +#include + +#include "config.h" +#include "settings.h" +#include "wifi_manager.h" +#include "power_manager.h" +#include "button_handler.h" +#include "immich_client.h" +#include "photo_queue.h" +#include "image_pipeline.h" +#include "display_manager.h" +#include "web_server.h" + +// Global instances +SettingsManager settingsManager; +WiFiManager wifiManager; +PowerManager powerManager; +ButtonHandler buttonHandler; +ImmichClient immichClient; +PhotoQueue photoQueue; +ImagePipeline imagePipeline; +DisplayManager displayManager; +AppWebServer webServer; + +// Shared state +SemaphoreHandle_t stateMutex; +volatile bool slideshowPlaying = true; +volatile bool refreshRequested = false; +volatile bool randomRequested = false; +volatile unsigned long lastRefreshTime = 0; + +// Task handles +TaskHandle_t displayTaskHandle = nullptr; + +void displayTask(void* param) { + Serial.println("[display_task] Started on Core 1"); + + // Initial sync + if (wifiManager.isConnected()) { + Settings s = settingsManager.get(); + immichClient.begin(s.immich_url, s.immich_key); + photoQueue.begin(immichClient, settingsManager); + + if (photoQueue.sync()) { + Serial.printf("[display_task] Queue ready: %d photos\n", photoQueue.size()); + } else { + displayManager.showMessage("No Photos", "Select albums in web UI"); + } + } + + while (true) { + unsigned long now = millis(); + Settings s = settingsManager.get(); + unsigned long intervalMs = s.interval_min * 60000UL; + + bool shouldRefresh = false; + bool wantsRandom = false; + + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { + if (refreshRequested) { + shouldRefresh = true; + refreshRequested = false; + } else if (randomRequested) { + shouldRefresh = true; + wantsRandom = true; + randomRequested = false; + } else if (slideshowPlaying && (now - lastRefreshTime >= intervalMs)) { + shouldRefresh = true; + } + xSemaphoreGive(stateMutex); + } + + // Periodic re-sync + if (photoQueue.needsResync() && wifiManager.isConnected()) { + photoQueue.sync(); + } + + if (shouldRefresh && photoQueue.size() > 0 && wifiManager.isConnected()) { + // Get next photo ID + String assetId; + if (wantsRandom) { + assetId = photoQueue.random(); + } else { + assetId = photoQueue.next(); + } + + if (assetId.length() > 0) { + Serial.printf("[display_task] Loading asset: %s\n", assetId.c_str()); + + // Fetch asset info for portrait detection and metadata + AssetInfo info = immichClient.fetchAssetInfo(assetId); + + // Download photo + uint8_t* jpegBuf = nullptr; + size_t jpegSize = 0; + bool downloaded = immichClient.downloadAsset( + assetId, s.img_quality, &jpegBuf, &jpegSize); + + if (downloaded && jpegBuf != nullptr) { + ProcessedImage img; + + if (info.isPortrait) { + // Try to find a portrait pair + String pairId = photoQueue.findPortraitPair( + settingsManager.get().queue_cursor); + if (pairId.length() > 0) { + AssetInfo pairInfo = immichClient.fetchAssetInfo(pairId); + if (pairInfo.isPortrait) { + uint8_t* jpeg2Buf = nullptr; + size_t jpeg2Size = 0; + if (immichClient.downloadAsset(pairId, s.img_quality, + &jpeg2Buf, &jpeg2Size)) { + img = imagePipeline.processPortraitPair( + jpegBuf, jpegSize, jpeg2Buf, jpeg2Size); + free(jpeg2Buf); + } else { + img = imagePipeline.process(jpegBuf, jpegSize); + } + } else { + img = imagePipeline.process(jpegBuf, jpegSize); + } + } else { + img = imagePipeline.process(jpegBuf, jpegSize); + } + } else { + img = imagePipeline.process(jpegBuf, jpegSize); + } + + free(jpegBuf); + + if (img.valid) { + displayManager.showImage(img); + // Show metadata overlay if enabled + if (s.meta_flags != 0) { + displayManager.showMetadata(info, s.meta_flags, s.meta_pos); + } + imagePipeline.freeImage(img); + } else { + Serial.println("[display_task] Image processing failed"); + } + } + + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { + lastRefreshTime = millis(); + xSemaphoreGive(stateMutex); + } + } + } + + // Yield — check every second + vTaskDelay(pdMS_TO_TICKS(1000)); + } +} + +void webActionCallback(const String& action) { + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { + if (action == "next") { + refreshRequested = true; + } else if (action == "random") { + randomRequested = true; + } else if (action == "pause") { + slideshowPlaying = false; + } else if (action == "play") { + slideshowPlaying = true; + } + xSemaphoreGive(stateMutex); + } +} + +void setup() { + auto cfg = M5.config(); + M5.begin(cfg); + + Serial.begin(115200); + Serial.println("[main] Immich Frame v1.0 booting..."); + + // Create state mutex + stateMutex = xSemaphoreCreateMutex(); + + // Initialize subsystems + powerManager.begin(); + settingsManager.begin(); + wifiManager.begin(settingsManager); + buttonHandler.begin(powerManager, settingsManager); + displayManager.begin(powerManager); + + if (!wifiManager.isAPMode()) { + // Station mode — set up Immich and web server + Settings s = settingsManager.get(); + immichClient.begin(s.immich_url, s.immich_key); + photoQueue.begin(immichClient, settingsManager); + } + + // Start web server (works in both AP and station modes) + webServer.begin(settingsManager, immichClient, powerManager, wifiManager); + webServer.setActionCallback(webActionCallback); + + if (wifiManager.isAPMode()) { + displayManager.showMessage("Setup Required", + "Connect to PaperColor-Setup WiFi"); + } else { + displayManager.showMessage("Immich Frame", "Loading photos..."); + + // Create display task on Core 1 + xTaskCreatePinnedToCore(displayTask, "display", 32768, nullptr, 1, + &displayTaskHandle, 1); + } + + // Enable light sleep for power saving + powerManager.enableLightSleep(); + + Serial.println("[main] Boot complete"); +} + +void loop() { + M5.update(); + powerManager.updateBatteryLED(); + + // Handle button events + ButtonEvent event = buttonHandler.poll(); + switch (event) { + case ButtonEvent::NextPhoto: + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { + refreshRequested = true; + xSemaphoreGive(stateMutex); + } + break; + case ButtonEvent::RandomPhoto: + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { + randomRequested = true; + xSemaphoreGive(stateMutex); + } + break; + case ButtonEvent::PlayPause: + if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { + slideshowPlaying = !slideshowPlaying; + Serial.printf("[main] Slideshow: %s\n", slideshowPlaying ? "playing" : "paused"); + xSemaphoreGive(stateMutex); + } + break; + case ButtonEvent::DeepSleep: + case ButtonEvent::FactoryReset: + break; + case ButtonEvent::None: + break; + default: { + ButtonEvent unreachable = event; + (void)unreachable; + break; + } + } + + delay(10); +} +``` + +- [ ] **Step 2: Build full firmware** + +Run: `pio run -e m5stack-papercolor` +Expected: BUILD SUCCESS with all modules linked + +- [ ] **Step 3: Upload filesystem and firmware** + +Run: `pio run -e m5stack-papercolor -t uploadfs && pio run -e m5stack-papercolor -t upload` +Expected: Both upload successfully + +- [ ] **Step 4: End-to-end test** + +1. Device boots into AP mode (first time) +2. Connect to `PaperColor-Setup`, configure WiFi + Immich API key +3. Device reboots, connects to WiFi +4. Navigate to `http://papercolor.local` — dashboard loads +5. Select albums in Albums page +6. Device begins displaying photos from slideshow +7. Press buttons: BTN_UP advances photo, BTN_TOP shows random, BTN_DOWN pauses + +Run: `pio device monitor` +Expected: Serial shows full lifecycle: boot → WiFi → Immich connect → queue sync → photo fetch → decode → dither → display + +- [ ] **Step 5: Commit** + +```bash +git add src/main.cpp +git commit -m "feat: full integration with FreeRTOS tasks and slideshow state machine" +``` + +--- + +### Task 12: Testing & Polish + +**Files:** +- Create: `test/README.md` +- Modify: Various files for bug fixes discovered during testing + +**Interfaces:** +- Consumes: Full firmware +- Produces: Working, tested firmware ready for daily use + +- [ ] **Step 1: Create test documentation** + +Create `test/README.md`: + +```markdown +# Testing + +## Native Tests (host machine) + +Run algorithm tests that don't require hardware: + +```bash +pio test -e native +``` + +Tests: +- `test_photo_queue.cpp` — queue cycling, favorites weighting +- `test_image_pipeline.cpp` — nearest color, dithering correctness + +## On-Device Testing + +Flash and monitor: + +```bash +pio run -e m5stack-papercolor -t upload && pio device monitor +``` + +### Verification Checklist + +- [ ] Device boots and prints version to serial +- [ ] AP mode activates on first boot +- [ ] Captive portal serves setup page at 192.168.4.1 +- [ ] WiFi credentials save and device reboots to station mode +- [ ] mDNS resolves at papercolor.local +- [ ] Web UI dashboard shows battery/WiFi/uptime +- [ ] Albums page lists Immich albums +- [ ] Album selection persists across reboots +- [ ] Photos download and display correctly +- [ ] Portrait photos pair side-by-side +- [ ] Slideshow advances at configured interval +- [ ] BTN_UP: next photo (LED flashes white) +- [ ] BTN_TOP: random photo (LED flashes white) +- [ ] BTN_DOWN: pause/play toggle +- [ ] BTN_TOP+BTN_DOWN 3s: deep sleep +- [ ] BTN_UP 5s: factory reset +- [ ] Battery LED: orange at 25%, red at 10% +- [ ] OTA upload succeeds via web UI +- [ ] Device recovers from WiFi disconnect (reconnect on next cycle) +``` + +- [ ] **Step 2: Run native tests** + +Run: `pio test -e native` +Expected: All tests PASS + +- [ ] **Step 3: Run full device verification checklist** + +Flash firmware and LittleFS, then work through the checklist above item by item. Fix any issues discovered. + +- [ ] **Step 4: Commit any fixes** + +```bash +git add -A +git commit -m "fix: testing corrections and polish" +``` + +- [ ] **Step 5: Tag release** + +```bash +git tag v1.0.0 +git log --oneline -10 +``` + +Expected: Clean commit history showing incremental feature additions. + +--- + +## Summary + +| Task | What It Builds | Key Files | +|------|---------------|-----------| +| 1 | Project scaffolding | `platformio.ini`, `config.h`, `main.cpp` | +| 2 | Settings persistence | `settings.h/cpp` | +| 3 | WiFi (station + AP) | `wifi_manager.h/cpp` | +| 4 | Power/battery/sleep | `power_manager.h/cpp` | +| 5 | Button input | `button_handler.h/cpp` | +| 6 | Immich API client | `immich_client.h/cpp` | +| 7 | Photo queue/cycling | `photo_queue.h/cpp` | +| 8 | Image decode+dither | `image_pipeline.h/cpp` | +| 9 | E-ink display output | `display_manager.h/cpp` | +| 10 | Web UI + REST API | `web_server.h/cpp`, `data/*` | +| 11 | Full integration | `main.cpp` (FreeRTOS tasks) | +| 12 | Test & polish | `test/README.md`, bug fixes |