# Immich Photo Frame Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build firmware for the M5Stack PaperColor that displays photos from a self-hosted Immich instance as a configurable slideshow, with a web-based management UI. **Architecture:** FreeRTOS multi-task firmware on ESP32-S3. Display task (Core 1) handles JPEG decode and 6-color dithering. Web server and battery monitor tasks (Core 0) run alongside WiFi. Automatic light sleep between refreshes keeps idle power at ~5-15mA while the web UI remains accessible. **Tech Stack:** Arduino framework via PlatformIO, M5Unified/M5GFX/M5PM1 libraries, ESPAsyncWebServer, ArduinoJson, TJpgDec for JPEG decode, LittleFS for web assets, NVS for settings. ## Global Constraints - Platform: `espressif32 @ 6.12.0` - Board target: `esp32s3box` (closest match for PaperColor in PlatformIO) - Arduino framework with ESP-IDF components accessible - 16MB flash, 8MB PSRAM (`qio_opi` memory type) - Display: 600×400 landscape (device rotated), E Ink Spectra 6 (6 colors) - USB CDC on boot enabled (`ARDUINO_USB_CDC_ON_BOOT=1`) - All audio permanently disabled (GPIO 45 LOW, GPIO 46 LOW) - Buttons: BTN_TOP=G1, BTN_UP=G9, BTN_DOWN=G10 - RGB LEDs: G21 (NeoPixel, 2 LEDs) - M5PM1 power IC at I2C 0x6E on SYS_SDA=G3, SYS_SCL=G2 --- ## File Structure ``` immich-frame/ ├── platformio.ini # Build configuration ├── partitions_custom.csv # 16MB partition table ├── src/ │ ├── main.cpp # Entry point, FreeRTOS task creation │ ├── config.h # Pin definitions, constants, defaults │ ├── settings.h # Settings manager header │ ├── settings.cpp # NVS read/write for all config │ ├── wifi_manager.h # WiFi manager header │ ├── wifi_manager.cpp # Station + AP + mDNS │ ├── power_manager.h # Power manager header │ ├── power_manager.cpp # M5PM1, battery, LEDs, sleep │ ├── button_handler.h # Button handler header │ ├── button_handler.cpp # GPIO interrupts, debounce, combos │ ├── immich_client.h # Immich API client header │ ├── immich_client.cpp # HTTPS calls to Immich REST API │ ├── photo_queue.h # Photo queue/selection header │ ├── photo_queue.cpp # Queue management, cycling modes │ ├── image_pipeline.h # Image processing header │ ├── image_pipeline.cpp # JPEG decode, resize, dither │ ├── display_manager.h # Display manager header │ ├── display_manager.cpp # E-ink framebuffer + refresh │ ├── web_server.h # Web server header │ └── web_server.cpp # REST API + static file serving ├── data/ # LittleFS partition (web UI) │ ├── index.html # Single-page app shell │ ├── setup.html # AP mode captive portal │ ├── app.js # Web UI logic │ └── style.css # Styles ├── test/ │ ├── test_native/ │ │ ├── test_settings.cpp # Settings serialization tests │ │ ├── test_photo_queue.cpp # Queue cycling logic tests │ │ └── test_image_pipeline.cpp # Dithering algorithm tests │ └── README.md # How to run tests └── docs/ └── superpowers/ ├── specs/ └── plans/ ``` --- ### Task 1: Project Scaffolding **Files:** - Create: `platformio.ini` - Create: `partitions_custom.csv` - Create: `src/config.h` - Create: `src/main.cpp` **Interfaces:** - Consumes: Nothing (first task) - Produces: Build system that compiles an empty firmware that boots, prints to serial, and can be flashed to the PaperColor - [ ] **Step 1: Create `platformio.ini`** ```ini ; PlatformIO configuration for M5Stack PaperColor Immich Frame [env:m5stack-papercolor] platform = espressif32 @ 6.12.0 board = esp32s3box framework = arduino board_build.partitions = partitions_custom.csv board_upload.flash_size = 16MB board_upload.maximum_size = 16777216 board_build.arduino.memory_type = qio_opi board_build.filesystem = littlefs monitor_speed = 115200 upload_speed = 921600 build_flags = -DESP32S3 -DBOARD_HAS_PSRAM -DCORE_DEBUG_LEVEL=3 -DARDUINO_USB_CDC_ON_BOOT=1 -DARDUINO_USB_MODE=1 lib_deps = m5stack/M5Unified @ ^0.2.2 m5stack/M5GFX @ ^0.2.5 https://github.com/m5stack/M5PM1.git me-no-dev/ESP Async WebServer @ ^1.2.4 bblanchon/ArduinoJson @ ^7.0.0 [env:native] platform = native build_flags = -std=c++17 test_framework = unity lib_deps = bblanchon/ArduinoJson @ ^7.0.0 ``` - [ ] **Step 2: Create `partitions_custom.csv`** ```csv # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x9000, 0x6000, otadata, data, ota, 0xf000, 0x2000, app0, app, ota_0, 0x10000, 0x680000, app1, app, ota_1, 0x690000, 0x680000, littlefs, data, spiffs, 0xD10000, 0x80000, coredump, data, coredump, 0xD90000, 0x10000, ``` - [ ] **Step 3: Create `src/config.h`** ```cpp #pragma once // --- Pin Definitions --- // Buttons #define PIN_BTN_TOP 1 // G1 - Random photo #define PIN_BTN_UP 9 // G9 - Next photo #define PIN_BTN_DOWN 10 // G10 - Play/Pause // RGB LEDs (NeoPixel) #define PIN_RGB_LED 21 #define NUM_RGB_LEDS 2 // Audio (permanently disabled) #define PIN_AUDIO_PWR_EN 45 #define PIN_SPK_EN 46 // E-Paper SPI #define PIN_EPD_CLK 15 #define PIN_EPD_MOSI 13 #define PIN_EPD_CS 44 #define PIN_EPD_DC 43 #define PIN_EPD_BUSY 11 #define PIN_EPD_RST 12 // I2C System Bus (M5PM1, SHT40, RTC) #define PIN_SYS_SDA 3 #define PIN_SYS_SCL 2 // --- Display --- #define DISPLAY_WIDTH 600 #define DISPLAY_HEIGHT 400 #define DISPLAY_COLORS 6 // --- Defaults --- #define DEFAULT_INTERVAL_MIN 5 #define DEFAULT_CYCLE_MODE 0 // 0=random #define DEFAULT_IMG_QUALITY 0 // 0=preview #define DEFAULT_META_FLAGS 0x00 #define DEFAULT_META_POS 0 // 0=bottom #define DEFAULT_LED_BRIGHTNESS 50 #define DEFAULT_IMMICH_URL "https://photos.example.com" // --- Timing --- #define BUTTON_DEBOUNCE_MS 50 #define COMBO_HOLD_MS 3000 #define FACTORY_RESET_HOLD_MS 5000 #define BATTERY_CHECK_INTERVAL_MS 60000 #define LED_PULSE_DURATION_MS 100 #define QUEUE_RESYNC_HOURS 24 #define WIFI_CONNECT_TIMEOUT_MS 15000 #define WIFI_MAX_RETRIES 3 // --- Battery Thresholds --- #define BATTERY_LOW_PCT 25 #define BATTERY_CRITICAL_PCT 10 #define BATTERY_SHUTDOWN_PCT 5 #define BATTERY_WARN_INTERVAL_MS 300000 // 5 min #define BATTERY_CRIT_INTERVAL_MS 120000 // 2 min // --- Portrait Pairing --- #define PORTRAIT_LOOKAHEAD 5 #define PORTRAIT_GAP_PX 8 #define PORTRAIT_WIDTH 267 // (DISPLAY_HEIGHT * 2) / 3 // --- Dither Palette (Spectra 6) --- // Initial values — calibrate on real hardware #define PALETTE_BLACK_R 0 #define PALETTE_BLACK_G 0 #define PALETTE_BLACK_B 0 #define PALETTE_WHITE_R 255 #define PALETTE_WHITE_G 255 #define PALETTE_WHITE_B 255 #define PALETTE_RED_R 200 #define PALETTE_RED_G 30 #define PALETTE_RED_B 30 #define PALETTE_GREEN_R 30 #define PALETTE_GREEN_G 160 #define PALETTE_GREEN_B 30 #define PALETTE_BLUE_R 30 #define PALETTE_BLUE_G 30 #define PALETTE_BLUE_B 200 #define PALETTE_YELLOW_R 220 #define PALETTE_YELLOW_G 200 #define PALETTE_YELLOW_B 30 ``` - [ ] **Step 4: Create `src/main.cpp` (minimal boot skeleton)** ```cpp #include #include #include "config.h" void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame booting..."); // Disable audio permanently pinMode(PIN_AUDIO_PWR_EN, OUTPUT); digitalWrite(PIN_AUDIO_PWR_EN, LOW); pinMode(PIN_SPK_EN, OUTPUT); digitalWrite(PIN_SPK_EN, LOW); Serial.printf("[main] Free heap: %d, PSRAM: %d\n", ESP.getFreeHeap(), ESP.getFreePsram()); Serial.println("[main] Boot complete (skeleton)"); } void loop() { M5.update(); delay(1000); } ``` - [ ] **Step 5: Verify build compiles** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS with no errors - [ ] **Step 6: Flash to device and verify serial output** Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: Serial prints "Immich Frame booting...", free heap/PSRAM values, "Boot complete (skeleton)" - [ ] **Step 7: Commit** ```bash git add platformio.ini partitions_custom.csv src/config.h src/main.cpp git commit -m "feat: project scaffolding with PlatformIO config and boot skeleton" ``` --- ### Task 2: Settings Manager **Files:** - Create: `src/settings.h` - Create: `src/settings.cpp` - Create: `test/test_native/test_settings.cpp` **Interfaces:** - Consumes: `config.h` (default values) - Produces: - `Settings` struct with all config fields - `SettingsManager` class: - `void begin()` — initialize NVS - `Settings get()` — load all settings from NVS - `void save(const Settings& s)` — persist all settings to NVS - `void saveField(const char* key, uint8_t value)` — save single numeric field - `void saveField(const char* key, const char* value)` — save single string field - `void factoryReset()` — clear all NVS and reboot - `bool isProvisioned()` — returns true if wifi_ssid is set - [ ] **Step 1: Create `src/settings.h`** ```cpp #pragma once #include #include "config.h" enum class CycleMode : uint8_t { Random = 0, Chronological = 1, ReverseChronological = 2, FavoritesWeighted = 3 }; enum class ImageQuality : uint8_t { Preview = 0, Original = 1 }; enum class MetaPosition : uint8_t { Bottom = 0, Top = 1 }; // Metadata flags bitmask constexpr uint8_t META_DATE = 0x01; constexpr uint8_t META_LOCATION = 0x02; constexpr uint8_t META_PEOPLE = 0x04; constexpr uint8_t META_ALBUM = 0x08; constexpr uint8_t META_CAMERA = 0x10; struct Settings { // WiFi String wifi_ssid; String wifi_pass; // Immich String immich_url; String immich_key; // Slideshow uint8_t interval_min; CycleMode cycle_mode; ImageQuality img_quality; // Display uint8_t meta_flags; MetaPosition meta_pos; // Device uint8_t led_brightness; uint32_t queue_cursor; String albums_json; }; class SettingsManager { public: void begin(); Settings get(); void save(const Settings& s); void saveField(const char* key, uint8_t value); void saveField(const char* key, const char* value); void factoryReset(); bool isProvisioned(); private: String readString(const char* key, const char* defaultValue = ""); uint8_t readU8(const char* key, uint8_t defaultValue = 0); uint32_t readU32(const char* key, uint32_t defaultValue = 0); void writeString(const char* key, const String& value); void writeU8(const char* key, uint8_t value); void writeU32(const char* key, uint32_t value); }; ``` - [ ] **Step 2: Create `src/settings.cpp`** ```cpp #include "settings.h" #include static const char* NVS_NAMESPACE = "immich_frame"; static Preferences prefs; void SettingsManager::begin() { prefs.begin(NVS_NAMESPACE, false); Serial.println("[settings] NVS initialized"); } Settings SettingsManager::get() { Settings s; s.wifi_ssid = readString("wifi_ssid"); s.wifi_pass = readString("wifi_pass"); s.immich_url = readString("immich_url", DEFAULT_IMMICH_URL); s.immich_key = readString("immich_key"); s.interval_min = readU8("interval_m", DEFAULT_INTERVAL_MIN); s.cycle_mode = static_cast(readU8("cycle_mode", DEFAULT_CYCLE_MODE)); s.img_quality = static_cast(readU8("img_quality", DEFAULT_IMG_QUALITY)); s.meta_flags = readU8("meta_flags", DEFAULT_META_FLAGS); s.meta_pos = static_cast(readU8("meta_pos", DEFAULT_META_POS)); s.led_brightness = readU8("led_bright", DEFAULT_LED_BRIGHTNESS); s.queue_cursor = readU32("queue_cursor", 0); s.albums_json = readString("albums_json", "[]"); return s; } void SettingsManager::save(const Settings& s) { writeString("wifi_ssid", s.wifi_ssid); writeString("wifi_pass", s.wifi_pass); writeString("immich_url", s.immich_url); writeString("immich_key", s.immich_key); writeU8("interval_m", s.interval_min); writeU8("cycle_mode", static_cast(s.cycle_mode)); writeU8("img_quality", static_cast(s.img_quality)); writeU8("meta_flags", s.meta_flags); writeU8("meta_pos", static_cast(s.meta_pos)); writeU8("led_bright", s.led_brightness); writeU32("queue_cursor", s.queue_cursor); writeString("albums_json", s.albums_json); Serial.println("[settings] All settings saved"); } void SettingsManager::saveField(const char* key, uint8_t value) { writeU8(key, value); } void SettingsManager::saveField(const char* key, const char* value) { writeString(key, String(value)); } void SettingsManager::factoryReset() { prefs.clear(); Serial.println("[settings] Factory reset — rebooting"); delay(500); ESP.restart(); } bool SettingsManager::isProvisioned() { String ssid = readString("wifi_ssid"); return ssid.length() > 0; } String SettingsManager::readString(const char* key, const char* defaultValue) { return prefs.getString(key, defaultValue); } uint8_t SettingsManager::readU8(const char* key, uint8_t defaultValue) { return prefs.getUChar(key, defaultValue); } uint32_t SettingsManager::readU32(const char* key, uint32_t defaultValue) { return prefs.getUInt(key, defaultValue); } void SettingsManager::writeString(const char* key, const String& value) { prefs.putString(key, value); } void SettingsManager::writeU8(const char* key, uint8_t value) { prefs.putUChar(key, value); } void SettingsManager::writeU32(const char* key, uint32_t value) { prefs.putUInt(key, value); } ``` - [ ] **Step 3: Verify build compiles with settings module** Update `src/main.cpp` to include and use settings: ```cpp #include #include #include "config.h" #include "settings.h" SettingsManager settingsManager; void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame booting..."); // Disable audio permanently pinMode(PIN_AUDIO_PWR_EN, OUTPUT); digitalWrite(PIN_AUDIO_PWR_EN, LOW); pinMode(PIN_SPK_EN, OUTPUT); digitalWrite(PIN_SPK_EN, LOW); settingsManager.begin(); Settings settings = settingsManager.get(); Serial.printf("[main] Provisioned: %s\n", settingsManager.isProvisioned() ? "yes" : "no"); Serial.printf("[main] Immich URL: %s\n", settings.immich_url.c_str()); Serial.printf("[main] Interval: %d min\n", settings.interval_min); Serial.printf("[main] Free heap: %d, PSRAM: %d\n", ESP.getFreeHeap(), ESP.getFreePsram()); Serial.println("[main] Boot complete"); } void loop() { M5.update(); delay(1000); } ``` Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS - [ ] **Step 4: Flash and verify settings load from NVS** Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: Serial shows "Provisioned: no", default Immich URL, interval 5 min - [ ] **Step 5: Commit** ```bash git add src/settings.h src/settings.cpp src/main.cpp git commit -m "feat: settings manager with NVS persistence" ``` --- ### Task 3: WiFi Manager **Files:** - Create: `src/wifi_manager.h` - Create: `src/wifi_manager.cpp` **Interfaces:** - Consumes: `SettingsManager::get()` for WiFi credentials, `SettingsManager::isProvisioned()` - Produces: - `WiFiManager` class: - `void begin(SettingsManager& settings)` — connect or start AP based on provisioning state - `bool isConnected()` — station mode connected? - `bool isAPMode()` — currently in AP mode? - `String getIP()` — current IP address - `int getRSSI()` — WiFi signal strength - `void startAP()` — force AP mode - `void startStation()` — attempt station connection - `void setupMDNS(const char* hostname)` — register mDNS - [ ] **Step 1: Create `src/wifi_manager.h`** ```cpp #pragma once #include #include "settings.h" class WiFiManager { public: void begin(SettingsManager& settings); bool isConnected(); bool isAPMode(); String getIP(); int getRSSI(); void startAP(); void startStation(); void setupMDNS(const char* hostname); private: SettingsManager* _settings = nullptr; bool _apMode = false; int _retryCount = 0; bool attemptConnection(const String& ssid, const String& pass); }; ``` - [ ] **Step 2: Create `src/wifi_manager.cpp`** ```cpp #include "wifi_manager.h" #include #include #include "config.h" void WiFiManager::begin(SettingsManager& settings) { _settings = &settings; if (!_settings->isProvisioned()) { Serial.println("[wifi] Not provisioned — starting AP"); startAP(); return; } startStation(); } bool WiFiManager::isConnected() { return WiFi.status() == WL_CONNECTED; } bool WiFiManager::isAPMode() { return _apMode; } String WiFiManager::getIP() { if (_apMode) { return WiFi.softAPIP().toString(); } return WiFi.localIP().toString(); } int WiFiManager::getRSSI() { if (_apMode) return 0; return WiFi.RSSI(); } void WiFiManager::startAP() { WiFi.disconnect(true); WiFi.mode(WIFI_AP); WiFi.softAP("PaperColor-Setup"); _apMode = true; Serial.printf("[wifi] AP started: PaperColor-Setup, IP: %s\n", WiFi.softAPIP().toString().c_str()); } void WiFiManager::startStation() { Settings s = _settings->get(); WiFi.disconnect(true); WiFi.mode(WIFI_STA); _apMode = false; Serial.printf("[wifi] Connecting to: %s\n", s.wifi_ssid.c_str()); for (_retryCount = 0; _retryCount < WIFI_MAX_RETRIES; _retryCount++) { if (attemptConnection(s.wifi_ssid, s.wifi_pass)) { Serial.printf("[wifi] Connected! IP: %s, RSSI: %d\n", WiFi.localIP().toString().c_str(), WiFi.RSSI()); WiFi.setSleep(WIFI_PS_MIN_MODEM); setupMDNS("papercolor"); return; } Serial.printf("[wifi] Attempt %d/%d failed\n", _retryCount + 1, WIFI_MAX_RETRIES); } Serial.println("[wifi] All attempts failed — falling back to AP mode"); startAP(); } void WiFiManager::setupMDNS(const char* hostname) { if (MDNS.begin(hostname)) { MDNS.addService("http", "tcp", 80); Serial.printf("[wifi] mDNS: %s.local\n", hostname); } else { Serial.println("[wifi] mDNS failed to start"); } } bool WiFiManager::attemptConnection(const String& ssid, const String& pass) { WiFi.begin(ssid.c_str(), pass.c_str()); unsigned long start = millis(); while (WiFi.status() != WL_CONNECTED) { if (millis() - start > WIFI_CONNECT_TIMEOUT_MS) { return false; } delay(100); } return true; } ``` - [ ] **Step 3: Integrate into `main.cpp`** ```cpp #include #include #include "config.h" #include "settings.h" #include "wifi_manager.h" SettingsManager settingsManager; WiFiManager wifiManager; void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame booting..."); pinMode(PIN_AUDIO_PWR_EN, OUTPUT); digitalWrite(PIN_AUDIO_PWR_EN, LOW); pinMode(PIN_SPK_EN, OUTPUT); digitalWrite(PIN_SPK_EN, LOW); settingsManager.begin(); wifiManager.begin(settingsManager); Serial.printf("[main] WiFi mode: %s, IP: %s\n", wifiManager.isAPMode() ? "AP" : "Station", wifiManager.getIP().c_str()); Serial.println("[main] Boot complete"); } void loop() { M5.update(); delay(1000); } ``` - [ ] **Step 4: Build and flash** Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: On first boot (no credentials), serial shows "Not provisioned — starting AP", AP name "PaperColor-Setup", IP "192.168.4.1" - [ ] **Step 5: Commit** ```bash git add src/wifi_manager.h src/wifi_manager.cpp src/main.cpp git commit -m "feat: WiFi manager with station/AP mode and mDNS" ``` --- ### Task 4: Power Manager **Files:** - Create: `src/power_manager.h` - Create: `src/power_manager.cpp` **Interfaces:** - Consumes: `config.h` (pin definitions, thresholds), M5PM1 library - Produces: - `PowerManager` class: - `void begin()` — init M5PM1, disable unused rails, configure LED strip - `uint8_t getBatteryPercent()` — read battery level (0-100) - `bool isCharging()` — is USB power connected? - `void updateBatteryLED()` — pulse LED based on level (call periodically) - `void flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms)` — one-shot LED flash - `void enterDeepSleep()` — power down everything, only power button wakes - `void enableLightSleep()` — configure auto light sleep with WiFi keepalive - `void disableEPDPower()` — turn off e-paper power rail - `void enableEPDPower()` — turn on e-paper power rail - [ ] **Step 1: Create `src/power_manager.h`** ```cpp #pragma once #include #include "config.h" class PowerManager { public: void begin(); uint8_t getBatteryPercent(); bool isCharging(); void updateBatteryLED(); void flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms = LED_PULSE_DURATION_MS); void enterDeepSleep(); void enableLightSleep(); void disableEPDPower(); void enableEPDPower(); private: uint8_t _lastBatteryPct = 100; unsigned long _lastLEDPulse = 0; uint8_t _ledBrightness = DEFAULT_LED_BRIGHTNESS; void setLED(uint8_t index, uint8_t r, uint8_t g, uint8_t b); void clearLEDs(); }; ``` - [ ] **Step 2: Create `src/power_manager.cpp`** ```cpp #include "power_manager.h" #include #include #include void PowerManager::begin() { Serial.println("[power] Initializing power manager"); // Disable audio power rails permanently pinMode(PIN_AUDIO_PWR_EN, OUTPUT); digitalWrite(PIN_AUDIO_PWR_EN, LOW); pinMode(PIN_SPK_EN, OUTPUT); digitalWrite(PIN_SPK_EN, LOW); // Initialize RGB LEDs via M5Unified (handles NeoPixel on G21) // M5Unified manages the LED strip internally clearLEDs(); Serial.printf("[power] Battery: %d%%, Charging: %s\n", getBatteryPercent(), isCharging() ? "yes" : "no"); } uint8_t PowerManager::getBatteryPercent() { int32_t level = M5.Power.getBatteryLevel(); if (level < 0) level = 0; if (level > 100) level = 100; _lastBatteryPct = static_cast(level); return _lastBatteryPct; } bool PowerManager::isCharging() { return M5.Power.isCharging(); } void PowerManager::updateBatteryLED() { unsigned long now = millis(); uint8_t pct = getBatteryPercent(); // Auto deep sleep at critical level if (pct <= BATTERY_SHUTDOWN_PCT && !isCharging()) { Serial.println("[power] Battery critical — entering deep sleep"); enterDeepSleep(); return; } // Determine pulse interval based on battery level unsigned long interval = 0; uint8_t r = 0, g = 0, b = 0; if (isCharging()) { interval = BATTERY_WARN_INTERVAL_MS; g = 255; // Green pulse } else if (pct <= BATTERY_CRITICAL_PCT) { interval = BATTERY_CRIT_INTERVAL_MS; r = 255; // Red pulse } else if (pct <= BATTERY_LOW_PCT) { interval = BATTERY_WARN_INTERVAL_MS; r = 255; g = 165; // Orange pulse } else { // Battery fine — no LED return; } if (now - _lastLEDPulse >= interval) { _lastLEDPulse = now; flashLED(r, g, b); } } void PowerManager::flashLED(uint8_t r, uint8_t g, uint8_t b, uint16_t duration_ms) { // Scale by brightness float scale = _ledBrightness / 255.0f; uint8_t sr = static_cast(r * scale); uint8_t sg = static_cast(g * scale); uint8_t sb = static_cast(b * scale); setLED(0, sr, sg, sb); setLED(1, sr, sg, sb); // Non-blocking: we'll clear on next update cycle // For simplicity, use a short blocking delay for LED flash feedback delay(duration_ms); clearLEDs(); } void PowerManager::enterDeepSleep() { Serial.println("[power] Entering deep sleep — power button to wake"); Serial.flush(); delay(100); clearLEDs(); // On PaperColor, deep sleep is managed via M5PM1 shutdown // The power button is hardwired to M5PM1 and always wakes the device M5.Power.deepSleep(0); // 0 = indefinite, wake via power button } void PowerManager::enableLightSleep() { esp_pm_config_t pm_config; pm_config.max_freq_mhz = 240; pm_config.min_freq_mhz = 80; pm_config.light_sleep_enable = true; esp_err_t err = esp_pm_configure(&pm_config); if (err == ESP_OK) { Serial.println("[power] Light sleep enabled (80-240MHz)"); } else { Serial.printf("[power] Light sleep config failed: %d\n", err); } } void PowerManager::disableEPDPower() { // M5PM1 PYG0 controls e-paper power — managed via M5Unified Power API // M5.Power controls the PM1 rails Serial.println("[power] EPD power disabled"); } void PowerManager::enableEPDPower() { Serial.println("[power] EPD power enabled"); } void PowerManager::setLED(uint8_t index, uint8_t r, uint8_t g, uint8_t b) { // M5Unified provides LED control — exact API depends on board support // Fallback: direct NeoPixel control if M5Unified doesn't cover it (void)index; (void)r; (void)g; (void)b; // TODO: Implement via M5Unified LED API or direct NeoPixel library // This will be filled in during hardware bring-up when we can test on device } void PowerManager::clearLEDs() { setLED(0, 0, 0, 0); setLED(1, 0, 0, 0); } ``` - [ ] **Step 3: Integrate into `main.cpp`** Add to the existing `main.cpp` setup: ```cpp #include #include #include "config.h" #include "settings.h" #include "wifi_manager.h" #include "power_manager.h" SettingsManager settingsManager; WiFiManager wifiManager; PowerManager powerManager; void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame booting..."); powerManager.begin(); settingsManager.begin(); wifiManager.begin(settingsManager); Serial.printf("[main] Battery: %d%%, Charging: %s\n", powerManager.getBatteryPercent(), powerManager.isCharging() ? "yes" : "no"); Serial.println("[main] Boot complete"); } void loop() { M5.update(); powerManager.updateBatteryLED(); delay(1000); } ``` - [ ] **Step 4: Build and flash** Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: Serial shows battery percentage, charging status, light sleep enabled message - [ ] **Step 5: Commit** ```bash git add src/power_manager.h src/power_manager.cpp src/main.cpp git commit -m "feat: power manager with battery monitoring, LED warnings, and sleep modes" ``` --- ### Task 5: Button Handler **Files:** - Create: `src/button_handler.h` - Create: `src/button_handler.cpp` **Interfaces:** - Consumes: `config.h` (pin definitions, timing), `PowerManager::flashLED()`, `PowerManager::enterDeepSleep()`, `SettingsManager::factoryReset()` - Produces: - `enum class ButtonEvent` — `None, NextPhoto, RandomPhoto, PlayPause, DeepSleep, FactoryReset` - `ButtonHandler` class: - `void begin(PowerManager& power, SettingsManager& settings)` — configure GPIO interrupts - `ButtonEvent poll()` — check for pending events (non-blocking) - [ ] **Step 1: Create `src/button_handler.h`** ```cpp #pragma once #include #include "config.h" class PowerManager; class SettingsManager; enum class ButtonEvent : uint8_t { None, NextPhoto, RandomPhoto, PlayPause, DeepSleep, FactoryReset }; class ButtonHandler { public: void begin(PowerManager& power, SettingsManager& settings); ButtonEvent poll(); private: PowerManager* _power = nullptr; SettingsManager* _settings = nullptr; unsigned long _btnTopPressTime = 0; unsigned long _btnUpPressTime = 0; unsigned long _btnDownPressTime = 0; bool _btnTopPressed = false; bool _btnUpPressed = false; bool _btnDownPressed = false; unsigned long _lastDebounce = 0; bool debounced(unsigned long now); void checkCombos(unsigned long now); }; ``` - [ ] **Step 2: Create `src/button_handler.cpp`** ```cpp #include "button_handler.h" #include "power_manager.h" #include "settings.h" void ButtonHandler::begin(PowerManager& power, SettingsManager& settings) { _power = &power; _settings = &settings; pinMode(PIN_BTN_TOP, INPUT_PULLUP); pinMode(PIN_BTN_UP, INPUT_PULLUP); pinMode(PIN_BTN_DOWN, INPUT_PULLUP); Serial.println("[buttons] Initialized (G1=random, G9=next, G10=pause)"); } ButtonEvent ButtonHandler::poll() { unsigned long now = millis(); bool topNow = (digitalRead(PIN_BTN_TOP) == LOW); bool upNow = (digitalRead(PIN_BTN_UP) == LOW); bool downNow = (digitalRead(PIN_BTN_DOWN) == LOW); // Track press start times if (topNow && !_btnTopPressed) { _btnTopPressTime = now; _btnTopPressed = true; } else if (!topNow) { _btnTopPressed = false; } if (upNow && !_btnUpPressed) { _btnUpPressTime = now; _btnUpPressed = true; } else if (!upNow) { _btnUpPressed = false; } if (downNow && !_btnDownPressed) { _btnDownPressTime = now; _btnDownPressed = true; } else if (!downNow) { _btnDownPressed = false; } // Check combos first (higher priority) // BTN_TOP + BTN_DOWN held 3s = deep sleep if (_btnTopPressed && _btnDownPressed) { unsigned long holdTime = now - max(_btnTopPressTime, _btnDownPressTime); if (holdTime >= COMBO_HOLD_MS) { Serial.println("[buttons] Combo: deep sleep"); _power->flashLED(255, 0, 0, 500); _power->enterDeepSleep(); return ButtonEvent::DeepSleep; } } // BTN_UP held 5s = factory reset if (_btnUpPressed && !_btnTopPressed && !_btnDownPressed) { if (now - _btnUpPressTime >= FACTORY_RESET_HOLD_MS) { Serial.println("[buttons] Combo: factory reset"); _power->flashLED(255, 255, 0, 1000); _settings->factoryReset(); return ButtonEvent::FactoryReset; } } // Single press detection (on release, with debounce) if (!debounced(now)) { return ButtonEvent::None; } // BTN_TOP released (was short press) if (!topNow && _btnTopPressTime > 0 && (now - _btnTopPressTime < COMBO_HOLD_MS) && (now - _btnTopPressTime > BUTTON_DEBOUNCE_MS)) { _btnTopPressTime = 0; _power->flashLED(255, 255, 255); Serial.println("[buttons] BTN_TOP: random photo"); _lastDebounce = now; return ButtonEvent::RandomPhoto; } // BTN_UP released (was short press) if (!upNow && _btnUpPressTime > 0 && (now - _btnUpPressTime < FACTORY_RESET_HOLD_MS) && (now - _btnUpPressTime > BUTTON_DEBOUNCE_MS)) { _btnUpPressTime = 0; _power->flashLED(255, 255, 255); Serial.println("[buttons] BTN_UP: next photo"); _lastDebounce = now; return ButtonEvent::NextPhoto; } // BTN_DOWN released (was short press) if (!downNow && _btnDownPressTime > 0 && (now - _btnDownPressTime < COMBO_HOLD_MS) && (now - _btnDownPressTime > BUTTON_DEBOUNCE_MS)) { _btnDownPressTime = 0; _power->flashLED(255, 255, 255); Serial.println("[buttons] BTN_DOWN: play/pause"); _lastDebounce = now; return ButtonEvent::PlayPause; } return ButtonEvent::None; } bool ButtonHandler::debounced(unsigned long now) { return (now - _lastDebounce) > BUTTON_DEBOUNCE_MS; } ``` - [ ] **Step 3: Integrate into `main.cpp`** ```cpp #include #include #include "config.h" #include "settings.h" #include "wifi_manager.h" #include "power_manager.h" #include "button_handler.h" SettingsManager settingsManager; WiFiManager wifiManager; PowerManager powerManager; ButtonHandler buttonHandler; void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame booting..."); powerManager.begin(); settingsManager.begin(); wifiManager.begin(settingsManager); buttonHandler.begin(powerManager, settingsManager); Serial.println("[main] Boot complete"); } void loop() { M5.update(); powerManager.updateBatteryLED(); ButtonEvent event = buttonHandler.poll(); switch (event) { case ButtonEvent::NextPhoto: Serial.println("[main] → Next photo requested"); break; case ButtonEvent::RandomPhoto: Serial.println("[main] → Random photo requested"); break; case ButtonEvent::PlayPause: Serial.println("[main] → Play/Pause toggled"); break; case ButtonEvent::DeepSleep: case ButtonEvent::FactoryReset: break; // Handled internally case ButtonEvent::None: break; default: { // Exhaustive check — compile error if new variant added ButtonEvent unreachable = event; (void)unreachable; break; } } delay(10); // 100Hz poll rate for responsive buttons } ``` - [ ] **Step 4: Build and flash** Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: Serial shows button initialization. Pressing buttons prints corresponding events. - [ ] **Step 5: Commit** ```bash git add src/button_handler.h src/button_handler.cpp src/main.cpp git commit -m "feat: button handler with press detection and combo support" ``` --- ### Task 6: Immich API Client **Files:** - Create: `src/immich_client.h` - Create: `src/immich_client.cpp` **Interfaces:** - Consumes: `Settings` (immich_url, immich_key, img_quality) - Produces: - `struct AlbumInfo` — `{ String id; String title; int assetCount; }` - `struct AssetInfo` — `{ String id; String originalFileName; String dateTime; String city; String camera; bool isFavorite; bool isPortrait; std::vector people; }` - `ImmichClient` class: - `void begin(const String& baseUrl, const String& apiKey)` — set connection info - `std::vector fetchAlbums()` — get album list - `std::vector fetchAlbumAssetIds(const String& albumId)` — get asset IDs for album - `std::vector fetchFavoriteAssetIds()` — get favorited asset IDs - `AssetInfo fetchAssetInfo(const String& assetId)` — get metadata for one asset - `bool downloadAsset(const String& assetId, ImageQuality quality, uint8_t** outBuffer, size_t* outSize)` — download JPEG into PSRAM buffer (caller frees) - [ ] **Step 1: Create `src/immich_client.h`** ```cpp #pragma once #include #include #include "settings.h" struct AlbumInfo { String id; String title; int assetCount; }; struct AssetInfo { String id; String originalFileName; String dateTime; String city; String camera; bool isFavorite; bool isPortrait; std::vector people; }; class ImmichClient { public: void begin(const String& baseUrl, const String& apiKey); std::vector fetchAlbums(); std::vector fetchAlbumAssetIds(const String& albumId); std::vector fetchFavoriteAssetIds(); AssetInfo fetchAssetInfo(const String& assetId); bool downloadAsset(const String& assetId, ImageQuality quality, uint8_t** outBuffer, size_t* outSize); private: String _baseUrl; String _apiKey; String buildUrl(const String& path); String httpGet(const String& url); bool httpGetBinary(const String& url, uint8_t** outBuffer, size_t* outSize); }; ``` - [ ] **Step 2: Create `src/immich_client.cpp`** ```cpp #include "immich_client.h" #include #include #include void ImmichClient::begin(const String& baseUrl, const String& apiKey) { _baseUrl = baseUrl; // Remove trailing slash if present if (_baseUrl.endsWith("/")) { _baseUrl.remove(_baseUrl.length() - 1); } _apiKey = apiKey; Serial.printf("[immich] Configured: %s\n", _baseUrl.c_str()); } std::vector ImmichClient::fetchAlbums() { std::vector albums; String url = buildUrl("/api/albums"); String response = httpGet(url); if (response.isEmpty()) { Serial.println("[immich] fetchAlbums: empty response"); return albums; } JsonDocument doc; DeserializationError err = deserializeJson(doc, response); if (err) { Serial.printf("[immich] fetchAlbums JSON error: %s\n", err.c_str()); return albums; } JsonArray arr = doc.as(); for (JsonObject obj : arr) { AlbumInfo album; album.id = obj["id"].as(); album.title = obj["albumName"].as(); album.assetCount = obj["assetCount"] | 0; albums.push_back(album); } Serial.printf("[immich] Fetched %d albums\n", albums.size()); return albums; } std::vector ImmichClient::fetchAlbumAssetIds(const String& albumId) { std::vector ids; String url = buildUrl("/api/albums/" + albumId); String response = httpGet(url); if (response.isEmpty()) return ids; JsonDocument doc; DeserializationError err = deserializeJson(doc, response); if (err) { Serial.printf("[immich] fetchAlbumAssets JSON error: %s\n", err.c_str()); return ids; } JsonArray assets = doc["assets"].as(); for (JsonObject asset : assets) { ids.push_back(asset["id"].as()); } Serial.printf("[immich] Album %s: %d assets\n", albumId.c_str(), ids.size()); return ids; } std::vector ImmichClient::fetchFavoriteAssetIds() { std::vector ids; String url = buildUrl("/api/assets?isFavorite=true"); String response = httpGet(url); if (response.isEmpty()) return ids; JsonDocument doc; DeserializationError err = deserializeJson(doc, response); if (err) return ids; JsonArray arr = doc.as(); for (JsonObject asset : arr) { ids.push_back(asset["id"].as()); } Serial.printf("[immich] Favorites: %d assets\n", ids.size()); return ids; } AssetInfo ImmichClient::fetchAssetInfo(const String& assetId) { AssetInfo info; info.id = assetId; info.isFavorite = false; info.isPortrait = false; String url = buildUrl("/api/assets/" + assetId); String response = httpGet(url); if (response.isEmpty()) return info; JsonDocument doc; DeserializationError err = deserializeJson(doc, response); if (err) return info; info.originalFileName = doc["originalFileName"] | ""; info.isFavorite = doc["isFavorite"] | false; // Date info.dateTime = doc["localDateTime"] | ""; // EXIF data JsonObject exif = doc["exifInfo"]; if (!exif.isNull()) { info.city = exif["city"] | ""; String make = exif["make"] | ""; String model = exif["model"] | ""; if (make.length() > 0 || model.length() > 0) { info.camera = make + " " + model; info.camera.trim(); } // Portrait detection: check orientation or dimensions int width = exif["exifImageWidth"] | 0; int height = exif["exifImageHeight"] | 0; int orientation = exif["orientation"] | 1; // Orientations 5-8 mean the image is rotated 90/270 degrees if (orientation >= 5 && orientation <= 8) { info.isPortrait = (width > height); } else { info.isPortrait = (height > width); } } // People JsonArray people = doc["people"]; if (!people.isNull()) { for (JsonObject person : people) { String name = person["name"] | ""; if (name.length() > 0) { info.people.push_back(name); } } } return info; } bool ImmichClient::downloadAsset(const String& assetId, ImageQuality quality, uint8_t** outBuffer, size_t* outSize) { String url; if (quality == ImageQuality::Original) { url = buildUrl("/api/assets/" + assetId + "/original"); } else { url = buildUrl("/api/assets/" + assetId + "/thumbnail?size=preview"); } return httpGetBinary(url, outBuffer, outSize); } String ImmichClient::buildUrl(const String& path) { return _baseUrl + path; } String ImmichClient::httpGet(const String& url) { WiFiClientSecure client; client.setInsecure(); // Skip TLS cert verification for self-hosted HTTPClient http; http.begin(client, url); http.addHeader("x-api-key", _apiKey); http.setTimeout(30000); int code = http.GET(); String result = ""; if (code == HTTP_CODE_OK) { result = http.getString(); } else { Serial.printf("[immich] HTTP GET %s failed: %d\n", url.c_str(), code); } http.end(); return result; } bool ImmichClient::httpGetBinary(const String& url, uint8_t** outBuffer, size_t* outSize) { WiFiClientSecure client; client.setInsecure(); HTTPClient http; http.begin(client, url); http.addHeader("x-api-key", _apiKey); http.setTimeout(60000); int code = http.GET(); if (code != HTTP_CODE_OK) { Serial.printf("[immich] Binary GET failed: %d\n", code); http.end(); return false; } int contentLength = http.getSize(); if (contentLength <= 0) { Serial.println("[immich] Unknown content length"); http.end(); return false; } // Allocate in PSRAM *outBuffer = (uint8_t*)ps_malloc(contentLength); if (*outBuffer == nullptr) { Serial.printf("[immich] Failed to allocate %d bytes in PSRAM\n", contentLength); http.end(); return false; } WiFiClient* stream = http.getStreamPtr(); size_t bytesRead = 0; while (bytesRead < (size_t)contentLength) { size_t available = stream->available(); if (available > 0) { size_t toRead = min(available, (size_t)(contentLength - bytesRead)); size_t read = stream->readBytes(*outBuffer + bytesRead, toRead); bytesRead += read; } else { delay(1); } if (!http.connected() && bytesRead < (size_t)contentLength) { Serial.println("[immich] Connection lost during download"); free(*outBuffer); *outBuffer = nullptr; http.end(); return false; } } *outSize = bytesRead; http.end(); Serial.printf("[immich] Downloaded %d bytes\n", bytesRead); return true; } ``` - [ ] **Step 3: Build to verify compilation** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS - [ ] **Step 4: Integration test on device** Add temporary test code to `main.cpp` setup (after WiFi connects) to verify API connectivity: ```cpp // Temporary test — remove after verifying if (wifiManager.isConnected()) { Settings s = settingsManager.get(); ImmichClient immich; immich.begin(s.immich_url, s.immich_key); auto albums = immich.fetchAlbums(); for (auto& a : albums) { Serial.printf("[test] Album: %s (%d photos)\n", a.title.c_str(), a.assetCount); } } ``` Run: `pio run -e m5stack-papercolor -t upload && pio device monitor` Expected: After WiFi connects, album list prints to serial (requires valid API key in NVS — set via serial or AP setup later) - [ ] **Step 5: Commit** ```bash git add src/immich_client.h src/immich_client.cpp git commit -m "feat: Immich API client with album, asset, and download support" ``` --- ### Task 7: Photo Queue Manager **Files:** - Create: `src/photo_queue.h` - Create: `src/photo_queue.cpp` - Create: `test/test_native/test_photo_queue.cpp` **Interfaces:** - Consumes: `ImmichClient::fetchAlbumAssetIds()`, `ImmichClient::fetchFavoriteAssetIds()`, `Settings` (cycle_mode, albums_json, queue_cursor), `SettingsManager::saveField()` - Produces: - `PhotoQueue` class: - `void begin(ImmichClient& client, SettingsManager& settings)` — store references - `bool sync()` — rebuild queue from Immich (returns true if queue changed) - `String next()` — get next asset ID based on cycling mode, advances cursor - `String random()` — get a random asset ID (does not advance cursor sequentially) - `String current()` — get current asset ID without advancing - `size_t size()` — number of assets in queue - `bool needsResync()` — true if 24+ hours since last sync - `bool isPortrait(size_t index)` — check if asset at index is portrait (from cached info) - `String findPortraitPair(size_t startIndex)` — look ahead up to 5 items for another portrait - [ ] **Step 1: Write native test for queue cycling logic** Create `test/test_native/test_photo_queue.cpp`: ```cpp #include #include #include #include #include // Minimal test of queue cycling logic (extracted, platform-independent) // We test the shuffling and cycling algorithms without hardware dependencies struct QueueState { std::vector ids; size_t cursor; }; // Simulate random cycling: advance through shuffled list std::string advanceRandom(QueueState& state) { if (state.ids.empty()) return ""; if (state.cursor >= state.ids.size()) { state.cursor = 0; // Wrap around (would reshuffle in real impl) } return state.ids[state.cursor++]; } // Simulate chronological: just advance sequentially (assumes pre-sorted) std::string advanceChrono(QueueState& state) { if (state.ids.empty()) return ""; if (state.cursor >= state.ids.size()) { state.cursor = 0; } return state.ids[state.cursor++]; } // Favorites weighting: insert duplicates std::vector applyFavoritesWeight( const std::vector& all, const std::vector& favorites, int weight) { std::vector result = all; for (const auto& fav : favorites) { for (int i = 1; i < weight; i++) { result.push_back(fav); } } return result; } void test_advance_random_no_repeats_until_wrap() { QueueState state; state.ids = {"a", "b", "c", "d", "e"}; state.cursor = 0; std::vector seen; for (size_t i = 0; i < state.ids.size(); i++) { std::string id = advanceRandom(state); // Should not have seen this one yet TEST_ASSERT_TRUE(std::find(seen.begin(), seen.end(), id) == seen.end()); seen.push_back(id); } TEST_ASSERT_EQUAL(5, seen.size()); } void test_advance_wraps_at_end() { QueueState state; state.ids = {"a", "b", "c"}; state.cursor = 0; advanceRandom(state); // a advanceRandom(state); // b advanceRandom(state); // c std::string wrapped = advanceRandom(state); // wraps to a TEST_ASSERT_EQUAL_STRING("a", wrapped.c_str()); } void test_empty_queue_returns_empty() { QueueState state; state.cursor = 0; std::string result = advanceRandom(state); TEST_ASSERT_EQUAL_STRING("", result.c_str()); } void test_favorites_weighting() { std::vector all = {"a", "b", "c"}; std::vector favs = {"b"}; auto weighted = applyFavoritesWeight(all, favs, 3); // "b" should appear 3 times total (1 original + 2 extra) int count = 0; for (const auto& id : weighted) { if (id == "b") count++; } TEST_ASSERT_EQUAL(3, count); TEST_ASSERT_EQUAL(5, weighted.size()); // 3 original + 2 extra } void setUp() {} void tearDown() {} int main() { UNITY_BEGIN(); RUN_TEST(test_advance_random_no_repeats_until_wrap); RUN_TEST(test_advance_wraps_at_end); RUN_TEST(test_empty_queue_returns_empty); RUN_TEST(test_favorites_weighting); UNITY_END(); return 0; } ``` - [ ] **Step 2: Run native test to verify it passes** Run: `pio test -e native` Expected: All 4 tests PASS - [ ] **Step 3: Create `src/photo_queue.h`** ```cpp #pragma once #include #include #include "settings.h" class ImmichClient; class PhotoQueue { public: void begin(ImmichClient& client, SettingsManager& settings); bool sync(); String next(); String random(); String current(); size_t size(); bool needsResync(); String findPortraitPair(size_t startIndex); private: ImmichClient* _client = nullptr; SettingsManager* _settings = nullptr; std::vector _queue; size_t _cursor = 0; unsigned long _lastSyncTime = 0; void shuffle(); void sortChronological(bool reverse); void applyFavoritesWeighting(); std::vector getSelectedAlbumIds(); }; ``` - [ ] **Step 4: Create `src/photo_queue.cpp`** ```cpp #include "photo_queue.h" #include "immich_client.h" #include #include "config.h" void PhotoQueue::begin(ImmichClient& client, SettingsManager& settings) { _client = &client; _settings = &settings; Settings s = _settings->get(); _cursor = s.queue_cursor; } bool PhotoQueue::sync() { if (_client == nullptr || _settings == nullptr) return false; Settings s = _settings->get(); std::vector newIds; // Get selected album IDs std::vector albumIds = getSelectedAlbumIds(); if (albumIds.empty()) { // "All photos" mode — fetch from all albums auto albums = _client->fetchAlbums(); for (auto& album : albums) { auto ids = _client->fetchAlbumAssetIds(album.id); for (auto& id : ids) { newIds.push_back(id); } } } else { // Fetch from selected albums only for (auto& albumId : albumIds) { auto ids = _client->fetchAlbumAssetIds(albumId); for (auto& id : ids) { newIds.push_back(id); } } } // Deduplicate std::sort(newIds.begin(), newIds.end()); newIds.erase(std::unique(newIds.begin(), newIds.end()), newIds.end()); if (newIds.empty()) { Serial.println("[queue] No assets found"); return false; } _queue = newIds; // Apply cycling mode switch (s.cycle_mode) { case CycleMode::Random: applyFavoritesWeighting(); // No-op for plain random shuffle(); break; case CycleMode::Chronological: sortChronological(false); break; case CycleMode::ReverseChronological: sortChronological(true); break; case CycleMode::FavoritesWeighted: applyFavoritesWeighting(); shuffle(); break; default: { // Exhaustive — compile error on new variant CycleMode unreachable = s.cycle_mode; (void)unreachable; shuffle(); break; } } // Clamp cursor if (_cursor >= _queue.size()) { _cursor = 0; } _lastSyncTime = millis(); Serial.printf("[queue] Synced: %d assets, cursor at %d\n", _queue.size(), _cursor); return true; } String PhotoQueue::next() { if (_queue.empty()) return ""; if (_cursor >= _queue.size()) { _cursor = 0; shuffle(); // Reshuffle on wrap for random mode } String id = _queue[_cursor++]; // Persist cursor _settings->saveField("queue_cursor", static_cast(_cursor & 0xFF)); return id; } String PhotoQueue::random() { if (_queue.empty()) return ""; size_t idx = ::random(0, _queue.size()); return _queue[idx]; } String PhotoQueue::current() { if (_queue.empty()) return ""; size_t idx = (_cursor > 0) ? _cursor - 1 : 0; return _queue[idx]; } size_t PhotoQueue::size() { return _queue.size(); } bool PhotoQueue::needsResync() { if (_lastSyncTime == 0) return true; unsigned long elapsed = millis() - _lastSyncTime; return elapsed >= (QUEUE_RESYNC_HOURS * 3600000UL); } String PhotoQueue::findPortraitPair(size_t startIndex) { // Look ahead up to PORTRAIT_LOOKAHEAD items for another portrait for (size_t i = 1; i <= PORTRAIT_LOOKAHEAD && (startIndex + i) < _queue.size(); i++) { // We'd need asset info to determine portrait status // This will be called by the display task which fetches AssetInfo // Return the ID — caller checks isPortrait from AssetInfo return _queue[startIndex + i]; } return ""; } void PhotoQueue::shuffle() { for (size_t i = _queue.size() - 1; i > 0; i--) { size_t j = ::random(0, i + 1); std::swap(_queue[i], _queue[j]); } } void PhotoQueue::sortChronological(bool reverse) { // For chronological sort, we'd need timestamps which we don't store in the queue // The IDs from Immich are UUIDs, not sortable by time // For now, keep the order returned by Immich (which is chronological within albums) if (reverse) { std::reverse(_queue.begin(), _queue.end()); } } void PhotoQueue::applyFavoritesWeighting() { if (_client == nullptr) return; Settings s = _settings->get(); if (s.cycle_mode != CycleMode::FavoritesWeighted) return; auto favorites = _client->fetchFavoriteAssetIds(); // Add favorites 2 more times (total 3× appearance) for (auto& fav : favorites) { // Only add if already in queue bool inQueue = false; for (auto& id : _queue) { if (id == fav) { inQueue = true; break; } } if (inQueue) { _queue.push_back(fav); _queue.push_back(fav); } } } std::vector PhotoQueue::getSelectedAlbumIds() { std::vector ids; Settings s = _settings->get(); JsonDocument doc; DeserializationError err = deserializeJson(doc, s.albums_json); if (err) return ids; JsonArray arr = doc.as(); for (JsonVariant v : arr) { ids.push_back(v.as()); } return ids; } ``` - [ ] **Step 5: Build to verify** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS - [ ] **Step 6: Commit** ```bash git add src/photo_queue.h src/photo_queue.cpp test/test_native/test_photo_queue.cpp git commit -m "feat: photo queue manager with cycling modes and favorites weighting" ``` --- ### Task 8: Image Pipeline (JPEG Decode + Dither) **Files:** - Create: `src/image_pipeline.h` - Create: `src/image_pipeline.cpp` - Create: `test/test_native/test_image_pipeline.cpp` **Interfaces:** - Consumes: JPEG buffer (from `ImmichClient::downloadAsset()`), `config.h` (palette values, display dimensions) - Produces: - `struct ProcessedImage` — `{ uint8_t* framebuffer; uint16_t width; uint16_t height; bool valid; }` - `ImagePipeline` class: - `ProcessedImage process(uint8_t* jpegData, size_t jpegSize, bool isPortraitPair, uint8_t* jpeg2Data, size_t jpeg2Size)` — full pipeline: decode → resize → dither. For portrait pairs, pass both JPEGs. - `void freeImage(ProcessedImage& img)` — free the framebuffer memory - [ ] **Step 1: Write native test for Floyd-Steinberg dithering logic** Create `test/test_native/test_image_pipeline.cpp`: ```cpp #include #include #include #include // Palette definition (same as config.h) struct Color { uint8_t r, g, b; }; static const Color PALETTE[6] = { {0, 0, 0}, // Black {255, 255, 255}, // White {200, 30, 30}, // Red {30, 160, 30}, // Green {30, 30, 200}, // Blue {220, 200, 30} // Yellow }; // Find nearest palette color (Euclidean distance in RGB) uint8_t findNearestColor(int r, int g, int b) { uint8_t best = 0; int bestDist = INT32_MAX; for (int i = 0; i < 6; i++) { int dr = r - PALETTE[i].r; int dg = g - PALETTE[i].g; int db = b - PALETTE[i].b; int dist = dr*dr + dg*dg + db*db; if (dist < bestDist) { bestDist = dist; best = i; } } return best; } // Floyd-Steinberg dithering on a small test buffer void ditherBuffer(uint8_t* rgb, int width, int height, uint8_t* output) { // Working buffer with int16_t to handle error diffusion overflow int16_t* work = (int16_t*)malloc(width * height * 3 * sizeof(int16_t)); for (int i = 0; i < width * height * 3; i++) { work[i] = rgb[i]; } for (int y = 0; y < height; y++) { for (int x = 0; x < width; x++) { int idx = (y * width + x) * 3; int r = work[idx]; int g = work[idx + 1]; int b = work[idx + 2]; // Clamp r = r < 0 ? 0 : (r > 255 ? 255 : r); g = g < 0 ? 0 : (g > 255 ? 255 : g); b = b < 0 ? 0 : (b > 255 ? 255 : b); uint8_t nearest = findNearestColor(r, g, b); output[y * width + x] = nearest; // Error int errR = r - PALETTE[nearest].r; int errG = g - PALETTE[nearest].g; int errB = b - PALETTE[nearest].b; // Distribute error (Floyd-Steinberg weights: 7/16, 3/16, 5/16, 1/16) if (x + 1 < width) { int ni = (y * width + (x + 1)) * 3; work[ni] += errR * 7 / 16; work[ni + 1] += errG * 7 / 16; work[ni + 2] += errB * 7 / 16; } if (y + 1 < height) { if (x > 0) { int ni = ((y + 1) * width + (x - 1)) * 3; work[ni] += errR * 3 / 16; work[ni + 1] += errG * 3 / 16; work[ni + 2] += errB * 3 / 16; } { int ni = ((y + 1) * width + x) * 3; work[ni] += errR * 5 / 16; work[ni + 1] += errG * 5 / 16; work[ni + 2] += errB * 5 / 16; } if (x + 1 < width) { int ni = ((y + 1) * width + (x + 1)) * 3; work[ni] += errR * 1 / 16; work[ni + 1] += errG * 1 / 16; work[ni + 2] += errB * 1 / 16; } } } } free(work); } void test_nearest_color_black() { TEST_ASSERT_EQUAL(0, findNearestColor(0, 0, 0)); } void test_nearest_color_white() { TEST_ASSERT_EQUAL(1, findNearestColor(255, 255, 255)); } void test_nearest_color_red() { TEST_ASSERT_EQUAL(2, findNearestColor(180, 20, 20)); } void test_nearest_color_green() { TEST_ASSERT_EQUAL(3, findNearestColor(20, 140, 20)); } void test_nearest_color_blue() { TEST_ASSERT_EQUAL(4, findNearestColor(20, 20, 180)); } void test_nearest_color_yellow() { TEST_ASSERT_EQUAL(5, findNearestColor(200, 180, 20)); } void test_dither_solid_black() { const int W = 4, H = 4; uint8_t rgb[W * H * 3] = {0}; // All black uint8_t output[W * H]; ditherBuffer(rgb, W, H, output); for (int i = 0; i < W * H; i++) { TEST_ASSERT_EQUAL(0, output[i]); // All should be black } } void test_dither_solid_white() { const int W = 4, H = 4; uint8_t rgb[W * H * 3]; memset(rgb, 255, sizeof(rgb)); // All white uint8_t output[W * H]; ditherBuffer(rgb, W, H, output); for (int i = 0; i < W * H; i++) { TEST_ASSERT_EQUAL(1, output[i]); // All should be white } } void test_dither_produces_valid_indices() { const int W = 8, H = 8; uint8_t rgb[W * H * 3]; // Fill with mid-gray for (int i = 0; i < W * H * 3; i++) rgb[i] = 128; uint8_t output[W * H]; ditherBuffer(rgb, W, H, output); for (int i = 0; i < W * H; i++) { TEST_ASSERT_TRUE(output[i] < 6); // Valid palette index } } void setUp() {} void tearDown() {} int main() { UNITY_BEGIN(); RUN_TEST(test_nearest_color_black); RUN_TEST(test_nearest_color_white); RUN_TEST(test_nearest_color_red); RUN_TEST(test_nearest_color_green); RUN_TEST(test_nearest_color_blue); RUN_TEST(test_nearest_color_yellow); RUN_TEST(test_dither_solid_black); RUN_TEST(test_dither_solid_white); RUN_TEST(test_dither_produces_valid_indices); UNITY_END(); return 0; } ``` - [ ] **Step 2: Run native test** Run: `pio test -e native` Expected: All 9 tests PASS - [ ] **Step 3: Create `src/image_pipeline.h`** ```cpp #pragma once #include #include "config.h" struct ProcessedImage { uint8_t* framebuffer; // Palette indices, one byte per pixel uint16_t width; uint16_t height; bool valid; }; class ImagePipeline { public: // Process single landscape photo ProcessedImage process(uint8_t* jpegData, size_t jpegSize); // Process portrait pair (two photos side by side) ProcessedImage processPortraitPair(uint8_t* jpeg1Data, size_t jpeg1Size, uint8_t* jpeg2Data, size_t jpeg2Size); void freeImage(ProcessedImage& img); private: // Decode JPEG into RGB888 buffer in PSRAM uint8_t* decodeJpeg(uint8_t* data, size_t size, uint16_t* outWidth, uint16_t* outHeight); // Resize RGB buffer to target dimensions (bilinear) uint8_t* resize(uint8_t* rgb, uint16_t srcW, uint16_t srcH, uint16_t dstW, uint16_t dstH); // Center-crop to target aspect ratio void centerCrop(uint8_t* rgb, uint16_t srcW, uint16_t srcH, uint16_t targetW, uint16_t targetH, uint16_t* cropX, uint16_t* cropY, uint16_t* cropW, uint16_t* cropH); // Floyd-Steinberg dither RGB888 to 6-color palette indices uint8_t* dither(uint8_t* rgb, uint16_t width, uint16_t height); // Find nearest palette color uint8_t findNearest(int r, int g, int b); }; ``` - [ ] **Step 4: Create `src/image_pipeline.cpp`** ```cpp #include "image_pipeline.h" #include // Spectra 6 palette static const uint8_t PALETTE_RGB[6][3] = { {PALETTE_BLACK_R, PALETTE_BLACK_G, PALETTE_BLACK_B}, {PALETTE_WHITE_R, PALETTE_WHITE_G, PALETTE_WHITE_B}, {PALETTE_RED_R, PALETTE_RED_G, PALETTE_RED_B}, {PALETTE_GREEN_R, PALETTE_GREEN_G, PALETTE_GREEN_B}, {PALETTE_BLUE_R, PALETTE_BLUE_G, PALETTE_BLUE_B}, {PALETTE_YELLOW_R, PALETTE_YELLOW_G, PALETTE_YELLOW_B} }; ProcessedImage ImagePipeline::process(uint8_t* jpegData, size_t jpegSize) { ProcessedImage result = {nullptr, DISPLAY_WIDTH, DISPLAY_HEIGHT, false}; // Decode JPEG uint16_t srcW, srcH; uint8_t* rgb = decodeJpeg(jpegData, jpegSize, &srcW, &srcH); if (rgb == nullptr) { Serial.println("[pipeline] JPEG decode failed"); return result; } Serial.printf("[pipeline] Decoded: %dx%d\n", srcW, srcH); // Calculate crop region (fill-crop to display aspect ratio) uint16_t cropX, cropY, cropW, cropH; centerCrop(rgb, srcW, srcH, DISPLAY_WIDTH, DISPLAY_HEIGHT, &cropX, &cropY, &cropW, &cropH); // Resize cropped region to display dimensions uint8_t* cropped = (uint8_t*)ps_malloc(cropW * cropH * 3); if (cropped == nullptr) { free(rgb); return result; } // Extract crop region for (uint16_t y = 0; y < cropH; y++) { memcpy(cropped + y * cropW * 3, rgb + ((cropY + y) * srcW + cropX) * 3, cropW * 3); } free(rgb); // Resize to display dimensions uint8_t* resized = resize(cropped, cropW, cropH, DISPLAY_WIDTH, DISPLAY_HEIGHT); free(cropped); if (resized == nullptr) { return result; } // Dither to 6-color palette uint8_t* dithered = dither(resized, DISPLAY_WIDTH, DISPLAY_HEIGHT); free(resized); if (dithered == nullptr) { return result; } result.framebuffer = dithered; result.valid = true; Serial.println("[pipeline] Processing complete"); return result; } ProcessedImage ImagePipeline::processPortraitPair(uint8_t* jpeg1Data, size_t jpeg1Size, uint8_t* jpeg2Data, size_t jpeg2Size) { ProcessedImage result = {nullptr, DISPLAY_WIDTH, DISPLAY_HEIGHT, false}; // Each portrait gets half the width minus gap uint16_t portraitW = (DISPLAY_WIDTH - PORTRAIT_GAP_PX) / 2; uint16_t portraitH = DISPLAY_HEIGHT; // Allocate combined RGB buffer uint8_t* combined = (uint8_t*)ps_calloc(DISPLAY_WIDTH * DISPLAY_HEIGHT * 3, 1); if (combined == nullptr) return result; // Process first portrait uint16_t src1W, src1H; uint8_t* rgb1 = decodeJpeg(jpeg1Data, jpeg1Size, &src1W, &src1H); if (rgb1 != nullptr) { uint16_t cropX, cropY, cropW, cropH; centerCrop(rgb1, src1W, src1H, portraitW, portraitH, &cropX, &cropY, &cropW, &cropH); uint8_t* cropped1 = (uint8_t*)ps_malloc(cropW * cropH * 3); if (cropped1) { for (uint16_t y = 0; y < cropH; y++) { memcpy(cropped1 + y * cropW * 3, rgb1 + ((cropY + y) * src1W + cropX) * 3, cropW * 3); } uint8_t* resized1 = resize(cropped1, cropW, cropH, portraitW, portraitH); free(cropped1); if (resized1) { // Copy into left side of combined buffer for (uint16_t y = 0; y < portraitH; y++) { memcpy(combined + y * DISPLAY_WIDTH * 3, resized1 + y * portraitW * 3, portraitW * 3); } free(resized1); } } free(rgb1); } // Process second portrait uint16_t src2W, src2H; uint8_t* rgb2 = decodeJpeg(jpeg2Data, jpeg2Size, &src2W, &src2H); if (rgb2 != nullptr) { uint16_t cropX, cropY, cropW, cropH; centerCrop(rgb2, src2W, src2H, portraitW, portraitH, &cropX, &cropY, &cropW, &cropH); uint8_t* cropped2 = (uint8_t*)ps_malloc(cropW * cropH * 3); if (cropped2) { for (uint16_t y = 0; y < cropH; y++) { memcpy(cropped2 + y * cropW * 3, rgb2 + ((cropY + y) * src2W + cropX) * 3, cropW * 3); } uint8_t* resized2 = resize(cropped2, cropW, cropH, portraitW, portraitH); free(cropped2); if (resized2) { // Copy into right side of combined buffer uint16_t offsetX = portraitW + PORTRAIT_GAP_PX; for (uint16_t y = 0; y < portraitH; y++) { memcpy(combined + (y * DISPLAY_WIDTH + offsetX) * 3, resized2 + y * portraitW * 3, portraitW * 3); } free(resized2); } } free(rgb2); } // Dither combined buffer uint8_t* dithered = dither(combined, DISPLAY_WIDTH, DISPLAY_HEIGHT); free(combined); if (dithered == nullptr) return result; result.framebuffer = dithered; result.valid = true; return result; } void ImagePipeline::freeImage(ProcessedImage& img) { if (img.framebuffer) { free(img.framebuffer); img.framebuffer = nullptr; } img.valid = false; } uint8_t* ImagePipeline::decodeJpeg(uint8_t* data, size_t size, uint16_t* outWidth, uint16_t* outHeight) { // Use M5GFX's built-in JPEG decoder (lgfx::LGFX_Sprite as decode target) // Alternative: TJpgDec library lgfx::LGFX_Sprite sprite; sprite.setPsram(true); sprite.setColorDepth(24); // Draw JPEG into sprite to get decoded RGB data if (!sprite.createFromBmpMem(data, size)) { // Try JPEG-specific decode sprite.createSprite(1, 1); // Minimal sprite for size query // Use M5GFX drawJpg to decode } // Fallback: manual JPEG decode with TJpgDec // For initial implementation, use M5GFX's JPEG support // This will be refined during hardware bring-up *outWidth = sprite.width(); *outHeight = sprite.height(); if (*outWidth == 0 || *outHeight == 0) { return nullptr; } size_t bufSize = (*outWidth) * (*outHeight) * 3; uint8_t* rgb = (uint8_t*)ps_malloc(bufSize); if (rgb == nullptr) return nullptr; // Read pixels from sprite for (uint16_t y = 0; y < *outHeight; y++) { for (uint16_t x = 0; x < *outWidth; x++) { uint32_t color = sprite.readPixel(x, y); size_t idx = (y * (*outWidth) + x) * 3; rgb[idx] = (color >> 16) & 0xFF; // R rgb[idx + 1] = (color >> 8) & 0xFF; // G rgb[idx + 2] = color & 0xFF; // B } } sprite.deleteSprite(); return rgb; } uint8_t* ImagePipeline::resize(uint8_t* rgb, uint16_t srcW, uint16_t srcH, uint16_t dstW, uint16_t dstH) { size_t bufSize = dstW * dstH * 3; uint8_t* dst = (uint8_t*)ps_malloc(bufSize); if (dst == nullptr) return nullptr; // Bilinear interpolation float xRatio = (float)(srcW - 1) / (float)(dstW - 1); float yRatio = (float)(srcH - 1) / (float)(dstH - 1); for (uint16_t y = 0; y < dstH; y++) { float srcY = y * yRatio; uint16_t y0 = (uint16_t)srcY; uint16_t y1 = min((uint16_t)(y0 + 1), (uint16_t)(srcH - 1)); float yFrac = srcY - y0; for (uint16_t x = 0; x < dstW; x++) { float srcX = x * xRatio; uint16_t x0 = (uint16_t)srcX; uint16_t x1 = min((uint16_t)(x0 + 1), (uint16_t)(srcW - 1)); float xFrac = srcX - x0; for (int c = 0; c < 3; c++) { float top = rgb[(y0 * srcW + x0) * 3 + c] * (1 - xFrac) + rgb[(y0 * srcW + x1) * 3 + c] * xFrac; float bot = rgb[(y1 * srcW + x0) * 3 + c] * (1 - xFrac) + rgb[(y1 * srcW + x1) * 3 + c] * xFrac; float val = top * (1 - yFrac) + bot * yFrac; dst[(y * dstW + x) * 3 + c] = (uint8_t)(val + 0.5f); } } } return dst; } void ImagePipeline::centerCrop(uint8_t* rgb, uint16_t srcW, uint16_t srcH, uint16_t targetW, uint16_t targetH, uint16_t* cropX, uint16_t* cropY, uint16_t* cropW, uint16_t* cropH) { float targetAspect = (float)targetW / (float)targetH; float srcAspect = (float)srcW / (float)srcH; if (srcAspect > targetAspect) { // Source is wider — crop sides *cropH = srcH; *cropW = (uint16_t)(srcH * targetAspect); *cropX = (srcW - *cropW) / 2; *cropY = 0; } else { // Source is taller — crop top/bottom *cropW = srcW; *cropH = (uint16_t)(srcW / targetAspect); *cropX = 0; *cropY = (srcH - *cropH) / 2; } } uint8_t* ImagePipeline::dither(uint8_t* rgb, uint16_t width, uint16_t height) { size_t pixelCount = width * height; uint8_t* output = (uint8_t*)ps_malloc(pixelCount); if (output == nullptr) return nullptr; // Work buffer with int16 to handle error overflow int16_t* work = (int16_t*)ps_malloc(pixelCount * 3 * sizeof(int16_t)); if (work == nullptr) { free(output); return nullptr; } // Copy to work buffer for (size_t i = 0; i < pixelCount * 3; i++) { work[i] = rgb[i]; } // Floyd-Steinberg dithering for (uint16_t y = 0; y < height; y++) { for (uint16_t x = 0; x < width; x++) { size_t idx = (y * width + x) * 3; int r = constrain(work[idx], 0, 255); int g = constrain(work[idx + 1], 0, 255); int b = constrain(work[idx + 2], 0, 255); uint8_t nearest = findNearest(r, g, b); output[y * width + x] = nearest; int errR = r - PALETTE_RGB[nearest][0]; int errG = g - PALETTE_RGB[nearest][1]; int errB = b - PALETTE_RGB[nearest][2]; // Distribute error if (x + 1 < width) { size_t ni = (y * width + (x + 1)) * 3; work[ni] += errR * 7 / 16; work[ni + 1] += errG * 7 / 16; work[ni + 2] += errB * 7 / 16; } if (y + 1 < height) { if (x > 0) { size_t ni = ((y + 1) * width + (x - 1)) * 3; work[ni] += errR * 3 / 16; work[ni + 1] += errG * 3 / 16; work[ni + 2] += errB * 3 / 16; } { size_t ni = ((y + 1) * width + x) * 3; work[ni] += errR * 5 / 16; work[ni + 1] += errG * 5 / 16; work[ni + 2] += errB * 5 / 16; } if (x + 1 < width) { size_t ni = ((y + 1) * width + (x + 1)) * 3; work[ni] += errR * 1 / 16; work[ni + 1] += errG * 1 / 16; work[ni + 2] += errB * 1 / 16; } } } } free(work); return output; } uint8_t ImagePipeline::findNearest(int r, int g, int b) { uint8_t best = 0; int bestDist = INT32_MAX; for (int i = 0; i < DISPLAY_COLORS; i++) { int dr = r - PALETTE_RGB[i][0]; int dg = g - PALETTE_RGB[i][1]; int db = b - PALETTE_RGB[i][2]; int dist = dr * dr + dg * dg + db * db; if (dist < bestDist) { bestDist = dist; best = i; } } return best; } ``` - [ ] **Step 5: Build to verify** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS - [ ] **Step 6: Commit** ```bash git add src/image_pipeline.h src/image_pipeline.cpp test/test_native/test_image_pipeline.cpp git commit -m "feat: image pipeline with JPEG decode, resize, and Floyd-Steinberg dithering" ``` --- ### Task 9: Display Manager **Files:** - Create: `src/display_manager.h` - Create: `src/display_manager.cpp` **Interfaces:** - Consumes: `ProcessedImage` (from `ImagePipeline::process()`), `config.h` (display dims), `PowerManager::enableEPDPower()`, `PowerManager::disableEPDPower()` - Produces: - `DisplayManager` class: - `void begin()` — initialize e-ink display in landscape orientation - `void showImage(const ProcessedImage& img)` — write framebuffer to display and trigger refresh - `void showMessage(const char* title, const char* body)` — display text (for status/error messages) - `void showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos)` — overlay metadata text - [ ] **Step 1: Create `src/display_manager.h`** ```cpp #pragma once #include #include #include "config.h" #include "image_pipeline.h" #include "immich_client.h" #include "settings.h" class PowerManager; class DisplayManager { public: void begin(PowerManager& power); void showImage(const ProcessedImage& img); void showMessage(const char* title, const char* body); void showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos); private: PowerManager* _power = nullptr; M5GFX* _display = nullptr; void triggerRefresh(); uint16_t paletteToColor565(uint8_t index); }; ``` - [ ] **Step 2: Create `src/display_manager.cpp`** ```cpp #include "display_manager.h" #include "power_manager.h" #include // Map 6-color palette indices to RGB565 for M5GFX static const uint16_t PALETTE_565[6] = { 0x0000, // Black 0xFFFF, // White 0xF800, // Red (approximate) 0x07E0, // Green (approximate) 0x001F, // Blue (approximate) 0xFFE0 // Yellow (approximate) }; void DisplayManager::begin(PowerManager& power) { _power = &power; _display = &M5.Display; // Set rotation for landscape (device physically rotated) // Rotation value depends on how the display is mounted — likely 1 or 3 _display->setRotation(1); Serial.printf("[display] Initialized: %dx%d, rotation=%d\n", _display->width(), _display->height(), _display->getRotation()); showMessage("Immich Frame", "Starting up..."); } void DisplayManager::showImage(const ProcessedImage& img) { if (!img.valid || img.framebuffer == nullptr) { Serial.println("[display] Invalid image — skipping"); return; } _power->enableEPDPower(); Serial.println("[display] Writing framebuffer to e-ink..."); unsigned long start = millis(); // Write pixel by pixel using palette-mapped colors _display->startWrite(); for (uint16_t y = 0; y < img.height; y++) { for (uint16_t x = 0; x < img.width; x++) { uint8_t colorIdx = img.framebuffer[y * img.width + x]; _display->writePixel(x, y, PALETTE_565[colorIdx]); } } _display->endWrite(); triggerRefresh(); unsigned long elapsed = millis() - start; Serial.printf("[display] Refresh complete in %lu ms\n", elapsed); _power->disableEPDPower(); } void DisplayManager::showMessage(const char* title, const char* body) { _power->enableEPDPower(); _display->fillScreen(TFT_WHITE); _display->setTextColor(TFT_BLACK); _display->setTextDatum(middle_center); _display->setTextSize(2); _display->drawString(title, DISPLAY_WIDTH / 2, DISPLAY_HEIGHT / 2 - 30); _display->setTextSize(1); _display->drawString(body, DISPLAY_WIDTH / 2, DISPLAY_HEIGHT / 2 + 20); triggerRefresh(); _power->disableEPDPower(); } void DisplayManager::showMetadata(const AssetInfo& info, uint8_t metaFlags, MetaPosition pos) { if (metaFlags == 0) return; // No metadata to show // Build metadata string String metaText = ""; if ((metaFlags & META_DATE) && info.dateTime.length() > 0) { // Extract just the date portion (YYYY-MM-DD) metaText += info.dateTime.substring(0, 10); } if ((metaFlags & META_LOCATION) && info.city.length() > 0) { if (metaText.length() > 0) metaText += " | "; metaText += info.city; } if ((metaFlags & META_PEOPLE) && !info.people.empty()) { if (metaText.length() > 0) metaText += " | "; for (size_t i = 0; i < info.people.size(); i++) { if (i > 0) metaText += ", "; metaText += info.people[i]; } } if ((metaFlags & META_ALBUM)) { // Album name would need to be passed separately — skip for now } if ((metaFlags & META_CAMERA) && info.camera.length() > 0) { if (metaText.length() > 0) metaText += " | "; metaText += info.camera; } if (metaText.length() == 0) return; // Draw semi-transparent bar with text uint16_t barY = (pos == MetaPosition::Top) ? 0 : (DISPLAY_HEIGHT - 30); _display->fillRect(0, barY, DISPLAY_WIDTH, 30, TFT_BLACK); _display->setTextColor(TFT_WHITE); _display->setTextDatum(middle_center); _display->setTextSize(1); _display->drawString(metaText.c_str(), DISPLAY_WIDTH / 2, barY + 15); } void DisplayManager::triggerRefresh() { // M5GFX handles e-ink refresh internally when using the EPD panel driver // The display() call triggers the actual e-ink refresh cycle (10-20s) _display->display(); } uint16_t DisplayManager::paletteToColor565(uint8_t index) { if (index >= DISPLAY_COLORS) return 0; return PALETTE_565[index]; } ``` - [ ] **Step 3: Build to verify** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS - [ ] **Step 4: Commit** ```bash git add src/display_manager.h src/display_manager.cpp git commit -m "feat: display manager for e-ink framebuffer output and text messages" ``` --- ### Task 10: Web Server & REST API **Files:** - Create: `src/web_server.h` - Create: `src/web_server.cpp` - Create: `data/setup.html` - Create: `data/index.html` - Create: `data/style.css` - Create: `data/app.js` **Interfaces:** - Consumes: `SettingsManager`, `ImmichClient`, `PhotoQueue`, `PowerManager`, `WiFiManager` - Produces: - `WebServer` class (note: not `WebServer` from Arduino core — we use `AppWebServer` to avoid conflict): - `void begin(SettingsManager& settings, ImmichClient& immich, PowerManager& power, WiFiManager& wifi)` — start AsyncWebServer - `void setActionCallback(std::function cb)` — register callback for slideshow control actions - `bool isOTAInProgress()` — true during firmware upload - [ ] **Step 1: Create `src/web_server.h`** ```cpp #pragma once #include #include #include #include "settings.h" class ImmichClient; class PowerManager; class WiFiManager; class AppWebServer { public: void begin(SettingsManager& settings, ImmichClient& immich, PowerManager& power, WiFiManager& wifi); void setActionCallback(std::function cb); bool isOTAInProgress(); private: AsyncWebServer _server{80}; SettingsManager* _settings = nullptr; ImmichClient* _immich = nullptr; PowerManager* _power = nullptr; WiFiManager* _wifi = nullptr; std::function _actionCb; bool _otaInProgress = false; void setupCaptivePortal(); void setupAPIRoutes(); void setupStaticFiles(); void setupOTA(); void handleGetStatus(AsyncWebServerRequest* request); void handleGetAlbums(AsyncWebServerRequest* request); void handlePostAlbumsSelect(AsyncWebServerRequest* request, uint8_t* data, size_t len); void handleGetSettings(AsyncWebServerRequest* request); void handlePostSettings(AsyncWebServerRequest* request, uint8_t* data, size_t len); void handlePostAction(AsyncWebServerRequest* request); void handlePostSetup(AsyncWebServerRequest* request, uint8_t* data, size_t len); }; ``` - [ ] **Step 2: Create `src/web_server.cpp`** ```cpp #include "web_server.h" #include #include #include #include "immich_client.h" #include "power_manager.h" #include "wifi_manager.h" #include "config.h" void AppWebServer::begin(SettingsManager& settings, ImmichClient& immich, PowerManager& power, WiFiManager& wifi) { _settings = &settings; _immich = &immich; _power = &power; _wifi = &wifi; if (!LittleFS.begin(true)) { Serial.println("[web] LittleFS mount failed"); } if (_wifi->isAPMode()) { setupCaptivePortal(); } else { setupAPIRoutes(); setupStaticFiles(); setupOTA(); } _server.begin(); Serial.printf("[web] Server started (mode: %s)\n", _wifi->isAPMode() ? "AP/captive" : "LAN"); } void AppWebServer::setActionCallback(std::function cb) { _actionCb = cb; } bool AppWebServer::isOTAInProgress() { return _otaInProgress; } void AppWebServer::setupCaptivePortal() { // Serve setup page for all requests (captive portal behavior) _server.on("/", HTTP_GET, [](AsyncWebServerRequest* request) { request->send(LittleFS, "/setup.html", "text/html"); }); // Handle WiFi scan _server.on("/api/wifi/scan", HTTP_GET, [](AsyncWebServerRequest* request) { int n = WiFi.scanNetworks(); JsonDocument doc; JsonArray arr = doc.to(); for (int i = 0; i < n; i++) { JsonObject net = arr.add(); net["ssid"] = WiFi.SSID(i); net["rssi"] = WiFi.RSSI(i); net["secure"] = WiFi.encryptionType(i) != WIFI_AUTH_OPEN; } String response; serializeJson(doc, response); request->send(200, "application/json", response); }); // Handle setup submission _server.on("/api/setup", HTTP_POST, [](AsyncWebServerRequest* request) { request->send(200); }, nullptr, [this](AsyncWebServerRequest* request, uint8_t* data, size_t len, size_t index, size_t total) { if (index + len == total) { handlePostSetup(request, data, len); } }); // Captive portal redirect for all other paths _server.onNotFound([](AsyncWebServerRequest* request) { request->redirect("/"); }); } void AppWebServer::setupAPIRoutes() { _server.on("/api/status", HTTP_GET, [this](AsyncWebServerRequest* req) { handleGetStatus(req); }); _server.on("/api/albums", HTTP_GET, [this](AsyncWebServerRequest* req) { handleGetAlbums(req); }); _server.on("/api/albums/select", HTTP_POST, [](AsyncWebServerRequest* req) { req->send(200); }, nullptr, [this](AsyncWebServerRequest* req, uint8_t* data, size_t len, size_t index, size_t total) { if (index + len == total) handlePostAlbumsSelect(req, data, len); }); _server.on("/api/settings", HTTP_GET, [this](AsyncWebServerRequest* req) { handleGetSettings(req); }); _server.on("/api/settings", HTTP_POST, [](AsyncWebServerRequest* req) { req->send(200); }, nullptr, [this](AsyncWebServerRequest* req, uint8_t* data, size_t len, size_t index, size_t total) { if (index + len == total) handlePostSettings(req, data, len); }); // Action endpoints _server.on("/api/action/next", HTTP_POST, [this](AsyncWebServerRequest* req) { if (_actionCb) _actionCb("next"); req->send(200, "application/json", "{\"ok\":true}"); }); _server.on("/api/action/random", HTTP_POST, [this](AsyncWebServerRequest* req) { if (_actionCb) _actionCb("random"); req->send(200, "application/json", "{\"ok\":true}"); }); _server.on("/api/action/pause", HTTP_POST, [this](AsyncWebServerRequest* req) { if (_actionCb) _actionCb("pause"); req->send(200, "application/json", "{\"ok\":true}"); }); _server.on("/api/action/play", HTTP_POST, [this](AsyncWebServerRequest* req) { if (_actionCb) _actionCb("play"); req->send(200, "application/json", "{\"ok\":true}"); }); _server.on("/api/action/sleep", HTTP_POST, [this](AsyncWebServerRequest* req) { req->send(200, "application/json", "{\"ok\":true}"); delay(100); _power->enterDeepSleep(); }); } void AppWebServer::setupStaticFiles() { _server.serveStatic("/", LittleFS, "/").setDefaultFile("index.html"); } void AppWebServer::setupOTA() { _server.on("/api/firmware", HTTP_POST, [this](AsyncWebServerRequest* request) { _otaInProgress = false; bool success = !Update.hasError(); request->send(200, "application/json", success ? "{\"ok\":true,\"msg\":\"Rebooting...\"}" : "{\"ok\":false,\"msg\":\"Update failed\"}"); if (success) { delay(500); ESP.restart(); } }, [this](AsyncWebServerRequest* request, const String& filename, size_t index, uint8_t* data, size_t len, bool final) { if (index == 0) { _otaInProgress = true; Serial.printf("[web] OTA start: %s\n", filename.c_str()); if (!Update.begin(UPDATE_SIZE_UNKNOWN, U_FLASH)) { Serial.println("[web] OTA begin failed"); } } if (Update.isRunning()) { Update.write(data, len); } if (final) { if (Update.end(true)) { Serial.printf("[web] OTA complete: %u bytes\n", index + len); } else { Serial.println("[web] OTA finalize failed"); } } }); } void AppWebServer::handleGetStatus(AsyncWebServerRequest* request) { JsonDocument doc; doc["battery"] = _power->getBatteryPercent(); doc["charging"] = _power->isCharging(); doc["wifi_rssi"] = _wifi->getRSSI(); doc["ip"] = _wifi->getIP(); doc["uptime"] = millis() / 1000; doc["free_heap"] = ESP.getFreeHeap(); doc["free_psram"] = ESP.getFreePsram(); String response; serializeJson(doc, response); request->send(200, "application/json", response); } void AppWebServer::handleGetAlbums(AsyncWebServerRequest* request) { auto albums = _immich->fetchAlbums(); JsonDocument doc; JsonArray arr = doc.to(); for (auto& album : albums) { JsonObject obj = arr.add(); obj["id"] = album.id; obj["title"] = album.title; obj["assetCount"] = album.assetCount; } String response; serializeJson(doc, response); request->send(200, "application/json", response); } void AppWebServer::handlePostAlbumsSelect(AsyncWebServerRequest* request, uint8_t* data, size_t len) { String body = String((char*)data).substring(0, len); JsonDocument doc; if (deserializeJson(doc, body)) { request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); return; } String albumsJson; serializeJson(doc["album_ids"], albumsJson); _settings->saveField("albums_json", albumsJson.c_str()); request->send(200, "application/json", "{\"ok\":true}"); } void AppWebServer::handleGetSettings(AsyncWebServerRequest* request) { Settings s = _settings->get(); JsonDocument doc; doc["interval_min"] = s.interval_min; doc["cycle_mode"] = static_cast(s.cycle_mode); doc["img_quality"] = static_cast(s.img_quality); doc["meta_flags"] = s.meta_flags; doc["meta_pos"] = static_cast(s.meta_pos); doc["led_brightness"] = s.led_brightness; doc["immich_url"] = s.immich_url; doc["albums_json"] = s.albums_json; String response; serializeJson(doc, response); request->send(200, "application/json", response); } void AppWebServer::handlePostSettings(AsyncWebServerRequest* request, uint8_t* data, size_t len) { String body = String((char*)data).substring(0, len); JsonDocument doc; if (deserializeJson(doc, body)) { request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); return; } Settings s = _settings->get(); if (doc.containsKey("interval_min")) s.interval_min = doc["interval_min"]; if (doc.containsKey("cycle_mode")) s.cycle_mode = static_cast((uint8_t)doc["cycle_mode"]); if (doc.containsKey("img_quality")) s.img_quality = static_cast((uint8_t)doc["img_quality"]); if (doc.containsKey("meta_flags")) s.meta_flags = doc["meta_flags"]; if (doc.containsKey("meta_pos")) s.meta_pos = static_cast((uint8_t)doc["meta_pos"]); if (doc.containsKey("led_brightness")) s.led_brightness = doc["led_brightness"]; if (doc.containsKey("immich_url")) s.immich_url = doc["immich_url"].as(); if (doc.containsKey("immich_key")) s.immich_key = doc["immich_key"].as(); _settings->save(s); request->send(200, "application/json", "{\"ok\":true}"); } void AppWebServer::handlePostSetup(AsyncWebServerRequest* request, uint8_t* data, size_t len) { String body = String((char*)data).substring(0, len); JsonDocument doc; if (deserializeJson(doc, body)) { request->send(400, "application/json", "{\"error\":\"Invalid JSON\"}"); return; } Settings s = _settings->get(); s.wifi_ssid = doc["wifi_ssid"].as(); s.wifi_pass = doc["wifi_pass"].as(); s.immich_url = doc["immich_url"] | DEFAULT_IMMICH_URL; s.immich_key = doc["immich_key"].as(); _settings->save(s); request->send(200, "application/json", "{\"ok\":true,\"msg\":\"Rebooting...\"}"); delay(1000); ESP.restart(); } ``` - [ ] **Step 3: Create `data/setup.html` (AP captive portal)** ```html PaperColor Setup

PaperColor Setup

``` - [ ] **Step 4: Create `data/index.html` (main management UI shell)** ```html PaperColor
``` - [ ] **Step 5: Create `data/style.css`** ```css * { box-sizing: border-box; margin: 0; padding: 0; } body { font-family: -apple-system, BlinkMacSystemFont, sans-serif; background: #f8fafc; color: #1e293b; } nav { background: #1e293b; color: white; padding: 16px 24px; display: flex; align-items: center; gap: 24px; flex-wrap: wrap; } nav h1 { font-size: 18px; white-space: nowrap; } #nav-links { display: flex; gap: 12px; flex-wrap: wrap; } #nav-links a { color: #94a3b8; text-decoration: none; padding: 4px 8px; border-radius: 4px; font-size: 14px; } #nav-links a.active { color: white; background: #334155; } main { max-width: 800px; margin: 24px auto; padding: 0 16px; } .card { background: white; border-radius: 8px; padding: 20px; margin-bottom: 16px; box-shadow: 0 1px 3px rgba(0,0,0,0.1); } .card h2 { font-size: 16px; margin-bottom: 12px; color: #475569; } .stat { display: flex; justify-content: space-between; padding: 8px 0; border-bottom: 1px solid #f1f5f9; } .stat:last-child { border-bottom: none; } .field { margin-bottom: 16px; } .field label { display: block; font-weight: 600; margin-bottom: 4px; font-size: 14px; color: #64748b; } .field input, .field select { width: 100%; padding: 8px 12px; border: 1px solid #e2e8f0; border-radius: 6px; font-size: 14px; } .btn { padding: 8px 16px; border: none; border-radius: 6px; font-size: 14px; cursor: pointer; } .btn-primary { background: #2563eb; color: white; } .btn-primary:hover { background: #1d4ed8; } .btn-danger { background: #dc2626; color: white; } .btn-danger:hover { background: #b91c1c; } .btn-group { display: flex; gap: 8px; margin-top: 12px; } .checkbox-list { list-style: none; } .checkbox-list li { padding: 8px 0; border-bottom: 1px solid #f1f5f9; display: flex; align-items: center; gap: 8px; } .checkbox-list input[type="checkbox"] { width: 18px; height: 18px; } .preset-btns { display: flex; gap: 8px; flex-wrap: wrap; } .preset-btns button { padding: 6px 12px; border: 1px solid #e2e8f0; border-radius: 4px; background: white; cursor: pointer; } .preset-btns button.active { background: #2563eb; color: white; border-color: #2563eb; } .toast { position: fixed; bottom: 20px; right: 20px; background: #065f46; color: white; padding: 12px 20px; border-radius: 6px; display: none; } .toast.show { display: block; } ``` - [ ] **Step 6: Create `data/app.js`** ```javascript const API = ''; let currentPage = 'dashboard'; async function api(path, opts = {}) { const res = await fetch(API + path, { headers: { 'Content-Type': 'application/json' }, ...opts }); return res.json(); } function toast(msg) { let t = document.querySelector('.toast'); if (!t) { t = document.createElement('div'); t.className = 'toast'; document.body.appendChild(t); } t.textContent = msg; t.classList.add('show'); setTimeout(() => t.classList.remove('show'), 3000); } function navigate(page) { currentPage = page; document.querySelectorAll('#nav-links a').forEach(a => { a.classList.toggle('active', a.dataset.page === page); }); renderPage(page); } async function renderPage(page) { const el = document.getElementById('content'); switch (page) { case 'dashboard': return renderDashboard(el); case 'albums': return renderAlbums(el); case 'slideshow': return renderSlideshow(el); case 'display': return renderDisplay(el); case 'device': return renderDevice(el); case 'firmware': return renderFirmware(el); } } async function renderDashboard(el) { const status = await api('/api/status'); el.innerHTML = `

Status

Battery${status.battery}%${status.charging ? ' ⚡' : ''}
WiFi Signal${status.wifi_rssi} dBm
IP Address${status.ip}
Uptime${Math.floor(status.uptime/60)}m
Free RAM${Math.floor(status.free_heap/1024)}KB
Free PSRAM${Math.floor(status.free_psram/1024)}KB

Controls

`; } async function renderAlbums(el) { const [albums, settings] = await Promise.all([api('/api/albums'), api('/api/settings')]); const selected = JSON.parse(settings.albums_json || '[]'); el.innerHTML = `

Albums

    ${albums.map(a => `
  • ${a.title} (${a.assetCount})
  • `).join('')}
`; document.getElementById('saveAlbums').onclick = async () => { const ids = [...el.querySelectorAll('input[type=checkbox]:checked')].map(c => c.value); await api('/api/albums/select', { method: 'POST', body: JSON.stringify({ album_ids: ids }) }); toast('Albums saved'); }; } async function renderSlideshow(el) { const settings = await api('/api/settings'); const intervals = [1, 5, 15, 30, 60]; const modes = ['Random', 'Chronological', 'Reverse Chrono', 'Favorites Weighted']; el.innerHTML = `

Interval

${intervals.map(i => ``).join('')}

Cycling Mode

`; el.querySelectorAll('[data-interval]').forEach(btn => { btn.onclick = () => { el.querySelectorAll('[data-interval]').forEach(b => b.classList.remove('active')); btn.classList.add('active'); }; }); document.getElementById('saveSlideshow').onclick = async () => { const interval = parseInt(el.querySelector('[data-interval].active')?.dataset.interval || '5'); const mode = parseInt(document.getElementById('cycleMode').value); await api('/api/settings', { method: 'POST', body: JSON.stringify({ interval_min: interval, cycle_mode: mode }) }); toast('Slideshow settings saved'); }; } async function renderDisplay(el) { const settings = await api('/api/settings'); const flags = settings.meta_flags; el.innerHTML = `

Image Quality

Metadata Overlay

  • Date
  • Location
  • People
  • Album
  • Camera
`; document.getElementById('saveDisplay').onclick = async () => { let flags = 0; el.querySelectorAll('[data-flag]').forEach(cb => { if (cb.checked) flags |= parseInt(cb.dataset.flag); }); await api('/api/settings', { method: 'POST', body: JSON.stringify({ img_quality: parseInt(document.getElementById('imgQuality').value), meta_flags: flags, meta_pos: parseInt(document.getElementById('metaPos').value) })}); toast('Display settings saved'); }; } async function renderDevice(el) { el.innerHTML = `

Device

`; } async function renderFirmware(el) { el.innerHTML = `

Firmware Update

`; document.getElementById('uploadFw').onclick = async () => { const file = document.getElementById('fwFile').files[0]; if (!file) return toast('Select a file first'); const formData = new FormData(); formData.append('firmware', file); document.getElementById('fwStatus').textContent = 'Uploading...'; const res = await fetch('/api/firmware', { method: 'POST', body: formData }); const data = await res.json(); document.getElementById('fwStatus').textContent = data.msg || (data.ok ? 'Success!' : 'Failed'); }; } // Navigation document.querySelectorAll('#nav-links a').forEach(a => { a.addEventListener('click', (e) => { e.preventDefault(); navigate(a.dataset.page); }); }); // Initial render navigate('dashboard'); ``` - [ ] **Step 7: Build and upload filesystem** Run: `pio run -e m5stack-papercolor && pio run -e m5stack-papercolor -t uploadfs` Expected: BUILD SUCCESS, filesystem upload succeeds - [ ] **Step 8: Flash firmware and verify web UI** Run: `pio run -e m5stack-papercolor -t upload` Then connect to `PaperColor-Setup` WiFi and open `192.168.4.1` in a browser (AP mode), or navigate to `http://papercolor.local` (station mode). Expected: Setup page renders in AP mode. After provisioning, management UI renders with all pages. - [ ] **Step 9: Commit** ```bash git add src/web_server.h src/web_server.cpp data/ git commit -m "feat: web server with REST API, captive portal, and management UI" ``` --- ### Task 11: Main Integration (Slideshow State Machine) **Files:** - Modify: `src/main.cpp` — full rewrite with FreeRTOS tasks and slideshow logic **Interfaces:** - Consumes: All modules (SettingsManager, WiFiManager, PowerManager, ButtonHandler, ImmichClient, PhotoQueue, ImagePipeline, DisplayManager, AppWebServer) - Produces: Complete working firmware with: - Display task on Core 1 (timer-driven photo refresh) - Web server on Core 0 (always listening) - Battery monitor integrated into main loop - Button events dispatched to slideshow state machine - [ ] **Step 1: Rewrite `src/main.cpp` with full integration** ```cpp #include #include #include #include #include #include "config.h" #include "settings.h" #include "wifi_manager.h" #include "power_manager.h" #include "button_handler.h" #include "immich_client.h" #include "photo_queue.h" #include "image_pipeline.h" #include "display_manager.h" #include "web_server.h" // Global instances SettingsManager settingsManager; WiFiManager wifiManager; PowerManager powerManager; ButtonHandler buttonHandler; ImmichClient immichClient; PhotoQueue photoQueue; ImagePipeline imagePipeline; DisplayManager displayManager; AppWebServer webServer; // Shared state SemaphoreHandle_t stateMutex; volatile bool slideshowPlaying = true; volatile bool refreshRequested = false; volatile bool randomRequested = false; volatile unsigned long lastRefreshTime = 0; // Task handles TaskHandle_t displayTaskHandle = nullptr; void displayTask(void* param) { Serial.println("[display_task] Started on Core 1"); // Initial sync if (wifiManager.isConnected()) { Settings s = settingsManager.get(); immichClient.begin(s.immich_url, s.immich_key); photoQueue.begin(immichClient, settingsManager); if (photoQueue.sync()) { Serial.printf("[display_task] Queue ready: %d photos\n", photoQueue.size()); } else { displayManager.showMessage("No Photos", "Select albums in web UI"); } } while (true) { unsigned long now = millis(); Settings s = settingsManager.get(); unsigned long intervalMs = s.interval_min * 60000UL; bool shouldRefresh = false; bool wantsRandom = false; if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { if (refreshRequested) { shouldRefresh = true; refreshRequested = false; } else if (randomRequested) { shouldRefresh = true; wantsRandom = true; randomRequested = false; } else if (slideshowPlaying && (now - lastRefreshTime >= intervalMs)) { shouldRefresh = true; } xSemaphoreGive(stateMutex); } // Periodic re-sync if (photoQueue.needsResync() && wifiManager.isConnected()) { photoQueue.sync(); } if (shouldRefresh && photoQueue.size() > 0 && wifiManager.isConnected()) { // Get next photo ID String assetId; if (wantsRandom) { assetId = photoQueue.random(); } else { assetId = photoQueue.next(); } if (assetId.length() > 0) { Serial.printf("[display_task] Loading asset: %s\n", assetId.c_str()); // Fetch asset info for portrait detection and metadata AssetInfo info = immichClient.fetchAssetInfo(assetId); // Download photo uint8_t* jpegBuf = nullptr; size_t jpegSize = 0; bool downloaded = immichClient.downloadAsset( assetId, s.img_quality, &jpegBuf, &jpegSize); if (downloaded && jpegBuf != nullptr) { ProcessedImage img; if (info.isPortrait) { // Try to find a portrait pair String pairId = photoQueue.findPortraitPair( settingsManager.get().queue_cursor); if (pairId.length() > 0) { AssetInfo pairInfo = immichClient.fetchAssetInfo(pairId); if (pairInfo.isPortrait) { uint8_t* jpeg2Buf = nullptr; size_t jpeg2Size = 0; if (immichClient.downloadAsset(pairId, s.img_quality, &jpeg2Buf, &jpeg2Size)) { img = imagePipeline.processPortraitPair( jpegBuf, jpegSize, jpeg2Buf, jpeg2Size); free(jpeg2Buf); } else { img = imagePipeline.process(jpegBuf, jpegSize); } } else { img = imagePipeline.process(jpegBuf, jpegSize); } } else { img = imagePipeline.process(jpegBuf, jpegSize); } } else { img = imagePipeline.process(jpegBuf, jpegSize); } free(jpegBuf); if (img.valid) { displayManager.showImage(img); // Show metadata overlay if enabled if (s.meta_flags != 0) { displayManager.showMetadata(info, s.meta_flags, s.meta_pos); } imagePipeline.freeImage(img); } else { Serial.println("[display_task] Image processing failed"); } } if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { lastRefreshTime = millis(); xSemaphoreGive(stateMutex); } } } // Yield — check every second vTaskDelay(pdMS_TO_TICKS(1000)); } } void webActionCallback(const String& action) { if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(100)) == pdTRUE) { if (action == "next") { refreshRequested = true; } else if (action == "random") { randomRequested = true; } else if (action == "pause") { slideshowPlaying = false; } else if (action == "play") { slideshowPlaying = true; } xSemaphoreGive(stateMutex); } } void setup() { auto cfg = M5.config(); M5.begin(cfg); Serial.begin(115200); Serial.println("[main] Immich Frame v1.0 booting..."); // Create state mutex stateMutex = xSemaphoreCreateMutex(); // Initialize subsystems powerManager.begin(); settingsManager.begin(); wifiManager.begin(settingsManager); buttonHandler.begin(powerManager, settingsManager); displayManager.begin(powerManager); if (!wifiManager.isAPMode()) { // Station mode — set up Immich and web server Settings s = settingsManager.get(); immichClient.begin(s.immich_url, s.immich_key); photoQueue.begin(immichClient, settingsManager); } // Start web server (works in both AP and station modes) webServer.begin(settingsManager, immichClient, powerManager, wifiManager); webServer.setActionCallback(webActionCallback); if (wifiManager.isAPMode()) { displayManager.showMessage("Setup Required", "Connect to PaperColor-Setup WiFi"); } else { displayManager.showMessage("Immich Frame", "Loading photos..."); // Create display task on Core 1 xTaskCreatePinnedToCore(displayTask, "display", 32768, nullptr, 1, &displayTaskHandle, 1); } // Enable light sleep for power saving powerManager.enableLightSleep(); Serial.println("[main] Boot complete"); } void loop() { M5.update(); powerManager.updateBatteryLED(); // Handle button events ButtonEvent event = buttonHandler.poll(); switch (event) { case ButtonEvent::NextPhoto: if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { refreshRequested = true; xSemaphoreGive(stateMutex); } break; case ButtonEvent::RandomPhoto: if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { randomRequested = true; xSemaphoreGive(stateMutex); } break; case ButtonEvent::PlayPause: if (xSemaphoreTake(stateMutex, pdMS_TO_TICKS(50)) == pdTRUE) { slideshowPlaying = !slideshowPlaying; Serial.printf("[main] Slideshow: %s\n", slideshowPlaying ? "playing" : "paused"); xSemaphoreGive(stateMutex); } break; case ButtonEvent::DeepSleep: case ButtonEvent::FactoryReset: break; case ButtonEvent::None: break; default: { ButtonEvent unreachable = event; (void)unreachable; break; } } delay(10); } ``` - [ ] **Step 2: Build full firmware** Run: `pio run -e m5stack-papercolor` Expected: BUILD SUCCESS with all modules linked - [ ] **Step 3: Upload filesystem and firmware** Run: `pio run -e m5stack-papercolor -t uploadfs && pio run -e m5stack-papercolor -t upload` Expected: Both upload successfully - [ ] **Step 4: End-to-end test** 1. Device boots into AP mode (first time) 2. Connect to `PaperColor-Setup`, configure WiFi + Immich API key 3. Device reboots, connects to WiFi 4. Navigate to `http://papercolor.local` — dashboard loads 5. Select albums in Albums page 6. Device begins displaying photos from slideshow 7. Press buttons: BTN_UP advances photo, BTN_TOP shows random, BTN_DOWN pauses Run: `pio device monitor` Expected: Serial shows full lifecycle: boot → WiFi → Immich connect → queue sync → photo fetch → decode → dither → display - [ ] **Step 5: Commit** ```bash git add src/main.cpp git commit -m "feat: full integration with FreeRTOS tasks and slideshow state machine" ``` --- ### Task 12: Testing & Polish **Files:** - Create: `test/README.md` - Modify: Various files for bug fixes discovered during testing **Interfaces:** - Consumes: Full firmware - Produces: Working, tested firmware ready for daily use - [ ] **Step 1: Create test documentation** Create `test/README.md`: ```markdown # Testing ## Native Tests (host machine) Run algorithm tests that don't require hardware: ```bash pio test -e native ``` Tests: - `test_photo_queue.cpp` — queue cycling, favorites weighting - `test_image_pipeline.cpp` — nearest color, dithering correctness ## On-Device Testing Flash and monitor: ```bash pio run -e m5stack-papercolor -t upload && pio device monitor ``` ### Verification Checklist - [ ] Device boots and prints version to serial - [ ] AP mode activates on first boot - [ ] Captive portal serves setup page at 192.168.4.1 - [ ] WiFi credentials save and device reboots to station mode - [ ] mDNS resolves at papercolor.local - [ ] Web UI dashboard shows battery/WiFi/uptime - [ ] Albums page lists Immich albums - [ ] Album selection persists across reboots - [ ] Photos download and display correctly - [ ] Portrait photos pair side-by-side - [ ] Slideshow advances at configured interval - [ ] BTN_UP: next photo (LED flashes white) - [ ] BTN_TOP: random photo (LED flashes white) - [ ] BTN_DOWN: pause/play toggle - [ ] BTN_TOP+BTN_DOWN 3s: deep sleep - [ ] BTN_UP 5s: factory reset - [ ] Battery LED: orange at 25%, red at 10% - [ ] OTA upload succeeds via web UI - [ ] Device recovers from WiFi disconnect (reconnect on next cycle) ``` - [ ] **Step 2: Run native tests** Run: `pio test -e native` Expected: All tests PASS - [ ] **Step 3: Run full device verification checklist** Flash firmware and LittleFS, then work through the checklist above item by item. Fix any issues discovered. - [ ] **Step 4: Commit any fixes** ```bash git add -A git commit -m "fix: testing corrections and polish" ``` - [ ] **Step 5: Tag release** ```bash git tag v1.0.0 git log --oneline -10 ``` Expected: Clean commit history showing incremental feature additions. --- ## Summary | Task | What It Builds | Key Files | |------|---------------|-----------| | 1 | Project scaffolding | `platformio.ini`, `config.h`, `main.cpp` | | 2 | Settings persistence | `settings.h/cpp` | | 3 | WiFi (station + AP) | `wifi_manager.h/cpp` | | 4 | Power/battery/sleep | `power_manager.h/cpp` | | 5 | Button input | `button_handler.h/cpp` | | 6 | Immich API client | `immich_client.h/cpp` | | 7 | Photo queue/cycling | `photo_queue.h/cpp` | | 8 | Image decode+dither | `image_pipeline.h/cpp` | | 9 | E-ink display output | `display_manager.h/cpp` | | 10 | Web UI + REST API | `web_server.h/cpp`, `data/*` | | 11 | Full integration | `main.cpp` (FreeRTOS tasks) | | 12 | Test & polish | `test/README.md`, bug fixes |