mirror of
https://github.com/cottongin/cardputerzero-sync.git
synced 2026-11-20 14:44:47 -06:00
docs: add photo sync design spec and implementation plan
This commit is contained in:
722
docs/superpowers/plans/2026-11-18-cardputerzero-photo-sync.md
Normal file
722
docs/superpowers/plans/2026-11-18-cardputerzero-photo-sync.md
Normal file
@@ -0,0 +1,722 @@
|
||||
# CardputerZero Photo Sync Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build an automatic one-way photo syncing system from an M5Stack CardputerZero (CM0 ARM64 Linux) over Tailscale to a macOS receiver, complete with offline queueing, a `cp0-mac-sync` Mac CLI/LaunchAgent, and a `shadcn/ui`-styled 320×170 LVGL v9 companion app for `APPLaunch`.
|
||||
|
||||
**Architecture:** A zero-dependency Python 3 `systemd --user` background daemon (`cp0-syncd`) watches user-configured directories via `inotify` + startup walk, queues photos in SQLite (WAL mode), probes the Mac receiver (`cp0-mac-receiver`) over Tailscale, and uploads atomically with SHA-256 verification. On the CardputerZero, a C++17/LVGL v9 companion UI (`cp0-sync-ui`) communicates with `cp0-syncd` over a Unix domain socket (with read-only SQLite fallback) and ships alongside `cp0-syncd` in a local `arm64` `.deb` package.
|
||||
|
||||
**Tech Stack:** Python 3.9+ standard library (`sqlite3`, `http.server`, `urllib`, `ctypes` `libc` `inotify`), C++17, LVGL v9, FreeType (`Inter` + `JetBrains Mono`), SDL2 (macOS simulator), Linux `fbdev`/`evdev` (device), CMake + CPack / `dpkg-deb`.
|
||||
|
||||
**Spec:** [docs/superpowers/specs/2026-11-18-cardputerzero-photo-sync-design.md](../specs/2026-11-18-cardputerzero-photo-sync-design.md)
|
||||
|
||||
---
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **Target Device:** M5Stack CardputerZero (Raspberry Pi CM0, quad-core Cortex-A53, 512 MB RAM, ARM64 Debian Linux, 320×170 non-touch LCD, 46-key `cp0-keys` matrix keyboard, `APPLaunch.service` running under `systemd --user`).
|
||||
- **Python Dependencies:** Zero third-party `pip` packages (`python3 (>= 3.9)` stdlib only) for both `cp0-syncd` and `cp0-mac-receiver`.
|
||||
- **Default Paths:** CardputerZero first-run watched folder is `/home/pi/Pictures/DCIM/Camera` (resolved via `~/Pictures/DCIM/Camera`, label `Camera`); Mac receiver default root is `~/Pictures/CardputerZero/<folder_label>/<rel_path>`.
|
||||
- **Network & Security:** Tailscale strictly private tailnet; no pairing `auth_token` or `X-CP0-Token`. Mac receiver listens on `0.0.0.0:8765` (configurable) and enforces Tailscale CIDR allowlist (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`) + loopback (`127.0.0.1`, `::1`).
|
||||
- **Sync Semantics:** Global one-way "upload once" keyed by `(folder_id, rel_path, sha256)`; never re-uploads if moved/deleted on Mac; never overwrites a different photo on filename collision (`<stem>_<YYYYMMDD_HHMMSS>_<sha8><ext>`).
|
||||
- **UI Theme:** `shadcn/ui` Zinc Dark (`#09090b` bg, `#18181b` muted, `#27272a` 1px border, `#fafafa` fg) and balanced Zinc Light (`#fafafa` bg, `#ffffff` card, `#f4f4f5` muted, `#e4e4e7` 1px border, `#71717a` focus ring, `#09090b` fg), toggleable live via `[T]` and persisted in `config.json`.
|
||||
- **Packaging:** Personal local build `cp0-sync_1.0.0_m5stack1_arm64.deb` (no `czdev` / `CardputerZeroRepository` AppStore submission constraints).
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **Percent-encoding of spaces/UTF-8 paths in HTTP headers:** `X-CP0-Folder-Label` and `X-CP0-Rel-Path` can contain spaces or non-ASCII characters (e.g., `Trip Photos/café.jpg`); both `Uploader` and `cp0-mac-receiver` must percent-encode/decode header values (`urllib.parse.quote`/`unquote`) rather than failing HTTP header Latin-1 encoding. *(Pinned in Task 1 & Task 4)*
|
||||
2. **Zero-byte media files from crashed camera writes:** A `0`-byte `.jpg` written before a crash must be skipped with a `WARN` log by `SyncStore`/`Watcher` rather than uploaded as a corrupt empty file. *(Pinned in Task 3)*
|
||||
3. **Queued `PENDING` file deleted on CardputerZero before Mac wakes:** `Uploader` must catch `FileNotFoundError` when opening a `PENDING` file, delete the stale row from `state.db`, log a warning, and proceed with the remaining queue. *(Pinned in Task 4)*
|
||||
4. **Stale Unix socket file after unclean shutdown:** `IpcServer` must unlink any pre-existing stale socket path before `bind()` so `cp0-syncd` restarts cleanly after power loss. *(Pinned in Task 4)*
|
||||
5. **Corrupt or truncated `config.json` on disk:** Both `MacConfig` and `DaemonConfig` must save atomically (`.tmp` + `os.replace`) and fall back safely to defaults on malformed JSON rather than crashing on startup. *(Pinned in Task 1 & Task 3)*
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Mac Receiver Config, Tailscale IP Filter, & Atomic HTTP Server
|
||||
|
||||
**Files:**
|
||||
- Create: `mac-receiver/cp0_mac_receiver/__init__.py`
|
||||
- Create: `mac-receiver/cp0_mac_receiver/config.py`
|
||||
- Create: `mac-receiver/cp0_mac_receiver/server.py`
|
||||
- Test: `mac-receiver/tests/test_server.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `MacConfig(dest_root: str, listen_host: str, listen_port: int, notify_on_batch: bool, tailscale_only: bool)`
|
||||
- `load_mac_config(path: Path | None = None) -> MacConfig`
|
||||
- `save_mac_config(cfg: MacConfig, path: Path | None = None) -> None`
|
||||
- `is_allowed_client_ip(ip_str: str, tailscale_only: bool) -> bool`
|
||||
- `MacReceiverServer(config_path: Path, notifier: Callable[[int, Path], None] | None = None)` with `.start()`, `.shutdown()`, `.server_address`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for config, IP allowlist, `/health`, `/upload`, dedup, collision rename, path traversal, and percent-encoded headers**
|
||||
|
||||
```python
|
||||
# mac-receiver/tests/test_server.py
|
||||
import hashlib, json, os, tempfile, threading, unittest
|
||||
from pathlib import Path
|
||||
from urllib.error import HTTPError
|
||||
from urllib.parse import quote
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
from cp0_mac_receiver.config import MacConfig, load_mac_config, save_mac_config
|
||||
from cp0_mac_receiver.server import MacReceiverServer, is_allowed_client_ip
|
||||
|
||||
class TestMacConfigAndFilter(unittest.TestCase):
|
||||
def test_corrupt_config_falls_back_to_defaults(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
cfg_path = Path(tmp) / "config.json"
|
||||
cfg_path.write_text("{broken json")
|
||||
cfg = load_mac_config(cfg_path)
|
||||
self.assertEqual(cfg.listen_port, 8765)
|
||||
self.assertTrue(cfg.tailscale_only)
|
||||
|
||||
def test_tailscale_ip_filter(self):
|
||||
self.assertTrue(is_allowed_client_ip("100.101.102.103", True))
|
||||
self.assertTrue(is_allowed_client_ip("fd7a:115c:a1e0::1", True))
|
||||
self.assertTrue(is_allowed_client_ip("127.0.0.1", True))
|
||||
self.assertFalse(is_allowed_client_ip("192.168.1.50", True))
|
||||
self.assertTrue(is_allowed_client_ip("192.168.1.50", False))
|
||||
|
||||
class TestMacReceiverServer(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dest_root = Path(self.tmp.name) / "CardputerZero"
|
||||
self.cfg_path = Path(self.tmp.name) / "config.json"
|
||||
save_mac_config(MacConfig(dest_root=str(self.dest_root), listen_host="127.0.0.1", listen_port=0, notify_on_batch=True, tailscale_only=True), self.cfg_path)
|
||||
self.notified = []
|
||||
self.srv = MacReceiverServer(self.cfg_path, notifier=lambda n, d: self.notified.append((n, d)), debounce_sec=0.05)
|
||||
self.thread = threading.Thread(target=self.srv.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.base_url = f"http://127.0.0.1:{self.srv.server_port}"
|
||||
|
||||
def tearDown(self):
|
||||
self.srv.shutdown()
|
||||
self.srv.server_close()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _upload(self, label: str, rel_path: str, data: bytes, mtime: float = 1700000000.0):
|
||||
sha = hashlib.sha256(data).hexdigest()
|
||||
req = Request(f"{self.base_url}/upload", data=data, method="POST", headers={
|
||||
"Content-Length": str(len(data)),
|
||||
"X-CP0-Folder-Label": quote(label, safe=""),
|
||||
"X-CP0-Rel-Path": quote(rel_path, safe="/"),
|
||||
"X-CP0-SHA256": sha,
|
||||
"X-CP0-Mtime": str(mtime),
|
||||
})
|
||||
return urlopen(req)
|
||||
|
||||
def test_health_returns_dest_root(self):
|
||||
with urlopen(f"{self.base_url}/health") as resp:
|
||||
self.assertEqual(resp.status, 200)
|
||||
body = json.loads(resp.read())
|
||||
self.assertEqual(body["status"], "ok")
|
||||
self.assertEqual(body["dest_root"], str(self.dest_root))
|
||||
|
||||
def test_upload_dedup_collision_and_utf8_headers(self):
|
||||
payload = b"jpeg-bytes-1"
|
||||
r1 = json.loads(self._upload("Trip Photos", "2026/café 1.jpg", payload, 1700000123.0).read())
|
||||
self.assertEqual(r1["status"], "saved")
|
||||
target = self.dest_root / "Trip Photos" / "2026" / "café 1.jpg"
|
||||
self.assertEqual(target.read_bytes(), payload)
|
||||
self.assertEqual(int(target.stat().st_mtime), 1700000123)
|
||||
|
||||
r2 = json.loads(self._upload("Trip Photos", "2026/café 1.jpg", payload, 1700000123.0).read())
|
||||
self.assertEqual(r2["status"], "already_exists")
|
||||
|
||||
r3 = json.loads(self._upload("Trip Photos", "2026/café 1.jpg", b"jpeg-bytes-different", 1700000456.0).read())
|
||||
self.assertEqual(r3["status"], "saved")
|
||||
self.assertNotEqual(Path(r3["saved_path"]), target)
|
||||
self.assertEqual(target.read_bytes(), payload)
|
||||
|
||||
def test_rejects_path_traversal_and_bad_sha256(self):
|
||||
with self.assertRaises(HTTPError) as ctx:
|
||||
self._upload("../etc", "passwd", b"bad")
|
||||
self.assertEqual(ctx.exception.code, 400)
|
||||
|
||||
req = Request(f"{self.base_url}/upload", data=b"abc", method="POST", headers={
|
||||
"Content-Length": "3", "X-CP0-Folder-Label": "Camera",
|
||||
"X-CP0-Rel-Path": "a.jpg", "X-CP0-SHA256": "0" * 64, "X-CP0-Mtime": "1700000000",
|
||||
})
|
||||
with self.assertRaises(HTTPError) as ctx2:
|
||||
urlopen(req)
|
||||
self.assertEqual(ctx2.exception.code, 400)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v`
|
||||
Expected: FAIL (`ModuleNotFoundError: No module named 'cp0_mac_receiver'`).
|
||||
|
||||
- [ ] **Step 3: Implement `config.py` and `server.py` in `mac-receiver/cp0_mac_receiver/`**
|
||||
|
||||
- `config.py`: `MacConfig` dataclass with default `dest_root = str(Path.home() / "Pictures" / "CardputerZero")`, `listen_host = "0.0.0.0"`, `listen_port = 8765`, `notify_on_batch = True`, `tailscale_only = True`; atomic `.tmp` + `os.replace` writes; safe fallback on missing/corrupt JSON; default path `~/.config/cp0-mac-sync/config.json` (overrideable via `CP0_MAC_CONFIG`).
|
||||
- `server.py`: `is_allowed_client_ip()` using Python `ipaddress` against `100.64.0.0/10`, `fd7a:115c:a1e0::/48`, `127.0.0.0/8`, `::1/128` (handling IPv4-mapped IPv6 `::ffff:`); `MacReceiverServer(ThreadingHTTPServer)` reloading config per request so `set-dir` takes effect immediately, streaming `64 KB` chunks to `.cp0-<uuid>.part`, validating `X-CP0-SHA256` and path segments (no `..` / absolute paths after `unquote`), deduplicating identical SHA-256, renaming collisions to `<stem>_<YYYYMMDD_HHMMSS>_<sha8><ext>`, applying `os.utime`, and triggering a debounced `osascript` batch notification.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v`
|
||||
Expected: PASS (all tests in `test_server.py` OK).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add .gitignore mac-receiver/cp0_mac_receiver/__init__.py mac-receiver/cp0_mac_receiver/config.py mac-receiver/cp0_mac_receiver/server.py mac-receiver/tests/test_server.py
|
||||
git commit -m "feat(mac-receiver): add config store, Tailscale IP filter, and atomic HTTP receiver"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Mac CLI (`cp0-mac-sync`) & `LaunchAgent` Service Installer
|
||||
|
||||
**Files:**
|
||||
- Create: `mac-receiver/cp0_mac_receiver/cli.py`
|
||||
- Create: `mac-receiver/bin/cp0-mac-receiver`
|
||||
- Create: `mac-receiver/bin/cp0-mac-sync`
|
||||
- Test: `mac-receiver/tests/test_cli.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `MacConfig`, `load_mac_config`, `save_mac_config` from `cp0_mac_receiver.config`
|
||||
- Produces:
|
||||
- `LAUNCH_AGENT_LABEL = "com.cardputerzero.photosync"`
|
||||
- `render_launch_agent_plist(receiver_bin: Path, log_path: Path, config_path: Path) -> str`
|
||||
- `run_cli(argv: list[str], config_path: Path | None = None, plist_path: Path | None = None, runner: Callable[..., CompletedProcess] | None = None) -> int`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for `cp0-mac-sync` subcommands (`set-dir`, `set-port`, `install-service`, `uninstall-service`, `status`, `logs`)**
|
||||
|
||||
```python
|
||||
# mac-receiver/tests/test_cli.py
|
||||
import tempfile, unittest
|
||||
from pathlib import Path
|
||||
from subprocess import CompletedProcess
|
||||
|
||||
from cp0_mac_receiver.cli import run_cli
|
||||
from cp0_mac_receiver.config import load_mac_config
|
||||
|
||||
class TestMacCli(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.cfg_path = Path(self.tmp.name) / "config.json"
|
||||
self.plist_path = Path(self.tmp.name) / "com.cardputerzero.photosync.plist"
|
||||
self.calls = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _fake_runner(self, cmd, **kwargs):
|
||||
self.calls.append(cmd)
|
||||
return CompletedProcess(cmd, 0, stdout="", stderr="")
|
||||
|
||||
def test_set_dir_creates_directory_and_updates_config(self):
|
||||
new_dir = Path(self.tmp.name) / "Photos" / "CP0"
|
||||
rc = run_cli(["set-dir", str(new_dir)], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner)
|
||||
self.assertEqual(rc, 0)
|
||||
self.assertTrue(new_dir.is_dir())
|
||||
self.assertEqual(load_mac_config(self.cfg_path).dest_root, str(new_dir.resolve()))
|
||||
|
||||
def test_install_and_uninstall_service_writes_and_removes_plist(self):
|
||||
rc = run_cli(["install-service"], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner)
|
||||
self.assertEqual(rc, 0)
|
||||
plist_text = self.plist_path.read_text()
|
||||
self.assertIn("com.cardputerzero.photosync", plist_text)
|
||||
self.assertIn("KeepAlive", plist_text)
|
||||
self.assertTrue(any("launchctl" in c[0] for c in self.calls))
|
||||
|
||||
rc2 = run_cli(["uninstall-service"], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner)
|
||||
self.assertEqual(rc2, 0)
|
||||
self.assertFalse(self.plist_path.exists())
|
||||
|
||||
def test_set_port_validates_range_and_restarts_service(self):
|
||||
run_cli(["install-service"], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner)
|
||||
self.calls.clear()
|
||||
self.assertNotEqual(run_cli(["set-port", "99999"], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner), 0)
|
||||
rc = run_cli(["set-port", "9100"], config_path=self.cfg_path, plist_path=self.plist_path, runner=self._fake_runner)
|
||||
self.assertEqual(rc, 0)
|
||||
self.assertEqual(load_mac_config(self.cfg_path).listen_port, 9100)
|
||||
self.assertTrue(any("kickstart" in " ".join(c) or "bootstrap" in " ".join(c) for c in self.calls))
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v`
|
||||
Expected: FAIL (`ModuleNotFoundError: No module named 'cp0_mac_receiver.cli'`).
|
||||
|
||||
- [ ] **Step 3: Implement `cli.py` and executable entrypoints `bin/cp0-mac-receiver` and `bin/cp0-mac-sync`**
|
||||
|
||||
- `cli.py`: `argparse` subcommands `install-service`, `uninstall-service`, `set-dir <path>`, `set-port <port>`, `status`, `logs [--follow]`; generates plist via `plistlib.dumps`, manages `launchctl bootout`/`bootstrap`/`kickstart` for `gui/<uid>`, queries `tailscale status --json` (optional best-effort) and local `/health` in `status`.
|
||||
- `bin/cp0-mac-receiver`: `#!/usr/bin/env python3` entrypoint setting up rotating logger (`~/Library/Logs/cp0-mac-sync.log`, 5 MB × 2 backups) and running `MacReceiverServer`.
|
||||
- `bin/cp0-mac-sync`: `#!/usr/bin/env python3` entrypoint invoking `run_cli(sys.argv[1:])`, `chmod +x` both scripts.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v`
|
||||
Expected: PASS (`test_server.py` and `test_cli.py` all OK).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add mac-receiver/cp0_mac_receiver/cli.py mac-receiver/bin/cp0-mac-receiver mac-receiver/bin/cp0-mac-sync mac-receiver/tests/test_cli.py
|
||||
git commit -m "feat(mac-receiver): add cp0-mac-sync CLI and LaunchAgent management"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: On-Device Config & SQLite Queue Store (`device-daemon/`)
|
||||
|
||||
**Files:**
|
||||
- Create: `device-daemon/cp0_syncd/__init__.py`
|
||||
- Create: `device-daemon/cp0_syncd/config.py`
|
||||
- Create: `device-daemon/cp0_syncd/store.py`
|
||||
- Test: `device-daemon/tests/test_store_and_config.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `WatchedFolder(id: str, path: str, label: str, enabled: bool)`
|
||||
- `DaemonConfig(target_host: str, target_port: int, theme: str, watched_folders: list[WatchedFolder], extensions: list[str], ignore_prefixes: list[str], ignore_suffixes: list[str])`
|
||||
- `load_daemon_config(path: Path | None = None, home_dir: Path | None = None) -> DaemonConfig`
|
||||
- `save_daemon_config(cfg: DaemonConfig, path: Path | None = None) -> None`
|
||||
- `add_watched_folder(cfg: DaemonConfig, folder_path: str, label: str | None = None) -> tuple[DaemonConfig, WatchedFolder]` (raises `ValueError` on overlap or non-existent dir; auto-disambiguates duplicate labels)
|
||||
- `remove_watched_folder(cfg: DaemonConfig, folder_id: str) -> DaemonConfig`
|
||||
- `SyncStore(db_path: Path)` with WAL mode, `enqueue_file(folder_id, abs_path, rel_path, size_bytes, mtime_ns, sha256) -> str`, `next_pending() -> dict | None`, `mark_uploading(file_id)`, `mark_synced(file_id)`, `mark_retry(file_id, err)`, `mark_failed(file_id, err)`, `delete_file(file_id)`, `purge_pending_for_folder(folder_id)`, `retry_all_failed() -> int`, `log(level, message)` (pruned to 500 rows), `set_meta(key, value)`, `get_status_snapshot(watched_folders) -> dict`, `is_media_file(path: Path, cfg: DaemonConfig) -> bool`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for default config, label disambiguation, overlap rejection, zero-byte skipping, WAL SQLite dedup, and 500-row log cap**
|
||||
|
||||
```python
|
||||
# device-daemon/tests/test_store_and_config.py
|
||||
import tempfile, unittest
|
||||
from pathlib import Path
|
||||
|
||||
from cp0_syncd.config import (
|
||||
DaemonConfig, add_watched_folder, load_daemon_config, save_daemon_config,
|
||||
)
|
||||
from cp0_syncd.store import SyncStore, compute_sha256, is_media_file
|
||||
|
||||
class TestDaemonConfig(unittest.TestCase):
|
||||
def test_first_run_creates_default_camera_folder_and_theme(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
home = Path(tmp) / "home"
|
||||
cfg_path = Path(tmp) / "config.json"
|
||||
cfg = load_daemon_config(cfg_path, home_dir=home)
|
||||
self.assertEqual(cfg.theme, "dark")
|
||||
self.assertEqual(cfg.target_port, 8765)
|
||||
self.assertEqual(len(cfg.watched_folders), 1)
|
||||
self.assertEqual(cfg.watched_folders[0].path, str(home / "Pictures" / "DCIM" / "Camera"))
|
||||
self.assertEqual(cfg.watched_folders[0].label, "Camera")
|
||||
|
||||
def test_corrupt_config_falls_back_to_defaults(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
cfg_path = Path(tmp) / "config.json"
|
||||
cfg_path.write_text("{corrupt")
|
||||
cfg = load_daemon_config(cfg_path, home_dir=Path(tmp))
|
||||
self.assertEqual(cfg.theme, "dark")
|
||||
|
||||
def test_overlap_rejected_and_duplicate_labels_disambiguated(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
d1 = Path(tmp) / "a" / "Camera"
|
||||
d2 = Path(tmp) / "b" / "Camera"
|
||||
child = d1 / "sub"
|
||||
d1.mkdir(parents=True); d2.mkdir(parents=True); child.mkdir()
|
||||
cfg = DaemonConfig(watched_folders=[])
|
||||
cfg, wf1 = add_watched_folder(cfg, str(d1))
|
||||
cfg, wf2 = add_watched_folder(cfg, str(d2))
|
||||
self.assertEqual(wf1.label, "Camera")
|
||||
self.assertEqual(wf2.label, "Camera-2")
|
||||
with self.assertRaises(ValueError):
|
||||
add_watched_folder(cfg, str(child))
|
||||
|
||||
class TestSyncStore(unittest.TestCase):
|
||||
def test_dedup_zero_byte_skip_and_log_cap(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db = SyncStore(Path(tmp) / "state.db")
|
||||
img = Path(tmp) / "IMG_0001.jpg"
|
||||
img.write_bytes(b"photo-bytes")
|
||||
zero = Path(tmp) / "empty.jpg"
|
||||
zero.write_bytes(b"")
|
||||
cfg = DaemonConfig()
|
||||
|
||||
self.assertTrue(is_media_file(img, cfg))
|
||||
self.assertFalse(is_media_file(zero, cfg))
|
||||
|
||||
sha = compute_sha256(img)
|
||||
self.assertEqual(db.enqueue_file("cam", str(img), "IMG_0001.jpg", 11, 1, sha), "enqueued")
|
||||
row = db.next_pending()
|
||||
db.mark_synced(row["id"])
|
||||
self.assertEqual(db.enqueue_file("cam", str(img), "IMG_0001.jpg", 11, 1, sha), "already_synced")
|
||||
|
||||
for i in range(550):
|
||||
db.log("INFO", f"msg {i}")
|
||||
self.assertEqual(db.log_count(), 500)
|
||||
db.close()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `PYTHONPATH=device-daemon python3 -m unittest discover -s device-daemon/tests -v`
|
||||
Expected: FAIL (`ModuleNotFoundError: No module named 'cp0_syncd'`).
|
||||
|
||||
- [ ] **Step 3: Implement `config.py` and `store.py` in `device-daemon/cp0_syncd/`**
|
||||
|
||||
- `config.py`: `WatchedFolder` + `DaemonConfig` dataclasses, atomic save (`.tmp` + `os.replace`), safe fallback on corrupt JSON, first-run default `<home>/Pictures/DCIM/Camera` (label `"Camera"`), canonical `realpath` ancestor/descendant overlap detection in `add_watched_folder`, and `Camera-2` label disambiguation.
|
||||
- `store.py`: `sqlite3` connection in `WAL` mode (`busy_timeout=5000`), `files` table with unique index on `(folder_id, rel_path, sha256)`, `logs` table pruned to newest 500 rows, `meta` key-value table, and `is_media_file()` filtering extensions, hidden/temp prefixes/suffixes, and `0`-byte files.
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `PYTHONPATH=device-daemon python3 -m unittest discover -s device-daemon/tests -v`
|
||||
Expected: PASS (`test_store_and_config.py` OK).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add device-daemon/cp0_syncd/__init__.py device-daemon/cp0_syncd/config.py device-daemon/cp0_syncd/store.py device-daemon/tests/test_store_and_config.py
|
||||
git commit -m "feat(device-daemon): add daemon config, overlap guard, and SQLite WAL queue store"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: On-Device `inotify` Watcher, Wake-Probe Uploader, & Unix Socket IPC
|
||||
|
||||
**Files:**
|
||||
- Create: `device-daemon/cp0_syncd/watcher.py`
|
||||
- Create: `device-daemon/cp0_syncd/uploader.py`
|
||||
- Create: `device-daemon/cp0_syncd/ipc.py`
|
||||
- Create: `device-daemon/cp0_syncd/daemon.py`
|
||||
- Create: `device-daemon/bin/cp0-syncd`
|
||||
- Test: `device-daemon/tests/test_daemon_integration.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `DaemonConfig`, `load_daemon_config`, `save_daemon_config`, `add_watched_folder`, `remove_watched_folder`, `SyncStore`, `compute_sha256`, `is_media_file` from Task 3
|
||||
- Produces:
|
||||
- `scan_folder(folder: WatchedFolder, cfg: DaemonConfig, store: SyncStore) -> int`
|
||||
- `InotifyWatcher(on_file_ready: Callable[[WatchedFolder, Path], None], on_dir_created: Callable[[WatchedFolder, Path], None] | None = None)` (uses `ctypes` `libc.so.6` on Linux with graceful `PollingWatcher` fallback for non-Linux/macOS unit tests; tracks nearest existing ancestor for unmounted paths)
|
||||
- `Uploader(store: SyncStore, get_config: Callable[[], DaemonConfig], wake_event: threading.Event, probe_timeout: float = 3.0)` with `process_once() -> str` (backoff sequence `[5, 10, 20, 40, 60]`)
|
||||
- `IpcServer(sock_path: Path, handler: Callable[[dict], dict])` (unlinks stale socket before bind, newline-delimited JSON)
|
||||
- `SyncDaemon` orchestrating watcher + uploader + IPC server for `bin/cp0-syncd`
|
||||
|
||||
- [ ] **Step 1: Write failing integration tests covering sleep/wake retry, missing-file purge, 403/500 vs 413 handling, stale socket cleanup, IPC mutations, and end-to-end upload to `MacReceiverServer`**
|
||||
|
||||
```python
|
||||
# device-daemon/tests/test_daemon_integration.py
|
||||
import hashlib, json, socket, tempfile, threading, unittest
|
||||
from pathlib import Path
|
||||
|
||||
from cp0_syncd.config import DaemonConfig, WatchedFolder, save_daemon_config
|
||||
from cp0_syncd.daemon import SyncDaemon
|
||||
from cp0_syncd.ipc import IpcServer
|
||||
from cp0_syncd.store import SyncStore
|
||||
|
||||
class TestDaemonIntegration(unittest.TestCase):
|
||||
def test_stale_socket_unlinked_and_ipc_commands_work(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
sock_path = Path(tmp) / "cp0-sync.sock"
|
||||
sock_path.write_text("stale")
|
||||
srv = IpcServer(sock_path, lambda req: {"ok": True, "cmd": req["cmd"]})
|
||||
srv.start()
|
||||
try:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as c:
|
||||
c.connect(str(sock_path))
|
||||
c.sendall(b'{"cmd":"status"}\n')
|
||||
resp = json.loads(c.makefile().readline())
|
||||
self.assertEqual(resp, {"ok": True, "cmd": "status"})
|
||||
finally:
|
||||
srv.stop()
|
||||
|
||||
def test_sleep_wake_end_to_end_and_missing_file_cleanup(self):
|
||||
import sys
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "mac-receiver"))
|
||||
from cp0_mac_receiver.config import MacConfig, save_mac_config
|
||||
from cp0_mac_receiver.server import MacReceiverServer
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
cam_dir = Path(tmp) / "Camera"
|
||||
dest_root = Path(tmp) / "MacDest"
|
||||
cam_dir.mkdir()
|
||||
p1 = cam_dir / "IMG_1.jpg"
|
||||
p2 = cam_dir / "IMG_2.jpg"
|
||||
p1.write_bytes(b"real-photo-1")
|
||||
p2.write_bytes(b"real-photo-2")
|
||||
|
||||
# Allocate ephemeral unused port, then close before starting MacReceiverServer later
|
||||
probe_sock = socket.socket()
|
||||
probe_sock.bind(("127.0.0.1", 0))
|
||||
port = probe_sock.getsockname()[1]
|
||||
probe_sock.close()
|
||||
|
||||
cfg_path = Path(tmp) / "daemon.json"
|
||||
db_path = Path(tmp) / "state.db"
|
||||
sock_path = Path(tmp) / "cp0.sock"
|
||||
save_daemon_config(DaemonConfig(
|
||||
target_host="127.0.0.1",
|
||||
target_port=port,
|
||||
watched_folders=[WatchedFolder(id="cam", path=str(cam_dir), label="Camera")],
|
||||
), cfg_path)
|
||||
|
||||
daemon = SyncDaemon(cfg_path, db_path, sock_path)
|
||||
daemon.scan_all()
|
||||
daemon.uploader.process_once()
|
||||
self.assertEqual(daemon.store.get_meta("mac_online"), "false")
|
||||
|
||||
# Delete IMG_2.jpg while Mac is sleeping, then wake Mac receiver
|
||||
p2.unlink()
|
||||
mac_cfg = Path(tmp) / "mac.json"
|
||||
save_mac_config(MacConfig(dest_root=str(dest_root), listen_host="127.0.0.1", listen_port=port, notify_on_batch=False, tailscale_only=True), mac_cfg)
|
||||
srv = MacReceiverServer(mac_cfg)
|
||||
t = threading.Thread(target=srv.serve_forever, daemon=True)
|
||||
t.start()
|
||||
try:
|
||||
daemon.uploader.process_once()
|
||||
self.assertEqual(daemon.store.get_meta("mac_online"), "true")
|
||||
self.assertEqual((dest_root / "Camera" / "IMG_1.jpg").read_bytes(), b"real-photo-1")
|
||||
self.assertFalse((dest_root / "Camera" / "IMG_2.jpg").exists())
|
||||
self.assertIsNone(daemon.store.next_pending())
|
||||
finally:
|
||||
srv.shutdown()
|
||||
srv.server_close()
|
||||
daemon.store.close()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `PYTHONPATH=device-daemon:mac-receiver python3 -m unittest discover -s device-daemon/tests -v`
|
||||
Expected: FAIL (`ModuleNotFoundError: No module named 'cp0_syncd.watcher'`).
|
||||
|
||||
- [ ] **Step 3: Implement `watcher.py`, `uploader.py`, `ipc.py`, `daemon.py`, and `bin/cp0-syncd`**
|
||||
|
||||
- `watcher.py`: Recursive `scan_folder()` (skips if path missing); `InotifyWatcher` via `ctypes` `libc.so.6` (`inotify_init1`, `inotify_add_watch`, `inotify_rm_watch`, `IN_CLOSE_WRITE | IN_MOVED_TO | IN_CREATE | IN_DELETE_SELF`) attaching watches to subdirectories dynamically and tracking nearest existing ancestor when a configured SD card mount is missing, plus a clean `PollingWatcher` fallback when not on Linux.
|
||||
- `uploader.py`: `GET /health` probe (`3s` timeout) caching `mac_dest_root` in `meta`; FIFO `POST /upload` streaming via `http.client` / `urllib` with percent-encoded `X-CP0-Folder-Label` & `X-CP0-Rel-Path`; catches `FileNotFoundError` to remove vanished files cleanly; keeps `PENDING` + pauses queue on network/`5xx`/`401`/`403`; marks `FAILED` only on `400`/`413`/`422`; adaptive backoff `[5, 10, 20, 40, 60]` interrupted immediately by `wake_event`.
|
||||
- `ipc.py` & `daemon.py`: Unix stream server (unlinks existing socket before `bind()`), implements `status`, `sync_now`, `add_folder`, `remove_folder`, `set_target`, `set_theme`, `retry_failed`.
|
||||
- `bin/cp0-syncd`: `#!/usr/bin/env python3` entrypoint handling `SIGTERM`/`SIGINT` cleanly.
|
||||
|
||||
- [ ] **Step 4: Run all Python tests across both `mac-receiver` and `device-daemon`**
|
||||
|
||||
Run: `PYTHONPATH=device-daemon:mac-receiver python3 -m unittest discover -s device-daemon/tests -v && PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v`
|
||||
Expected: PASS (all daemon and Mac receiver tests OK).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add device-daemon/cp0_syncd/watcher.py device-daemon/cp0_syncd/uploader.py device-daemon/cp0_syncd/ipc.py device-daemon/cp0_syncd/daemon.py device-daemon/bin/cp0-syncd device-daemon/tests/test_daemon_integration.py
|
||||
git commit -m "feat(device-daemon): add inotify watcher, adaptive wake uploader, and IPC daemon"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Companion UI IPC Client, Tailscale Peer Discovery, & `shadcn/ui` Zinc Theme Tokens
|
||||
|
||||
**Files:**
|
||||
- Create: `device-ui/CMakeLists.txt`
|
||||
- Create: `device-ui/src/ipc_client.hpp`
|
||||
- Create: `device-ui/src/ipc_client.cpp`
|
||||
- Create: `device-ui/src/theme.hpp`
|
||||
- Create: `device-ui/src/theme.cpp`
|
||||
- Test: `device-ui/tests/test_ipc_and_theme.cpp`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Unix socket JSON protocol from Task 4 (`status`, `sync_now`, `add_folder`, `remove_folder`, `set_target`, `set_theme`, `retry_failed`) and SQLite WAL fallback `~/.local/share/cp0-sync/state.db`
|
||||
- Produces:
|
||||
- `struct DaemonStatus`, `struct FolderStatus`, `struct LogEntry`, `struct TailscalePeer`
|
||||
- `class IpcClient` (`fetch_status()`, `send_command(json)`, `read_sqlite_fallback()`, `parse_tailscale_peers(json_str)`, `list_tailscale_peers()`)
|
||||
- `enum class ThemeMode { Dark, Light }`, `struct ShadcnPalette` (exact hex tokens from Spec Section 5.4), `get_palette(ThemeMode)`, `toggle_theme(ThemeMode)`
|
||||
|
||||
- [ ] **Step 1: Write failing C++ tests for IPC JSON parsing, SQLite fallback reading, Tailscale peer sorting (macOS/Darwin first, self excluded), and Zinc Dark/Light palette tokens**
|
||||
|
||||
```cpp
|
||||
// device-ui/tests/test_ipc_and_theme.cpp
|
||||
#include "ipc_client.hpp"
|
||||
#include "theme.hpp"
|
||||
#include <cassert>
|
||||
|
||||
int main() {
|
||||
auto dark = cp0::get_palette(cp0::ThemeMode::Dark);
|
||||
auto light = cp0::get_palette(cp0::ThemeMode::Light);
|
||||
assert(dark.background == 0x09090b && dark.border == 0x27272a && dark.foreground == 0xfafafa);
|
||||
assert(light.background == 0xfafafa && light.card == 0xffffff && light.border == 0xe4e4e7 && light.ring == 0x71717a);
|
||||
assert(cp0::toggle_theme(cp0::ThemeMode::Dark) == cp0::ThemeMode::Light);
|
||||
|
||||
const char* ts_json = R"({"Self":{"ID":"self1"},"Peer":{"p1":{"HostName":"linux-box","DNSName":"linux.ts.net.","OS":"linux","Online":true,"TailscaleIPs":["100.64.0.2"]},"p2":{"HostName":"macbook","DNSName":"macbook.ts.net.","OS":"macOS","Online":true,"TailscaleIPs":["100.64.0.3"]}}})";
|
||||
auto peers = cp0::parse_tailscale_peers(ts_json);
|
||||
assert(peers.size() == 2);
|
||||
assert(peers[0].hostname == "macbook" && peers[0].os == "macOS");
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
*(Include tests for `IpcClient` round-tripping against a real temporary `AF_UNIX` server socket and reading fallback rows from a temporary `sqlite3` DB.)*
|
||||
|
||||
- [ ] **Step 2: Configure CMake and run tests to verify failure**
|
||||
|
||||
Run: `cmake -S device-ui -B device-ui/build -DCP0_UI_BUILD_TESTS=ON -DCP0_UI_BUILD_APP=OFF && cmake --build device-ui/build`
|
||||
Expected: FAIL (source files do not exist yet).
|
||||
|
||||
- [ ] **Step 3: Implement `CMakeLists.txt`, `theme.hpp/.cpp`, and `ipc_client.hpp/.cpp`**
|
||||
|
||||
- `CMakeLists.txt`: C++17 project `cp0-sync-ui`; options `CP0_UI_BUILD_TESTS` (default `ON`) and `CP0_UI_BUILD_APP` (default `ON`); links `SQLite::SQLite3` for core library (`cp0_ui_core`); when `CP0_UI_BUILD_APP=ON`, links LVGL v9 (`FetchContent` or local `LVGL_DIR`), FreeType2, and SDL2 (macOS/sim) or Linux `fbdev`/`evdev` (`CP0_UI_TARGET_DEVICE=ON`).
|
||||
- `theme.hpp/.cpp`: Exact `ShadcnPalette` color tables for Zinc Dark and balanced Zinc Light matching Spec Section 5.4.
|
||||
- `ipc_client.hpp/.cpp`: Non-blocking `AF_UNIX` client (`1s` timeout), minimal dependency-free JSON parser/serializer (or single-header `nlohmann/json.hpp` vendored or small parser—keep zero external network/pip requirements), `sqlite3` read-only fallback (`SQLITE_OPEN_READONLY`), and `tailscale status --json` peer extraction prioritizing `macOS`/`darwin` online peers at top.
|
||||
|
||||
- [ ] **Step 4: Build and run `ctest` to verify tests pass**
|
||||
|
||||
Run: `cmake -S device-ui -B device-ui/build -DCP0_UI_BUILD_TESTS=ON -DCP0_UI_BUILD_APP=OFF && cmake --build device-ui/build && ctest --test-dir device-ui/build --output-on-failure`
|
||||
Expected: PASS (`test_ipc_and_theme` passes).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add device-ui/CMakeLists.txt device-ui/src/ipc_client.hpp device-ui/src/ipc_client.cpp device-ui/src/theme.hpp device-ui/src/theme.cpp device-ui/tests/test_ipc_and_theme.cpp
|
||||
git commit -m "feat(device-ui): add C++ IPC client, Tailscale peer parser, and shadcn Zinc themes"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Companion UI 320×170 LVGL Views, Modals, & Keyboard Controller
|
||||
|
||||
**Files:**
|
||||
- Create: `device-ui/src/ui_controller.hpp`
|
||||
- Create: `device-ui/src/ui_controller.cpp`
|
||||
- Create: `device-ui/src/app_ui.hpp`
|
||||
- Create: `device-ui/src/app_ui.cpp`
|
||||
- Create: `device-ui/src/main.cpp`
|
||||
- Test: `device-ui/tests/test_ui_navigation.cpp`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `IpcClient`, `DaemonStatus`, `ThemeMode`, `ShadcnPalette`, `get_palette`, `toggle_theme` from Task 5
|
||||
- Produces:
|
||||
- `enum class ActiveTab { Status = 0, Folders = 1, Logs = 2 }`
|
||||
- `enum class ModalKind { None, TargetPicker, DirPicker, ConfirmRemove, TextInput }`
|
||||
- `class UiController`: pure state/keyboard state machine handling `1`/`2`/`3`, `Enter`, `T`, `C`, `A`, `D`/`Delete`, `R`, `j`/`k`/arrow keys, `Space`, `Backspace`, `Esc`/`Q`, directory traversal, and emitting `IpcAction` requests
|
||||
- `class LvglAppView` (`app_ui.hpp/.cpp`): 320×170 LVGL v9 Bento card + segmented `TabsList` renderer with live `apply_theme(ThemeMode)` switching and FreeType font loading (`Inter` + `JetBrains Mono`, falling back to LVGL built-in Montserrat/Unscii if `.ttf` absent)
|
||||
- `cp0-sync-ui` executable (`main.cpp`) supporting `--sim` / macOS SDL2 window (`320×170`) and Linux `fbdev` + `evdev` (`cp0-keys`)
|
||||
|
||||
- [ ] **Step 1: Write failing C++ tests for keyboard state transitions (`1/2/3`, `[T]` theme toggle outside vs inside modal, `[A]` directory drill-down & `Space` watch, `[D]` confirm Y/N, `[C]` peer selection, `[R]` retry failed, `Esc`/`Q`)**
|
||||
|
||||
```cpp
|
||||
// device-ui/tests/test_ui_navigation.cpp
|
||||
#include "ui_controller.hpp"
|
||||
#include <cassert>
|
||||
#include <filesystem>
|
||||
|
||||
int main() {
|
||||
cp0::UiController ctrl;
|
||||
assert(ctrl.state().tab == cp0::ActiveTab::Status);
|
||||
assert(ctrl.state().theme == cp0::ThemeMode::Dark);
|
||||
|
||||
// 'T' toggles theme on main tab, emits SetTheme action
|
||||
auto action = ctrl.handle_key('t', cp0::KeyCode::Char);
|
||||
assert(ctrl.state().theme == cp0::ThemeMode::Light);
|
||||
assert(action.type == cp0::ActionType::SetTheme);
|
||||
|
||||
// '2' jumps to Folders, 'A' opens DirPicker, 'T' inside modal opens TextInput instead of toggling theme
|
||||
ctrl.handle_key('2', cp0::KeyCode::Char);
|
||||
assert(ctrl.state().tab == cp0::ActiveTab::Folders);
|
||||
ctrl.handle_key('a', cp0::KeyCode::Char);
|
||||
assert(ctrl.state().modal == cp0::ModalKind::DirPicker);
|
||||
auto modal_t = ctrl.handle_key('t', cp0::KeyCode::Char);
|
||||
assert(ctrl.state().modal == cp0::ModalKind::TextInput);
|
||||
assert(ctrl.state().theme == cp0::ThemeMode::Light); // unchanged inside modal
|
||||
(void)modal_t;
|
||||
|
||||
// 'Esc' closes modal, 'Esc' again requests Quit
|
||||
ctrl.handle_key(0, cp0::KeyCode::Escape);
|
||||
assert(ctrl.state().modal == cp0::ModalKind::None);
|
||||
auto quit_act = ctrl.handle_key(0, cp0::KeyCode::Escape);
|
||||
assert(quit_act.type == cp0::ActionType::Quit);
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
*(Also test `[D]` -> `[Y]` folder removal confirmation and `DirPicker` `Enter` / `Backspace` / `Space` directory selection against a `std::filesystem::temp_directory_path()` folder tree.)*
|
||||
|
||||
- [ ] **Step 2: Build and run `ctest` to verify failure**
|
||||
|
||||
Run: `cmake --build device-ui/build && ctest --test-dir device-ui/build --output-on-failure`
|
||||
Expected: FAIL (`ui_controller.hpp` missing).
|
||||
|
||||
- [ ] **Step 3: Implement `ui_controller.hpp/.cpp`, `app_ui.hpp/.cpp`, `main.cpp`, and verify SDL2 simulator build**
|
||||
|
||||
- Implement `UiController` cleanly separated from LVGL objects so every keyboard shortcut and modal flow is unit-tested in `ctest`.
|
||||
- Implement `LvglAppView` creating the 320×170 widget hierarchy matching Spec Section 5.4 (`Top bar 24px`, `Main content 126px`, `Footer 20px` with `<kbd>` keycaps), wiring `lv_style_t` styles for Zinc Dark and Zinc Light so `[T]` swaps styles cleanly without recreating the screen, and loading FreeType fonts (`Inter` + `JetBrains Mono`) from `/usr/share/cp0-sync/fonts/`, `/usr/share/fonts/`, or `assets/fonts/` when present, with graceful LVGL built-in Montserrat/Unscii fallback when TTFs are absent.
|
||||
- Wire `main.cpp` using `lv_sdl_window_create(320, 170)` + `lv_sdl_keyboard_create()` on macOS/desktop simulator and `lv_linux_fbdev_create()` + `lv_evdev_create()` on CardputerZero Linux, polling `IpcClient::fetch_status()` every `1000 ms` via `lv_timer_create`.
|
||||
|
||||
- [ ] **Step 4: Build both `ctest` unit tests and full SDL2 app target (`CP0_UI_BUILD_APP=ON`)**
|
||||
|
||||
Run: `cmake -S device-ui -B device-ui/build -DCP0_UI_BUILD_TESTS=ON -DCP0_UI_BUILD_APP=ON && cmake --build device-ui/build && ctest --test-dir device-ui/build --output-on-failure`
|
||||
Expected: PASS (`test_ipc_and_theme` and `test_ui_navigation` pass; `cp0-sync-ui` binary links cleanly).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add device-ui/CMakeLists.txt device-ui/src/ui_controller.hpp device-ui/src/ui_controller.cpp device-ui/src/app_ui.hpp device-ui/src/app_ui.cpp device-ui/src/main.cpp device-ui/tests/test_ui_navigation.cpp
|
||||
git commit -m "feat(device-ui): add 320x170 LVGL shadcn Bento UI, modals, and keyboard controller"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Debian (`arm64` `.deb`) Packaging, `APPLaunch` Integration, & `systemd --user` Unit
|
||||
|
||||
**Files:**
|
||||
- Create: `packaging/cp0-syncd.service`
|
||||
- Create: `packaging/cp0-sync.desktop`
|
||||
- Create: `packaging/cp0-sync.png` (generated cleanly via Python `struct`/`zlib` PNG writer or SVG→PNG so zero extra assets needed)
|
||||
- Create: `packaging/debian/control`
|
||||
- Create: `packaging/debian/postinst`
|
||||
- Create: `packaging/debian/prerm`
|
||||
- Create: `packaging/build-deb.sh`
|
||||
- Create: `packaging/Dockerfile.arm64`
|
||||
- Test: `packaging/tests/test_packaging.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `device-daemon/bin/cp0-syncd`, `device-daemon/cp0_syncd/`, `device-ui/` CMake build
|
||||
- Produces: `cp0-sync_1.0.0_m5stack1_arm64.deb` containing `/usr/bin/cp0-sync-ui`, `/usr/bin/cp0-syncd`, `/usr/lib/cp0-sync/cp0_syncd/...`, `/usr/lib/systemd/user/cp0-syncd.service`, `/usr/share/APPLaunch/applications/cp0-sync.desktop`, `/usr/share/APPLaunch/share/images/cp0-sync.png`
|
||||
|
||||
- [ ] **Step 1: Write failing validation tests verifying packaging metadata, systemd user unit paths, desktop entry fields, maintainer script permissions/hooks, and `cp0-sync.png` validity**
|
||||
|
||||
```python
|
||||
# packaging/tests/test_packaging.py
|
||||
import struct, unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
class TestPackagingAssets(unittest.TestCase):
|
||||
def test_systemd_user_unit_and_desktop_entry(self):
|
||||
unit = (ROOT / "packaging" / "cp0-syncd.service").read_text()
|
||||
self.assertIn("ExecStart=/usr/bin/cp0-syncd", unit)
|
||||
self.assertIn("WantedBy=default.target", unit)
|
||||
|
||||
desktop = (ROOT / "packaging" / "cp0-sync.desktop").read_text()
|
||||
self.assertIn("Exec=/usr/bin/cp0-sync-ui", desktop)
|
||||
self.assertIn("Icon=/usr/share/APPLaunch/share/images/cp0-sync.png", desktop)
|
||||
|
||||
def test_debian_control_and_maintainer_scripts(self):
|
||||
ctrl = (ROOT / "packaging" / "debian" / "control").read_text()
|
||||
self.assertIn("Package: cp0-sync", ctrl)
|
||||
self.assertIn("Architecture: arm64", ctrl)
|
||||
self.assertIn("python3 (>= 3.9)", ctrl)
|
||||
|
||||
postinst = (ROOT / "packaging" / "debian" / "postinst").read_text()
|
||||
self.assertIn("systemctl --global enable cp0-syncd.service", postinst)
|
||||
|
||||
png = (ROOT / "packaging" / "cp0-sync.png").read_bytes()
|
||||
self.assertEqual(png[:8], b"\x89PNG\r\n\x1a\n")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `python3 -m unittest discover -s packaging/tests -v`
|
||||
Expected: FAIL (`FileNotFoundError`).
|
||||
|
||||
- [ ] **Step 3: Create packaging files, icon PNG, maintainer scripts, and ARM64 Docker `.deb` build script**
|
||||
|
||||
- `packaging/cp0-syncd.service`: `systemd --user` service (`Restart=on-failure`, `RestartSec=5`, `ExecStart=/usr/bin/cp0-syncd`, `WantedBy=default.target`).
|
||||
- `packaging/cp0-sync.desktop`: `Type=Application`, `Name=Photo Sync`, `Exec=/usr/bin/cp0-sync-ui`, `Icon=/usr/share/APPLaunch/share/images/cp0-sync.png`, `Terminal=false`.
|
||||
- `packaging/cp0-sync.png`: crisp 64×64 zinc/white camera-sync icon PNG.
|
||||
- `packaging/debian/control`, `postinst` (`chmod 0755`, runs `systemctl --global enable cp0-syncd.service` and starts active user units safely with `|| true`), `prerm` (`chmod 0755`, stops/disables user service on remove).
|
||||
- `packaging/Dockerfile.arm64` & `packaging/build-deb.sh`: builds `cp0-sync_1.0.0_m5stack1_arm64.deb` in `--platform linux/arm64` Debian Bookworm container (and also supports running directly on CardputerZero over SSH).
|
||||
|
||||
- [ ] **Step 4: Run packaging tests (and build `.deb` if Docker `linux/arm64` daemon is available)**
|
||||
|
||||
Run: `python3 -m unittest discover -s packaging/tests -v`
|
||||
Expected: PASS (`test_packaging.py` OK).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add packaging/
|
||||
git commit -m "feat(packaging): add systemd user unit, APPLaunch entry, and arm64 deb builder"
|
||||
```
|
||||
@@ -0,0 +1,290 @@
|
||||
# CardputerZero Photo Sync — Design Specification
|
||||
|
||||
**Date:** 2026-11-18
|
||||
**Project Size:** Project (3 coordinated deliverables: on-device background sync daemon, on-device LVGL companion UI packaged as an `arm64` `.deb`, and macOS receiver LaunchAgent + CLI)
|
||||
|
||||
---
|
||||
|
||||
## 1. Intent & Why
|
||||
|
||||
The M5Stack CardputerZero (powered by the Raspberry Pi CM0, running ARM64 Debian Linux — *not* the ESP32-based Cardputer/ADV) includes an 8MP Sony IMX219 camera and built-in Camera app, plus support for third-party camera tools and SD card storage. Getting photos off the device currently requires manual copying or plugging in cables, and fails whenever the user's Mac is asleep or away from home Wi-Fi.
|
||||
|
||||
This project provides automatic, reliable, one-way photo syncing from the CardputerZero to the user's Mac over Tailscale:
|
||||
|
||||
- Photos taken with the native Camera app (or any other camera/screenshot app saving into a watched directory) are detected automatically in the background.
|
||||
- When the Mac is asleep or offline, photos remain safely queued on the CardputerZero and upload automatically as soon as the Mac becomes reachable again.
|
||||
- An `APPLaunch` companion app gives the user an on-device frontend on the 320×170 screen to configure watched folders, pick the target Mac, trigger a manual sync, and view queue status and logs.
|
||||
|
||||
---
|
||||
|
||||
## 2. Goals, Non-Goals, and Anti-Goals
|
||||
|
||||
### Goals
|
||||
- **App-agnostic background watching:** Watch one or more user-defined directories on the CardputerZero so photos from the built-in Camera app or any other app are captured automatically without opening the sync UI.
|
||||
- **Resilient offline queueing:** Persist queued uploads across CardputerZero reboots and Mac sleep/wake cycles; resume cleanly with zero manual intervention once the Mac wakes.
|
||||
- **Tailscale transport:** Sync transparently over the user's existing Tailscale network (both on LAN and remotely).
|
||||
- **One-way "upload once" guarantees:** Track uploaded files locally so moving, renaming, or deleting photos on the Mac after arrival never causes re-uploads.
|
||||
- **Native `APPLaunch` companion UI:** Provide a keyboard-first 320×170 LVGL UI (`1:Status`, `2:Folders`, `3:Logs`) integrated into the CardputerZero launcher.
|
||||
- **Configurable Mac destination:** Store received photos on the Mac in an easily changeable root folder, defaulting to `~/Pictures/CardputerZero/<watched-folder-name>/`.
|
||||
|
||||
### Non-Goals
|
||||
- Building a custom camera capture UI or replacing the native Camera app.
|
||||
- Two-way sync or deleting photos on the CardputerZero when deleted on the Mac.
|
||||
- Cloud relay servers, external object storage, or multi-Mac sync tracking **(User Decision)** — sync targets a single configured Mac; if multi-Mac sync is ever needed later, it is out of scope for v1.
|
||||
- Publishing or submitting packages to the official `CardputerZeroRepository` AppStore via `CardputerZero-AppBuilder` (`czdev`) **(User Decision)**.
|
||||
|
||||
### Anti-Goals
|
||||
- Uploading partial/in-progress writes while the camera is still flushing an image to disk.
|
||||
- Silently overwriting different photos that share a filename (e.g., `IMG_0001.jpg` after a counter reset).
|
||||
- Requiring heavy typing of Linux file paths or Tailscale MagicDNS hostnames on the 46-key matrix keyboard (`fn`/`sym` layers) when interactive pickers can handle selection.
|
||||
|
||||
---
|
||||
|
||||
## 3. Constraints
|
||||
|
||||
- **Device Hardware (CardputerZero):**
|
||||
- Broadcom BCM2837 / RP3A0 (Raspberry Pi CM0, quad-core ARM Cortex-A53 @ 1 GHz), **512 MB LPDDR2 RAM**.
|
||||
- **1.9-inch 320×170 LCD**, **no touchscreen**.
|
||||
- **46-key physical QWERTY matrix keyboard** (`cp0-keys`) where `fn`, `shift`, and `sym` modify keys via hold, tap-latch, or double-tap lock (e.g., arrows are `fn + F/X/Z/C`, Delete is `fn + Backspace`).
|
||||
- **Device OS & Packaging:**
|
||||
- ARM64 Debian Linux running **`APPLaunch.service`** as a `systemd --user` service.
|
||||
- Apps register a `.desktop` file under `/usr/share/APPLaunch/applications/` and icon assets under `/usr/share/APPLaunch/share/images/`.
|
||||
- Distributed as a local `arm64` Debian package (`cp0-sync_1.0.0_m5stack1_arm64.deb`) built via CMake/CPack (in an ARM64 Docker container or directly on the CardputerZero over SSH) for personal installation via `sudo apt install ./cp0-sync_1.0.0_m5stack1_arm64.deb` **(User Decision)**.
|
||||
- **Network & Mac Environment:**
|
||||
- Tailscale is already installed and running on both the CardputerZero and the Mac on a **strictly private tailnet of authorized devices** **(User Decision)**.
|
||||
- Mac receiver runs on macOS as a user background agent (`LaunchAgent`).
|
||||
|
||||
---
|
||||
|
||||
## 4. Approaches Considered & Selected
|
||||
|
||||
Three architectural approaches were evaluated:
|
||||
|
||||
1. **Approach 1 (Selected by User): Official `CardputerZero/Template` C++ (LVGL UI) + Zero-Dependency Python 3 Sync Daemon & Mac Receiver**
|
||||
- *How it works:* The `APPLaunch` frontend (`cp0-sync-ui`) is built in C++ / LVGL using `CardputerZero/Template` (CMake + CPack `.deb`, with local macOS SDL2 simulator support). Both the on-device background `systemd` service (`cp0-syncd`) and the Mac receiver (`cp0-mac-receiver` + `cp0-mac-sync` CLI) are written in pure Python 3 standard library (`sqlite3`, `http.server`/`urllib`, `ctypes` Linux `inotify`).
|
||||
- *Tradeoffs:* Native LVGL performance and full `CardputerZero/Template` / `APPLaunch` compatibility without AppStore/CI submission constraints, while keeping networking, hashing, queueing, and filesystem watching in zero-dependency Python (~12 MB RSS) that shares protocol logic cleanly between device and Mac and is easy to inspect/debug on-device.
|
||||
|
||||
2. **Approach 2: All-C++ Device Package (LVGL UI + C++ Daemon) + Python Mac Receiver**
|
||||
- *How it works:* Both `cp0-sync-ui` and `cp0-syncd` are compiled C++ binaries using `libcurl`, `libsqlite3`, and `<sys/inotify.h>`.
|
||||
- *Tradeoffs:* Lowest device RAM usage (~3 MB RSS) and single-language device codebase, but requires `libcurl`/`sqlite3` ARM64 cross-compile sysroot dependencies and recompiling the `.deb` for any daemon change.
|
||||
|
||||
3. **Approach 3: All-Python Stack (Python Framebuffer/SDL UI + Python Daemon + Python Mac Receiver)**
|
||||
- *How it works:* Everything, including the 320×170 UI, is written in Python (`pygame`/`pillow`).
|
||||
- *Tradeoffs:* Avoids C++ cross-compilation, but diverges from the official `CardputerZero/Template` LVGL UX and adds heavy Python graphics dependencies.
|
||||
|
||||
---
|
||||
|
||||
## 5. Detailed Architecture & Component Design
|
||||
|
||||
> **Attribution key:** Items marked **(User Decision)** were explicitly requested or confirmed by the user. Items marked **(Our Call)** are design decisions filled in during specification with rationale provided.
|
||||
|
||||
### 5.1 Repository Layout **(Our Call)**
|
||||
Monorepo under `cardputerzero-sync/`:
|
||||
- `device-daemon/` — Python 3 `cp0-syncd` service, SQLite queue store, `inotify` watcher, HTTP/Tailscale uploader, Unix socket control server, and unit tests.
|
||||
- `device-ui/` — `CardputerZero/Template`-based C++/LVGL companion app (`cp0-sync-ui`), SDL desktop simulator preset, `APPLaunch` `.desktop` + icon assets, and CMake/CPack `.deb` packaging config.
|
||||
- `mac-receiver/` — Python 3 `cp0-mac-receiver` HTTP server, `cp0-mac-sync` CLI helper, `LaunchAgent` plist installer, and unit tests.
|
||||
- `packaging/` — Debian maintainer scripts (`postinst`, `prerm`, `postrm`), `systemd` user unit file (`cp0-syncd.service`), and shared protocol schema docs.
|
||||
|
||||
---
|
||||
|
||||
### 5.2 On-Device Background Sync Daemon (`cp0-syncd`)
|
||||
|
||||
#### Service Lifecycle & Shared Storage **(Our Call)**
|
||||
- Installed at `/usr/bin/cp0-syncd` with a **`systemd --user` service** at `/usr/lib/systemd/user/cp0-syncd.service` (matching `APPLaunch.service`, which also runs under `systemd --user`).
|
||||
- Debian `postinst` runs `systemctl --global enable cp0-syncd.service` so the daemon starts automatically on boot for the device's desktop user under the exact same `$HOME` and UID as `cp0-sync-ui`.
|
||||
- **Shared state paths** (also overrideable via environment variables `CP0_SYNC_CONFIG`, `CP0_SYNC_DB`, `CP0_SYNC_SOCK` for testing):
|
||||
- Config file: `~/.config/cp0-sync/config.json`
|
||||
- SQLite database: `~/.local/share/cp0-sync/state.db` (WAL mode enabled so `cp0-sync-ui` can read live status concurrently without locking the daemon)
|
||||
- Control Unix domain socket: `${XDG_RUNTIME_DIR:-/tmp}/cp0-sync.sock`
|
||||
|
||||
#### Configuration Schema (`~/.config/cp0-sync/config.json`) **(Our Call + User Decisions)**
|
||||
- `target_host`: Tailscale MagicDNS hostname or `100.x.y.z` IP of the Mac (initially empty until configured in UI or CLI; daemon stays idle and logs `No target host configured` without error loops).
|
||||
- `target_port`: Integer, default `8765`.
|
||||
- `theme`: `"dark"` or `"light"` (default `"dark"`), controlling the UI's Zinc Dark vs. Zinc Light palette **(User Decision)**.
|
||||
- **No `auth_token` required **(User Decision)**:** Because the tailnet contains strictly private, authorized devices, transport security relies on Tailscale WireGuard encryption plus the Mac receiver's Tailscale CIDR allowlist (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`, and loopback).
|
||||
- `watched_folders`: List of objects `{ "id": "<uuid-or-slug>", "path": "/abs/path", "label": "Camera", "enabled": true }`.
|
||||
- **Label uniqueness:** Derived initially from folder basename (`Camera`, `DCIM`, `Screenshots`), automatically disambiguated if two watched paths share a basename (e.g., `DCIM`, `DCIM-2`) so Mac subfolders never collide.
|
||||
- **First-run default **(User Decision)**:** On first boot when no config exists, `cp0-syncd` pre-populates a single default watched folder: `~/Pictures/DCIM/Camera` (i.e., `/home/pi/Pictures/DCIM/Camera`, label `Camera`), matching the native CardputerZero Camera app. Additional folders (such as SD card paths or other app output directories) can be added anytime via Tab `2:Folders`.
|
||||
- **Overlap prevention:** `add_folder` resolves canonical paths (`os.path.realpath`) and warns/blocks (or skips duplicate ingestion) if a newly added folder overlaps an already-watched ancestor/descendant directory.
|
||||
- `extensions`: Case-insensitive media allowlist, defaulting to `[".jpg", ".jpeg", ".png", ".webp", ".gif", ".bmp", ".dng", ".raw", ".mp4", ".mov"]`.
|
||||
- `ignore_prefixes`: `[".", "~"]`, `ignore_suffixes`: `[".part", ".tmp", ".crdownload"]`.
|
||||
|
||||
#### File Watching & Queue Ingestion **(Our Call)**
|
||||
- **Startup reconciliation:** On daemon startup (and whenever a watched folder is added), recursively walk each enabled watched folder (`os.walk` with `followlinks=False`). Any matching file not already recorded as `SYNCED` (same path, size, and `mtime_ns` — or matching SHA-256) is enqueued as `PENDING`.
|
||||
- **Real-time `inotify` watcher & mount resilience:** Pure-Python `ctypes` bindings against `libc.so.6` (`inotify_init1`, `inotify_add_watch`, `inotify_rm_watch`) watching each directory and dynamically attaching watches to newly created subdirectories (`IN_CREATE | IN_ISDIR`).
|
||||
- If a configured watched folder does not currently exist (e.g., an unmounted SD card path), the daemon keeps it in config (showing `! unmounted/missing` in UI) and watches its nearest existing ancestor directory (or re-checks on a periodic timer) so inserting an SD card later automatically attaches watches and triggers a scan.
|
||||
- Only reacts to **`IN_CLOSE_WRITE`** and **`IN_MOVED_TO`** events (never `IN_CREATE` for files), guaranteeing the camera app has finished writing and closed the file descriptor before hashing or uploading.
|
||||
- Enqueues newly closed/moved media files immediately into SQLite (`status='PENDING'`) and wakes the uploader thread.
|
||||
|
||||
#### SQLite State Schema (`state.db`) **(Our Call)**
|
||||
- `files` table:
|
||||
- `id` INTEGER PRIMARY KEY
|
||||
- `folder_id` TEXT NOT NULL
|
||||
- `abs_path` TEXT NOT NULL
|
||||
- `rel_path` TEXT NOT NULL (relative to watched folder root, preserving any camera date subdirectories)
|
||||
- `size_bytes` INTEGER NOT NULL
|
||||
- `mtime_ns` INTEGER NOT NULL
|
||||
- `sha256` TEXT NOT NULL
|
||||
- `status` TEXT NOT NULL (`PENDING`, `UPLOADING`, `SYNCED`, `FAILED`)
|
||||
- `retry_count` INTEGER DEFAULT 0
|
||||
- `last_error` TEXT
|
||||
- `discovered_at` INTEGER NOT NULL (Unix epoch ms)
|
||||
- `synced_at` INTEGER
|
||||
- Unique index on `(folder_id, rel_path, sha256)` — ensures global one-way "upload once" behavior for the configured single-Mac target **(User Decision)** even if files are later moved or deleted on the Mac, while re-uploading if a file at the same path is genuinely modified/replaced with different content.
|
||||
- `logs` table:
|
||||
- `id` INTEGER PRIMARY KEY AUTOINCREMENT
|
||||
- `ts` INTEGER NOT NULL (Unix epoch ms)
|
||||
- `level` TEXT NOT NULL (`INFO`, `WARN`, `ERROR`)
|
||||
- `message` TEXT NOT NULL
|
||||
- Automatically pruned to the newest 500 rows to bound SD/eMMC wear and disk usage.
|
||||
- `meta` table:
|
||||
- Key-value runtime summary (`mac_online`, `mac_dest_root`, `last_probe_ts`, `next_retry_ts`, `last_sync_ts`, `last_synced_file`) written by the daemon so Tab `1:Status` renders instantly without expensive queries.
|
||||
|
||||
#### Wake Probe, Upload Loop & Backoff **(Our Call)**
|
||||
- When `PENDING` rows exist and `target_host` is configured:
|
||||
1. Daemon sends `GET http://<target_host>:<target_port>/health` with a short **3-second connect/read timeout**.
|
||||
2. If unreachable (Mac sleeping, offline, or Tailscale tunnel asleep): marks `mac_online = false` in `meta`, sleeps with adaptive backoff (**5s → 10s → 20s → 40s → 60s max**, resetting to 5s whenever a new photo is captured), and keeps everything queued safely in SQLite.
|
||||
3. If `200 OK`: marks `mac_online = true`, caches `mac_dest_root` from the health JSON (for display on Tab `1:Status`), and streams `PENDING` files sequentially in FIFO order (`discovered_at ASC`) via `POST /upload`.
|
||||
4. **Failure handling:** Network drop, `5xx`, or auth/access rejection (`401`/`403`) keeps the file as `PENDING` (pausing the upload queue with backoff and surfacing an error state in `meta`/`logs` rather than poisoning individual files). Only per-file payload rejections (`400`/`413`/`422`) mark that specific file `FAILED` with `last_error` so it doesn't block the rest of the queue; the user can retry failed items from Tab `3:Logs` (`[R]`).
|
||||
5. **Immediate wake on command:** `Sync Now` or config edits over the Unix socket interrupt the backoff sleep immediately (`threading.Event.set()`), resetting backoff and probing at once.
|
||||
|
||||
#### Local Control IPC Protocol (`/tmp/.../cp0-sync.sock`) **(Our Call)**
|
||||
Newline-delimited JSON request/response over Unix stream socket:
|
||||
- `{"cmd": "status"}` → returns current `meta` + queue counts + watched folder list with per-folder `pending`/`synced` counts.
|
||||
- `{"cmd": "sync_now"}` → interrupts sleep, resets retry backoff, triggers immediate folder rescan + `/health` probe + queue flush.
|
||||
- `{"cmd": "add_folder", "path": "...", "label": "..."}` → validates directory exists, saves config, attaches `inotify` watch, triggers rescan.
|
||||
- `{"cmd": "remove_folder", "folder_id": "..."}` → removes folder from `config.json`, detaches `inotify` watch, purges un-synced `PENDING` queue entries for that folder (never touches files on disk).
|
||||
- `{"cmd": "set_target", "host": "...", "port": 8765}` → saves `config.json` and immediately probes new target.
|
||||
- `{"cmd": "set_theme", "theme": "dark" | "light"}` → persists the active UI theme in `config.json`.
|
||||
- `{"cmd": "retry_failed"}` → flips `FAILED` rows back to `PENDING` and triggers `sync_now`.
|
||||
|
||||
---
|
||||
|
||||
### 5.3 Mac Receiver & CLI (`cp0-mac-receiver` & `cp0-mac-sync`)
|
||||
|
||||
#### User-Specified Requirements **(User Decision)**
|
||||
- Lightweight receiver daemon running on the Mac (`LaunchAgent`), communicating over Tailscale.
|
||||
- Stores synced photos under `~/Pictures/CardputerZero/` by default, grouped into subfolders by watched folder label (`~/Pictures/CardputerZero/<folder_label>/<rel_path>`).
|
||||
- Destination root path on the Mac must be easy to set and change.
|
||||
|
||||
#### Configuration & Service Management **(Our Call)**
|
||||
- **Config file:** `~/.config/cp0-mac-sync/config.json`:
|
||||
```json
|
||||
{
|
||||
"dest_root": "/Users/<user>/Pictures/CardputerZero",
|
||||
"listen_host": "0.0.0.0",
|
||||
"listen_port": 8765,
|
||||
"notify_on_batch": true,
|
||||
"tailscale_only": true
|
||||
}
|
||||
```
|
||||
- **Logging:** `~/Library/Logs/cp0-mac-sync.log` (rotated at 5 MB, keeping 2 backups).
|
||||
- **`LaunchAgent`:** `~/Library/LaunchAgents/com.cardputerzero.photosync.plist` with `RunAtLoad = true` and `KeepAlive = true`, managed via `launchctl bootstrap`/`bootout`.
|
||||
- **`cp0-mac-sync` CLI** (installed to `/usr/local/bin/cp0-mac-sync` or `~/.local/bin/cp0-mac-sync`):
|
||||
- `cp0-mac-sync install-service` — initializes config, creates default `~/Pictures/CardputerZero`, installs and loads the `LaunchAgent`.
|
||||
- `cp0-mac-sync uninstall-service` — unloads and removes the `LaunchAgent` (keeps synced photos intact).
|
||||
- `cp0-mac-sync set-dir <path>` — validates/creates `<path>`, updates `config.json`, and reloads/notifies the receiver immediately (no manual plist editing).
|
||||
- `cp0-mac-sync set-port <port>` — updates port and restarts the `LaunchAgent`.
|
||||
- `cp0-mac-sync status` — prints service running state, Tailscale IP/MagicDNS name (via `tailscale status --json` if available), current `dest_root`, and recent upload count.
|
||||
- `cp0-mac-sync logs [--follow]` — displays or tails `~/Library/Logs/cp0-mac-sync.log`.
|
||||
|
||||
#### HTTP Receiver Protocol & Security **(Our Call + User Decision)**
|
||||
Pure Python `ThreadingHTTPServer`:
|
||||
- **Source IP filtering **(User Decision + Our Call)**:** Because the user's tailnet contains strictly private/authorized devices, no pairing token is used. When `tailscale_only` is `true` (default), the receiver rejects any client IP outside Tailscale CGNAT (`100.64.0.0/10`), Tailscale IPv6 (`fd7a:115c:a1e0::/48`), or loopback (`127.0.0.1`, `::1`) with `403 Forbidden`. Listening on `0.0.0.0:8765` avoids binding failures if the Tailscale interface comes up after user login, while the CIDR allowlist guarantees LAN/public interfaces cannot reach it.
|
||||
- **`GET /health`:** Returns `200 OK` JSON:
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "1.0.0",
|
||||
"hostname": "<mac-hostname>",
|
||||
"dest_root": "/Users/<user>/Pictures/CardputerZero"
|
||||
}
|
||||
```
|
||||
- **`POST /upload`:**
|
||||
- **Headers:** `Content-Length`, `X-CP0-Folder-Label`, `X-CP0-Rel-Path`, `X-CP0-SHA256`, `X-CP0-Mtime` (Unix epoch seconds float/int).
|
||||
- **Path traversal sanitization:** Rejects (`400`) empty/absolute paths or any `..` segment in `folder_label` or `rel_path`; enforces resolved destination strictly inside `dest_root`.
|
||||
- **Streaming & atomic commit:**
|
||||
1. Streams request body in 64 KB chunks to `<target_dir>/.cp0-<uuid>.part` while computing SHA-256 incrementally.
|
||||
2. If stream disconnects prematurely or computed SHA-256 != `X-CP0-SHA256`, deletes `.part` file and returns `400`/`499`.
|
||||
3. If destination path already exists:
|
||||
- Computes existing file's SHA-256; if identical, deletes `.part` and returns `200 {"status": "already_exists"}` (deduplication on retry).
|
||||
- If different content exists at that name, renames destination to `<stem>_<YYYYMMDD_HHMMSS>_<sha8><ext>` so no photo is ever overwritten.
|
||||
4. Atomically renames `.part` → final path (`os.replace`) and applies original capture timestamp via `os.utime`.
|
||||
5. Returns `200 {"status": "saved", "saved_path": "..."}`.
|
||||
- **Batch macOS Notification:** A 5-second debounce timer fires a single native macOS notification via `osascript` (`display notification "Synced N photos to CardputerZero" with title "CardputerZero Sync"`) once a burst of uploads settles, rather than spamming one banner per photo.
|
||||
|
||||
---
|
||||
|
||||
### 5.4 CardputerZero Companion UI (`cp0-sync-ui`)
|
||||
|
||||
#### Visual & Design System — `shadcn/ui` Zinc Style for LVGL v9 **(User Decision + Our Calls)**
|
||||
- Built on `CardputerZero/Template` using C++ / LVGL v9 for the **320×170** display **(User Decision)**.
|
||||
- **Layout & Component Geometry (Option A: Bento Cards + Segmented Tabs) **(User Decision)**:**
|
||||
- **Top bar (24 px):** Segmented `TabsList` pill container (`radius: 5px`, `padding: 2px`) on the left (`1 Status`, `2 Folders`, `3 Logs` with `radius: 4px` active pill) + right-aligned pill status `Badge` (`radius: 999px`, `1px` border: `● Mac Online` / `● Mac Sleeping` / `↻ Syncing` / `○ No Target`).
|
||||
- **Main content (126 px):** Full 320px width dedicated to the active tab:
|
||||
- `1 Status`: Two side-by-side bordered `Card` stat tiles (`Pending Queue` and `Total Synced`, `radius: 6px`, `1px` border) above a full-width bordered `Target` summary card (`Target <host>`, retry/last-sync indicator, and `→ <dest_root>/<folder>`).
|
||||
- `2 Folders`: Stacked bordered `Card` rows (`radius: 6px`, `1px` border) showing full path on the left and a pill `Badge` (`queued` / `synced` / `missing`) on the right.
|
||||
- `3 Logs`: Monospace log list separated by `1px` hairline row dividers (`HH:MM:SS`, `OK`/`IN`/`WN`/`ER` tag, message).
|
||||
- **Bottom key-hint bar (20 px):** `<kbd>` keycap badges (`radius: 3px`, `1px` border) paired with context-sensitive shortcut labels.
|
||||
- **Dual Toggleable Theme Tokens (Zinc Dark & Standard Zinc Light) **(User Decision)**:**
|
||||
- **Zinc Dark (`"theme": "dark"`, default):**
|
||||
- `--background`: `#09090b` (`zinc-950`), `--card`: `#09090b`, `--muted`: `#18181b` (`zinc-900`), `--border`: `#27272a` (`1px`), `--ring` (focused row/control border): `#d4d4d8`, `--foreground`: `#fafafa`, `--muted-foreground`: `#a1a1aa`, active tab pill: `#27272a` fill / `#fafafa` text.
|
||||
- Semantic accents (badges/dots only): Online/Synced `#4ade80`, Sleeping/Queued `#fbbf24`, Missing/Error `#f87171`.
|
||||
- **Zinc Light (`"theme": "light"`, balanced standard `shadcn/ui` light skin):**
|
||||
- `--background`: `#fafafa` (`zinc-50`), `--card`: `#ffffff`, `--muted`: `#f4f4f5` (`zinc-100`), `--border`: `#e4e4e7` (`1px`), `--ring` (focused row/control border): `#71717a` (balanced neutral focus ring rather than stark high-contrast black), `--foreground`: `#09090b`, `--muted-foreground`: `#71717a`, active tab pill: `#ffffff` fill with `#e4e4e7` border / `#09090b` text.
|
||||
- Semantic accents (badges/dots only): Online/Synced `#16a34a` (`#dcfce7` badge fill), Sleeping/Queued `#d97706` (`#fef3c7` badge fill), Missing/Error `#dc2626` (`#fee2e2` badge fill).
|
||||
- **Typography **(Our Call)**:** Proportional sans-serif (`Inter` bundled and loaded via LVGL FreeType) for UI chrome, headers, tabs, and badges paired with monospace (`JetBrains Mono` via FreeType) for paths, stat counts, hostnames, `<kbd>` keycaps, and log timestamps.
|
||||
- **Data access **(Our Call)**:** Polls `cp0-syncd` over the Unix socket every 1 second while open (with direct read-only SQLite fallback to display cached state even if the daemon is restarting) and sends mutations (`sync_now`, `add_folder`, `remove_folder`, `set_target`, `set_theme`, `retry_failed`) over the socket.
|
||||
|
||||
#### Tab Views & Keyboard Map **(Our Call)**
|
||||
Designed around the `cp0-keys` 46-key matrix so common actions do not require multi-key `fn` chords unless desired:
|
||||
|
||||
- **Global shortcuts (when no modal is open):**
|
||||
- `1` / `2` / `3`: Jump directly to `Status`, `Folders`, or `Logs` tab.
|
||||
- `Enter`: Trigger `Sync Now` (`sync_now` IPC) and flash feedback in the status banner.
|
||||
- `T`: Toggle live between **Zinc Dark** and **Zinc Light** and persist `"theme"` via `set_theme` IPC **(User Decision)**. (When inside the Target Config or Directory Picker modal, `T` retains its modal role of opening manual text input.)
|
||||
- `Esc` or `Q`: Quit `cp0-sync-ui` back to `APPLaunch` (leaving `cp0-syncd` running in the background).
|
||||
- Navigation: Both `LV_KEY_UP`/`DOWN` (`fn+F`/`fn+X`) and `k`/`j` move selection/scroll; `LV_KEY_LEFT`/`RIGHT` (`fn+Z`/`fn+C`) also switch tabs.
|
||||
|
||||
- **Tab `1:Status`:**
|
||||
- Displays: `QUEUE: <N> pending (<X.X> MB)` | `SYNCED: <M>`, `TARGET: <host>:<port>`, `DEST: <mac_dest_root>/<folder>` (fetched from `/health`), and a status strip showing `Last sync: ...` or `Next retry in <N>s` / current upload filename.
|
||||
- `[C]` opens **Target Mac Config Modal**:
|
||||
- Runs `tailscale status --json` locally to list online/offline Tailscale peers (filtering to non-self nodes, prioritizing macOS/Darwin peers at the top), so the user can pick their Mac with arrow/`j`/`k` + `Enter` without typing `.ts.net` addresses.
|
||||
- Also includes a `[T] Type host manually` option with an LVGL text input field + port field.
|
||||
|
||||
- **Tab `2:Folders`:**
|
||||
- Scrollable list of watched folders showing path (left) and status badge (`3 queued` amber / `✓ synced` green / `! missing` red).
|
||||
- `[A]` opens **Keyboard Directory Picker Modal**:
|
||||
- Root choices `/home`, `/media`, `/` (or starts in `$HOME`).
|
||||
- Arrow/`j`/`k` moves cursor, `Enter` drills into directory, `Backspace` goes up parent directory (`..`), **`Space`** watches the highlighted directory (or current directory), **`[T]`** opens a manual path text input fallback.
|
||||
- `[D]` or `Delete` (`fn+Backspace`): Opens a brief `[Y/N]` confirmation bar to remove the selected folder from the watch list (explicitly noting local files on disk are not deleted).
|
||||
|
||||
- **Tab `3:Logs`:**
|
||||
- Scrollable monospace log viewer reading the newest entries from `state.db` (`HH:MM:SS` + concise message, color-coded by `INFO`/`WARN`/`ERROR`).
|
||||
- `[R]` retries all `FAILED` files (`retry_failed` IPC).
|
||||
|
||||
---
|
||||
|
||||
### 5.5 Packaging & Launcher Integration **(Our Call + User Decision)**
|
||||
- **Package name, version, & build path:** `cp0-sync_1.0.0_m5stack1_arm64.deb` generated locally via CMake/CPack (ARM64 Docker build script or native build on CardputerZero over SSH) for personal installation via `sudo apt install ./cp0-sync_1.0.0_m5stack1_arm64.deb` **(User Decision)**. No `CardputerZeroRepository` or `czdev` CI submission layout constraints apply.
|
||||
- **Files installed on CardputerZero:**
|
||||
- `/usr/bin/cp0-sync-ui` (compiled C++/LVGL binary)
|
||||
- `/usr/bin/cp0-syncd` (Python 3 daemon entrypoint, `chmod +x`)
|
||||
- `/usr/lib/systemd/user/cp0-syncd.service` (`WantedBy=default.target`, `After=network-online.target`)
|
||||
- `/usr/share/APPLaunch/applications/cp0-sync.desktop`
|
||||
- `/usr/share/APPLaunch/share/images/cp0-sync.png` (custom app icon sized for `APPLaunch`)
|
||||
- **Debian `Depends`:** `python3 (>= 3.9)` (zero third-party pip packages required) plus standard LVGL/runtime shared deps resolved via `dpkg-shlibdeps`.
|
||||
- **Maintainer scripts:**
|
||||
- `postinst`: runs `systemctl --global enable cp0-syncd.service` and restarts the user service for active graphical sessions if present.
|
||||
- `prerm`: stops/disables `cp0-syncd.service` cleanly on package removal.
|
||||
|
||||
---
|
||||
|
||||
## 6. What's Left to the Builder
|
||||
|
||||
- LVGL v9 `lv_style_t` token table, bundled FreeType `.ttf` subsetting/loading (`Inter` + `JetBrains Mono`), and live theme switcher implementing the specified `shadcn/ui` Zinc Dark and Zinc Light palettes on the 320×170 display.
|
||||
- CMake toolchain/preset configuration for both local macOS SDL2 simulator testing and local ARM64 `.deb` packaging (Docker `linux/arm64` build script or native CardputerZero SSH build).
|
||||
- Concrete implementation of Python modules (`cp0-syncd`, `cp0-mac-receiver`, `cp0-mac-sync`), plist template, and `postinst`/`prerm` packaging scripts.
|
||||
- Automated unit and end-to-end integration tests:
|
||||
- Python tests for `inotify` close-write handling, SQLite queue persistence/dedup, atomic `.part` transfer, collision renaming, Tailscale CIDR allowlist, and sleep/wake retry recovery.
|
||||
- C++ UI state/IPC parsing tests and SDL2 simulator verification at 320×170.
|
||||
Reference in New Issue
Block a user