docs: add build, installation, usage, and troubleshooting documentation

This commit is contained in:
2026-11-18 09:35:38 -05:00
parent 19bb127974
commit 59c34603cc
4 changed files with 522 additions and 0 deletions

View File

@@ -0,0 +1,206 @@
# Building & Installation Guide
This guide covers building the ARM64 Debian package for **CardputerZero**, installing the background receiver on **macOS**, running the desktop **SDL2 UI simulator**, and running all test suites.
---
## 1. Prerequisites
### macOS Host
- **Python:** `3.9+` (macOS system/Homebrew Python 3; both `mac-receiver` and `device-daemon` use the Python standard library with zero third-party `pip` dependencies).
- **Tailscale:** Signed into the same private tailnet as your CardputerZero.
- **Docker Desktop (for building the CardputerZero `.deb` on Mac):** Apple Silicon or `linux/arm64` binfmt/emulation support enabled.
- **Homebrew packages (only needed for building/running the macOS SDL2 desktop UI simulator & C++ tests):**
```bash
brew install cmake pkg-config sdl2 freetype sqlite3
```
### CardputerZero Device
- M5Stack CardputerZero running Raspberry Pi CM0 **ARM64 Debian Bookworm** (or compatible `arm64` Debian) with `systemd --user`, `APPLaunch`, and `tailscale` connected to your tailnet.
- Runtime packages declared by `packaging/debian/control`: `python3 (>= 3.9)`, `libsqlite3-0`, `libfreetype6`.
- Native build packages (only required if compiling `.deb` directly on the CardputerZero instead of via Docker):
```bash
sudo apt-get update
sudo apt-get install -y build-essential cmake pkg-config libsqlite3-dev libfreetype-dev dpkg-dev python3
```
---
## 2. Optional: Bundling TrueType Fonts (`Inter` + `JetBrains Mono`)
At runtime, `cp0-sync-ui` searches for Sans and Mono fonts in this order (`device-ui/src/app_ui.cpp`):
1. `CP0_FONT_SANS` / `CP0_FONT_MONO` environment variable overrides
2. `/usr/share/cp0-sync/fonts/` (`Inter-Regular.ttf` / `Inter.ttf` and `JetBrainsMono-Regular.ttf` / `JetBrainsMono.ttf`)
3. `assets/fonts/` relative to working directory
4. System Debian (`DejaVuSans.ttf`, `LiberationSans-Regular.ttf`, `/usr/share/fonts/truetype/inter/...`) or macOS (`/System/Library/Fonts/SFNS.ttf`, `Menlo.ttc`, etc.)
5. Built-in crisp 5×7 pixel bitmap font (`kFont5x7`) if no `.ttf` / `.ttc` font is found
If you want `Inter` and `JetBrains Mono` bundled inside the `.deb` at `/usr/share/cp0-sync/fonts/`, place `.ttf` or `.otf` font files in `packaging/fonts/` (or `device-ui/assets/fonts/`) **before** running `packaging/build-deb.sh`:
```text
packaging/fonts/
├── Inter-Regular.ttf
└── JetBrainsMono-Regular.ttf
```
> Note: When building the CardputerZero target inside a clean container or minimal Bookworm image without bundled fonts, installing `fonts-dejavu-core` on the device also activates FreeType anti-aliased rendering automatically without needing custom files in `packaging/fonts/`.
---
## 3. Building the CardputerZero ARM64 `.deb`
### Method A: Cross-Build on macOS via Docker (Recommended)
`packaging/build-deb.sh` uses [`packaging/Dockerfile.arm64`](../packaging/Dockerfile.arm64) (`--platform linux/arm64`, `debian:bookworm-slim`) to compile `device-ui` with `-DCP0_UI_TARGET_DEVICE=ON` (Linux `fbdev` + `evdev`), run C++ unit tests inside the container, and package `cp0-sync_1.0.0_m5stack1_arm64.deb`:
```bash
./packaging/build-deb.sh
```
Output artifact:
```text
packaging/dist/cp0-sync_1.0.0_m5stack1_arm64.deb
```
`(packaging/dist/ and device-ui/build-arm64/ are gitignored.)`
#### `build-deb.sh` Options
| Flag | Description |
|---|---|
| *(none)* | Build inside `--platform linux/arm64` Docker container and copy `.deb` to `packaging/dist/` |
| `--local` | Compile `device-ui` (`CP0_UI_TARGET_DEVICE=ON`) and build `.deb` directly on a Linux ARM64 host |
| `--stage-only <dir>` | Stage package file tree without calling `dpkg-deb` (requires existing binary or `--ui-bin`) |
| `--ui-bin <path>` | Use a prebuilt `cp0-sync-ui` binary instead of recompiling |
| `--out-dir <dir>` | Write `.deb` into a custom directory instead of `packaging/dist/` |
### Method B: Build Directly on CardputerZero (or ARM64 Linux Host)
```bash
git clone <your-repo-url> cardputerzero-sync
cd cardputerzero-sync
sudo apt-get update && sudo apt-get install -y build-essential cmake pkg-config libsqlite3-dev libfreetype-dev dpkg-dev python3
./packaging/build-deb.sh --local
```
### Package Contents (`cp0-sync_1.0.0_m5stack1_arm64.deb`)
- `/usr/bin/cp0-sync-ui` — 320×170 companion UI (`fbdev` + `evdev` binary)
- `/usr/bin/cp0-syncd` — Background Python sync daemon CLI entrypoint
- `/usr/lib/cp0-sync/cp0_syncd/*.py` — Sync daemon Python modules
- `/usr/lib/systemd/user/cp0-syncd.service` — `systemd --user` service unit
- `/usr/share/APPLaunch/applications/cp0-sync.desktop` — `Photo Sync` app launcher entry
- `/usr/share/APPLaunch/share/images/cp0-sync.png` — `80×80` app icon
- `/usr/share/cp0-sync/fonts/*` — Optional bundled fonts (if provided at build time)
---
## 4. Installing & Uninstalling on CardputerZero
Copy the `.deb` over Tailscale and install:
```bash
scp packaging/dist/cp0-sync_1.0.0_m5stack1_arm64.deb pi@<cardputerzero-host>:/tmp/
ssh pi@<cardputerzero-host> "sudo dpkg -i /tmp/cp0-sync_1.0.0_m5stack1_arm64.deb"
```
If `dpkg` reports missing shared runtime dependencies (`libfreetype6` or `libsqlite3-0`), resolve them with:
```bash
sudo apt-get install -f -y
```
### What `postinst` / `prerm` Do
- **`postinst` (`configure`):** Runs `systemctl --global enable cp0-syncd.service || true` and best-effort restarts `cp0-syncd.service` for active logged-in user sessions via `loginctl`.
- **`prerm` (`remove` / `deconfigure`):** Stops active user instances, runs `systemctl --global disable cp0-syncd.service || true`, and cleans Python `__pycache__` files under `/usr/lib/cp0-sync` without deleting your `~/.config/cp0-sync/` or `~/.local/share/cp0-sync/` data.
Verify the background service is active under your user session:
```bash
systemctl --user status cp0-syncd.service
```
To uninstall later:
```bash
sudo dpkg -r cp0-sync
```
---
## 5. Installing the Mac Receiver
On your Mac, run the `LaunchAgent` installer from the repository:
```bash
./mac-receiver/bin/cp0-mac-sync install-service
./mac-receiver/bin/cp0-mac-sync status
```
This creates and loads `~/Library/LaunchAgents/com.cardputerzero.photosync.plist` (`RunAtLoad=true`, `KeepAlive=true`), starting `mac-receiver/bin/cp0-mac-receiver` automatically at login.
*(Optional)* Symlink the CLI scripts into your `PATH` (both entrypoints resolve the `mac-receiver/` package cleanly through symlinks):
```bash
mkdir -p ~/.local/bin
ln -sf "$(pwd)/mac-receiver/bin/cp0-mac-sync" ~/.local/bin/cp0-mac-sync
ln -sf "$(pwd)/mac-receiver/bin/cp0-mac-receiver" ~/.local/bin/cp0-mac-receiver
```
To stop and remove the `LaunchAgent`:
```bash
./mac-receiver/bin/cp0-mac-sync uninstall-service
```
---
## 6. Building & Running the Desktop Simulator (`device-ui`)
You can run and visually inspect the 320×170 UI natively on macOS using SDL2 without CardputerZero hardware:
```bash
cmake -S device-ui -B device-ui/build -DCP0_UI_BUILD_TESTS=ON -DCP0_UI_BUILD_APP=ON
cmake --build device-ui/build
```
Run a scaled `3×` simulator window (connected to `~/.config/cp0-sync/` or offline fallback):
```bash
./device-ui/build/cp0-sync-ui --sim --scale 3
```
Render a headless `320×170` PPM snapshot without opening a window:
```bash
./device-ui/build/cp0-sync-ui --render-test /tmp/status_dark.ppm --theme dark --tab status
./device-ui/build/cp0-sync-ui --render-test /tmp/folders_light.ppm --theme light --tab folders
```
### CMake Options Reference (`device-ui/CMakeLists.txt`)
| Option | Default | Notes |
|---|---|---|
| `CP0_UI_BUILD_TESTS` | `ON` | Builds `test_ipc_and_theme` and `test_ui_navigation` |
| `CP0_UI_BUILD_APP` | `ON` | Builds the `cp0-sync-ui` executable (set `OFF` to build only `cp0_ui_core` + tests) |
| `CP0_UI_TARGET_DEVICE` | `OFF` | Linux only: targets `/dev/fb0` + `/dev/input/event*` instead of SDL2 (errors cleanly on non-Linux hosts) |
| *(Optional LVGL)* | auto-detected | If system `lvgl` (`find_package(lvgl)` / `pkg-config lvgl`) is present, enables `CP0_HAS_LVGL=1` style hooks alongside the built-in `320×170` renderer; never downloads over network via `FetchContent` |
---
## 7. Running All Test Suites
All test suites use zero third-party Python dependencies (`unittest` stdlib) and CTest:
```bash
# 1. Mac Receiver unit & HTTP server tests (24 tests)
PYTHONPATH=mac-receiver python3 -m unittest discover -s mac-receiver/tests -v
# 2. Device Daemon config, SQLite store, watcher, uploader & IPC tests (25 tests)
PYTHONPATH=device-daemon:mac-receiver python3 -W error -m unittest discover -s device-daemon/tests -v
# 3. Companion UI C++ tests (2 CTest targets: test_ipc_and_theme, test_ui_navigation)
cmake --build device-ui/build && ctest --test-dir device-ui/build --output-on-failure -V
# 4. Packaging validation tests (7 tests)
python3 -m unittest discover -s packaging/tests -v
```
*(Note on macOS sandboxing: `test_server.py`, `test_daemon_integration.py`, and `test_ipc_and_theme` open local TCP loopback or `AF_UNIX` sockets; if running inside a restricted sandbox that blocks sockets with `Operation not permitted` / `EPERM`, run them outside the sandbox.)*

View File

@@ -0,0 +1,180 @@
# Usage, Configuration & Operations Guide
---
## 1. Mac Receiver Operations (`cp0-mac-sync`)
Manage where photos land, check Tailscale/receiver health, view logs, and control the macOS background service using `cp0-mac-sync`.
### Command Reference
| Command | Description |
|---|---|
| `cp0-mac-sync status` | Show `LaunchAgent` state (`com.cardputerzero.photosync`), live HTTP `/health` check (`0.0.0.0:<port>`), destination directory, Tailscale CLI status, and active config path. |
| `cp0-mac-sync set-dir <path>` | Create `<path>` if needed and update `dest_root` in `config.json`. Picked up **automatically on the next upload** (no daemon restart required). |
| `cp0-mac-sync set-port <1-65535>` | Update `listen_port` in `config.json` and restart the `LaunchAgent` if installed (`launchctl kickstart -k`). Default: `8765`. |
| `cp0-mac-sync logs [-n LINES] [-f]` | Print recent lines (`-n 100` default) or tail live receiver logs (`~/Library/Logs/cp0-mac-sync.log`). |
| `cp0-mac-sync install-service` | Write `~/Library/LaunchAgents/com.cardputerzero.photosync.plist` and load/enable the service (`gui/<uid>/com.cardputerzero.photosync`). |
| `cp0-mac-sync uninstall-service` | Stop and unload the `LaunchAgent` and remove its `.plist`. |
### Running `cp0-mac-receiver` Manually (Foreground / Debug)
For foreground debugging without `launchd`:
```bash
./mac-receiver/bin/cp0-mac-receiver --config ~/.config/cp0-mac-sync/config.json --log-file /tmp/cp0-rec.log
```
### Destination Layout & Collision Safety
Uploaded files are written into:
```text
<dest_root>/<folder_label>/<rel_path>
```
For example, if `dest_root` is `~/Pictures/CardputerZero` and CardputerZero watches `/home/pi/Pictures/DCIM/Camera` (label `Camera`):
- `IMG_0042.JPG` → `~/Pictures/CardputerZero/Camera/IMG_0042.JPG`
- `2026-11/IMG_0043.JPG` → `~/Pictures/CardputerZero/Camera/2026-11/IMG_0043.JPG`
Collision & safety guarantees (`mac-receiver/cp0_mac_receiver/server.py`):
- **Atomic staging:** Body streams to a `.cp0-*.part` temp file, hashes SHA-256, validates `X-CP0-SHA256`, applies `os.utime` (`X-CP0-Mtime`) when valid, `fsync`s, and atomically replaces into place (`60s` socket timeout cleans up `.part` files if a connection drops mid-upload).
- **Idempotent duplicate:** If a file already exists at the destination with the **same SHA-256**, it returns `200 OK` (`status: "duplicate"`) without creating extra copies.
- **Collision rename:** If a file exists at the destination with a **different SHA-256**, it is saved alongside it as `<stem>_<YYYYMMDD_HHMMSS>_<sha8><ext>` (never overwriting).
- **Tailscale IP allowlist (`tailscale_only: true`):** Only accepts HTTP requests from Tailscale CGNAT (`100.64.0.0/10`), Tailscale IPv6 (`fd7a:115c:a1e0::/48`), or loopback (`127.0.0.0/8`, `::1/128`); all other client IPs receive `403 Forbidden`.
- **Notifications:** When `notify_on_batch: true`, debounced macOS Notification Center alerts (`2s` debounce window) summarize batch uploads (e.g., `"Synced 5 photos from CardputerZero"`).
---
## 2. CardputerZero Companion UI (`cp0-sync-ui`)
Launch **Photo Sync** from `APPLaunch` or run `cp0-sync-ui` from a shell. The UI refreshes daemon state every `1000 ms` over the daemon's Unix domain socket (falling back cleanly to read-only SQLite + `config.json` display when `cp0-syncd` is stopped).
### Top-Bar Status Pills
| Badge | Meaning |
|---|---|
| **`Mac Online`** | Daemon running, target Mac configured, and last upload/probe succeeded. |
| **`Syncing`** | Daemon is actively streaming queued photos over Tailscale. |
| **`Mac Sleeping`** | Target Mac unreachable; queued photos waiting safely until next wake-probe or `[Enter]`. |
| **`No Target`** | Daemon running, no Mac host configured yet — press **`C`** on `1 Status`. |
| **`Daemon Off`** | `cp0-syncd` IPC socket unreachable (shows read-only cached counts/logs below if `state.db` exists). |
### Tabs & Workflows
#### Tab `1 Status`
Displays two Bento summary tiles (**Pending Queue** file count + total bytes, **Total Synced** count + 1-way verified status), your **Target** host (`host:port`) with retry countdown/uploading/cached indicator, destination directory on the Mac, and last synced filename or error summary.
- Press **`C`** to open the **Target Mac Picker**:
- Automatically queries `tailscale status --json` and sorts **online macOS peers first**.
- Navigate with `j`/`k` or `Up`/`Down`, press **`Enter`** to select `<peer-ip-or-dns>:8765`.
- Press **`T`** inside the picker to manually enter `<host>` or `<host>:<port>` (or IPv6 `[host]:port`).
#### Tab `2 Folders`
Lists all watched directories with badges (`N queued`, `synced`, `N err`, or `! missing`). On first run, `cp0-syncd` creates `/home/pi/Pictures/DCIM/Camera` (label `Camera`) automatically.
- **Add folder (`A`):** Opens the on-device filesystem directory picker starting at your home directory (`~`):
- `j`/`k` or `Up`/`Down`: move highlight
- `Enter`: drill down into highlighted subdirectory
- `Backspace`: go up to parent directory
- `Space`: select & add the highlighted directory
- `.`: select & add the directory you are currently inside
- `T`: switch to manual path typing (supports `~/...` expansion)
- Nested or overlapping watched directories (parent or child of an existing watched folder) are rejected cleanly with an error toast to avoid double-uploading; duplicate base folder names (e.g., two `Camera` directories) automatically get unique labels (`Camera`, `Camera_2`, ...).
- **Remove folder (`D` or `Delete`):** Opens confirmation modal **`[Y] Confirm / [N] Cancel`**. Removing a watched folder stops watching it and purges `PENDING`/`UPLOADING` queue items, **never deleting photos on disk** and retaining historical `SYNCED` dedup records so re-adding the folder later won't re-upload old files.
#### Tab `3 Logs`
Displays timestamped `INFO`, `WARN`, and `ERROR` entries from SQLite (`state.db` retains the most recent 500 log rows).
- `j`/`k` or `Up`/`Down`: scroll through recent log rows.
- **`R`:** Reset all **`FAILED`** rows back to `PENDING` and trigger an immediate sync/wake probe.
#### Theme Toggle (`T`) & Exit (`Esc` / `Q`)
- Press **`T`** on any main tab (with no modal open) to toggle live between **Zinc Dark** (`#09090b`) and **Zinc Light** (`#fafafa`). The choice is saved to `config.json` via `cp0-syncd`.
- Press **`Esc`** or **`Q`** on main tabs to exit `cp0-sync-ui` cleanly (leaving `cp0-syncd` running in the background) and return to `APPLaunch`.
---
## 3. Daemon CLI & Runtime Behavior (`cp0-syncd`)
### `cp0-syncd` CLI Flags
```bash
cp0-syncd [--config PATH] [--db PATH] [--sock PATH] [--verbose]
```
### File Filtering & Queue Lifecycle
- **Watched media extensions (`DaemonConfig.extensions`):** `.jpg`, `.jpeg`, `.png`, `.webp`, `.gif`, `.bmp`, `.dng`, `.raw`, `.mp4`, `.mov` (case-insensitive).
- **Ignored names:** Prefixes `.` or `~` (`DEFAULT_IGNORE_PREFIXES`), suffixes `.part`, `.tmp`, `.crdownload` (`DEFAULT_IGNORE_SUFFIXES`), and **0-byte files** (logged once as `WARN` without enqueueing).
- **Watch engine:** On Linux, uses native `ctypes` `inotify` (`IN_CLOSE_WRITE | IN_MOVED_TO | IN_CREATE | IN_DELETE_SELF | IN_MOVE_SELF`) plus `IN_ONLYDIR` ancestor watches so removable SD card mounts are detected automatically; falls back to periodic directory polling on non-Linux hosts.
- **Wake-probe backoff:** When a Mac sleeps or refuses connections, `Uploader` pauses uploads, logs cleanly, probes `GET /health` (`3s` timeout) on backoff `(5s → 10s → 20s → 40s → 60s max)` or wakes immediately on new captured photos or **`Enter`** in the UI.
- **HTTP failure triage:** Network errors, `403`, `408`, and `5xx` pause the queue and retry once the Mac recovers; `400`, `413`, and `422` (and missing local files) mark only that single file `FAILED` so later photos keep flowing.
- **Crash recovery:** On startup, any rows left in `UPLOADING` state from an unexpected power loss automatically reset to `PENDING` before scanning begins.
---
## 4. Configuration Files & Environment Variables
### On Mac
| Item | Default Path | Override Env Var |
|---|---|---|
| Config JSON | `~/.config/cp0-mac-sync/config.json` | `CP0_MAC_CONFIG` |
| Rotating Logs | `~/Library/Logs/cp0-mac-sync.log` *(5 MB × 2 backups)* | — |
| LaunchAgent | `~/Library/LaunchAgents/com.cardputerzero.photosync.plist` | — |
| Synced Photos | `~/Pictures/CardputerZero/<folder_label>/...` | *(via `cp0-mac-sync set-dir`)* |
**Mac `config.json` schema (`MacConfig`):**
```json
{
"dest_root": "/Users/you/Pictures/CardputerZero",
"listen_host": "0.0.0.0",
"listen_port": 8765,
"notify_on_batch": true,
"tailscale_only": true
}
```
### On CardputerZero
| Item | Default Path | Override Env Var |
|---|---|---|
| Config JSON | `~/.config/cp0-sync/config.json` | `CP0_SYNC_CONFIG` |
| SQLite WAL DB | `~/.local/share/cp0-sync/state.db` | `CP0_SYNC_DB` |
| Unix IPC Socket | `$XDG_RUNTIME_DIR/cp0-sync.sock` (fallback `/tmp/cp0-sync.sock`) | `CP0_SYNC_SOCK` |
| Sans / Mono Fonts | `/usr/share/cp0-sync/fonts/` → system fonts → 5×7 fallback | `CP0_FONT_SANS`, `CP0_FONT_MONO` |
| Framebuffer (`device` build) | `/dev/fb0` | `CP0_FB_DEVICE` |
| Keyboard `evdev` (`device` build) | Auto-scans `/dev/input/event*` for `EV_KEY` | `CP0_EVDEV_DEVICE` |
---
## 5. Troubleshooting
### UI displays **`Daemon Off`**
Check whether the background `systemd --user` service is running on CardputerZero:
```bash
systemctl --user status cp0-syncd.service
journalctl --user -u cp0-syncd.service -n 50 --no-pager
```
Restart it if needed:
```bash
systemctl --user restart cp0-syncd.service
```
### Status stays on **`Mac Sleeping`** while the Mac is awake
1. On Mac, verify the receiver is running and listening:
```bash
cp0-mac-sync status
```
2. Verify macOS Firewall / Little Snitch isn't blocking incoming connections to Python on TCP port `8765`.
3. From CardputerZero over SSH, test reachability over Tailscale:
```bash
curl -i http://<mac-tailscale-ip-or-magicdns>:8765/health
```
A healthy response returns HTTP `200` with `{"ok": true, "version": "1.0.0", "dest_root": "..."}`. If you get `403`, confirm traffic is traveling over Tailscale (`100.64.0.0/10`) rather than an unallowed LAN interface.
### One photo shows **`FAILED`** in Logs
Open **`3 Logs`** in `cp0-sync-ui` to view the error reason (e.g., permission denied, missing source file, destination path conflict), resolve the issue, and press **`R`** to re-queue all `FAILED` entries.
### Re-uploading previously synced photos
Because one-way sync tracks `(folder_id, rel_path, sha256)` in `state.db` so deleted files on your Mac aren't re-copied, intentionally re-syncing everything requires resetting `state.db`:
```bash
systemctl --user stop cp0-syncd.service
rm ~/.local/share/cp0-sync/state.db*
systemctl --user start cp0-syncd.service
```