9.3 KiB
Important
This project was developed entirely with AI coding assistance (Google Neon) and has not undergone rigorous review. It is provided as-is and may require adjustments for other environments.
Architecture & Camera Protocol Reference
czcamera separates user-space UI, image processing, media assembly, and storage management in C++17 from Raspberry Pi hardware capture in Python 3 (python3-picamera2) through an explicit backend abstraction (cz::ICameraBackend).
1. System Topology & Decoupling Boundary
+-------------------------------------------------------------------+
| C++17 /usr/bin/czcamera |
| - AppView (320x170 ShadcnCanvas: FreeType2 + 5x7 fallback) |
| - UiController, TimelapseEngine, GalleryStore, VideoPlayer |
| - Filters RGB24 + libjpeg still pipeline, BacklightController |
| - Linux (/dev/fb0 + cp0-keys evdev) OR Desktop SDL2 (--sim) |
| | |
| cz::ICameraBackend |
| +---------------+---------------+ |
| | | |
| SubprocessCameraBackend SimCameraBackend |
| (device / --mock-driver) (--sim / --mock-cam / tests) |
+--------------|----------------------------------------------------+
| stdin/stdout JSON v1 + dedicated CZFR fd pipe
v
+-------------------------------------------------------------------+
| Python 3 /usr/bin/czcamera-cam (czcamera_cam) |
| - JsonProtocolServer + FramePipeWriter (latest-slot drop-oldest) |
| - Picam2Worker (main + lores YUV420 dual streams, H264Encoder) |
| - MockCameraDriver (--mock-driver deterministic test driver) |
+-------------------------------------------------------------------+
| python3-picamera2 / libcamera IPA (rpi/vc4)
v
Sony IMX219 (3280x2464) + ES8389 MEMS Mic (ALSA default)
Replacing picamera2 with Native libcamera Later
czcamera C++ sources have zero #include <libcamera/...> or CMake linkage to libcamera.so. To swap python3-picamera2 for a native C++ libcamera backend (or rpicam-apps), implement app/src/camera_backend.hpp (cz::ICameraBackend) and return it from create_camera_backend() in app/src/sim_backend.cpp without touching UiController, AppView, filters, timelapse, or gallery code.
2. Camera Helper Wire Protocol v1
SubprocessCameraBackend launches /usr/bin/czcamera-cam --frame-fd <N> (overridable via CZCAMERA_CAM_BIN, passing --mock-driver in protocol tests) with an inherited read/write pipe (frame_fd).
Control Channel (stdin / stdout Newline-Delimited JSON)
Requests from C++ → Python carry "id" (uint64) and "cmd":
| Command | Payload Fields | Reply Extra Fields ({"id", "ok": true, ...}) |
|---|---|---|
configure |
mode ("photo"/"video"/"timelapse"), main_w, main_h, preview_w (226), preview_h (170), fps, zoom, ev, transform ("auto"/"0"/"180"/"hflip"/"vflip") |
mode, main_w, main_h, preview_w, preview_h (226x128 for 16:9 modes), transform, is_16_9 |
set_controls |
Optional zoom (1.0..4.0), ev (-2.0..+2.0) |
zoom, ev |
capture_still |
tmp_path, width, height |
bytes (temporarily stops video / switches to still format if needed and restores stream config) |
start_video |
tmp_path, width, height, fps, bitrate, audio (bool) |
audio_recorded (bool) |
stop_video |
— | duration_sec, bytes, audio_recorded |
pause |
— | Stops recording if active, closes camera & releases IMX219 lock |
resume |
— | Re-opens camera and restores last configure spec |
shutdown |
— | Tears down writer & camera cleanly and exits 0 |
Asynchronous Events from Python → C++ (no "id"):
{"event": "ready", "protocol": 1, "backend": "picamera2"|"mock", "sensor": "...", "has_audio": true|false}{"event": "warn", "code": "...", "message": "..."}(for example"audio_unavailable","invalid_json","control_error"){"event": "error", "code": "...", "message": "..."}
Binary Preview Frame Channel (--frame-fd <N>, 16-Byte CZFR Header)
Every preview frame packet starts with a fixed 16-byte little-endian header (struct.Struct("<4sHHBBHI")) followed immediately by payload_len bytes:
| Offset | Size | C Type | Field | Description |
|---|---|---|---|---|
0 |
4 |
char[4] |
magic |
"CZFR" (0x43 0x5A 0x46 0x52) |
4 |
2 |
uint16_t |
width |
Negotiated preview frame width (e.g., 226) |
6 |
2 |
uint16_t |
height |
Preview frame height (170 for 4:3, 128 for 16:9) |
8 |
1 |
uint8_t |
pixfmt |
1 = YUV420, 2 = RGB24 |
9 |
1 |
uint8_t |
flags |
Bit 0 (0x01): FLAG_ASPECT_16_9 |
10 |
2 |
uint16_t |
reserved |
Always 0 |
12 |
4 |
uint32_t |
payload_len |
width * height * 3 / 2 (YUV420) or width * height * 3 (RGB24) |
Backpressure Contract (Review Focus #1):
FramePipeWriteruses a single-slot bounded queue (_pending). When the UI consumer slows down,push_frame()drops any unconsumed queued packet and replaces it with the newest frame, keeping camera/H.264 encoder threads non-blocking.Picam2Worker._extract_lores_yuv420()inspects the negotiatedloresstride(handling 64-byte DMA padding such as256for226px) and strips row padding before packing tightYUV420payloads.
3. Target Hardware Constraints & Media Pipelines
- Sony IMX219 Sensor Modes:
640×480(4:3, crop(1000,752)/1280×960),1640×1232(4:3 full-FOV 2×2 binned),1920×1080(16:9 crop),3280×2464(4:3 8MP full sensor). - Video Output (
FfmpegOutput):_FfmpegOutputTargetoverrides.split()cleanly aroundpicamera2.outputs.FfmpegOutput's whitespace.split()so staging paths containing spaces or special characters (and ending in.mp4.part, with-f mp4 -movflags +faststart) remain exactargvtokens. - Timelapse Compilation:
TimelapseEngineinvokesffmpegwith-threads 1 -filter_threads 1and scales4:3(2mp/8mp) to1024:768or16:9(1080p) to1280:720(bilinear,setsar=1) to stay safely within CardputerZero's 512 MB RAM / 64 MB CMA budget and VideoCore IV/dev/video11constraints, encoding withh264_v4l2m2m(-num_output_buffers 4,5M,yuv420p) and falling back to low-memorylibx264(-threads 2 -preset ultrafast -crf 23 -x264-params rc-lookahead=0:ref=1:bframes=0:threads=2:lookahead-threads=1:sync-lookahead=0) on failure. - Gallery Playback:
VideoPlayerprobes duration & audio presence viaffprobe, spawnsffmpegrawrgb24frames timed cleanly to PTS, and drives audio through ALSAaplayon Linux orset_pcm_sink(SDL2 audio queue) on desktop—allowing silent.mp4files (compiled timelapses or muted clips) to play smoothly without audio failure.
4. 320×170 UI Renderer & Zinc Color Tokens
AppView renders into an offscreen 320×170 ARGB8888 canvas with SDF antialiased rounded primitives and FreeType2 + 5×7 bitmap fallback, then blits via fb_blit.hpp (RGB565 / RGB24 / XRGB8888) or SDL2 texture.
Exact palette tokens defined in app/src/theme.cpp:
| Token | Zinc Dark (ThemeMode::Dark) |
Zinc Light (ThemeMode::Light) |
|---|---|---|
background / card |
#09090b / #09090b |
#fafafa / #ffffff |
muted |
#18181b |
#f4f4f5 |
border |
#27272a |
#e4e4e7 |
ring |
#d4d4d8 |
#71717a |
foreground / muted_foreground |
#fafafa / #a1a1aa |
#09090b / #71717a |
active_tab / active_tab_foreground |
#27272a / #fafafa |
#ffffff / #09090b |
status_online / status_online_bg |
#4ade80 / #11261a |
#16a34a / #dcfce7 |
status_sleeping / status_sleeping_bg |
#fbbf24 / #271d0b |
#d97706 / #fef3c7 |
status_error / status_error_bg |
#f87171 / #2b1214 |
#dc2626 / #fee2e2 |
viewfinder_surround |
#050507 |
#050507 |
5. Source Directory Map
app/src/:theme.{hpp,cpp},fb_blit.hpp,config.{hpp,cpp},storage.{hpp,cpp}filters.{hpp,cpp},jpeg_pipeline.{hpp,cpp}camera_backend.hpp,subprocess_backend.{hpp,cpp},sim_backend.{hpp,cpp}backlight.{hpp,cpp},timelapse_engine.{hpp,cpp},gallery_model.{hpp,cpp},video_player.{hpp,cpp}ui_controller.{hpp,cpp},app_ui.{hpp,cpp},evdev_repeat_guard.hpp,main.cpp
app/tests/:test_config_and_storage.cpp,test_filters_and_jpeg.cpp,test_backend_protocol.cpp,test_timelapse_and_gallery.cpp,test_ui_controller.cppcam-helper/:bin/czcamera-cam,czcamera_cam/{__init__,protocol,picam2_worker}.py,tests/test_{protocol,picam2_worker}.pypackaging/:Dockerfile.arm64,build-deb.sh,czcamera.desktop,czcamera.png,debian/{control,postinst,prerm},tests/test_packaging.pydocs/: User/developer guides (building-and-installation.md,usage-and-controls.md,architecture-and-protocol.md)LICENSE.md: Apache License, Version 2.0