cottongin/crosspoint-reader-mod
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

mod: Phase 1 - bring forward mod-exclusive files with ActivityManager migration

Browse Source 
Brings ~55 mod-exclusive files to the upstream-based mod/master-resync branch:

Activities (migrated to new ActivityManager pattern):
- Clock/Time: SetTimeActivity, SetTimezoneOffsetActivity, NtpSyncActivity
- Dictionary: DictionaryDefinitionActivity, DictionarySuggestionsActivity,
  DictionaryWordSelectActivity, LookedUpWordsActivity
- Bookmark: EpubReaderBookmarkSelectionActivity
- Book management: BookManageMenuActivity, EndOfBookMenuActivity
- OPDS: OpdsServerListActivity, OpdsSettingsActivity
- Utility: DirectoryPickerActivity, NumericStepperActivity

Utilities (unchanged):
- BookManager, BookSettings, BookmarkStore, BootNtpSync
- Dictionary, LookupHistory, TimeSync, OpdsServerStore

Libraries: PlaceholderCover, TableData, ChapterXPathIndexer
Scripts: inject_mod_version, generate_book_icon, preview_placeholder_cover
Docs: KOReader sync XPath mapping

Migration changes:
- ActivityWithSubactivity -> Activity base class
- Callback constructors -> finish()/setResult() pattern
- enterNewActivity() -> startActivityForResult()
- Activity::RenderLock&& -> RenderLock&&

These files won't compile yet - they reference mod settings and I18n
strings that will be added in subsequent phases.

Made-with: Cursor
This commit is contained in:
cottongin
2026-03-07 15:10:00 -05:00
parent 170cc25774
commit dfbc931c14
147 changed files with 112771 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
mod/docs/The World in a Grain_ The Story of Sand an - Vince Beiser.epub Normal file
View File

Binary file not shown.

9
mod/docs/bookmark-svgrepo-com.svg Normal file
View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="utf-8"?><!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg version="1.1" id="Uploaded to svgrepo.com" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
width="800px" height="800px" viewBox="0 0 32 32" xml:space="preserve">
<style type="text/css">
.puchipuchi_een{fill:#111918;}
</style>
<path class="puchipuchi_een" d="M23.293,30.705l-6.586-6.586c-0.391-0.391-1.024-0.391-1.414,0l-6.586,6.586
C8.077,31.335,7,30.889,7,29.998V3c0-1.105,0.895-2,2-2h14c1.105,0,2,0.895,2,2v26.998C25,30.889,23.923,31.335,23.293,30.705z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 630 B

280
mod/docs/ci-build-and-code-style.md Normal file
View File

@@ -0,0 +1,280 @@
# CI, Build, Release & Code Style
This document covers the CrossPoint Reader build system, CI pipeline, release process, code formatting rules, static analysis, and contribution guidelines.
## Build System
The project uses **PlatformIO** with the Arduino framework targeting the ESP32-C3.
### Build Environments
Defined in `platformio.ini`:
| Environment | Purpose | Version String |
|---|---|---|
| `default` | Local development builds | `1.0.0-dev` |
| `gh_release` | Official tagged releases | `1.0.0` |
| `gh_release_rc` | Release candidates | `1.0.0-rc+{7-char SHA}` |
### Build Command
```bash
# Development build
pio run
# Release build
pio run -e gh_release
# Release candidate build (requires CROSSPOINT_RC_HASH env var)
CROSSPOINT_RC_HASH=abc1234 pio run -e gh_release_rc
```
### Build Flags
All environments share a common set of flags (`[base]` section):
| Flag | Purpose |
|---|---|
| `-std=c++2a` | C++20 standard |
| `-DARDUINO_USB_MODE=1` | USB mode selection |
| `-DARDUINO_USB_CDC_ON_BOOT=1` | Enable USB CDC serial on boot |
| `-DMINIZ_NO_ZLIB_COMPATIBLE_NAMES=1` | Avoid miniz/zlib symbol conflicts |
| `-DEINK_DISPLAY_SINGLE_BUFFER_MODE=1` | Single frame buffer (saves RAM) |
| `-DDISABLE_FS_H_WARNING=1` | Suppress Arduino FS.h warning |
| `-DXML_GE=0` | Disable expat general entity expansion |
| `-DXML_CONTEXT_BYTES=1024` | Expat context buffer size |
| `-DUSE_UTF8_LONG_NAMES=1` | Enable UTF-8 long filenames in SdFat |
### Pre-Build Step
`scripts/build_html.py` runs before compilation (configured via `extra_scripts = pre:scripts/build_html.py`). It:
1. Finds all `.html` files under `src/`.
2. Minifies them (strips comments, collapses whitespace, preserves `<pre>`, `<code>`, `<textarea>`, `<script>`, `<style>` blocks).
3. Generates `.generated.h` files containing `constexpr char ...Html[] PROGMEM = R"rawliteral(...)rawliteral";` strings.
### Dependencies
**SDK libraries** (symlinked from `open-x4-sdk` submodule):
- `BatteryMonitor`
- `InputManager`
- `EInkDisplay`
- `SDCardManager`
**External libraries** (managed by PlatformIO):
- `bblanchon/ArduinoJson @ 7.4.2` -- JSON parsing
- `ricmoo/QRCode @ 0.0.1` -- QR code generation
- `links2004/WebSockets @ 2.7.3` -- WebSocket server
### Tool Versions
| Tool | Version | Notes |
|---|---|---|
| PlatformIO | Latest (via pip) | `espressif32 @ 6.12.0` platform |
| Python | 3.14 | Used in CI and build scripts |
| clang-format | 21 | From LLVM apt repository |
| cppcheck | Latest (via PlatformIO) | Static analysis |
## CI Pipeline
All CI workflows are in `.github/workflows/`.
### `ci.yml` -- Main CI
**Triggers:** Push to `master`, all pull requests.
Runs **4 jobs in parallel** (the first 3 are independent; the 4th aggregates results):
#### 1. `clang-format` -- Code Formatting Check
1. Installs `clang-format-21` from the LLVM apt repository.
2. Runs `bin/clang-format-fix` on the full codebase.
3. Checks `git diff --exit-code` -- fails if any file was reformatted.
#### 2. `cppcheck` -- Static Analysis
1. Installs PlatformIO.
2. Runs `pio check --fail-on-defect low --fail-on-defect medium --fail-on-defect high`.
3. Fails on any low, medium, or high severity defect.
#### 3. `build` -- Compilation
1. Installs PlatformIO.
2. Runs `pio run` (default environment).
3. Extracts RAM and Flash usage stats into the GitHub step summary.
4. Uploads `firmware.bin` as a build artifact.
#### 4. `test-status` -- PR Gate
- Depends on all three jobs above.
- Fails if any dependency failed or was cancelled.
- This is the required status check for pull requests, decoupling CI steps from PR merge requirements.
### `pr-formatting-check.yml` -- PR Title Validation
**Triggers:** Pull request `opened`, `reopened`, `edited` events.
Uses `amannn/action-semantic-pull-request@v6` to enforce semantic PR title format (e.g., `feat: add dark mode`, `fix: correct page numbering`).
### `release.yml` -- Official Release
**Trigger:** Push of any git tag.
1. Builds using the `gh_release` environment.
2. Uploads release artifacts named `CrossPoint-{tag}`:
- `bootloader.bin`
- `firmware.bin`
- `firmware.elf`
- `firmware.map`
- `partitions.bin`
### `release_candidate.yml` -- RC Build
**Trigger:** Manual `workflow_dispatch`, restricted to branches matching `release/*`.
1. Extracts the 7-character short SHA and branch suffix (e.g., `v1.0.0` from `release/v1.0.0`).
2. Sets `CROSSPOINT_RC_HASH` env var.
3. Builds using the `gh_release_rc` environment.
4. Uploads artifacts named `CrossPoint-RC-{suffix}`.
## Release Process
### Official Release
1. Create and push a git tag (e.g., `v1.0.0`).
2. The `release.yml` workflow triggers automatically.
3. Artifacts are uploaded to the GitHub Actions run.
### Release Candidate
1. Create a branch named `release/{identifier}` (e.g., `release/v1.0.0`).
2. Navigate to Actions in GitHub and manually trigger `Compile Release Candidate` on that branch.
3. The RC version string includes the commit SHA for traceability.
### Version Scheme
```
1.0.0-dev # Local development (default env)
1.0.0 # Official release (gh_release env)
1.0.0-rc+a1b2c3d # Release candidate (gh_release_rc env)
```
The version base is set in `platformio.ini` under `[crosspoint] version = 1.0.0`.
## Code Style
### Formatting: clang-format
The project uses **clang-format 21** with a `.clang-format` config at the repository root. Key rules:
| Setting | Value | Meaning |
|---|---|---|
| `IndentWidth` | `2` | 2-space indentation |
| `TabWidth` / `UseTab` | `8` / `Never` | Spaces only (no tabs) |
| `ColumnLimit` | `120` | Maximum line width |
| `BreakBeforeBraces` | `Attach` | K&R brace style (opening brace on same line) |
| `PointerAlignment` | `Left` | `int* ptr` not `int *ptr` |
| `ReferenceAlignment` | `Pointer` | References follow pointer style |
| `ContinuationIndentWidth` | `4` | 4-space continuation indent |
| `AllowShortFunctionsOnASingleLine` | `All` | Short functions may be single-line |
| `AllowShortIfStatementsOnASingleLine` | `WithoutElse` | `if (x) return;` allowed |
| `AllowShortLoopsOnASingleLine` | `true` | Short loops may be single-line |
| `BreakBeforeTernaryOperators` | `true` | `?` and `:` start new lines |
| `BreakBeforeBinaryOperators` | `None` | Binary operators stay at end of line |
| `SortIncludes` | `Enabled` | Includes sorted lexicographically |
| `IncludeBlocks` | `Regroup` | Includes grouped by category |
| `ReflowComments` | `Always` | Long comments are rewrapped |
| `SpacesBeforeTrailingComments` | `2` | Two spaces before `// comment` |
| `LineEnding` | `DeriveLF` | Unix-style line endings |
### Include Order
Includes are regrouped into priority categories:
1. **Priority 1:** System headers with `.h` extension (`<foo.h>`)
2. **Priority 2:** Other system headers (`<foo>`) and extension headers (`<ext/foo.h>`)
3. **Priority 3:** Project-local headers (`"foo.h"`)
### Running the Formatter
```bash
# Format all tracked source files
./bin/clang-format-fix
# Format only modified (staged or unstaged) files
./bin/clang-format-fix -g
```
The script formats `.c`, `.cpp`, `.h`, `.hpp` files tracked by git, **excluding** `lib/EpdFont/builtinFonts/` (script-generated font headers).
### Static Analysis: cppcheck
Configuration in `platformio.ini`:
```
check_tool = cppcheck
check_flags = --enable=all
--suppress=missingIncludeSystem
--suppress=unusedFunction
--suppress=unmatchedSuppression
--suppress=*:*/.pio/*
--inline-suppr
```
- `--enable=all` enables all checks.
- Suppressed: missing system includes, unused functions (common in library code), PlatformIO build directory.
- `--inline-suppr` allows `// cppcheck-suppress` comments in source.
- CI fails on **any** defect severity (low, medium, or high).
Run locally:
```bash
pio check
```
## Contribution Guidelines
### PR Requirements
1. **Semantic PR title** -- enforced by CI. Must follow conventional format:
- `feat: ...`, `fix: ...`, `refactor: ...`, `docs: ...`, `chore: ...`, etc.
2. **PR template** (`.github/PULL_REQUEST_TEMPLATE.md`) must be filled out:
- **Summary:** Goal and changes included.
- **Additional Context:** Performance implications, risks, focus areas.
- **AI Usage:** Transparent disclosure -- `YES`, `PARTIALLY`, or `NO`.
3. **Code formatting** -- run `bin/clang-format-fix` before pushing. CI will reject unformatted code.
4. **Static analysis** -- ensure `pio check` passes without low/medium/high defects.
5. **Build** -- confirm `pio run` compiles without errors.
### Project Scope
All contributions must align with the project's core mission: **focused reading on the Xteink X4**. See `SCOPE.md` for full details.
**In scope:** UX improvements, document rendering (EPUB), typography, e-ink driver refinement, library management, local file transfer, language support.
**Out of scope:** Interactive apps, active connectivity (RSS/news/browsers), media playback, complex reader features (highlighting, notes, dictionary).
When unsure, open a Discussion before coding.
### Governance
From `GOVERNANCE.md`:
- **Assume good intent** in technical discussions.
- **Critique code, not people.**
- **Public by default** -- decisions happen in Issues, PRs, and Discussions.
- Maintainers guide technical direction for ESP32-C3 hardware constraints.
- Report harassment privately to `@daveallie`.
### Checklist Before Submitting
- [ ] Code compiles: `pio run`
- [ ] Code is formatted: `bin/clang-format-fix`
- [ ] Static analysis passes: `pio check`
- [ ] PR title follows semantic format
- [ ] PR template filled out (summary, context, AI disclosure)
- [ ] Changes align with project scope (`SCOPE.md`)

BIN
mod/docs/cover_worldgrain.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

101144
mod/docs/dictionary.dict Executable file
View File

File diff suppressed because one or more lines are too long

BIN
mod/docs/dictionary.idx Executable file
View File

Binary file not shown.

11
mod/docs/dictionary.ifo Executable file
View File

@@ -0,0 +1,11 @@
StarDict's dict ifo file
version=3.0.0
bookname=reader.dict EN
wordcount=894535
idxfilesize=17310252
sametypesequence=h
synwordcount=504743
website=https://www.reader-dict.com
date=2026-01-01
description=© reader.dict 2026
lang=en-en

306
mod/docs/file-structure.md Normal file
View File

@@ -0,0 +1,306 @@
# CrossPoint Reader -- Project File Structure
This document maps the repository layout and describes where key pieces of code can be found.
## Root Directory
```
crosspoint-reader-mod/
├── platformio.ini # PlatformIO build config (environments, flags, deps)
├── partitions.csv # ESP32 flash partition table
├── .clang-format # C++ code formatting rules (clang-format 21)
├── .clangd # Language server configuration
├── .gitmodules # Git submodule reference (open-x4-sdk)
├── README.md # Project overview and getting started
├── LICENSE # MIT License
├── GOVERNANCE.md # Community governance principles
├── SCOPE.md # Project vision and feature scope
├── USER_GUIDE.md # End-user manual
├── bin/ # Developer scripts
├── docs/ # Upstream documentation
├── lib/ # Project libraries (HAL, EPUB, fonts, etc.)
├── src/ # Main application source code
├── scripts/ # Build and utility scripts (Python)
├── test/ # Unit/evaluation tests
├── open-x4-sdk/ # Hardware SDK (git submodule)
├── include/ # PlatformIO include directory (empty)
├── mod/ # Local modifications workspace
└── .github/ # CI workflows, templates, funding
```
## `src/` -- Application Source
The main firmware application code. Entry point is `main.cpp`.
```
src/
├── main.cpp # Arduino setup()/loop(), activity routing, font init
├── CrossPointSettings.cpp/.h # User settings (load/save to SD, defaults)
├── CrossPointState.cpp/.h # Runtime application state (open book, sleep tracking)
├── MappedInputManager.cpp/.h # User-remappable button abstraction over HalGPIO
├── Battery.h # Global BatteryMonitor instance
├── fontIds.h # Numeric font family identifiers
├── SettingsList.h # Setting definitions and enums
├── RecentBooksStore.cpp/.h # Recently opened books persistence
├── WifiCredentialStore.cpp/.h # Saved WiFi network credentials
│
├── activities/ # UI screens (Activity pattern, see below)
├── components/ # Shared UI components and themes
├── network/ # Web server, HTTP, OTA updater
├── images/ # Logo assets (SVG, PNG, compiled header)
└── util/ # String and URL utility functions
```
### `src/activities/` -- UI Activity System
Each screen is an `Activity` subclass with `onEnter()`, `onExit()`, and `loop()` lifecycle methods. Activities are swapped by the routing functions in `main.cpp`.
```
activities/
├── Activity.h # Base Activity interface
├── ActivityWithSubactivity.cpp/.h # Composition: activity that hosts a child
│
├── boot_sleep/
│ ├── BootActivity.cpp/.h # Boot splash screen
│ └── SleepActivity.cpp/.h # Sleep screen (cover image, clock, etc.)
│
├── browser/
│ └── OpdsBookBrowserActivity.cpp/.h # OPDS catalog browsing and download
│
├── home/
│ ├── HomeActivity.cpp/.h # Main menu / home screen
│ ├── MyLibraryActivity.cpp/.h # File browser for books on SD card
│ └── RecentBooksActivity.cpp/.h # Recently opened books list
│
├── network/
│ ├── CrossPointWebServerActivity.cpp/.h # Web server file transfer mode
│ ├── CalibreConnectActivity.cpp/.h # Calibre wireless device connection
│ ├── NetworkModeSelectionActivity.cpp/.h # WiFi/AP/Calibre mode picker
│ └── WifiSelectionActivity.cpp/.h # WiFi network scanner and connector
│
├── reader/
│ ├── ReaderActivity.cpp/.h # Format dispatcher (EPUB, TXT, XTC)
│ ├── EpubReaderActivity.cpp/.h # EPUB reading engine
│ ├── EpubReaderMenuActivity.cpp/.h # In-reader menu overlay
│ ├── EpubReaderChapterSelectionActivity.cpp/.h # Table of contents navigation
│ ├── EpubReaderPercentSelectionActivity.cpp/.h # Jump-to-percentage navigation
│ ├── TxtReaderActivity.cpp/.h # Plain text reader
│ ├── XtcReaderActivity.cpp/.h # XTC format reader
│ ├── XtcReaderChapterSelectionActivity.cpp/.h # XTC chapter navigation
│ └── KOReaderSyncActivity.cpp/.h # KOReader reading progress sync
│
├── settings/
│ ├── SettingsActivity.cpp/.h # Main settings screen
│ ├── ButtonRemapActivity.cpp/.h # Front button remapping UI
│ ├── CalibreSettingsActivity.cpp/.h # Calibre server configuration
│ ├── ClearCacheActivity.cpp/.h # Cache clearing utility
│ ├── KOReaderAuthActivity.cpp/.h # KOReader auth credential entry
│ ├── KOReaderSettingsActivity.cpp/.h # KOReader sync settings
│ └── OtaUpdateActivity.cpp/.h # Over-the-air firmware update UI
│
└── util/
├── FullScreenMessageActivity.cpp/.h # Full-screen status/error messages
└── KeyboardEntryActivity.cpp/.h # On-screen keyboard for text input
```
### `src/components/` -- UI Components and Themes
```
components/
├── UITheme.cpp/.h # Theme manager singleton
└── themes/
├── BaseTheme.cpp/.h # Default theme implementation
└── lyra/
└── LyraTheme.cpp/.h # Alternative "Lyra" theme
```
### `src/network/` -- Networking
```
network/
├── CrossPointWebServer.cpp/.h # HTTP web server (file management, upload, settings)
├── HttpDownloader.cpp/.h # HTTP/HTTPS file downloader
├── OtaUpdater.cpp/.h # OTA firmware update client
└── html/
├── HomePage.html # Web UI home page
├── FilesPage.html # Web UI file browser / upload page
└── SettingsPage.html # Web UI settings page
```
HTML files are minified at build time by `scripts/build_html.py` into `PROGMEM` C++ headers (`.generated.h`).
### `src/util/` -- Utilities
```
util/
├── StringUtils.cpp/.h # String manipulation helpers
└── UrlUtils.cpp/.h # URL encoding/decoding
```
## `lib/` -- Project Libraries
PlatformIO-managed libraries local to the project.
### `lib/hal/` -- Hardware Abstraction Layer
The HAL wraps the SDK drivers behind a stable API the rest of the firmware consumes.
| File | Purpose |
|---|---|
| `HalDisplay.cpp/.h` | E-ink display control (refresh, buffer ops, grayscale, sleep) |
| `HalGPIO.cpp/.h` | Button input, battery reading, USB detection, deep sleep, wakeup |
| `HalStorage.cpp/.h` | SD card file operations (`Storage` singleton) |
### `lib/Epub/` -- EPUB Library
The core document parsing and rendering engine.
```
Epub/
├── Epub.cpp/.h # Top-level EPUB interface
├── Epub/
│ ├── Page.cpp/.h # Single rendered page
│ ├── Section.cpp/.h # Book section (chapter)
│ ├── ParsedText.cpp/.h # Parsed text representation
│ ├── BookMetadataCache.cpp/.h # Metadata caching to SD card
│ │
│ ├── blocks/
│ │ ├── Block.h # Base block type
│ │ ├── BlockStyle.h # Block styling
│ │ └── TextBlock.cpp/.h # Text block rendering
│ │
│ ├── css/
│ │ ├── CssParser.cpp/.h # CSS subset parser
│ │ └── CssStyle.h # Parsed CSS style representation
│ │
│ ├── hyphenation/
│ │ ├── Hyphenator.cpp/.h # Main hyphenation API
│ │ ├── LiangHyphenation.cpp/.h # Liang algorithm implementation
│ │ ├── HyphenationCommon.cpp/.h # Shared types
│ │ ├── LanguageHyphenator.h # Per-language interface
│ │ ├── LanguageRegistry.cpp/.h # Language detection and routing
│ │ ├── SerializedHyphenationTrie.h # Trie data format
│ │ └── generated/ # Pre-built hyphenation tries
│ │ ├── hyph-de.trie.h # German
│ │ ├── hyph-en.trie.h # English
│ │ ├── hyph-es.trie.h # Spanish
│ │ ├── hyph-fr.trie.h # French
│ │ └── hyph-ru.trie.h # Russian
│ │
│ └── parsers/
│ ├── ChapterHtmlSlimParser.cpp/.h # Chapter HTML content parser
│ ├── ContainerParser.cpp/.h # META-INF/container.xml parser
│ ├── ContentOpfParser.cpp/.h # content.opf (spine, manifest) parser
│ ├── TocNavParser.cpp/.h # EPUB3 nav TOC parser
│ └── TocNcxParser.cpp/.h # EPUB2 NCX TOC parser
```
### `lib/EpdFont/` -- Font Rendering
```
EpdFont/
├── EpdFont.cpp/.h # Font rendering engine
├── EpdFontData.h # Raw font data structure
├── EpdFontFamily.cpp/.h # Font family (regular, bold, italic, bold-italic)
├── builtinFonts/
│ ├── all.h # Aggregate include for all built-in fonts
│ ├── bookerly_*.h # Bookerly at sizes 12, 14, 16, 18
│ ├── notosans_*.h # Noto Sans at sizes 8, 12, 14, 16, 18
│ ├── opendyslexic_*.h # OpenDyslexic at sizes 8, 10, 12, 14
│ ├── ubuntu_*.h # Ubuntu at sizes 10, 12 (UI font)
│ └── source/ # Original TTF/OTF font files
└── scripts/
├── fontconvert.py # TTF/OTF to compiled header converter
├── convert-builtin-fonts.sh # Batch conversion script
├── build-font-ids.sh # Generate fontIds.h
└── requirements.txt # Python dependencies for font tools
```
### Other Libraries
| Library | Path | Purpose |
|---|---|---|
| `expat/` | `lib/expat/` | XML parser (used by EPUB parsers) |
| `miniz/` | `lib/miniz/` | Compression/decompression (ZIP support) |
| `ZipFile/` | `lib/ZipFile/` | ZIP file reading (EPUB is a ZIP) |
| `picojpeg/` | `lib/picojpeg/` | Lightweight JPEG decoder |
| `JpegToBmpConverter/` | `lib/JpegToBmpConverter/` | JPEG to bitmap conversion for display |
| `GfxRenderer/` | `lib/GfxRenderer/` | Graphics renderer, bitmap helpers |
| `Txt/` | `lib/Txt/` | Plain text (.txt) file reader |
| `Xtc/` | `lib/Xtc/` | XTC format parser and types |
| `OpdsParser/` | `lib/OpdsParser/` | OPDS feed XML parser and streaming |
| `KOReaderSync/` | `lib/KOReaderSync/` | KOReader sync client, credential store, progress mapping |
| `Utf8/` | `lib/Utf8/` | UTF-8 string utilities |
| `Serialization/` | `lib/Serialization/` | Binary serialization helper |
| `FsHelpers/` | `lib/FsHelpers/` | File system utility functions |
## `open-x4-sdk/` -- Hardware SDK (Git Submodule)
Community SDK providing low-level hardware drivers. Referenced in `.gitmodules` from `https://github.com/open-x4-epaper/community-sdk.git`.
Supplies four libraries linked as symlinks in `platformio.ini`:
| Library | SDK Path | Purpose |
|---|---|---|
| `BatteryMonitor` | `libs/hardware/BatteryMonitor` | Battery voltage ADC reading |
| `InputManager` | `libs/hardware/InputManager` | Physical button GPIO management |
| `EInkDisplay` | `libs/display/EInkDisplay` | E-ink display SPI driver |
| `SDCardManager` | `libs/hardware/SDCardManager` | SD card SPI initialization |
## `scripts/` -- Build and Utility Scripts
| Script | Language | Purpose |
|---|---|---|
| `build_html.py` | Python | **Pre-build step.** Minifies HTML files in `src/` into `PROGMEM` C++ headers (`.generated.h`). Strips comments, collapses whitespace, preserves `<pre>`, `<code>`, `<script>`, `<style>` content. |
| `generate_hyphenation_trie.py` | Python | Generates serialized hyphenation trie headers from TeX hyphenation patterns. |
| `debugging_monitor.py` | Python | Enhanced serial monitor for debugging output. |
## `test/` -- Tests
```
test/
├── README # PlatformIO test directory info
├── run_hyphenation_eval.sh # Test runner script
└── hyphenation_eval/
├── HyphenationEvaluationTest.cpp # Hyphenation accuracy evaluation
└── resources/
├── english_hyphenation_tests.txt
├── french_hyphenation_tests.txt
├── german_hyphenation_tests.txt
├── russian_hyphenation_tests.txt
├── spanish_hyphenation_tests.txt
└── generate_hyphenation_test_data.py
```
## `docs/` -- Upstream Documentation
| File | Topic |
|---|---|
| `comparison.md` | Feature comparison with stock firmware |
| `file-formats.md` | Supported document formats |
| `hyphenation-trie-format.md` | Hyphenation trie binary format specification |
| `troubleshooting.md` | Common issues and fixes |
| `webserver.md` | Web server feature guide |
| `webserver-endpoints.md` | Web server HTTP API reference |
| `images/` | Screenshots and cover image |
## `.github/` -- GitHub Configuration
```
.github/
├── workflows/
│ ├── ci.yml # Main CI: format check, cppcheck, build
│ ├── release.yml # Tag-triggered release build
│ ├── release_candidate.yml # Manual RC build (release/* branches)
│ └── pr-formatting-check.yml # Semantic PR title validation
├── PULL_REQUEST_TEMPLATE.md # PR template (summary, context, AI usage)
├── ISSUE_TEMPLATE/
│ └── bug_report.yml # Bug report form
└── FUNDING.yml # GitHub Sponsors configuration
```
## `bin/` -- Developer Tools
| Script | Purpose |
|---|---|
| `clang-format-fix` | Runs `clang-format` on all tracked `.c`, `.cpp`, `.h`, `.hpp` files (excluding generated font headers). Pass `-g` to format only modified files. |

174
mod/docs/hardware.md Normal file
View File

@@ -0,0 +1,174 @@
# Xteink X4 Hardware Capabilities
This document describes the hardware present on the Xteink X4 e-ink device and what the CrossPoint Reader firmware can leverage.
## CPU / Microcontroller
- **MCU:** ESP32-C3 (RISC-V, single-core)
- **Usable RAM:** ~380 KB
- **Architecture:** Single-core -- background tasks (WiFi, etc.) compete with the main application loop for CPU time.
> Source: `platformio.ini` (`board = esp32-c3-devkitm-1`), `README.md`
## Flash Memory
| Region | Offset | Size | Purpose |
|---|---|---|---|
| NVS | `0x9000` | 20 KB | Non-volatile key/value storage (settings, credentials) |
| OTA Data | `0xE000` | 8 KB | OTA boot-selection metadata |
| App Partition 0 (ota_0) | `0x10000` | 6.25 MB | Primary firmware slot |
| App Partition 1 (ota_1) | `0x650000` | 6.25 MB | Secondary firmware slot (OTA target) |
| SPIFFS | `0xC90000` | 3.375 MB | On-flash file system |
| Core Dump | `0xFF0000` | 64 KB | Crash dump storage |
- **Total Flash:** 16 MB
- **Flash Mode:** DIO (Dual I/O)
- **Dual OTA partitions** enable over-the-air firmware updates: one slot runs the active firmware while the other receives the new image.
> Source: `partitions.csv`, `platformio.ini`
## Display
- **Type:** E-ink (E-paper)
- **Resolution:** 480 x 800 pixels (portrait orientation)
- **Color Depth:** 2-bit grayscale (4 levels: white, light gray, dark gray, black)
- **Frame Buffer Size:** 48,000 bytes (480/8 x 800), single-buffer mode
### Refresh Modes
| Mode | Description |
|---|---|
| `FULL_REFRESH` | Complete waveform -- best quality, slowest |
| `HALF_REFRESH` | Balanced quality/speed (~1720 ms) |
| `FAST_REFRESH` | Custom LUT -- fastest, used as the default |
### Grayscale Support
The display driver exposes separate LSB and MSB grayscale buffers (`copyGrayscaleBuffers`, `copyGrayscaleLsbBuffers`, `copyGrayscaleMsbBuffers`) for 2-bit (4-level) grayscale rendering.
### SPI Pin Mapping
| Signal | GPIO | Description |
|---|---|---|
| `EPD_SCLK` | 8 | SPI Clock |
| `EPD_MOSI` | 10 | SPI MOSI (Master Out Slave In) |
| `EPD_CS` | 21 | Chip Select |
| `EPD_DC` | 4 | Data/Command |
| `EPD_RST` | 5 | Reset |
| `EPD_BUSY` | 6 | Busy signal |
| `SPI_MISO` | 7 | SPI MISO (shared with SD card) |
> Source: `lib/hal/HalDisplay.h`, `lib/hal/HalGPIO.h`
## Buttons / Input
The device has **7 physical buttons**:
| Index | Constant | Location | Notes |
|---|---|---|---|
| 0 | `BTN_BACK` | Front | Remappable |
| 1 | `BTN_CONFIRM` | Front | Remappable |
| 2 | `BTN_LEFT` | Front | Page navigation, Remappable |
| 3 | `BTN_RIGHT` | Front | Page navigation, Remappable |
| 4 | `BTN_UP` | Side | Page navigation, Remappable |
| 5 | `BTN_DOWN` | Side | Page navigation, Remappable |
| 6 | `BTN_POWER` | Side | Wakes from deep sleep |
### Input Features
- Press, release, and hold-time detection via `InputManager` (SDK) and `HalGPIO`.
- The 4 front buttons and 2 side buttons are user-remappable through `MappedInputManager`.
- `MappedInputManager` adds logical button aliases (`PageBack`, `PageForward`) and a label-mapping system for UI hints.
- Side button layout can be swapped for left-handed use.
- Configurable power button hold duration for sleep/wake.
> Source: `lib/hal/HalGPIO.h`, `src/MappedInputManager.h`
## Storage (SD Card)
- **Interface:** SPI (shares the SPI bus with the display; `SPI_MISO` on GPIO 7 is common)
- **File System Library:** SdFat with UTF-8 long filename support (`USE_UTF8_LONG_NAMES=1`)
- **Driver:** `SDCardManager` from `open-x4-sdk`
- **Abstraction:** `HalStorage` provides a singleton (`Storage`) for all file operations -- listing, reading, writing, streaming, and directory management.
- **Usage:** EPUB storage, metadata caching to SD, settings persistence, book progress, and file transfer via web server.
> Source: `lib/hal/HalStorage.h`, `platformio.ini` build flags
## Battery
- **Monitoring Pin:** GPIO 0 (`BAT_GPIO0`)
- **Library:** `BatteryMonitor` from `open-x4-sdk`
- **Output:** Battery percentage (0-100%)
- **Access:** Via `HalGPIO::getBatteryPercentage()` or the global `battery` instance in `Battery.h`.
> Source: `src/Battery.h`, `lib/hal/HalGPIO.h`
## WiFi / Networking
- **Radio:** Built-in ESP32-C3 WiFi (802.11 b/g/n, 2.4 GHz)
- **Supported Modes:**
- **Station (STA):** Connect to an existing wireless network.
- **Access Point (AP):** Create a local hotspot for direct device connection.
### Network Features
| Feature | Description |
|---|---|
| Web Server | File management UI, book upload/download (`CrossPointWebServer`) |
| OTA Updates | HTTPS firmware download and flash (`OtaUpdater`) |
| WebSocket | Real-time communication with the web UI (`WebSockets @ 2.7.3`) |
| Calibre Connect | Direct wireless connection to Calibre library software |
| KOReader Sync | Reading progress sync with KOReader-compatible servers |
| OPDS Browser | Browse and download books from OPDS catalog feeds |
### Power Considerations
WiFi is power-hungry on the single-core ESP32-C3. The firmware disables WiFi sleep during active transfers and yields CPU time to the network stack when a web server activity is running (`skipLoopDelay()`).
> Source: `src/network/`, `src/activities/network/`, `platformio.ini` lib_deps
## USB
- **Connector:** USB-C
- **Connection Detection:** `UART0_RXD` (GPIO 20) reads HIGH when USB is connected.
- **Firmware Upload Speed:** 921,600 baud
- **Serial Monitor:** 115,200 baud (only initialized when USB is detected)
- **CDC on Boot:** Enabled (`ARDUINO_USB_CDC_ON_BOOT=1`)
> Source: `lib/hal/HalGPIO.h`, `platformio.ini`
## Power Management
### Deep Sleep
- Entered via `HalGPIO::startDeepSleep()` after a configurable inactivity timeout or power button hold.
- Wakeup is triggered by the power button GPIO.
- Before sleeping, application state (open book, reader position) is persisted to SD card.
### Wakeup Reasons
The firmware distinguishes between wakeup causes to decide boot behavior:
| Reason | Behavior |
|---|---|
| `PowerButton` | Normal boot -- verifies hold duration before proceeding |
| `AfterFlash` | Post-flash boot -- proceeds directly |
| `AfterUSBPower` | USB plug caused cold boot -- returns to sleep immediately |
| `Other` | Fallback -- proceeds to boot |
> Source: `lib/hal/HalGPIO.h`, `src/main.cpp`
## SDK Hardware Libraries
The `open-x4-sdk` git submodule (community SDK) provides the low-level drivers that the HAL wraps:
| Library | Path in SDK | Purpose |
|---|---|---|
| `BatteryMonitor` | `libs/hardware/BatteryMonitor` | Battery voltage reading |
| `InputManager` | `libs/hardware/InputManager` | Physical button input |
| `EInkDisplay` | `libs/display/EInkDisplay` | E-ink display driver |
| `SDCardManager` | `libs/hardware/SDCardManager` | SD card SPI management |
These are linked into the build as symlinks via `platformio.ini` `lib_deps`.
> Source: `.gitmodules`, `platformio.ini`

216
mod/docs/project-summary.md Normal file
View File

@@ -0,0 +1,216 @@
# CrossPoint Reader -- Project Summary
> This document is a condensed reference for quickly re-familiarizing with the CrossPoint Reader firmware project. It consolidates hardware specs, codebase layout, build/CI details, architecture patterns, and current mod work. For full detail see `hardware.md`, `file-structure.md`, and `ci-build-and-code-style.md` in this directory.
## What This Is
CrossPoint Reader is an open-source, community-driven firmware replacement for the **Xteink X4** e-paper reader. It is **not affiliated with Xteink**. The project's sole mission is focused, high-quality ebook reading -- no apps, games, browsers, or audio. See `SCOPE.md` for the full boundary definition.
Repository: `https://github.com/crosspoint-reader/crosspoint-reader`
License: MIT
Governance: `GOVERNANCE.md` (critique code not people, public-by-default decisions, `@daveallie` for moderation)
## Hardware at a Glance
| Component | Spec |
|---|---|
| MCU | ESP32-C3, RISC-V, **single-core** |
| RAM | ~380 KB usable (aggressive SD caching required) |
| Flash | 16 MB total, DIO mode |
| Display | 480x800 e-ink, 2-bit grayscale (4 levels), SPI |
| Refresh | FULL (best quality), HALF (~1720ms), FAST (default, custom LUT) |
| Frame buffer | 48,000 bytes, single-buffer mode |
| Buttons | 7 total: 4 front (remappable), 2 side (page nav), 1 power (deep-sleep wake) |
| Storage | SD card via SPI (shared bus with display), SdFat, UTF-8 long filenames |
| Battery | GPIO 0, BatteryMonitor SDK lib, 0-100% |
| WiFi | 802.11 b/g/n 2.4GHz, STA + AP modes |
| USB | USB-C, CDC on boot, connection detection via GPIO 20 |
| Power | Deep sleep with GPIO wakeup, configurable inactivity timeout |
### Flash Partition Layout
```
NVS 0x9000 20KB Key/value storage
OTA Data 0xE000 8KB Boot selection
ota_0 0x10000 6.25MB Primary firmware
ota_1 0x650000 6.25MB Secondary firmware (OTA)
SPIFFS 0xC90000 3.375MB On-flash filesystem
Coredump 0xFF0000 64KB Crash dumps
```
### Key GPIO Assignments
Display SPI: SCLK=8, MOSI=10, CS=21, DC=4, RST=5, BUSY=6, MISO=7 (shared with SD)
Battery: GPIO 0 | USB detect: GPIO 20
## Architecture Overview
### Application Loop
Standard Arduino `setup()` / `loop()` in `src/main.cpp`. The loop polls buttons via `HalGPIO`, delegates to the current `Activity`, handles auto-sleep timeout, and power button hold detection. Memory stats are logged every 10s over serial (when USB connected).
### Activity System
UI screens are `Activity` subclasses with lifecycle methods `onEnter()`, `onExit()`, `loop()`. Only one activity is active at a time. Routing functions in `main.cpp` (`onGoHome()`, `onGoToReader()`, etc.) manage transitions by deleting the old activity and creating the new one.
`ActivityWithSubactivity` provides composition for activities that host child activities (menus, overlays).
### Hardware Abstraction Layer (lib/hal/)
Three classes wrap the SDK drivers:
- `HalDisplay` -- display refresh, buffer ops, grayscale, deep sleep
- `HalGPIO` -- buttons, battery, USB detection, deep sleep, wakeup reason
- `HalStorage` -- SD card file ops via singleton `Storage`
### Rendering Pipeline
`GfxRenderer` wraps `HalDisplay` for drawing operations. Fonts are registered by ID at startup from compiled headers in `lib/EpdFont/builtinFonts/`. The renderer supports a `fadingFix` setting.
### Data Caching
EPUB data is aggressively cached to SD under `.crosspoint/epub_<hash>/` to conserve RAM:
- `progress.bin` -- reading position
- `cover.bmp` -- extracted cover image
- `book.bin` -- metadata (title, author, spine, TOC)
- `sections/*.bin` -- pre-parsed chapter layout data
Cache is NOT auto-cleaned on book deletion. Moving a book file resets its cache/progress.
## Key Source Locations
### Where to find things
| What | Where |
|---|---|
| Entry point | `src/main.cpp` -- `setup()`, `loop()`, activity routing, font init |
| User settings | `src/CrossPointSettings.cpp/.h` -- load/save to SD |
| App state | `src/CrossPointState.cpp/.h` -- open book path, sleep tracking |
| Button mapping | `src/MappedInputManager.cpp/.h` -- remappable buttons, labels |
| Boot screen | `src/activities/boot_sleep/BootActivity.cpp/.h` |
| Sleep screen | `src/activities/boot_sleep/SleepActivity.cpp/.h` |
| Home screen | `src/activities/home/HomeActivity.cpp/.h` |
| Library browser | `src/activities/home/MyLibraryActivity.cpp/.h` |
| EPUB reader | `src/activities/reader/EpubReaderActivity.cpp/.h` |
| Reader menu | `src/activities/reader/EpubReaderMenuActivity.cpp/.h` |
| TXT reader | `src/activities/reader/TxtReaderActivity.cpp/.h` |
| XTC reader | `src/activities/reader/XtcReaderActivity.cpp/.h` |
| Settings | `src/activities/settings/SettingsActivity.cpp/.h` |
| Button remap | `src/activities/settings/ButtonRemapActivity.cpp/.h` |
| OTA update UI | `src/activities/settings/OtaUpdateActivity.cpp/.h` |
| Web server | `src/network/CrossPointWebServer.cpp/.h` |
| OTA client | `src/network/OtaUpdater.cpp/.h` |
| HTTP download | `src/network/HttpDownloader.cpp/.h` |
| Web UI HTML | `src/network/html/{HomePage,FilesPage,SettingsPage}.html` |
| Themes | `src/components/themes/BaseTheme.cpp/.h`, `lyra/LyraTheme.cpp/.h` |
| Theme manager | `src/components/UITheme.cpp/.h` |
### Library code (lib/)
| What | Where |
|---|---|
| EPUB engine | `lib/Epub/` -- parsing, rendering, sections, pages |
| EPUB parsers | `lib/Epub/Epub/parsers/` -- container, OPF, HTML, TOC (nav+ncx) |
| CSS parser | `lib/Epub/Epub/css/CssParser.cpp/.h` |
| Hyphenation | `lib/Epub/Epub/hyphenation/` -- Liang algorithm, 5 language tries |
| Display HAL | `lib/hal/HalDisplay.cpp/.h` |
| GPIO HAL | `lib/hal/HalGPIO.cpp/.h` |
| Storage HAL | `lib/hal/HalStorage.cpp/.h` |
| Font rendering | `lib/EpdFont/EpdFont.cpp/.h` |
| Built-in fonts | `lib/EpdFont/builtinFonts/` -- Bookerly, NotoSans, OpenDyslexic, Ubuntu |
| Graphics | `lib/GfxRenderer/` -- renderer, bitmap, bitmap helpers |
| XML parsing | `lib/expat/` |
| ZIP/compression | `lib/miniz/`, `lib/ZipFile/` |
| JPEG decoding | `lib/picojpeg/`, `lib/JpegToBmpConverter/` |
| OPDS parsing | `lib/OpdsParser/` |
| KOReader sync | `lib/KOReaderSync/` |
| UTF-8 utils | `lib/Utf8/` |
### SDK (open-x4-sdk/ submodule)
Provides: `BatteryMonitor`, `InputManager`, `EInkDisplay`, `SDCardManager`
Linked as symlinks in `platformio.ini` lib_deps.
## Build & CI Quick Reference
### Build locally
```bash
pio run # Dev build (version: 1.0.0-dev)
pio run -e gh_release # Release build (version: 1.0.0)
pio run --target upload # Build and flash via USB
pio check # Run cppcheck static analysis
```
### Code formatting
```bash
./bin/clang-format-fix # Format all tracked C/C++ files
./bin/clang-format-fix -g # Format only modified files
```
Key style rules: 2-space indent, 120-char column limit, K&R braces (Attach), pointer-left (`int* x`), sorted/regrouped includes, spaces not tabs, LF line endings.
Excludes `lib/EpdFont/builtinFonts/` (generated).
### CI (GitHub Actions)
On push to `master` or any PR, 4 parallel jobs run:
1. **clang-format** -- formatting check (clang-format-21)
2. **cppcheck** -- static analysis (fails on low/medium/high)
3. **build** -- compilation + firmware.bin artifact
4. **test-status** -- aggregated PR gate
PR titles must follow semantic format (`feat:`, `fix:`, `refactor:`, etc.).
### Releases
- **Official:** Push a git tag -> `release.yml` builds `gh_release` env -> uploads bootloader, firmware, ELF, map, partitions
- **RC:** Manually trigger `release_candidate.yml` on a `release/*` branch -> embeds commit SHA in version
### Version scheme
```
1.0.0-dev # default env (local dev)
1.0.0 # gh_release env (tagged release)
1.0.0-rc+a1b2c3d # gh_release_rc env (release candidate)
```
### Dependencies
| Dependency | Version | Purpose |
|---|---|---|
| espressif32 | 6.12.0 | PlatformIO platform |
| ArduinoJson | 7.4.2 | JSON parsing |
| QRCode | 0.0.1 | QR code generation |
| WebSockets | 2.7.3 | WebSocket server |
### Pre-build step
`scripts/build_html.py` minifies `src/network/html/*.html` -> `.generated.h` PROGMEM strings.
### Debugging
```bash
python3 scripts/debugging_monitor.py # Linux
python3 scripts/debugging_monitor.py /dev/cu.usbmodem2101 # macOS
```
Requires: `pyserial`, `colorama`, `matplotlib`.
## Supported Document Formats
- **EPUB** (primary) -- EPUB 2 and EPUB 3, CSS subset, hyphenation (EN, DE, ES, FR, RU)
- **TXT** -- plain text
- **XTC** -- proprietary format with chapter support
- Image support within EPUB is listed as incomplete (per README checklist)
## Constraints to Remember
- **~380KB RAM** -- everything large must be cached to SD card
- **Single-core CPU** -- WiFi and main loop compete for cycles; no true background tasks
- **6.25MB firmware slot** -- maximum firmware binary size
- **48KB frame buffer** -- 1-bit per pixel in normal mode, 2-bit for grayscale
- **Shared SPI bus** -- display and SD card share MISO; cannot access simultaneously
- **No PSRAM** -- all memory is internal SRAM
- **Font headers are generated** -- do not hand-edit `lib/EpdFont/builtinFonts/` (excluded from formatting)
- **HTML headers are generated** -- `.generated.h` files in `src/network/html/` are auto-built from `.html` files

123
mod/docs/session-summary.md Normal file
View File

@@ -0,0 +1,123 @@
# Session Context Dump (2026-02-09)
> Everything below is a snapshot of the conversation context at the time of writing. It captures what was explored, what was read, what was produced, and what the current state of work is. Use this to resume without re-exploring.
## Branch State
- **Current branch:** `mod/sleep-screen-tweaks`
- **Git status:** `.gitignore` modified (unstaged). No other tracked changes. All `mod/docs/` files are new/untracked (the `mod/` directory is gitignored).
## What Was Done This Session
1. **Explored the entire project structure** using parallel agents. Mapped every directory, identified all source files, read key headers and configs.
2. **Created 3 documentation files in `mod/docs/`:**
- `hardware.md` -- Full hardware capabilities (CPU, flash partitions, display specs with GPIO pins, buttons with indices, SD card, battery, WiFi, USB, power management, SDK libraries)
- `file-structure.md` -- Complete directory tree with descriptions of every file group (src/ activities, components, network; lib/ HAL, EPUB engine, fonts, supporting libs; scripts, tests, docs, .github CI)
- `ci-build-and-code-style.md` -- Build system (3 PlatformIO environments), all 4 CI workflows with triggers and steps, release process, clang-format rules table, cppcheck config, PR requirements, contribution guidelines
3. **Created `project-summary.md`** -- condensed reference consolidating all three docs plus architecture patterns, caching behavior, and constraints.
## Files Read During Exploration
The following files were read in full and their contents informed the documentation:
**Config/root:**
- `platformio.ini` -- all build environments, flags, dependencies, SDK symlinks
- `partitions.csv` -- flash partition table (6 rows)
- `.clang-format` -- 332 lines of formatting rules
- `.clangd` -- C++2a standard setting
- `.gitmodules` -- open-x4-sdk submodule reference
- `README.md` -- project overview, features checklist, install instructions, internals (caching), contributing
- `SCOPE.md` -- in-scope/out-of-scope feature boundaries
- `GOVERNANCE.md` -- community principles, moderation
**HAL layer:**
- `lib/hal/HalDisplay.h` -- display constants (480x800, buffer size 48000), refresh modes enum, grayscale buffer methods
- `lib/hal/HalGPIO.h` -- GPIO pin defines (EPD_SCLK=8, EPD_MOSI=10, EPD_CS=21, EPD_DC=4, EPD_RST=5, EPD_BUSY=6, SPI_MISO=7, BAT_GPIO0=0, UART0_RXD=20), button indices (0-6), WakeupReason enum
- `lib/hal/HalStorage.h` -- Storage singleton, file ops API, SDCardManager wrapper
**Application:**
- `src/main.cpp` -- 413 lines: setup/loop, activity routing (onGoHome, onGoToReader, etc.), font initialization (Bookerly 12/14/16/18, NotoSans 12/14/16/18, OpenDyslexic 8/10/12/14, Ubuntu 10/12), power button verification, deep sleep entry, auto-sleep timeout, memory logging
- `src/MappedInputManager.h` -- Button enum (Back, Confirm, Left, Right, Up, Down, Power, PageBack, PageForward), Labels struct, mapLabels()
- `src/Battery.h` -- global `BatteryMonitor battery(BAT_GPIO0)` instance
**CI/GitHub:**
- `.github/workflows/ci.yml` -- 4 jobs (clang-format, cppcheck, build, test-status), triggers, steps
- `.github/workflows/release.yml` -- tag trigger, gh_release env, 5 artifact files
- `.github/workflows/release_candidate.yml` -- workflow_dispatch, release/* branch gate, SHORT_SHA extraction
- `.github/workflows/pr-formatting-check.yml` -- semantic PR title check
- `.github/PULL_REQUEST_TEMPLATE.md` -- summary, context, AI usage disclosure
**Scripts:**
- `bin/clang-format-fix` -- bash script, git ls-files, grep for C/C++ extensions, excludes builtinFonts/, -g flag for modified-only
**Directory listings obtained:**
- Root, `src/`, `src/Activities/` (full recursive), `lib/` (all subdirectories), `lib/Epub/` (full recursive), `docs/`, `scripts/`, `test/`
## Current Mod Work Context
From `mod/docs/todo.md`, two tasks are planned for the sleep screen on branch `mod/sleep-screen-tweaks`:
**Task 1 -- Gradient fill for letterboxed images:**
When a sleep screen image doesn't match the 480x800 display aspect ratio, void/letterbox areas should be filled with a gradient sampled from the nearest ~20 pixels of the image edge. Relevant files:
- `src/activities/boot_sleep/SleepActivity.cpp/.h` -- sleep screen rendering logic
- `lib/GfxRenderer/BitmapHelpers.cpp/.h` -- bitmap manipulation utilities
- `lib/GfxRenderer/Bitmap.cpp/.h` -- bitmap data structure
- `lib/hal/HalDisplay.h` -- display dimensions (480x800), buffer operations
**Task 2 -- Fix "Fit" mode for small images:**
In "Fit" mode, images smaller than the display should be scaled UP to fit (maintaining aspect ratio). The current implementation only scales down larger images. Same relevant files as Task 1.
Neither task has been started yet -- only documentation/exploration was done this session.
## Key Architectural Details Observed in Source
**Activity lifecycle (from main.cpp):**
- `exitActivity()` calls `onExit()` then `delete` on the current activity
- `enterNewActivity()` sets `currentActivity` and calls `onEnter()`
- Activities receive `GfxRenderer&` and `MappedInputManager&` in constructors
- `ReaderActivity` also receives the epub path and callback functions for navigation
**Boot sequence (from main.cpp setup()):**
1. `gpio.begin()` -- init GPIO/SPI
2. Serial init only if USB connected
3. `Storage.begin()` -- SD card init (shows error screen on failure)
4. Load settings, KOReader store, UI theme
5. Check wakeup reason: PowerButton verifies hold duration, AfterUSBPower goes back to sleep, AfterFlash/Other proceed
6. Display + font setup
7. Show BootActivity (splash)
8. Load app state + recent books
9. Route to HomeActivity or ReaderActivity based on saved state
**Main loop (from main.cpp loop()):**
1. Poll GPIO
2. Apply fadingFix setting to renderer
3. Log memory every 10s (when serial active)
4. Track activity timer, auto-sleep on configurable timeout
5. Check power button hold for manual sleep
6. Run `currentActivity->loop()`
7. Delay 10ms normally, or `yield()` if activity requests fast response (e.g. web server)
**Settings system:**
- `CrossPointSettings` loaded from SD at boot via `SETTINGS` macro
- Includes: `shortPwrBtn` behavior, `getPowerButtonDuration()`, `getSleepTimeoutMs()`, `fadingFix`
- `CrossPointState` (`APP_STATE`) tracks: `openEpubPath`, `lastSleepFromReader`, `readerActivityLoadCount`
**Display details (from HalDisplay.h):**
- `DISPLAY_WIDTH` and `DISPLAY_HEIGHT` are `constexpr` sourced from `EInkDisplay`
- `DISPLAY_WIDTH_BYTES = DISPLAY_WIDTH / 8` (480/8 = 60)
- `BUFFER_SIZE = DISPLAY_WIDTH_BYTES * DISPLAY_HEIGHT` (60 * 800 = 48000)
- Grayscale uses separate LSB/MSB buffers
- `displayBuffer()` and `refreshDisplay()` take `RefreshMode` and optional `turnOffScreen`
- `drawImage()` supports PROGMEM source flag
## Companion Documentation Files
All in `mod/docs/`:
- `todo.md` -- active mod task list (sleep screen tweaks)
- `hardware.md` -- full hardware reference (175 lines)
- `file-structure.md` -- complete file tree with descriptions (307 lines)
- `ci-build-and-code-style.md` -- build/CI/style reference (281 lines)
- `project-summary.md` -- condensed project reference (217 lines)
- `session-summary.md` -- this file

16
mod/docs/todo.md Normal file
View File

@@ -0,0 +1,16 @@
- [X] Sleep screen tweaks
- [ ] Better home screen (covers/recents)
- [ ] Firmware flashing screen
- [X] Process/render all covers/thumbs when opening book for first time
- [ ] Companion Android app
- [ ] Smart fill letterbox
- [X] Bookmark skeleton
- [X] Dictionary skeleton
- [ ] Quick menu
- [X] Bookmarks
- [X] Dictionary
- [X] Rotate screen
- [ ] Archive/Delete book
Optional?
- [ ] Device bezel offsets

335
mod/docs/upstream-sync.md Normal file
View File

@@ -0,0 +1,335 @@
# Upstream Sync Guide
This document describes how to keep `mod/master` synchronized with `upstream/master` (the [crosspoint-reader/crosspoint-reader](https://github.com/crosspoint-reader/crosspoint-reader) repository) without creating duplicate code or divergence that becomes painful to resolve.
## Branch Model
```
upstream/master ──────●────●────●────●────●────●──── (upstream development)
│ │
│ │
mod/master ──────┴──M1──M2──M3───────────┴──M1'──M2'──M3'────
▲ ▲
mod features mod features
on old base re-applied on new base
```
- **`master`**: Mirror of `upstream/master`. Updated with `git fetch upstream && git merge upstream/master`. Never commit mod code here.
- **`mod/master`**: The mod branch. Based on `upstream/master` with mod-exclusive patches applied on top.
- **`mod/backup-*`**: Snapshot branches created before each major sync for safety.
## The Golden Rule
> **Never cherry-pick individual upstream PRs into `mod/master`.**
Instead, sync the full `upstream/master` branch and rebase/replay mod patches on top. Cherry-picking creates shadow copies of upstream commits that:
1. Produce false conflicts when the real PR is later merged upstream
2. Diverge silently if upstream amends the PR after merge (e.g., via a follow-up fix)
3. Make `git log --left-right` unreliable for tracking what's mod-only vs. upstream
4. Accumulate -- after 50 cherry-picks, you have 50 duplicate commits that must be identified and dropped during every future sync
If an upstream PR is not yet merged and you want its functionality, port the *feature* (adapted to the mod's codebase) as a mod-exclusive commit with a clear commit message referencing the upstream PR number. Do not copy the upstream commit verbatim. Example:
```
feat: port upstream word-width cache optimization
Adapted from upstream PR #1027 (not yet merged).
Re-implemented against mod/master's current text layout code.
If/when #1027 is merged upstream, this commit should be dropped
during the next sync and the upstream version used instead.
```
## Sync Procedure
### When to Sync
- **Weekly** during active upstream development periods
- **Immediately** after a major upstream refactor is merged (e.g., ActivityManager, settings migration)
- **Before** porting any new upstream PR -- sync first, then port against the latest base
### Step-by-Step
#### 1. Prepare
```bash
# Ensure clean working tree
git stash # if needed
# Fetch latest upstream
git fetch upstream
# Update the master mirror
git checkout master
git merge upstream/master # should always fast-forward
git push origin master
# Create a backup of current mod/master
git branch mod/backup-pre-sync-$(date +%Y-%m-%d) mod/master
```
#### 2. Identify What Changed
```bash
# How many new upstream commits since last sync?
git rev-list --count mod/master..upstream/master
# What are they?
git log --oneline mod/master..upstream/master
# Which mod commits are ahead of upstream?
git log --oneline upstream/master..mod/master
# Divergence summary
git rev-list --left-right --count mod/master...upstream/master
# Output: LEFT RIGHT (LEFT = mod-only commits, RIGHT = new upstream commits)
```
#### 3. Choose a Sync Strategy
**If divergence is small (< 20 new upstream commits, < 5 mod commits ahead):**
```bash
# Interactive rebase mod/master onto upstream/master
git checkout mod/master
git rebase -i upstream/master
# Resolve conflicts, drop any commits that are now in upstream
git push origin mod/master --force-with-lease
```
**If divergence is moderate (20-50 upstream commits, some conflicts expected):**
```bash
# Merge upstream into mod/master
git checkout mod/master
git merge upstream/master
# Resolve conflicts, favoring upstream for any cherry-picked PRs
git push origin mod/master
```
**If divergence is severe (50+ upstream commits, major refactors):**
Use the "fresh replay" approach. This is the nuclear option -- create a new branch from `upstream/master` and manually re-apply every mod feature. It produces the cleanest result but requires the most effort.
##### Fresh Replay Procedure
**a. Assessment**
Before starting, quantify the problem:
```bash
# Divergence stats
git rev-list --left-right --count mod/master...upstream/master
# Conflict preview (count conflict hunks without actually merging)
git merge-tree $(git merge-base mod/master upstream/master) mod/master upstream/master \
| grep -c "<<<<<<<"
# Files that both sides modified (highest conflict risk)
git merge-tree $(git merge-base mod/master upstream/master) mod/master upstream/master \
| grep "^changed in both" | wc -l
```
**b. Classify all mod commits**
Separate mod-only commits from upstream-ported commits. Each mod commit falls into one of three buckets:
1. **Upstream duplicate** -- cherry-picked/ported from a PR that is now merged in `upstream/master`. These are dropped entirely; the upstream version takes precedence.
2. **Upstream port (still unmerged)** -- ported from a PR that is still open/unmerged upstream. These must be re-applied, adapted to the new upstream code.
3. **Mod-exclusive** -- original mod features with no upstream equivalent. These must be re-applied.
```bash
# List all mod-only commits
git log --oneline upstream/master..mod/master
# Find commits referencing upstream PR numbers
git log --oneline upstream/master..mod/master | grep -iE "(#[0-9]+|port|upstream)"
# For each referenced PR, check if it's now in upstream
git log upstream/master --oneline | grep "#XXXX"
```
**c. Identify upstream architectural changes**
Check for major refactors that will affect how mod code must be re-applied. Common high-impact changes include:
- Activity system changes (e.g., ActivityManager migration -- see `docs/contributing/ActivityManager.md`)
- Settings format changes (e.g., binary to JSON migration)
- Theme system overhauls
- Font rendering pipeline changes
- Library replacements (e.g., image decoder swaps)
- Class/file renames
```bash
# See what upstream changed (summary)
git diff --stat $(git merge-base mod/master upstream/master)...upstream/master | tail -5
# Look for large refactors
git log --oneline upstream/master --not mod/master | grep -iE "(refactor|rename|migrate|replace|remove)"
```
**d. Create the replay branch**
```bash
# Backup current mod/master
git branch mod/backup-pre-sync-$(date +%Y-%m-%d) mod/master
# Create fresh branch from upstream/master
git checkout -b mod/master-resync upstream/master
```
**e. Replay in phases (low-risk to high-risk)**
Work through the mod features in order of conflict risk:
*Phase 1 -- New files (low risk):* Bring over mod-exclusive files that don't exist in upstream. These are purely additive and rarely conflict. Copy them from the backup branch:
```bash
# Example: bring over a new mod file
git checkout mod/backup-pre-sync-YYYY-MM-DD -- src/activities/settings/NtpSyncActivity.cpp
git checkout mod/backup-pre-sync-YYYY-MM-DD -- src/activities/settings/NtpSyncActivity.h
```
New files will need adaptation if they depend on APIs that changed upstream (e.g., Activity base class, settings system). Fix compilation errors as you go.
*Phase 2 -- Core file modifications (high risk):* Re-apply modifications to files that also changed upstream. Do NOT cherry-pick or copy entire files from the old mod branch -- instead, read the old mod's diff for each file and manually re-apply the mod's *intent* against the new upstream code. Key files (in typical priority order):
- `platformio.ini` -- mod build flags
- `src/main.cpp` -- mod activity registration
- Settings files -- mod settings in the new format
- Activity files modified by mod (HomeActivity, EpubReaderActivity, menus, etc.)
- Renderer and HAL files
- Theme files
```bash
# See what the mod changed in a specific file vs the merge-base
git diff $(git merge-base mod/master upstream/master)...mod/backup-pre-sync-YYYY-MM-DD \
-- src/activities/home/HomeActivity.cpp
# See what upstream changed in that same file
git diff $(git merge-base mod/master upstream/master)...upstream/master \
-- src/activities/home/HomeActivity.cpp
```
*Phase 3 -- Re-port unmerged upstream PRs:* For upstream PRs that were ported into the mod but aren't yet merged upstream, re-apply each one against the new codebase. Check the ported PR tracking table in `mod/prs/MERGED.md` for context on each port.
**f. Verify and finalize**
```bash
# Build
pio run
# Check for conflict markers
grep -r "<<<<<<" src/ lib/ --include="*.cpp" --include="*.h"
# Run clang-format
./bin/clang-format-fix
# Verify divergence: LEFT should be mod-only, RIGHT should be 0
git rev-list --left-right --count mod/master-resync...upstream/master
# When satisfied, update mod/master
git checkout mod/master
git reset --hard mod/master-resync
git push origin mod/master --force-with-lease
```
**g. Post-sync cleanup**
- Update `mod/prs/MERGED.md` -- remove entries for PRs now in upstream, update status for remaining ports
- Update mod README with new base commit
- Delete the backup branch after confirming everything works on-device
#### 4. Handle Previously-Ported Upstream PRs
During rebase or merge, you will encounter conflicts where the mod cherry-picked an upstream PR that has since been merged natively. Resolution:
- **If the upstream PR is now in `upstream/master`**: Drop the mod's cherry-pick commit entirely. The upstream version takes precedence because it may have been amended by follow-up commits.
- **If the upstream PR is still unmerged**: Keep the mod's port, but verify it still applies cleanly against the updated codebase.
To check which mod commits reference upstream PRs:
```bash
# List mod-only commits that reference upstream PR numbers
git log --oneline upstream/master..mod/master | grep -iE "(port|upstream|PR #)"
```
To check if a specific upstream PR has been merged:
```bash
# Check if PR #XXXX is in upstream/master
git log upstream/master --oneline | grep "#XXXX"
```
#### 5. Verify
```bash
# Build
pio run
# Check for leftover conflict markers
grep -r "<<<<<<" src/ lib/ --include="*.cpp" --include="*.h"
# Run clang-format
./bin/clang-format-fix
# Verify the divergence is what you expect
git rev-list --left-right --count mod/master...upstream/master
# LEFT should be only mod-exclusive commits, RIGHT should be 0
```
#### 6. Document
After a successful sync, update the mod README with the new base upstream commit and note any dropped or reworked ports.
## Commit Message Conventions for Mod Commits
Use these prefixes to make mod commits easy to identify and filter during future syncs:
- `mod:` -- Mod-specific feature or change (e.g., `mod: add clock settings tab`)
- `feat:` / `fix:` / `perf:` -- Standard prefixes, but for mod-original work
- `port:` -- Feature ported from an unmerged upstream PR (e.g., `port: upstream PR #1027 word-width cache`)
Always include in the commit body:
- Which upstream PR it's based on (if any)
- Whether it should be dropped when that PR merges upstream
## Tracking Ported PRs
Detailed documentation for each ported PR lives in [`mod/prs/MERGED.md`](../prs/MERGED.md). That file contains full context on what was changed, how it differs from the upstream PR, and notable discussion.
Additionally, keep the quick-reference status table below up to date during each sync. This table answers the question every sync needs answered: "which ports are still relevant?"
| Upstream PR | Description | Upstream Status | Competing/Related PRs | Action on Next Sync |
|---|---|---|---|---|
| #857 | Dictionary word lookup | OPEN | None | Keep until merged upstream |
| #1003 | Image placeholders during decode | OPEN | #1291 (MERGED, user image display setting) | Evaluate -- #1291 may cover this |
| #1019 | File extensions in file browser | OPEN | #1260 (MERGED, MyLibrary->FileBrowser rename) | Keep, adapt to rename |
| #1027 | Word-width cache + hyphenation early exit | OPEN | #1168 (MERGED, fixed-point layout), #873 (MERGED, kerning) | Needs complete rework against new text layout |
| #1038 | std::list to std::vector in text layout | MERGED | -- | DROP on next sync (now in upstream) |
| #1045 | Shorten "Forget Wifi" labels | MERGED | -- | DROP on next sync (now in upstream) |
| #1037 | Decomposed character hyphenation/rendering | MERGED | -- | DROP on next sync (now in upstream) |
| #1055 | Byte-level framebuffer writes | OPEN | #1141 (MERGED, wrapped text in GfxRender) | Keep, adapt to GfxRenderer changes |
| #1068 | URL hyphenation fix | OPEN | None | Keep until merged upstream |
| #1090 | KOReader push progress + sleep | OPEN | #946 (OPEN, sync streamlining) | Evaluate overlap with #946 |
| #1185 | KOReader document hash cache | OPEN | #1286 (OPEN, OPDS filename matching) | Keep until merged upstream |
| #1209 | Multiple OPDS servers | OPEN | #1214 (OPEN, author folders) | Keep until merged upstream |
| #1217 | KOReader sync improvements | OPEN | #946 (OPEN, sync streamlining) | Evaluate overlap with #946 |
*Last updated: 2026-03-07*
### How to update this table during a sync
1. For each row, check if the upstream PR has been merged: `git log upstream/master --oneline | grep "#XXXX"`
2. If merged: change Action to "DROP" and remove the port commit during the sync
3. If still open: check if competing/related PRs have merged that affect the port
4. After the sync: remove dropped rows, add any new ports, update the date
## What NOT to Do
1. **Don't cherry-pick upstream commits.** See golden rule above.
2. **Don't let mod/master fall more than ~2 weeks behind upstream/master.** The longer you wait, the harder the sync.
3. **Don't resolve conflicts by "taking ours" for upstream files without understanding why.** The upstream version is almost always correct for code that isn't mod-specific.
4. **Don't merge `mod/master` into `master`.** The `master` branch is a clean mirror of upstream.
5. **Don't port an upstream PR without first syncing.** You'll be porting against a stale base, making the next sync harder.
6. **Don't create mod feature branches off of `master`.** Always branch from `mod/master`.