cottongin/crosspoint-reader-mod
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

mod: Phase 1 - bring forward mod-exclusive files with ActivityManager migration

Browse Source 
Brings ~55 mod-exclusive files to the upstream-based mod/master-resync branch:

Activities (migrated to new ActivityManager pattern):
- Clock/Time: SetTimeActivity, SetTimezoneOffsetActivity, NtpSyncActivity
- Dictionary: DictionaryDefinitionActivity, DictionarySuggestionsActivity,
  DictionaryWordSelectActivity, LookedUpWordsActivity
- Bookmark: EpubReaderBookmarkSelectionActivity
- Book management: BookManageMenuActivity, EndOfBookMenuActivity
- OPDS: OpdsServerListActivity, OpdsSettingsActivity
- Utility: DirectoryPickerActivity, NumericStepperActivity

Utilities (unchanged):
- BookManager, BookSettings, BookmarkStore, BootNtpSync
- Dictionary, LookupHistory, TimeSync, OpdsServerStore

Libraries: PlaceholderCover, TableData, ChapterXPathIndexer
Scripts: inject_mod_version, generate_book_icon, preview_placeholder_cover
Docs: KOReader sync XPath mapping

Migration changes:
- ActivityWithSubactivity -> Activity base class
- Callback constructors -> finish()/setResult() pattern
- enterNewActivity() -> startActivityForResult()
- Activity::RenderLock&& -> RenderLock&&

These files won't compile yet - they reference mod settings and I18n
strings that will be added in subsequent phases.

Made-with: Cursor
This commit is contained in:
cottongin
2026-03-07 15:10:00 -05:00
parent 170cc25774
commit dfbc931c14
147 changed files with 112771 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

335
mod/docs/upstream-sync.md Normal file
View File

@@ -0,0 +1,335 @@
# Upstream Sync Guide
This document describes how to keep `mod/master` synchronized with `upstream/master` (the [crosspoint-reader/crosspoint-reader](https://github.com/crosspoint-reader/crosspoint-reader) repository) without creating duplicate code or divergence that becomes painful to resolve.
## Branch Model
```
upstream/master ──────●────●────●────●────●────●──── (upstream development)
│ │
│ │
mod/master ──────┴──M1──M2──M3───────────┴──M1'──M2'──M3'────
▲ ▲
mod features mod features
on old base re-applied on new base
```
- **`master`**: Mirror of `upstream/master`. Updated with `git fetch upstream && git merge upstream/master`. Never commit mod code here.
- **`mod/master`**: The mod branch. Based on `upstream/master` with mod-exclusive patches applied on top.
- **`mod/backup-*`**: Snapshot branches created before each major sync for safety.
## The Golden Rule
> **Never cherry-pick individual upstream PRs into `mod/master`.**
Instead, sync the full `upstream/master` branch and rebase/replay mod patches on top. Cherry-picking creates shadow copies of upstream commits that:
1. Produce false conflicts when the real PR is later merged upstream
2. Diverge silently if upstream amends the PR after merge (e.g., via a follow-up fix)
3. Make `git log --left-right` unreliable for tracking what's mod-only vs. upstream
4. Accumulate -- after 50 cherry-picks, you have 50 duplicate commits that must be identified and dropped during every future sync
If an upstream PR is not yet merged and you want its functionality, port the *feature* (adapted to the mod's codebase) as a mod-exclusive commit with a clear commit message referencing the upstream PR number. Do not copy the upstream commit verbatim. Example:
```
feat: port upstream word-width cache optimization
Adapted from upstream PR #1027 (not yet merged).
Re-implemented against mod/master's current text layout code.
If/when #1027 is merged upstream, this commit should be dropped
during the next sync and the upstream version used instead.
```
## Sync Procedure
### When to Sync
- **Weekly** during active upstream development periods
- **Immediately** after a major upstream refactor is merged (e.g., ActivityManager, settings migration)
- **Before** porting any new upstream PR -- sync first, then port against the latest base
### Step-by-Step
#### 1. Prepare
```bash
# Ensure clean working tree
git stash # if needed
# Fetch latest upstream
git fetch upstream
# Update the master mirror
git checkout master
git merge upstream/master # should always fast-forward
git push origin master
# Create a backup of current mod/master
git branch mod/backup-pre-sync-$(date +%Y-%m-%d) mod/master
```
#### 2. Identify What Changed
```bash
# How many new upstream commits since last sync?
git rev-list --count mod/master..upstream/master
# What are they?
git log --oneline mod/master..upstream/master
# Which mod commits are ahead of upstream?
git log --oneline upstream/master..mod/master
# Divergence summary
git rev-list --left-right --count mod/master...upstream/master
# Output: LEFT RIGHT (LEFT = mod-only commits, RIGHT = new upstream commits)
```
#### 3. Choose a Sync Strategy
**If divergence is small (< 20 new upstream commits, < 5 mod commits ahead):**
```bash
# Interactive rebase mod/master onto upstream/master
git checkout mod/master
git rebase -i upstream/master
# Resolve conflicts, drop any commits that are now in upstream
git push origin mod/master --force-with-lease
```
**If divergence is moderate (20-50 upstream commits, some conflicts expected):**
```bash
# Merge upstream into mod/master
git checkout mod/master
git merge upstream/master
# Resolve conflicts, favoring upstream for any cherry-picked PRs
git push origin mod/master
```
**If divergence is severe (50+ upstream commits, major refactors):**
Use the "fresh replay" approach. This is the nuclear option -- create a new branch from `upstream/master` and manually re-apply every mod feature. It produces the cleanest result but requires the most effort.
##### Fresh Replay Procedure
**a. Assessment**
Before starting, quantify the problem:
```bash
# Divergence stats
git rev-list --left-right --count mod/master...upstream/master
# Conflict preview (count conflict hunks without actually merging)
git merge-tree $(git merge-base mod/master upstream/master) mod/master upstream/master \
| grep -c "<<<<<<<"
# Files that both sides modified (highest conflict risk)
git merge-tree $(git merge-base mod/master upstream/master) mod/master upstream/master \
| grep "^changed in both" | wc -l
```
**b. Classify all mod commits**
Separate mod-only commits from upstream-ported commits. Each mod commit falls into one of three buckets:
1. **Upstream duplicate** -- cherry-picked/ported from a PR that is now merged in `upstream/master`. These are dropped entirely; the upstream version takes precedence.
2. **Upstream port (still unmerged)** -- ported from a PR that is still open/unmerged upstream. These must be re-applied, adapted to the new upstream code.
3. **Mod-exclusive** -- original mod features with no upstream equivalent. These must be re-applied.
```bash
# List all mod-only commits
git log --oneline upstream/master..mod/master
# Find commits referencing upstream PR numbers
git log --oneline upstream/master..mod/master | grep -iE "(#[0-9]+|port|upstream)"
# For each referenced PR, check if it's now in upstream
git log upstream/master --oneline | grep "#XXXX"
```
**c. Identify upstream architectural changes**
Check for major refactors that will affect how mod code must be re-applied. Common high-impact changes include:
- Activity system changes (e.g., ActivityManager migration -- see `docs/contributing/ActivityManager.md`)
- Settings format changes (e.g., binary to JSON migration)
- Theme system overhauls
- Font rendering pipeline changes
- Library replacements (e.g., image decoder swaps)
- Class/file renames
```bash
# See what upstream changed (summary)
git diff --stat $(git merge-base mod/master upstream/master)...upstream/master | tail -5
# Look for large refactors
git log --oneline upstream/master --not mod/master | grep -iE "(refactor|rename|migrate|replace|remove)"
```
**d. Create the replay branch**
```bash
# Backup current mod/master
git branch mod/backup-pre-sync-$(date +%Y-%m-%d) mod/master
# Create fresh branch from upstream/master
git checkout -b mod/master-resync upstream/master
```
**e. Replay in phases (low-risk to high-risk)**
Work through the mod features in order of conflict risk:
*Phase 1 -- New files (low risk):* Bring over mod-exclusive files that don't exist in upstream. These are purely additive and rarely conflict. Copy them from the backup branch:
```bash
# Example: bring over a new mod file
git checkout mod/backup-pre-sync-YYYY-MM-DD -- src/activities/settings/NtpSyncActivity.cpp
git checkout mod/backup-pre-sync-YYYY-MM-DD -- src/activities/settings/NtpSyncActivity.h
```
New files will need adaptation if they depend on APIs that changed upstream (e.g., Activity base class, settings system). Fix compilation errors as you go.
*Phase 2 -- Core file modifications (high risk):* Re-apply modifications to files that also changed upstream. Do NOT cherry-pick or copy entire files from the old mod branch -- instead, read the old mod's diff for each file and manually re-apply the mod's *intent* against the new upstream code. Key files (in typical priority order):
- `platformio.ini` -- mod build flags
- `src/main.cpp` -- mod activity registration
- Settings files -- mod settings in the new format
- Activity files modified by mod (HomeActivity, EpubReaderActivity, menus, etc.)
- Renderer and HAL files
- Theme files
```bash
# See what the mod changed in a specific file vs the merge-base
git diff $(git merge-base mod/master upstream/master)...mod/backup-pre-sync-YYYY-MM-DD \
-- src/activities/home/HomeActivity.cpp
# See what upstream changed in that same file
git diff $(git merge-base mod/master upstream/master)...upstream/master \
-- src/activities/home/HomeActivity.cpp
```
*Phase 3 -- Re-port unmerged upstream PRs:* For upstream PRs that were ported into the mod but aren't yet merged upstream, re-apply each one against the new codebase. Check the ported PR tracking table in `mod/prs/MERGED.md` for context on each port.
**f. Verify and finalize**
```bash
# Build
pio run
# Check for conflict markers
grep -r "<<<<<<" src/ lib/ --include="*.cpp" --include="*.h"
# Run clang-format
./bin/clang-format-fix
# Verify divergence: LEFT should be mod-only, RIGHT should be 0
git rev-list --left-right --count mod/master-resync...upstream/master
# When satisfied, update mod/master
git checkout mod/master
git reset --hard mod/master-resync
git push origin mod/master --force-with-lease
```
**g. Post-sync cleanup**
- Update `mod/prs/MERGED.md` -- remove entries for PRs now in upstream, update status for remaining ports
- Update mod README with new base commit
- Delete the backup branch after confirming everything works on-device
#### 4. Handle Previously-Ported Upstream PRs
During rebase or merge, you will encounter conflicts where the mod cherry-picked an upstream PR that has since been merged natively. Resolution:
- **If the upstream PR is now in `upstream/master`**: Drop the mod's cherry-pick commit entirely. The upstream version takes precedence because it may have been amended by follow-up commits.
- **If the upstream PR is still unmerged**: Keep the mod's port, but verify it still applies cleanly against the updated codebase.
To check which mod commits reference upstream PRs:
```bash
# List mod-only commits that reference upstream PR numbers
git log --oneline upstream/master..mod/master | grep -iE "(port|upstream|PR #)"
```
To check if a specific upstream PR has been merged:
```bash
# Check if PR #XXXX is in upstream/master
git log upstream/master --oneline | grep "#XXXX"
```
#### 5. Verify
```bash
# Build
pio run
# Check for leftover conflict markers
grep -r "<<<<<<" src/ lib/ --include="*.cpp" --include="*.h"
# Run clang-format
./bin/clang-format-fix
# Verify the divergence is what you expect
git rev-list --left-right --count mod/master...upstream/master
# LEFT should be only mod-exclusive commits, RIGHT should be 0
```
#### 6. Document
After a successful sync, update the mod README with the new base upstream commit and note any dropped or reworked ports.
## Commit Message Conventions for Mod Commits
Use these prefixes to make mod commits easy to identify and filter during future syncs:
- `mod:` -- Mod-specific feature or change (e.g., `mod: add clock settings tab`)
- `feat:` / `fix:` / `perf:` -- Standard prefixes, but for mod-original work
- `port:` -- Feature ported from an unmerged upstream PR (e.g., `port: upstream PR #1027 word-width cache`)
Always include in the commit body:
- Which upstream PR it's based on (if any)
- Whether it should be dropped when that PR merges upstream
## Tracking Ported PRs
Detailed documentation for each ported PR lives in [`mod/prs/MERGED.md`](../prs/MERGED.md). That file contains full context on what was changed, how it differs from the upstream PR, and notable discussion.
Additionally, keep the quick-reference status table below up to date during each sync. This table answers the question every sync needs answered: "which ports are still relevant?"
| Upstream PR | Description | Upstream Status | Competing/Related PRs | Action on Next Sync |
|---|---|---|---|---|
| #857 | Dictionary word lookup | OPEN | None | Keep until merged upstream |
| #1003 | Image placeholders during decode | OPEN | #1291 (MERGED, user image display setting) | Evaluate -- #1291 may cover this |
| #1019 | File extensions in file browser | OPEN | #1260 (MERGED, MyLibrary->FileBrowser rename) | Keep, adapt to rename |
| #1027 | Word-width cache + hyphenation early exit | OPEN | #1168 (MERGED, fixed-point layout), #873 (MERGED, kerning) | Needs complete rework against new text layout |
| #1038 | std::list to std::vector in text layout | MERGED | -- | DROP on next sync (now in upstream) |
| #1045 | Shorten "Forget Wifi" labels | MERGED | -- | DROP on next sync (now in upstream) |
| #1037 | Decomposed character hyphenation/rendering | MERGED | -- | DROP on next sync (now in upstream) |
| #1055 | Byte-level framebuffer writes | OPEN | #1141 (MERGED, wrapped text in GfxRender) | Keep, adapt to GfxRenderer changes |
| #1068 | URL hyphenation fix | OPEN | None | Keep until merged upstream |
| #1090 | KOReader push progress + sleep | OPEN | #946 (OPEN, sync streamlining) | Evaluate overlap with #946 |
| #1185 | KOReader document hash cache | OPEN | #1286 (OPEN, OPDS filename matching) | Keep until merged upstream |
| #1209 | Multiple OPDS servers | OPEN | #1214 (OPEN, author folders) | Keep until merged upstream |
| #1217 | KOReader sync improvements | OPEN | #946 (OPEN, sync streamlining) | Evaluate overlap with #946 |
*Last updated: 2026-03-07*
### How to update this table during a sync
1. For each row, check if the upstream PR has been merged: `git log upstream/master --oneline | grep "#XXXX"`
2. If merged: change Action to "DROP" and remove the port commit during the sync
3. If still open: check if competing/related PRs have merged that affect the port
4. After the sync: remove dropped rows, add any new ports, update the date
## What NOT to Do
1. **Don't cherry-pick upstream commits.** See golden rule above.
2. **Don't let mod/master fall more than ~2 weeks behind upstream/master.** The longer you wait, the harder the sync.
3. **Don't resolve conflicts by "taking ours" for upstream files without understanding why.** The upstream version is almost always correct for code that isn't mod-specific.
4. **Don't merge `mod/master` into `master`.** The `master` branch is a clean mirror of upstream.
5. **Don't port an upstream PR without first syncing.** You'll be porting against a stale base, making the next sync harder.
6. **Don't create mod feature branches off of `master`.** Always branch from `mod/master`.