crosspoint-reader-mod/mod/prs/MERGED.md

Merged Upstream PRs

Tracking document for upstream PRs ported into this mod.

• PR #1038 — Replace std::list with std::vector in text layout (znelson)
• PR #1045 — Shorten "Forget Wifi" button labels (lukestein)
• PR #1037 — Fix hyphenation and rendering of decomposed characters (jpirnay)
• PR #1019 — Display file extensions in File Browser (CaptainFrito)
• PR #1055 — Byte-level framebuffer writes for fillRect and drawLine (jpirnay)
• PR #1027 — Reduce ParsedText layout time 7–9% via word-width cache and hyphenation early exit (jpirnay)
• PR #1068 — Correct hyphenation of URLs (Uri-Tauber)

---

PR #1038: Replace std::list with std::vector in text layout

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1038
• Author: znelson (revision of BlindBat's #802)
• Status in upstream: Open (approved by osteotek, awaiting second review from daveallie)
• Method: Manual port (partial -- incremental fixes only)

Context

The core list-to-vector conversion was already performed in a prior mod sync. This port only brings in two targeted fixes from the PR that were missing locally.

Changes applied

1. Erase consumed words in layoutAndExtractLines (lib/Epub/Epub/ParsedText.cpp): Added .erase() calls after the line extraction loop to remove consumed words from words, wordStyles, wordContinues, and forceBreakAfter vectors. Without this, the 750-word early flush threshold fires on every subsequent addWord call instead of only ~3 times per large paragraph, causing ~1,670 redundant flushes.
2. Fix wordContinues flag in hyphenateWordAtIndex (lib/Epub/Epub/ParsedText.cpp): Corrected the attach-to-previous flag handling when splitting a word at a hyphenation point. The prefix now keeps its original flag and the remainder gets false, instead of the previous behavior that cleared the prefix's flag and transferred it to the remainder. This fix was identified by coderabbit review and accepted in commit 9bbe994.

Differences from upstream PR

• The forceBreakAfter vector erase was added (mod-only vector not present in upstream) alongside the three upstream vectors.
• The upstream PR's full list-to-vector conversion (TextBlock.h/.cpp, ParsedText.h/.cpp) was already done in a prior mod sync. Only the two fixes above were new.

Notable PR discussion

• znelson posted detailed benchmarks comparing list vs vector on ESP32-C3 with "Intermezzo" by Sally Rooney (chapter 14, 2,420-word paragraph): 11% faster chapter parse time, 89% more heap headroom (~50KB saved), 10-14% faster layout for medium paragraphs.
• The erase cost is ~90-133 microseconds per flush (0.1% of total flush time).

---

PR #1045: Shorten "Forget Wifi" button labels to fit on button

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1045
• Author: lukestein
• Status in upstream: Open (approved by osteotek)
• Resolves: upstream issue #1035
• Method: Manual port (direct string changes)

Changes applied

Updated STR_FORGET_BUTTON in all 9 translation yaml files under lib/I18n/translations/:

Language	Before	After
Czech	"Zapomenout na síť"	"Zapomenout"
English	"Forget network"	"Forget"
French	"Oublier le réseau"	"Oublier"
German	"WLAN entfernen"	"Entfernen"
Portuguese	"Esquecer rede"	"Esquecer"
Romanian	"Uitaţi reţeaua"	"Uitaţi"
Russian	"Забыть сеть"	"Забыть"
Spanish	"Olvidar la red"	"Olvidar"
Swedish	"Glöm nätverk"	"Glöm"

Prerequisite

Romanian translation file (romanian.yaml) was pulled from upstream/master (merged PR #987 by ariel-lindemann) as it was missing locally.

Differences from upstream PR

None. Changes are identical to the upstream PR.

Notable PR discussion

• ariel-lindemann requested the Romanian shortening be included (was added in a follow-up commit by lukestein).
• Translations were verified via Google Translate (lukestein is not a native speaker of non-English languages).

---

PR #1037: Fix hyphenation and rendering of decomposed characters

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1037
• Author: jpirnay
• Status in upstream: Open (no approvals yet; coderabbit reviewed with one actionable comment, resolved)
• Related to: upstream issue #998 (Minor hyphenation and text flow issues)
• Method: Manual port (4 files)

Changes applied

1. **lib/Utf8/Utf8.h**: Added utf8IsCombiningMark(uint32_t) inline function that identifies Unicode combining diacritical marks (ranges: 0x0300-0x036F, 0x1DC0-0x1DFF, 0x20D0-0x20FF, 0xFE20-0xFE2F).
2. **lib/EpdFont/EpdFont.cpp**: Added combining mark positioning in getTextBounds. Tracks last base glyph state (lastBaseX, lastBaseAdvance, lastBaseTop, hasBaseGlyph). Combining marks are centered over the base glyph's advance width and raised with a minimum 1px gap. Cursor does not advance for combining marks.
3. **lib/Epub/Epub/hyphenation/HyphenationCommon.cpp**: Added NFC-like precomposition in collectCodepoints(). When a combining diacritic follows a base character and a precomposed Latin-1/Latin-Extended scalar exists (grave, acute, circumflex, tilde, diaeresis, cedilla), the base is replaced with the precomposed form and the combining mark is skipped. This allows Liang hyphenation patterns to match decomposed text correctly.
4. **lib/GfxRenderer/GfxRenderer.cpp**: Added combining mark handling to drawText, drawTextRotated90CW, drawTextRotated90CCW, and getTextAdvanceX. Each text drawing function tracks base glyph state and renders combining marks at adjusted (raised, centered) positions without advancing the cursor. getTextAdvanceX skips combining marks entirely.

Differences from upstream PR

• **drawTextRotated90CCW**: This is a mod-only function (not present in upstream). Combining mark handling was added here following the same pattern as the CW rotation, with coordinate adjustments inverted for CCW direction (+raiseBy on X, +lastBaseAdvance/2 on Y instead of negatives).
• The upstream PR initially had a duplicate local isCombiningMark function in EpdFont.cpp (anonymous namespace) which was flagged by coderabbit and replaced with the shared utf8IsCombiningMark from Utf8.h. Our port uses the shared function directly throughout.

Notable PR discussion

• jpirnay noted this is not a 100% bullet-proof implementation -- it maps common base+combining sequences rather than implementing full Unicode NFC normalization.
• The render fix is more universal: it properly x-centers the compound glyph over the previous one and uses at least 1pt visual distance in Y.
• lukestein specifically expressed interest in the fix for hyphenation of already-hyphenated words (e.g., "US-Satellitensystem" was exclusively broken at the existing hyphen).
• osteotek approved the approach of breaking already-hyphenated words at additional Liang pattern points.

---

PR #1019: Display file extensions in File Browser

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1019
• Author: CaptainFrito
• Status in upstream: Open (no approvals; Eloren1 asked about long filename behavior)
• Method: Manual port (1 file)

Changes applied

In src/activities/home/MyLibraryActivity.cpp:

1. Added getFileExtension(std::string) helper function near the existing getFileName. Returns the file extension including the dot (e.g., ".epub"), or empty string for directories or files without extensions.
2. Updated the GUI.drawList call to pass the extension lambda as the rowValue parameter and false for highlightValue. The drawList signature already supported these parameters with defaults.

Differences from upstream PR

None. The implementation is functionally identical.

Notable PR discussion

• CaptainFrito questioned whether the multiple lambda approach (each independently indexing into the files array) is optimal, suggesting a single function returning a struct might be better.
• Eloren1 asked how long filenames look with extensions displayed. This remains an open question in the upstream PR. See mod enhancement below.
• The PR was force-pushed to squash commits.

Mod enhancement: Expandable selected row for long filenames

Addresses Eloren1's concern about long filenames. When the selected row's filename overflows the available text width, the row expands to 2 lines with smart text wrapping. The file extension moves to the bottom-right of the expanded area, baseline-aligned with the last text line. Non-selected rows retain single-line truncation. If the filename still overflows after 2 lines, the second line is truncated with "...".

Files modified:

• src/components/themes/BaseTheme.cpp: Added wrapTextToLines utility function with 3-tier break logic and truncateWithEllipsis helper. Modified drawList to detect selected-row overflow, draw expanded selection highlight at 2x row height, render wrapped title with row-height line spacing, and position the file extension on line 2 (right-aligned).
• src/components/themes/lyra/LyraTheme.cpp: Same helpers and analogous drawList modifications with Lyra-specific styling (rounded-rect selection highlight, icon aligned with line 1, scroll bar awareness).

Design decisions:

• Only the selected/highlighted row expands; other rows with long filenames continue to truncate normally.
• Expansion triggers when the title overflows the value-reduced text width (i.e., when it would be truncated in single-line mode). For titles that overflow the value area but still fit the full width, the expanded row shows the full title on line 1 with the extension on line 2 below.
• 3-tier text wrapping for natural line breaks:
  1. Preferred delimiters: breaks at " -- ", " - ", en-dash, or em-dash separators (common "Title - Author" naming convention). Uses last occurrence to maximize line 1 content.
  2. Word boundaries: breaks at last space or hyphen that fits on line 1.
  3. Character-level fallback: for long unbroken tokens without spaces or hyphens.
• Each wrapped line is placed at the same Y position as a normal list row (row-height spacing between lines), giving natural visual separation that matches the list's vertical rhythm.
• File extension is always positioned on line 2 of the expanded area (right-aligned), regardless of how many text lines are produced by wrapping.
• Icons in LyraTheme are aligned with line 1 (not centered in the expanded area).
• Pagination uses effectivePageItems (pageItems - 1 when expanding) for totalPages, scroll indicators, and page boundaries. This ensures hidden items appear on the next page with proper scroll indicators. To prevent items from the previous page "leaking" into view, the page start is clamped to never go before the original (non-expanded) page boundary. Edge case: when the selected item is the last on the original page, the start shifts forward minimally to keep it visible.
• Boundary item duplication: when navigating to a "real" page boundary (non-expanding path), if the item just before the page start would need expansion when selected, it is included at the top of the current page (the page start is decremented by 1). This prevents the "bumped" item from vanishing when the user navigates from it to the next page. The guard selectedIndex < pageStartIndex + pageItems - 1 ensures the selected item isn't pushed off the page by this adjustment.

---

PR #1055: Byte-level framebuffer writes for fillRect and axis-aligned drawLine

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1055
• Author: jpirnay
• Status in upstream: Open (coderabbit reviewed, no approvals yet)
• Method: Manual port (2 files)

Context

Eliminates per-pixel drawPixel calls for solid fills, axis-aligned lines, and dithered fills by writing directly to the framebuffer at byte granularity. Exploits the fact that one logical dimension always maps to a contiguous physical row in the framebuffer, allowing entire spans to be written with byte masking and memset instead of individual read-modify-write cycles per pixel. Measured 232–470x speedups on ESP32-C3 hardware for full-screen operations.

Changes applied

1. lib/GfxRenderer/GfxRenderer.h: Added two new private methods: fillPhysicalHSpanByte(int phyY, int phyX_start, int phyX_end, uint8_t patternByte) for writing a patterned horizontal span with byte-level edge blending, and fillPhysicalHSpan(int phyY, int phyX_start, int phyX_end, bool state) as a thin solid-fill wrapper.
2. lib/GfxRenderer/GfxRenderer.cpp: Added #include <cstring> for memset. Implemented fillPhysicalHSpanByte (bounds clamping, MSB-first bit packing, partial-byte masking at edges, memset for aligned middle) and fillPhysicalHSpan wrapper.
3. drawLine (axis-aligned cases): Logical vertical lines in Portrait/PortraitInverted now route through fillPhysicalHSpan instead of per-pixel loops. Logical horizontal lines in Landscape orientations similarly use fillPhysicalHSpan. Bresenham diagonal path is unchanged.
4. fillRect: Replaced the per-row drawLine loop with orientation-specific fast paths. Each orientation iterates over the logical dimension that maps to a constant physical row, writing the perpendicular extent as a single fillPhysicalHSpan call.
5. fillRectDither: Replaced per-pixel drawPixelDither loops for DarkGray and LightGray with orientation-aware fillPhysicalHSpanByte calls using pre-computed byte patterns. DarkGray uses checkerboard 0xAA/0x55 alternating by physical row parity. LightGray uses 1-in-4 pattern with row-skipping optimization for all-white rows.

Differences from upstream PR

• fillPolygon landscape optimization (coderabbit nitpick): The upstream PR's coderabbit review noted that fillPolygon's horizontal scanline inner loop could benefit from fillPhysicalHSpan for Landscape orientations. This was noted as a "future optimization opportunity" in the review and not implemented in the PR. We applied it here: for LandscapeCounterClockwise and LandscapeClockwise, the scanline fill now uses fillPhysicalHSpan with appropriate coordinate transforms, falling back to per-pixel for Portrait orientations (where horizontal logical lines map to vertical physical lines).

Notable PR discussion

• jpirnay posted hardware benchmarks: fillRect(480×800) went from 125,577 µs to 519 µs (242× speedup), vertical drawLine(800px) from 261 µs to 3 µs (87×), fillRectDither DarkGray 234× speedup, LightGray 470× speedup.
• coderabbit review approved all core changes with 8 LGTM comments; the only nitpick was the fillPolygon optimization opportunity (applied in this port).
• The fillRectDither LightGray change has a subtle semantic difference from upstream: the original drawPixelDither<LightGray> only wrote dark pixels (leaving white untouched), while the new span-based approach explicitly writes the full pattern. For fillRectDither (which fills a complete rectangle) this is semantically equivalent.

---

PR #1027: Reduce ParsedText layout time 7–9% via word-width cache and hyphenation early exit

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1027
• Author: jpirnay
• Status in upstream: Open (no approvals; osteotek requested separate benchmarks for cache vs early exit)
• Method: Manual port (1 file, adapted for vector-based code)

Context

Reduces CPU time spent in ParsedText::layoutAndExtractLines, the hot path for every page render on the ESP32. Two independent optimizations contribute: a direct-mapped word-width cache and an early exit in the hyphenation breakpoint loop. Benchmarked on device (50 iterations, 474 px justified viewport) showing 5–9% improvement depending on corpus and layout mode.

Changes applied

1. Word-width cache (lib/Epub/Epub/ParsedText.cpp): Added a 128-entry, 4 KB static array (BSS, not heap) that caches getTextAdvanceX results keyed by (word, fontId, style). Lookup is O(1) via FNV-1a hash + bitmask slot selection. Words >= 24 bytes bypass the cache (uncommon, unlikely to repeat). Hyphen-fragment measurements (never repeated) skip the cache entirely. calculateWordWidths now calls cachedMeasureWordWidth instead of measureWordWidth.
2. Hyphenation early exit (lib/Epub/Epub/ParsedText.cpp): In hyphenateWordAtIndex, when a candidate prefix is wider than the available space, the loop now breaks instead of continueing. Hyphenator::breakOffsets returns candidates in ascending byte-offset order, so prefix widths are non-decreasing -- all subsequent candidates will also be too wide. A reusable std::string prefix buffer replaces per-iteration substr allocations.
3. Reserve hint (lib/Epub/Epub/ParsedText.cpp): Added lineBreakIndices.reserve(totalWordCount / 8 + 1) in computeLineBreaks to avoid repeated reallocation.

Differences from upstream PR

• List-specific optimizations not applicable: The upstream PR includes std::list splice optimizations in extractLine and iterator changes (std::advance to std::next) throughout. Our mod already uses std::vector (from PR #1038), so these changes don't apply -- vector index access and move iterators are already in place.
• continuesVec sync removed: The upstream PR updates a separate continuesVec pointer in hyphenateWordAtIndex. Our mod modifies wordContinues directly (it's already a vector), so this indirection is unnecessary.
• Benchmark infrastructure excluded: The PR's final commit removed ParsedTextLegacy.h/.cpp, ParsedTextBenchmark.h/.cpp, and main.cpp benchmark hooks. These were development-only files not part of the deliverable.

Notable PR discussion

• osteotek noted he had previously tried a word-width cache with "almost zero on-device improvements" and requested separate benchmarks for each optimization.
• jpirnay posted individual contribution data: the cache dominates (7–8% for DP layout, 3–4% for greedy) while the early exit contributes 1–2%. The cache saves a getTextAdvanceX call on every word in calculateWordWidths (56–61 calls/iteration), whereas the early exit only fires on the handful of words per paragraph that trigger hyphenation.
• jpirnay's benchmark table (50 iterations, 474 px justified viewport):

Scenario	Early exit only	Cache only (derived)	Both combined
German DP	−1%	~−7%	−8%
English DP	−1%	~−8%	−9%
Combined DP	−1%	~−8%	−9%
German greedy	−2%	~−3%	−5%
English greedy	−2%	~−4%	−6%
Combined greedy	−2%	~−3%	−5%

---

PR #1068: Correct hyphenation of URLs

• URL: https://github.com/crosspoint-reader/crosspoint-reader/pull/1068
• Author: Uri-Tauber
• Status in upstream: Open (coderabbit reviewed with one nitpick, no approvals yet)
• Related to: upstream issue #1066 (Some epubs crash Crosspoint — long URLs cause rendering issues)
• Method: Manual port (2 files)

Context

Long URLs in EPUBs could not be line-wrapped because buildExplicitBreakInfos required alphabetic characters on both sides of an explicit hyphen marker. URLs contain /, digits, dots, and other non-alphabetic characters, so path separators were never recognized as break points.

Changes applied

1. lib/Epub/Epub/hyphenation/HyphenationCommon.cpp: Added case '/': to isExplicitHyphen, treating the URL path separator as an explicit hyphen delimiter alongside existing hyphen/dash characters.
2. lib/Epub/Epub/hyphenation/Hyphenator.cpp: Split the single combined filter in buildExplicitBreakInfos into a two-stage check. The isExplicitHyphen test is applied first. Then, for / and - specifically, the strict requirement that both adjacent codepoints must be alphabetic is relaxed — these separators can break next to digits, dots, or other URL characters. All other explicit hyphens retain the original TeX-style alphabetic-surround rule.

Differences from upstream PR

• Repeated-separator guard (mod enhancement): Added a guard from the coderabbit review nitpick (not yet addressed in the upstream PR) that skips break points between consecutive identical separators. This prevents a line break from being inserted between the two slashes in http:// or between double hyphens like --.

Notable PR discussion

• coderabbit approved the isExplicitHyphen change and flagged the repeated-separator edge case as a nitpick. The PR author has not yet addressed it.
• The PR resolves the URL portion of issue #1066, where MIT Press EPUBs with long URLs caused rendering problems.