## Summary
* Consolidate chapter page data into single file
* Header structure of the file stays the same, following the page count,
we now put a LUT offset
* The page data is all then appended to this file
* Finally the LUT is appended to the end of the file, and the page count
is updated
* This will also significantly improve the duration of cache cleanup
which takes a while to scan the directory and cleanup content
* Remove page file version as it's all tied up into the section file now
* Bumped section file version to 7
* Moved section content into sub directory
* Updated docs
## Additional Context
* Benchmarks:
* Generating 74 pages of content from a chapter in Jade Legacy took:
* master: 6,229ms
* this PR: 1,305ms
* Speedup of 79%
* Generating 207 pages of content from Livesuit book:
* With progress bar UI updates:
* master: 24,250ms
* this PR: 8,063ms
* Speedup of 67%
* Without progress bar UI updates:
* master: 13,055ms
* this PR: 3,600ms
* Speedup of 72%
## Summary
* **What is the goal of this PR?**
Add support for XTC (XTeink X4 native) ebook format, which contains
pre-rendered 480x800 1-bit bitmap pages optimized for e-ink displays.
* **What changes are included?**
- New `lib/Xtc/` library with XtcParser for reading XTC files
- XtcReaderActivity for displaying XTC pages on e-ink display
- XTC file detection in FileSelectionActivity
- Cover BMP generation from first XTC page
- Correct XTG page header structure (22 bytes) and bit polarity handling
## Additional Context
- XTC files contain pre-rendered bitmap pages with embedded status bar
(page numbers, progress %)
- XTG page header: 22 bytes (magic + dimensions + reserved fields +
bitmap size)
- Bit polarity: 0 = black, 1 = white
- No runtime text rendering needed - pages display directly on e-ink
- Faster page display compared to EPUB since no parsing/rendering
required
- Memory efficient: loads one page at a time (48KB per page)
- Tested with XTC files generated from https://x4converter.rho.sh/
- Verified correct page alignment and color rendering
- Please report any issues if you test with XTC files from other
sources.
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* Use font ascender height for baseline offset
* Previously was using font height, but when rendering the font (even
from y = 0), there would be a lot of top margin
* Font would also go below the "bottom of the line" as we were using the
full font height as the baseline
## Additional Context
* This caused some text to move around, I've fixed everything I can
* Notably it moves the first line of font a little closer to the top of
the page
• What is the goal of this PR?
Implement a horizontal EPUB reading mode so books can be read in
landscape orientation (both 90° and 270°), while keeping the rest of the
UI in portrait.
• What changes are included?
◦ Rendering / Display
▪ Added an orientation model to GfxRenderer (Portrait, LandscapeNormal,
LandscapeFlipped) and made:
▪ drawPixel, drawImage, displayWindow map logical coordinates
differently depending on orientation.
▪ getScreenWidth() / getScreenHeight() return orientation‑aware logical
dimensions (480×800 in portrait, 800×480 in landscape).
◦ Settings / Configuration
▪ Extended CrossPointSettings with:
▪ landscapeReading (toggle for portrait vs. landscape EPUB reading).
▪ landscapeFlipped (toggle to flip landscape 180° so both horizontal
holding directions are supported).
▪ Updated settings serialization/deserialization to persist these fields
while remaining backward‑compatible with existing settings files.
▪ Updated SettingsActivity to expose two new toggles:
▪ “Landscape Reading”
▪ “Flip Landscape (swap top/bottom)”
◦ EPUB Reader
▪ In EpubReaderActivity:
▪ On onEnter, set GfxRenderer orientation based on the new settings
(Portrait, LandscapeNormal, or LandscapeFlipped).
▪ On onExit, reset orientation back to Portrait so Home, WiFi, Settings,
etc. continue to render as before.
▪ Adjusted renderStatusBar to position the status bar and battery
indicator relative to GfxRenderer::getScreenHeight() instead of
hard‑coded Y coordinates, so it stays correctly at the bottom in both
portrait and landscape.
◦ EPUB Caching / Layout
▪ Extended Section cache metadata (section.bin) to include the logical
screenWidth and screenHeight used when pages were generated; bumped
SECTION_FILE_VERSION.
▪ Updated loadCacheMetadata to compare:
▪ font/margins/line compression/extraParagraphSpacing and screen
dimensions; mismatches now invalidate and clear the cache.
▪ Updated persistPageDataToSD and all call sites in EpubReaderActivity
to pass the current GfxRenderer::getScreenWidth() / getScreenHeight() so
portrait and landscape caches are kept separate and correctly sized.
Additional Context
• Cache behavior / migration
◦ Existing section.bin files (old SECTION_FILE_VERSION) will be detected
as incompatible and their caches cleared and rebuilt once per chapter
when first opened after this change.
◦ Within a given orientation, caches will be reused as before. Switching
orientation (portrait ↔ landscape) will cause a one‑time re‑index of
each chapter in the new orientation.
• Scope and risks
◦ Orientation changes are scoped to the EPUB reader; the Home screen,
Settings, WiFi selection, sleep screens, and web server UI continue to
assume portrait orientation.
◦ The renderer’s orientation is a static/global setting; if future code
uses GfxRenderer outside the reader while a reader instance is active,
it should be aware that orientation is no longer implicitly fixed.
◦ All drawing primitives now go through orientation‑aware coordinate
transforms; any code that previously relied on edge‑case behavior or
out‑of‑bounds writes might surface as logged “Outside range” warnings
instead.
• Testing suggestions / areas to focus on
◦ Verify in hardware:
▪ Portrait mode still renders correctly (boot, home, settings, WiFi,
reader).
▪ Landscape reading in both directions:
▪ Landscape Reading = ON, Flip Landscape = OFF.
▪ Landscape Reading = ON, Flip Landscape = ON.
▪ Status bar (page X/Y, % progress, battery icon) is fully visible and
aligned at the bottom in all three combinations.
◦ Open the same book:
▪ In portrait first, then switch to landscape and reopen it.
▪ Confirm that:
▪ Old portrait caches are rebuilt once for landscape (you should see the
“Indexing…” page).
▪ Progress save/restore still works (resume opens to the correct page in
the current orientation).
◦ Ensure grayscale rendering (the secondary pass in
EpubReaderActivity::renderContents) still looks correct in both
orientations.
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?**
Improve reliability and user experience during chapter indexing by
adding retry logic for SD card operations and a visual progress bar.
* **What changes are included?**
- **Retry logic**: Add 3 retry attempts with 50ms delay for ZIP to SD
card streaming to handle timing issues after display refresh
- **Progress bar**: Display a visual progress bar (0-100%) during
chapter indexing based on file read progress, updating every 10% to
balance responsiveness with e-ink display limitations
## Additional Context
* **Problem observed**: When navigating quickly through books with many
chapters (before chapter titles finish rendering), the "Indexing..."
screen would appear frozen. Checking the serial log revealed the
operation had silently failed, but the UI showed no indication of this.
Users would likely assume the device had crashed. Pressing the next
button again would resume operation, but this behavior was confusing and
unexpected.
* **Solution**:
- Retry logic handles transient SD card timing failures automatically,
so users don't need to manually retry
- Progress bar provides visual feedback so users know indexing is
actively working (not frozen)
* **Why timing issues occur**: After display refresh operations, there
can be timing conflicts when immediately starting SD card write
operations. This is more likely to happen when rapidly navigating
through chapters.
* **Progress bar design**: Updates every 10% to avoid excessive e-ink
refreshes while still providing meaningful feedback during long indexing
operations (especially for large chapters with CJK characters).
* **Performance**: Minimal overhead - progress calculation is simple
byte counting, and display updates use `FAST_REFRESH` mode.
## Summary
* **What is the goal of this PR?**
Replace simple threshold-based grayscale quantization with ordered
dithering using a 4x4 Bayer matrix. This eliminates color banding
artifacts and produces smoother gradients on e-ink display.
* **What changes are included?**
- Add 4x4 Bayer dithering matrix for 16-level threshold patterns
- Modify `grayscaleTo2Bit()` function to accept pixel coordinates and
apply position-based dithering
- Replace simple `grayscale >> 6` threshold with ordered dithering
algorithm that produces smoother gradients
## Additional Context
* Bayer matrix approach: The 4x4 Bayer matrix creates a repeating
pattern that distributes quantization error spatially, effectively
simulating 16 levels of gray using only 4 actual color levels (black,
dark gray, light gray, white).
* Cache invalidation: Existing cached `cover.bmp` files will need to be
deleted to see the improved rendering, as the converter only runs when
the cache is missing.
## Summary
* **What is the goal of this PR?** Fixes a bug where text disappears
after approximately 25 pages in long chapters during EPUB indexing.
* **What changes are included?**
- Removed the `MAX_LINES = 1000` hard limit in
`ParsedText::computeLineBreaks()`
- Added safer infinite loop prevention by checking if `nextBreakIndex <=
currentWordIndex` and forcing advancement by one word when stuck
## Additional Context
* **Root cause:** The `MAX_LINES = 1000` limit was introduced to prevent
infinite loops, but it truncates content in long chapters. For example,
a 93KB chapter that generates ~242 pages (~9,680 lines) gets cut off at
~1000 lines, causing blank pages after page 25-27.
* **Solution approach:** Instead of a hard line limit, I now detect when
the line break algorithm gets stuck (when `nextBreakIndex` doesn't
advance) and force progress by moving one word at a time. This preserves
the infinite loop protection while allowing all content to be rendered.
* **Testing:** Verified with a Korean EPUB containing a 93KB chapter -
all 242 pages now render correctly without text disappearing.
## Summary
This creates a `renderer.drawButtonHints` to make all of the "hints"
over buttons to match the home screen.
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks, specific areas to
focus on).
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
Replace linear O(n) search with binary search O(log n) for unicode
interval lookup. Korean fonts have many intervals (~30,000+ glyphs), so
this improves text rendering performance during page navigation.
## Summary
* **What is the goal of this PR?** (e.g., Fixes a bug in the user
authentication module, Implements the new feature for
file uploading.)
Replace linear `O(n)` glyph lookup with binary search `O(log n)` to
improve text rendering performance during page navigation.
* **What changes are included?**
- Modified `EpdFont::getGlyph()` to use binary search instead of linear
search for unicode interval lookup
- Added early return for empty interval count
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks, specific areas to
focus on).
- Performance implications: Fonts with many unicode intervals benefit
the most. Korean fonts have ~30,000+ glyphs across multiple intervals,
but any font with significant glyph coverage (CJK, extended Latin,
emoji, etc.) will see improvement.
- Complexity: from `O(n)` to `O(log n)` where n = number of unicode
intervals. For fonts with 10+ intervals, this reduces lookup iterations
significantly.
- Risk: Low - the binary search logic is straightforward and the
intervals are already sorted by unicode codepoint (required for the
original early-exit optimization).
## Summary
* Use single unified cache file for book spine, table of contents, and
core metadata (title, author, cover image)
* Use new temp item store file in OPF parsing to store items to be
rescaned when parsing spine
* This avoids us holding these items in memory
* Use new toc.bin.tmp and spine.bin.tmp to build out partial toc / spine
data as part of parsing content.opf and the NCX file
* These files are re-read multiple times to ultimately build book.bin
## Additional Context
* Spec for file format included below as an image
* This should help with:
* #10
* #60
* #99
## Summary
* Handle 16x16 MCU blocks in JPEG decoding
* We were only correctly handling 8x8 blocks, which means that we did
not correctly support a lot of JPGs leading to an interlacing style on
the images
## Additional Context
* Fixes https://github.com/daveallie/crosspoint-reader/issues/118
## Summary
* adds cyrillic glyphs to pixel arial font, used as Small font in UI
## Additional Context
* with recent changes pixel arial font lost cyrillic glyphs
## Summary
* Fix incorrect justification of last line in paragraph
* `words` is changing size due to the slice, so `isLastLine` would
rarely be right, either removing justification mid-paragraph, or
including it in the last line.
## Additional Context
* Introduced in #73
## Summary
- Add basic JPG image support
- Map JPG back to 2-bit BMP output
- Can be used to later render the BMP file from disk or directly pass to
output if wanted
- Give the 3 passes over the data needed to render grayscale content,
putting it on disk is preferred to outputting it multiple times
## Additional Context
- WIP, looking forward to BMP support from
https://github.com/daveallie/crosspoint-reader/pull/16
- Addresses some of #11
## Summary
* Extract EPUB TOC into temp file before parsing
* Streaming ZIP -> XML parser uses up a lot of memory as we're
allocating inflation buffers while also holding a few copies of the
buffer in different forms
* Instead, but streaming the inflated file down to the SD card (like we
do for HTML parsing, we can lower memory usage)
## Additional Context
* This should help with
https://github.com/daveallie/crosspoint-reader/issues/60 and
https://github.com/daveallie/crosspoint-reader/issues/10. It won't
remove those class of issues completely, but will allow for many more
books to be opened.
Still a bit raw, but gets the time required to determine the size of
each chapter (for reading progress) down from ~25ms to 0-1ms.
This is done by keeping the zipArchive open (so simple ;)).
Probably we don't need to cache the spine sizes anymore then...
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* Build out lines for pages when holding over 750 buffered words
* Should fix issues with parsing long blocks of text causing memory
crashes
## Problem
`drawBmp()` allocates two row buffers via `malloc()` but doesn't check
if allocations succeed. On low memory, this causes a crash when the NULL
pointers are dereferenced.
## Fix
Add NULL check after both `malloc()` calls. If either fails, log error
and return early.
Changed `lib/GfxRenderer/GfxRenderer.cpp`.
## Test
- Defensive addition only - no logic changes
- Manual device testing appreciated
## Problem
Three Epub getter functions can throw exceptions:
- `getCumulativeSpineItemSize()`: No bounds check before
`.at(spineIndex)`
- `getSpineItem()`: If spine is empty and index invalid, `.at(0)` throws
- `getTocItem()`: If toc is empty and index invalid, `.at(0)` throws
## Fix
- Add bounds check to `getCumulativeSpineItemSize()`, return 0 on error
- Add empty container checks to `getSpineItem()` and `getTocItem()`
- Use static fallback objects for safe reference returns on empty
containers
Changed `lib/Epub/Epub.cpp`.
## Test
- Defensive additions - follows existing bounds check patterns
- No logic changes for valid inputs
- Manual device testing appreciated
## Problem
`readFileToMemory()` allocates an output buffer via `malloc()` at line
120 but doesn't check if allocation succeeds. On low memory, the NULL
pointer is passed to `fread()` causing a crash.
## Fix
Add NULL check after `malloc()` for the output buffer. Follows the
existing pattern already used for `deflatedData` at line 141.
Changed `lib/ZipFile/ZipFile.cpp`.
## Test
- Follows existing validated pattern from same function
- Defensive addition only - no logic changes
## Problem
`invertScreen()`, `storeBwBuffer()`, and `restoreBwBuffer()` dereference
`frameBuffer` without NULL validation. If the display isn't initialized,
these functions will crash.
## Fix
Add NULL checks before using `frameBuffer` in all three functions.
Follows the existing pattern from `drawPixel()` (line 11) which already
validates the pointer.
Changed `lib/GfxRenderer/GfxRenderer.cpp`.
## Test
- Follows existing validated pattern from `drawPixel()`
- No logic changes - only adds early return on NULL
- Manual device testing appreciated
## Problem
Three `fopen()` calls in ZipFile.cpp did not check for NULL before using
the file handle. If files cannot be opened, `fseek`/`fread`/`fclose`
receive NULL and crash.
## Fix
Added NULL checks with appropriate error logging and early returns for
all three locations:
- `getDataOffset()`
- `readFileToMemory()`
- `readFileToStream()`
## Testing
- Builds successfully with `pio run`
- Affects: `lib/ZipFile/ZipFile.cpp`
## Problem
- `getBookSize()` calls `getCumulativeSpineItemSize(getSpineItemsCount()
- 1)` which passes -1 when spine is empty
- `calculateProgress()` then divides by zero when book size is 0
## Fix
- Return 0 from `getBookSize()` if spine is empty
- Return 0 from `calculateProgress()` if book size is 0
## Testing
- Builds successfully with `pio run`
- Affects: `lib/Epub/Epub.cpp`
## Problem
`getSpineIndexForTocIndex()` and `getTocIndexForSpineIndex()` access
`toc[tocIndex]` and `spine[spineIndex]` without validating indices are
within bounds. Malformed EPUBs or edge cases could trigger out-of-bounds
access.
## Fix
Added bounds validation at the start of both functions before accessing
the arrays.
## Testing
- Builds successfully with `pio run`
- Affects: `lib/Epub/Epub.cpp`
- Introduced hyphenationEnabled flag in ParsedText and Section classes.
- Updated constructors and methods to handle hyphenation settings.
- Modified settings file versioning to include hyphenationEnabled.
- Enhanced settings UI to allow toggling of hyphenation feature.
