99 Commits

Author SHA1 Message Date
cottongin
5464d9de3a
fix: webserver stability, memory leaks, epub underlining, flash screen
Webserver:
- Remove MD5 hash computation from file listings (caused EAGAIN errors)
- Implement JSON batching (2KB) with pacing for file listings
- Simplify sendContentSafe() flow control

Memory:
- Cache QR codes on server start instead of regenerating per render
- Optimize WiFi scan: vector deduplication, 20 network limit, early scanDelete()
- Fix 48KB cover buffer leak when navigating Home -> File Transfer

EPUB Reader:
- Fix errant underlining by flushing partWordBuffer before style changes
- Text before styled inline elements (e.g., <a> with CSS underline) no longer
  incorrectly receives the element's styling

Flash Screen:
- Fix version string buffer overflow (30 -> 50 char limit)
- Use half refresh for cleaner display
- Adjust pre_flash.py timing for half refresh completion
2026-01-30 22:00:15 -05:00
cottongin
dd630dcf72
Improve EPUB rendering and add end-of-book Start Over
EPUB rendering improvements:
- Add margin-left/padding-left CSS parsing for block indentation
- Add vertical bar and italic styling for blockquotes
- Add left margin indentation for list items (ol/ul)
- Fix ordered lists showing bullets instead of numbers
- Fix nested <p> inside <li> causing marker on separate line

End-of-book improvements:
- Add "Start Over" option to wrap to first page when pressing next
- Show "Start Over" button hint on finished book prompt
2026-01-29 19:45:58 -05:00
cottongin
6ceba56620
checkpoint: refactor TextBlock/ParsedText from std::list to std::vector
Reduces heap fragmentation by ~12x fewer allocations per TextBlock.
This fixes crashes when repeatedly navigating dictionary pages.

- Replace std::list with std::vector in TextBlock members
- Replace splice() with move+erase in ParsedText::extractLine()
- Use index-based access in hyphenateWordAtIndex()
2026-01-29 09:52:30 -05:00
cottongin
4db384edb6
fix: prevent Serial.printf from blocking when USB disconnected
All checks were successful
CI / build (push) Successful in 2m23s
On ESP32-C3 with USB CDC, Serial.printf() blocks indefinitely when USB
is not connected. This caused device freezes when booted without USB.

Solution: Call Serial.setTxTimeoutMs(0) after Serial.begin() to make
all Serial output non-blocking.

Also added if (Serial) guards to high-traffic logging paths in
EpubReaderActivity as belt-and-suspenders protection.

Includes documentation of the debugging process and Serial call inventory.

Also applies clang-format to fix pre-existing formatting issues.
2026-01-28 16:16:11 -05:00
cottongin
c2a966a6ea
refactor: Memory optimization and XTC format removal
Memory optimization:
- Add LOG_STACK_WATERMARK macro for task stack monitoring
- Add freeCoverBufferIfAllocated() and preloadCoverBuffer() for memory management
- Improve cover buffer reuse to reduce heap fragmentation
- Add grayscale buffer cleanup safety in GfxRenderer
- Make grayscale rendering conditional on successful buffer allocation
- Add detailed heap fragmentation logging with ESP-IDF API
- Add CSS parser memory usage estimation

XTC format removal:
- Remove entire lib/Xtc library (XTC parser and types)
- Remove XtcReaderActivity and XtcReaderChapterSelectionActivity
- Remove XTC file handling from HomeActivity, SleepActivity, ReaderActivity
- Remove .xtc/.xtch from supported extensions in BookManager
- Remove XTC cache prefix from Md5Utils
- Update web server and file browser to exclude XTC format
- Clear in-memory caches when disk cache is cleared
2026-01-27 20:35:32 -05:00
cottongin
c90304f59b
fix: #348 fit cover artifacts - merge crop parameter (#465)
Cherry-picked from upstream PR #465
Resolved conflicts: merged crop parameter with existing progressCallback,
kept local dimension calculation and edge luminance caching logic
2026-01-27 07:50:37 -05:00
cottongin
8920c62957
fix: line break - flush word before <br/> tag (#525)
Manual implementation of upstream PR #525
Added flushPartWordBuffer() function and call it before <br/> handling
to fix issue where preceding word was incorrectly wrapped to new line
2026-01-27 07:36:06 -05:00
cottongin
25b75b706f
fix: Allow line break after ellipsis and underscore (#425)
Cherry-picked from upstream PR #425
2026-01-27 07:31:26 -05:00
cottongin
1a38fd96af
image tweaks 2026-01-27 07:11:18 -05:00
cottongin
5f4fa3bebe
checkpoint 1 2026-01-26 19:48:21 -05:00
Martin Brook
41eabba0d4 style: apply clang-format-21 formatting 2026-01-26 22:16:41 +00:00
Martin Brook
99702a342c feat: epub add png jpeg support 2026-01-26 19:09:19 +00:00
cottongin
91c8cc67ce
granular position tracking 2026-01-25 00:24:54 -05:00
cottongin
2b2bc95cf2
fixed 2026-01-24 21:10:08 -05:00
cottongin
5c3828efe8
nice 2026-01-24 02:01:53 -05:00
pablohc
881f866d86
feat(sleep): pre-calculate EPUB cover dimensions for FIT/CROP modes
- Add JpegToBmpConverter::getJpegDimensions() to extract JPEG dimensions
- Fix getCoverBmpPath() ternary operator precedence bug
- Pre-calculate exact dimensions per mode:
  * FIT: width=480px, height=(480*jpegHeight)/jpegWidth
  * CROP: height=800px, width=(800*jpegWidth)/jpegHeight
- Generate separate cache files (cover_fit.bmp vs cover_crop.bmp)
- Eliminates runtime scaling artifacts, improves sleep screen clarity
2026-01-23 22:33:35 -05:00
cottongin
ac1251282b
initial version of custom fonts 2026-01-23 03:54:39 -05:00
Daniel
16caa66b4a
perf: batch ZIP size lookup in buildBookBin (O(n*m) → O(n log n))
Add ZipFile::fillUncompressedSizes() for single-pass ZIP central directory
scan with hash-based target matching.

Also apply clang-format fixes for CI.

Shadow Slave results:
- buildBookBin: 506s → 35s
- Total indexing: 8.7min → 50s
2026-01-22 15:53:00 -05:00
Daniel
cf16d33710
perf: optimize large EPUB indexing from O(n²) to O(n log n)
Three optimizations for EPUBs with many chapters (e.g. 2768 chapters):

1. OPF idref→href lookup: Build sorted hash index during manifest parsing,
   use binary search during spine resolution. Reduces ~4min to ~30-60s.

2. TOC href→spineIndex lookup: Build sorted hash index in beginTocPass(),
   use binary search in createTocEntry(). Reduces ~4min to ~30-60s.

3. ZIP central-dir cursor: Resume scanning from last position instead of
   restarting from beginning. Reduces ~8min to ~1-3min.

All optimizations only activate for large EPUBs (≥400 spine items).
Small books use unchanged code paths.

Memory impact: ~33KB + ~39KB temporary during indexing, freed after.
Expected total: ~17min → ~3-5min for Shadow Slave (2768 chapters).

Also adds phase timing logs for performance measurement.
2026-01-22 15:53:00 -05:00
Daniel
51a4faddd4
fix: remove OOM-causing hrefToSpineIndex map from TOC pass
The unordered_map with 2768 string keys (~100KB+) was causing OOM crashes
at beginTocPass() on ESP32-C3's limited ~380KB RAM.

Reverted createTocEntry() to use original O(n) spine file scan instead.
Kept the safe spineToTocIndex vector in buildBookBin() (only ~5.5KB).
2026-01-22 15:52:45 -05:00
Daniel
a91bb0b1b8
perf: optimize large EPUB indexing from O(n²) to O(n)
Replace O(n²) lookups with O(n) preprocessing:

1. createTocEntry(): Build href->spineIndex map once in beginTocPass()
   instead of scanning spine file for every TOC entry

2. buildBookBin(): Build spineIndex->tocIndex vector in single pass
   instead of scanning TOC file for every spine entry

For 2768-chapter EPUBs, this reduces:
- TOC pass: from ~7.6M file reads to ~5.5K reads
- buildBookBin: from ~7.6M file reads to ~5.5K reads

Memory impact: ~80KB for href map (acceptable trade-off for 10x+ speedup)
2026-01-22 15:52:29 -05:00
Daniel
481b8210fb
fix: prevent OOM crash when loading large EPUBs (2000+ chapters)
Remove the call to loadAllFileStatSlims() which pre-loads all ZIP central
directory entries into memory. For EPUBs with 2000+ chapters (like webnovels),
this exhausts the ESP32-C3's ~380KB RAM and causes abort().

The existing loadFileStatSlim() function already handles individual lookups
by scanning the central directory per-file when the cache is empty. This is
O(n*m) instead of O(n), but prevents memory exhaustion.

Fixes #134
2026-01-22 15:52:07 -05:00
Jonas Diemer
c166b89f7b
Make sure img alt text is treated as separate text block 2026-01-22 15:48:32 -05:00
cottongin
9493fb1f18
sort of working dictionary 2026-01-22 12:42:01 -05:00
Jake Kenneally
8f3d226bf3 increment versions to prevent error when opening cached EPUBs 2026-01-20 10:27:55 -06:00
Jake Kenneally
5c9412b141 fix compilation errors 2026-01-19 23:09:35 -06:00
Jake Kenneally
750a6ee1d8 rerun clang-format 2026-01-19 22:39:40 -06:00
Jake Kenneally
be2de1123b Merge remote-tracking branch 'origin' into feature/add-epub-css-parsing
* origin:
  fix: truncate chapter names that are too long (#422)
  feat: dict based Hyphenation (#305)
  fix: render U+FFFD replacement character instead of ? (#366)
  fix: Invert colors on home screen cover overlay when recent book is selected (#390)
  Adds KOReader Sync support (#232)
  feat: Change keyboard "caps" to "shift" & Wrap Keyboard (#377)
  fix: XTC 1-bit thumb BMP polarity inversion (#373)
2026-01-19 22:37:37 -06:00
Arthur Tazhitdinov
8824c87490
feat: dict based Hyphenation (#305)
## Summary

* Adds (optional) Hyphenation for English, French, German, Russian
languages

## Additional Context

* Included hyphenation dictionaries add approximately 280kb to the flash
usage (German alone takes 200kb)
* Trie encoded dictionaries are adopted from hypher project
(https://github.com/typst/hypher)
* Soft hyphens (and other explicit hyphens) take precedence over
dict-based hyphenation. Overall, the hyphenation rules are quite
aggressive, as I believe it makes more sense on our smaller screen.

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2026-01-19 12:56:26 +00:00
Justin Mitchell
f69cddf2cc
Adds KOReader Sync support (#232)
## Summary

- Adds KOReader progress sync integration, allowing CrossPoint to sync
reading positions with other
KOReader-compatible devices
- Stores credentials securely with XOR obfuscation
- Uses KOReader's partial MD5 document hashing for cross-device book
matching
  - Syncs position via percentage with estimated XPath for compatibility

# Features
- Settings: KOReader Username, Password, and Authenticate options
- Sync from chapters menu: "Sync Progress" option appears when
credentials are configured
- Bidirectional sync: Can apply remote progress or upload local progress

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2026-01-19 11:55:35 +00:00
Jake Kenneally
be10b90a71 formatting: run clang-format-fix 2026-01-17 18:35:44 -05:00
Jake Kenneally
94ce987f2c feat: Add CSS parsing and CSS support in EPUBs 2026-01-17 17:57:04 -05:00
Jonas Diemer
c1c94c0112
Feature: Show img alt text (#168)
Let's start small by showing the ALT text of IMG. This is rudimentary,
but avoids those otherwise completely blank chapters.

I feel we will need this even when we can render images if that
rendering takes >1s - I would then prefer rendering optional and showing
the ALT text first.
2026-01-15 23:21:46 +11:00
efenner
d45f355e87
feat: Add EPUB table omitted placeholder (#372)
## Summary

* **What is the goal of this PR?**: Fix the bug I reported in
https://github.com/daveallie/crosspoint-reader/issues/292
* **What changes are included?**: Instead of silently dropping table
content in EPUBs., replace with an italicized '[Table omitted]' message
where tables appear.

## Additional Context

* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
  specific areas to focus on).

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY **_

---------

Co-authored-by: Evan Fenner <evan@evanfenner.com>
Co-authored-by: Warp <agent@warp.dev>
2026-01-15 23:14:59 +11:00
Maeve Andrews
e517945aaa
Add a bullet to the beginning of any <li> (#368)
Currently there is no visual indication whatsoever if something is in a
list. An `<li>` is essentially just another paragraph.

As a partial remedy for this, add a bullet character to the beginning of
`<li>` text blocks so that the user can see that they're list items.
This is incomplete in that an `<ol>` should also have a counter so that
its list items can get numbers instead of bullets (right now I don't
think we track if we're in a `<ul>` or an `<ol>` at all), but it's
strictly better than the current situation.

Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
2026-01-14 23:23:03 +11:00
Maeve Andrews
489220832f
Only indent paragraphs for justify/left-align (#367)
Currently, when Extra Paragraph Spacing is off, an em-space is added to
the beginning of each ParsedText even for blocks like headers that are
centered. This whitespace makes the centering slightly off. Change the
calculation here to only add the em-space for left/justified text.

Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
2026-01-14 23:21:48 +11:00
Jonas Diemer
1c027ce2cd
Skip BOM character (sometimes used in front of em-dashes) (#340)
## Summary

Skip BOM character (sometimes used in front of em-dashes) - they are not
part of the glyph set and would render `?` otherwise.

---

### AI Usage

Did you use AI tools to help write this code? _**YES**_
2026-01-14 22:38:30 +11:00
Eunchurn Park
fecd1849b9
Add cover image display in *Continue Reading* card with framebuffer caching (#200)
## Summary

* **What is the goal of this PR?** (e.g., Fixes a bug in the user
authentication module,

Display the book cover image in the **"Continue Reading"** card on the
home screen, with fast navigation using framebuffer caching.

* **What changes are included?**

- Display book cover image in the "Continue Reading" card on home screen
- Load cover from cached BMP (same as sleep screen cover)
- Add framebuffer store/restore functions (`copyStoredBwBuffer`,
`freeStoredBwBuffer`) for fast navigation after initial render
- Fix `drawBitmap` scaling bug: apply scale to offset only, not to base
coordinates
- Add white text boxes behind title/author/continue reading label for
readability on cover
- Support both EPUB and XTC file cover images
- Increase HomeActivity task stack size from 2048 to 4096 for cover
image rendering

## Additional Context

* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks, specific areas to
focus on).

- Performance: First render loads cover from SD card (~800ms),
subsequent navigation uses cached framebuffer (~instant)
- Memory: Framebuffer cache uses ~48KB (6 chunks × 8KB) while on home
screen, freed on exit
- Fallback: If cover image is not available, falls back to standard
text-only display
- The `drawBitmap` fix corrects a bug where screenY = (y + offset) scale
was incorrectly scaling the base coordinates. Now correctly uses screenY
= y + (offset scale)
2026-01-14 21:24:02 +11:00
Dave Allie
8f3df7e10e
fix: Handle EPUB 3 TOC to spine mapping when nav file in subdirectory (#332)
## Summary

- Nav file in EPUB 3 file is a HTML file with relative hrefs
- If this file exists anywhere but in the same location as the
content.opf file, navigating in the book will fail
- Bump the book cache version to rebuild potentially broken books

## Additional Context

- Fixes https://github.com/daveallie/crosspoint-reader/issues/264

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code?

- [ ] Yes
- [ ] Partially
- [x] No
2026-01-13 00:57:34 +11:00
Jonas Diemer
a9242fe61f
Generate different .bmp for cropped covers so settings have effect. (#330)
Addresses
https://github.com/daveallie/crosspoint-reader/pull/225#issuecomment-3735150337
2026-01-12 20:55:47 +11:00
Dave Allie
2b12a65011
Remove HTML entity parsing (#274)
## Summary

* Remove HTML entity parsing
  * This has been completely useless since the introduction of expat
* expat tries to parse all entities in the document, but only knows of
HTML ones
* Parsing will never end with HTML entities in the text, so the
additional step to parse them that we had went completely unused
* We should figure out the best way to parse that content in the future,
but for now remove that module as it generates a lot of heap allocations
with its map and strings
2026-01-07 23:08:43 +11:00
Pavel Liashkov
0332e1103a
Add EPUB 3 nav.xhtml TOC support (#197)
## Summary

* **What is the goal of this PR?** Add EPUB 3 support by implementing
native navigation document (nav.xhtml) parsing with NCX fallback,
addressing issue Fixes: #143.

  * **What changes are included?**
- New `TocNavParser` for parsing EPUB 3 HTML5 navigation documents
(`<nav epub:type="toc">`)
- Detection of nav documents via `properties="nav"` attribute in OPF
manifest
- Fallback logic: try EPUB 3 nav first, fall back to NCX (EPUB 2) if
unavailable
- Graceful degradation: books without any TOC now load with a warning
instead of failing

  ## Additional Context

* The implementation follows the existing streaming XML parser pattern
using Expat to minimize RAM usage on the ESP32-C3
* EPUB 3 books that include both nav.xhtml and toc.ncx will prefer the
nav document (per EPUB 3 spec recommendation)
* No breaking changes - existing EPUB 2 books continue to work as before
* Tested on examples from
https://idpf.github.io/epub3-samples/30/samples.html
2026-01-03 19:10:35 +11:00
Maeve Andrews
5e9626eb2a
Add paragraph alignment setting (justify/left/center/right) (#191)
## Summary

* **What is the goal of this PR?** 

Add a new user setting for paragraph alignment, instead of hard-coding
full justification.

* **What changes are included?**

One new line in the settings screen, with 4 options
(justify/left/center/right). Default is justified since that's what it
was already. I personally only wanted to disable justification and use
"left", but I included the other options for completeness since they
were already supported.

## Additional Context

Tested on my X4 and looks as expected for each alignment.

Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
2026-01-02 18:21:48 +11:00
Jonas Diemer
39080c0e51
Skip soft hyphens. (#195)
For now, let's skip the soft hyphens (later, we can treat them in the
layouter). See
https://github.com/daveallie/crosspoint-reader/discussions/17#discussioncomment-15378475
2026-01-02 17:54:46 +11:00
Dave Allie
6e9ba1006a
Use sane smaller data types for data in section.bin (#188)
## Summary

* Update EpdFontFamily::Style to be u8 instead of u32 (saving 3 bytes
per word)
* Update layout width/height to be u16 from int
* Update page element count to be u16 from u32
* Update text block element count to be u16 from u32
* Bumped section bin version to version 8
2025-12-31 13:11:36 +11:00
Dave Allie
52a0b5bbe9
Small cleanups from https://github.com/juicecultus/crosspoint-reader-x4 2025-12-30 23:19:08 +11:00
Dave Allie
3abcd0d05d
Redesign home screen (#166)
## Summary

* Redesigned home screen with big option to continue reading and
slightly nicer options to navigate to core sections
* Attempt to use the cached EPUB details (title, author) if they exist,
otherwise fall back to file name
* Adjusted button hints on home screen, removed Back option and changed
left/right to up/down

## Additional Context

* Core of this work comes from @ChandhokTannay in
1d36a86ef1
2025-12-30 23:18:10 +11:00
Jonas Diemer
03f0ce04cc
Feature: go to text/start reference in epub guide section at first start (#156)
This parses the guide section in the content.opf for text/start
references and jumps to this on first open of the book.

Currently, this behavior will be repeated in case the reader manually
jumps to Chapter 0 and then re-opens the book. IMO, this is an
acceptable edge case (for which I couldn't see a good fix other than to
drag a "first open" boolean around).

---------

Co-authored-by: Sam Davis <sam@sjd.co>
Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-30 23:02:46 +11:00
Dave Allie
be1b5bad21
Parse the author name from content.opf file (#165)
## Summary

* Parse the author name from content.opf file
  * Listed in the dc:creator tag within the metadata section
2025-12-30 22:15:44 +11:00
Dave Allie
9f31f80c80
Show previous title for unnamed spines (#158)
## Summary

* Show previous title for unnamed spines
* The spec is a little unclear, but there are plenty of cases where
chapters are split up in parts and should show the previous chapter's
title
* List TOC items instead of spine items in chapter select
* Bump `BOOK_CACHE_VERSION` to `2` to force regeneration of spine item's
TOC indexes
2025-12-30 18:52:42 +11:00