8 Commits

Author SHA1 Message Date
Daniel
16caa66b4a
perf: batch ZIP size lookup in buildBookBin (O(n*m) → O(n log n))
Add ZipFile::fillUncompressedSizes() for single-pass ZIP central directory
scan with hash-based target matching.

Also apply clang-format fixes for CI.

Shadow Slave results:
- buildBookBin: 506s → 35s
- Total indexing: 8.7min → 50s
2026-01-22 15:53:00 -05:00
Daniel
cf16d33710
perf: optimize large EPUB indexing from O(n²) to O(n log n)
Three optimizations for EPUBs with many chapters (e.g. 2768 chapters):

1. OPF idref→href lookup: Build sorted hash index during manifest parsing,
   use binary search during spine resolution. Reduces ~4min to ~30-60s.

2. TOC href→spineIndex lookup: Build sorted hash index in beginTocPass(),
   use binary search in createTocEntry(). Reduces ~4min to ~30-60s.

3. ZIP central-dir cursor: Resume scanning from last position instead of
   restarting from beginning. Reduces ~8min to ~1-3min.

All optimizations only activate for large EPUBs (≥400 spine items).
Small books use unchanged code paths.

Memory impact: ~33KB + ~39KB temporary during indexing, freed after.
Expected total: ~17min → ~3-5min for Shadow Slave (2768 chapters).

Also adds phase timing logs for performance measurement.
2026-01-22 15:53:00 -05:00
Dave Allie
fb5fc32c5d
Add exFAT support (#150)
## Summary

* Swap to updated SDCardManager which uses SdFat
* Add exFAT support
  * Swap to using FsFile everywhere
* Use newly exposed `SdMan` macro to get to static instance of
SDCardManager
* Move a bunch of FsHelpers up to SDCardManager
2025-12-30 16:09:30 +11:00
Dave Allie
071ccb9d1b
Custom zip parsing (#140)
## Summary

* Use custom zip central directory parsing to lower memory usage when
loading zipped epub content
2025-12-29 21:17:29 +11:00
Jonas Diemer
926c786705
Keep ZipFile open to speed up getting file stats. (#76)
Still a bit raw, but gets the time required to determine the size of
each chapter (for reading progress) down from ~25ms to 0-1ms.

This is done by keeping the zipArchive open (so simple ;)).

Probably we don't need to cache the spine sizes anymore then...

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-21 14:38:51 +11:00
Dave Allie
c7a32fe41f
Remove tinyxml2 dependency replace with expat parsers (#9) 2025-12-13 19:36:01 +11:00
Dave Allie
de453fed1d
Stream inflated EPUB HTMLs down to disk instead of inflating in memory (#4)
* Downgrade miniz for stability

* Stream HTML from ZIP down to disk instead of loading all in mem
2025-12-08 00:39:17 +11:00
Dave Allie
2ccdbeecc8
Public release 2025-12-03 22:06:45 +11:00