## Summary
Currently, each activity has to manage their own `displayTaskLoop` which
adds redundant boilerplate code. The loop is a wait loop which is also
not the best practice, as the `updateRequested` boolean is not protected
by a mutex.
In this PR:
- Move `displayTaskLoop` to the super `Activity` class
- Replace `updateRequested` with freeRTOS's [direct to task
notification](https://www.freertos.org/Documentation/02-Kernel/02-Kernel-features/03-Direct-to-task-notifications/01-Task-notifications)
- For `ActivityWithSubactivity`, whenever a sub-activity is present, the
parent's `render()` automatically goes inactive
With this change, activities now only need to expose `render()`
function, and anywhere in the code base can call `requestUpdate()` to
request a new rendering pass.
## Additional Context
In theory, this change may also make the battery life a bit better,
since one wait loop is removed. Although the equipment in my home lab
wasn't been able to verify it (the electric current is too noisy and
small). Would appreciate if anyone has any insights on this subject.
Update: I managed to hack [a small piece of
code](https://github.com/ngxson/crosspoint-reader/tree/xsn/measure_cpu_usage)
that allow tracking CPU idle time.
The CPU load does decrease a bit (1.47% down to 1.39%), which make
sense, because the display task is now sleeping most of the time unless
notified. This should translate to a slightly increase in battery life
in the long run.
```
PR:
[40012] [MEM] Free: 185856 bytes, Total: 231004 bytes, Min Free: 123316 bytes
[40012] [IDLE] Idle time: 98.61% (CPU load: 1.39%)
[50017] [MEM] Free: 185856 bytes, Total: 231004 bytes, Min Free: 123316 bytes
[50017] [IDLE] Idle time: 98.61% (CPU load: 1.39%)
[60022] [MEM] Free: 185856 bytes, Total: 231004 bytes, Min Free: 123316 bytes
[60022] [IDLE] Idle time: 98.61% (CPU load: 1.39%)
master:
[20012] [MEM] Free: 195016 bytes, Total: 231532 bytes, Min Free: 132460 bytes
[20012] [IDLE] Idle time: 98.53% (CPU load: 1.47%)
[30017] [MEM] Free: 195016 bytes, Total: 231532 bytes, Min Free: 132460 bytes
[30017] [IDLE] Idle time: 98.53% (CPU load: 1.47%)
[40022] [MEM] Free: 195016 bytes, Total: 231532 bytes, Min Free: 132460 bytes
[40022] [IDLE] Idle time: 98.53% (CPU load: 1.47%)
```
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Streamlined rendering architecture by consolidating update mechanisms
across all activities, improving efficiency and consistency.
* Modernized synchronization patterns for display updates to ensure
reliable, conflict-free rendering.
* **Bug Fixes**
* Enhanced rendering stability through improved locking mechanisms and
explicit update requests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: znelson <znelson@users.noreply.github.com>
## Summary
Pre-compress the HTML file to save flash space. I'm using `gzip` because
it's supported everywhere (indeed, we are using the same optimization on
[llama.cpp server](https://github.com/ggml-org/llama.cpp), our HTML page
is huge 😅 ).
This free up ~40KB flash space.
Some users suggested using `brotli` which is known to further reduce 20%
in size, but it doesn't supported by firefox (only supports if served
via HTTPS), and some reverse proxy like nginx doesn't support it out of
the box (unrelated in this context, but just mention for completeness)
```
PR:
RAM: [=== ] 31.0% (used 101700 bytes from 327680 bytes)
Flash: [==========] 95.5% (used 6259244 bytes from 6553600 bytes)
master:
RAM: [=== ] 31.0% (used 101700 bytes from 327680 bytes)
Flash: [==========] 96.2% (used 6302416 bytes from 6553600 bytes)
```
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **PARTIALLY**, only the
python part
## Summary
* Definition and use of a central LOG function, that can later be
extended or completely be removed (for public use where debugging
information may not be required) to save flash by suppressing the
-DENABLE_SERIAL_LOG like in the slim branch
* **What changes are included?**
## Additional Context
* By using the central logger the usual:
```
#include <HardwareSerial.h>
...
Serial.printf("[%lu] [WCS] Obfuscating/deobfuscating %zu bytes\n", millis(), data.size());
```
would then become
```
#include <Logging.h>
...
LOG_DBG("WCS", "Obfuscating/deobfuscating %zu bytes", data.size());
```
You do have ``LOG_DBG`` for debug messages, ``LOG_ERR`` for error
messages and ``LOG_INF`` for informational messages. Depending on the
verbosity level defined (see below) soe of these message types will be
suppressed/not-compiled.
* The normal compilation (default) will create a firmware.elf file of
42.194.356 bytes, the same code via slim will create 42.024.048 bytes -
170.308 bytes less
* Firmware.bin : 6.469.984 bytes for default, 6.418.672 bytes for slim -
51.312 bytes less
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _NO_
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
## Summary
* Allow OTA update from RC build to full release
* If all the segments match, then also check if the current version
contains "-rc"
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
## Summary
This is an updated version of @itsthisjustin's #346 that builds on
current master and also deduplicates the settings list so we don't have
two copies of the settings. In the Web UI, it should organize the
settings a little closer to what you see on device.
## Additional Context
I tested this live on device and it seems to play nicely for me. It's
re-based on master since master's settings stuff has moved somewhat
since the original PR and addresses the sole review comment #346 - it
also means that I don't need to manually key in the URL for my OPDS
server. :)
---
### AI Usage
My changes were implemented with Claude Opus 4.5 and Claude Code 2.1.25.
I don't know if @itsthisjustin's original work used AI assistance.
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
Continue my changes to introduce the HAL infrastructure from
https://github.com/crosspoint-reader/crosspoint-reader/pull/522
This PR touches quite a lot of files, but most of them are just name
changing. It should not have any impacts to the end behavior.
## Additional Context
My plan is to firstly add this small shim layer, which sounds useless at
first, but then I'll implement an emulated driver which can be helpful
for testing and for development.
Currently, on my fork, I'm using a FS driver that allow "mounting" a
local directory from my computer to the device, much like the `-v` mount
option on docker. This allows me to quickly reset `.crosspoint`
directory if anything goes wrong. I plan to upstream this feature when
this PR get merged.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? NO
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
This adds renaming and moving files to the File Manager
* **What changes are included?**
New `/move` and `/rename` endpoints, and corresponding modals and icons
added. Uses the `file.rename()` function, after sanity checking.
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
Fixes#559, #661, #663. Only touches the File Manager, so low risk of
affecting other systems.
Simpler than #619, at the cost of not migrating the cache of renamed
books.
<img width="870" height="437" alt="image"
src="https://github.com/user-attachments/assets/73e0e750-dfc8-48e0-a7a6-9694470b7ded"
/>
<img width="575" height="318" alt="image"
src="https://github.com/user-attachments/assets/38c5fb19-c38a-436b-b3ad-75c1be7375ab"
/>
<img width="574" height="293" alt="image"
src="https://github.com/user-attachments/assets/1d2a2403-765d-473f-8c4f-c6968e9bbfeb"
/>
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
I used Codex for the implementation itself, and then carefully reviewed
the code myself. As this is a simple change and only to the webserver,
it is low risk.
## Summary
The main motivation behind this PR was because `uploadBuffer` is
statically allocated, but only used when web server is enabled. This
results in 4KB of memory sitting idle most of the time.
As expected, 4KB of initial RAM is freed with this PR:
```
master:
RAM: [=== ] 32.5% (used 106508 bytes from 327680 bytes)
PR:
RAM: [=== ] 31.2% (used 102276 bytes from 327680 bytes)
```
## Additional Context
This also highlights the importance of only using statically-allocated
buffer when absolutely needed (for example, if the component is active
most of the time). Otherwise, it makes more sense to tie the buffer's
life cycle to its activity.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? NO
## Summary
Resolves#562
Implements regex change to support valid characters discussed by
@daveallie in issue
[here](https://github.com/crosspoint-reader/crosspoint-reader/issues/562#issuecomment-3830809156).
Also rejects `.` and `..` as folder names which are invalid in FAT32 and
exFAT filesystems
## Additional Context
- Unsure on the wording for the alert, it feels overly explicit, but
that might be a good thing. Happy to change.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY >**_
## Summary
Finally, I have received my device and got to chance to work on OTA.
https://github.com/crosspoint-reader/crosspoint-reader/issues/176
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Existing OTA functionality is very buggy, many of times (I would say 8
out of 10) are end up with fail for me. When the time that it works it
is very slow and take ages. For others looks like end up with crash or
different issues.
* **What changes are included?**
To be honest, I'm not familiar with Arduino APIs of OTA process, but
looks like not good as much esp-idf itself. I always found Arduino APIs
very bulky for esp32. Wrappers and wrappers.
## Additional Context
Right now, OTA takes ~ 3min 10sec (of course depends on size of .bin
file). Can be tested with playing version info inside from
`platform.ini` file.
```
[crosspoint]
version = 0.14.0
```
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< NO >**_
Our esp32 consistently dropped the last few packets of the TCP transfer
in the old implementation. Only about 1/5 transfers would complete. I've
refactored that entire system into an actual Calibre Device Plugin that
basically uses the exact same system as the web server's file transfer
protocol. I kept them separate so that we don't muddy up the existing
file transfer stuff even if it's basically the same at the end of the
day I didn't want to limit our ability to change it later.
I've also added basic auth to OPDS and renamed that feature to OPDS
Browser to just disassociate it from Calibre.
---------
Co-authored-by: Arthur Tazhitdinov <lisnake@gmail.com>
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
- Rewrite OpdsParser to stream parsing instead of full content
- Fix OOM due to big http xml response
Closes#385
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
When uploading or downloading an updated ebook from SD/WebUI/OPDS with
same the filename the `.crosspoint` cache is not cleared. This can lead
to issues with the Table of Contents and hangs when switching between
chapters.
I encountered this issue in two places:
- When I need to do further ePub cleaning using Calibre after I load an
ePub and find that some of its formatting should be cleaned up. When I
reprocess the same book and want to place it back in the same location I
need a way to invalidate the cache.
- When syncing RSS feed generated epubs. I generate news ePubs with
filenames like `news-outlet.epub` and so every day when I fetch new news
the crosspoint cache needs to be cleared to load that file.
This change offers the following features:
- On web uploads, if the file already exists, the cache for that file is
cleared
- On OPDS downloads, if the file already exists, the cache for that file
is cleared
- There's now an action for `Clear Cache` in the Settings page which can
clear the cache for all books
Addresses
https://github.com/crosspoint-reader/crosspoint-reader/issues/281
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? PARTIALLY
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** Fix WiFi file transfer stability
issues (especially crashes during uploads) and improve upload speed via
WebSocket binary protocol. File transfers now don't really crash as
much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s.
* **What changes are included?**
- **WebSocket upload support** - Adds WebSocket binary protocol for file
uploads, achieving faster speeds 335 KB/s vs HTTP multipart)
- **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls
throughout upload path to prevent watchdog timeouts during:
- File creation (FAT allocation can be slow)
- SD card write operations
- HTTP header parsing
- WebSocket chunk processing
- **4KB write buffering** - Batches SD card writes to reduce I/O
overhead
- **WiFi health monitoring** - Detects WiFi disconnection in STA mode
and exits gracefully
- **Improved handleClient loop** - 500 iterations with periodic watchdog
resets and button checks for responsiveness
- **Progress bar improvements** - Fixed jumping/inaccurate progress by
capping local progress at 95% until server confirms completion
- **Exit button responsiveness** - Button now checked inside the
handleClient loop every 64 iterations
- **Reduced exit delays** - Decreased shutdown delays from ~850ms to
~140ms
**Files changed:**
- `platformio.ini` - Added WebSockets library dependency
- `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering,
watchdog resets
- `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop,
button handling
- `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback
## Additional Context
- WebSocket uses 4KB chunks with backpressure management to prevent
ESP32 buffer overflow
- Falls back to HTTP automatically if WebSocket connection fails
- The main bottleneck now is SD card write speed (~44% of transfer
time), not WiFi
- STA mode was more prone to crashes than AP mode due to external
network factors; WiFi health monitoring helps detect and handle
disconnections gracefully
---
### AI Usage
Did you use AI tools to help write this code? _**YES**_ Claude did it
ALL, I have no idea what I am doing, but my books transfer fast now.
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
Adds support for browsing and downloading books from a Calibre-web
server via OPDS.
How it works
1. Configure server URL in Settings → Calibre Web URL (e.g.,
https://myserver.com:port I use Cloudflare tunnel to make my server
accessible anywhere fwiw)
2. "Calibre Library" will now show on the the home screen
3. Browse the catalog - navigate through categories like "By Newest",
"By Author", "By Series", etc.
4. Download books - select a book and press Confirm to download the EPUB
to your device
Navigation
- Up/Down - Move through entries
- Confirm - Open folder or download book
- Back - Go to parent catalog, or exit to home if at root
- Navigation entries show with > prefix, books show title and author
- Button hints update dynamically ("Open" for folders, "Download" for
books)
Technical details
- Fetches OPDS catalog from {server_url}/opds
- Parses both navigation feeds (catalog links) and acquisition feeds
(downloadable books)
- Maintains navigation history stack for back navigation
- Handles absolute paths in OPDS links correctly (e.g.,
/books/opds/navcatalog/...)
- Downloads EPUBs directly to the SD card root
Note
The server URL should be typed to include https:// if the server
requires it - HTTP→HTTPS redirects may cause SSL errors on ESP32.
## Additional Context
* I also changed the home titles to use uppercase for each word and
added a setting to change the size of the side margins
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** (e.g., Fixes a bug in the user
authentication module, Implements the new feature for
file uploading.)
This PR refactors the semantic version comparison logic used during OTA
update checks.
Memory stats before :
RAM: [=== ] 30.8% (used 101068 bytes from 327680 bytes)
Flash: [========= ] 85.7% (used **5617830** bytes from 6553600 bytes)
Memory stats before :
RAM: [=== ] 30.8% (used 101068 bytes from 327680 bytes)
Flash: [========= ] 85.7% (used **5616870** bytes from 6553600 bytes)
* **What changes are included?**
Replaced std::string::substr() and std::stoi() based parsing with a
lightweight, heap-free approach.
Version parsing is now done in a single pass without creating temporary
std::string objects.
Behavior remains identical: versions are still compared as
MAJOR.MINOR.PATCH.
## Additional Context
`std::string::substr() ` creates a new string and performs heap
allocation
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks, specific areas to
focus on).
## Summary
* Swap to updated SDCardManager which uses SdFat
* Add exFAT support
* Swap to using FsFile everywhere
* Use newly exposed `SdMan` macro to get to static instance of
SDCardManager
* Move a bunch of FsHelpers up to SDCardManager
## Summary
* **What is the goal of this PR?** Adds WiFi Access Point (AP) mode
support for File Transfer, allowing the device to create its own WiFi
network that users can connect to directly - useful when no existing
WiFi network is available. And in my experience is faster when the
device is right next to your laptop (but maybe further from your wifi)
* **What changes are included?**
- New `NetworkModeSelectionActivity` - an interstitial screen asking
users to choose between:
- "Join a Network" - connects to an existing WiFi network (existing
behavior)
- "Create Hotspot" - creates a WiFi access point named
"CrossPoint-Reader"
- Modified `CrossPointWebServerActivity` to:
- Launch the network mode selection screen before proceeding
- Support starting an Access Point with mDNS (`crosspoint.local`) and
DNS server for captive portal behavior
- Display appropriate connection info for both modes
- Modified `CrossPointWebServer` to support starting when WiFi is in AP
mode (not just STA connected mode)
## Additional Context
* **AP Mode Details**: The device creates an open WiFi network named
"CrossPoint-Reader". Once connected, users can access the file transfer
page at `http://crosspoint.local/` or `http://192.168.4.1/`
* **DNS Captive Portal**: A DNS server redirects all domain requests to
the device's IP, enabling captive portal behavior on some devices
* **mDNS**: Hostname resolution via `crosspoint.local` is enabled for
both AP and STA modes
* **No breaking changes**: The "Join a Network" option preserves the
existing WiFi connection flow
* **Memory impact**: Minimal - the AP mode uses roughly the same
resources as STA mode
## Summary
* Adds support for OTA
* Gets latest firmware bin from latest GitHub release
* I have noticed it be a little flaky unpacking the JSON and
occasionally failing to start
## Summary
* HTML files are now static, streamed directly to the client without
modification
* For any dynamic values, load via JSON APIs
* For files page, we stream the JSON content as we scan the directory to
avoid holding onto too much data
## Additional details
* We were previously building up a very large string all generated on
the X4 directly, we should be leveraging the browser
* Fixes https://github.com/daveallie/crosspoint-reader/issues/94
