Commit Graph

28 Commits

Author SHA1 Message Date
Claude
c75cc5cd31 Migrate to ESPAsyncWebServer for ~3-5x faster uploads
Replace synchronous WebServer with ESPAsyncWebServer:
- Fully callback-based, no polling loop needed
- Handles HTTP parsing asynchronously in background
- Larger chunk sizes from async TCP handling
- Known to achieve 500KB-1MB/s vs 200KB/s with sync WebServer

Key changes:
- platformio.ini: Add ESP Async WebServer dependency
- CrossPointWebServer: Rewrite for async API
- CrossPointWebServerActivity: Remove handleClient loop

The sync WebServer was capped at ~200KB/s due to its
synchronous design and small chunk sizes.
2026-01-11 01:43:06 +00:00
Claude
09393a34ac Increase handleClient iterations to 200 for better throughput 2026-01-11 01:38:33 +00:00
Claude
278d4b5d68 Simplify upload to direct SD writes with enhanced diagnostics
Major architectural changes:
- Remove complex 64KB circular buffer (was adding overhead)
- Use simple direct SD writes (ESP32 WebServer already buffers)
- Add detailed timing diagnostics (write count, avg write time, % time in writes)

WiFi optimizations enhanced:
- Explicit WIFI_PS_NONE power save disable
- Force 802.11b/g/n protocol for better throughput
- Log actual TX power and RSSI for debugging

The circular buffer was copying data twice (WiFi→buffer→SD) and adding
complexity. Direct writes let the WebServer's internal buffering work
optimally. Diagnostics will help identify if SD card speed is the bottleneck.
2026-01-10 21:05:58 +00:00
Claude
953df1f3f9 Optimize buffer operations for better upload performance
- Replace byte-by-byte copies with memcpy (10-100x faster)
- Increase SD write chunk size from 4KB to 16KB
- Use static buffer for SD writes to reduce stack usage
- Remove unnecessary yield() from handleClient loop

The byte-by-byte circular buffer operations were a major bottleneck.
Using memcpy with proper wrap-around handling significantly improves
throughput.
2026-01-10 21:02:38 +00:00
Claude
cc666e5c18 Implement aggressive WiFi upload performance optimizations
Add 64KB circular buffer to decouple WiFi receive from SD card writes,
enabling continuous data reception while SD writes complete in batches.

Key optimizations:
- 64KB upload buffer with 32KB batch SD writes
- CPU boost to 240MHz during uploads (from 160MHz)
- WiFi sleep disabled + max TX power (19.5dBm)
- Task priority 5 (from 1) with 6KB stack (from 2KB)
- 50 handleClient iterations per loop (from 10)
- Thread-safe upload status tracking with mutex
- Real-time speed calculation every 500ms

Expected improvement: ~100-300 KB/s → 1+ MB/s upload speed in STA mode
2026-01-10 20:51:07 +00:00
Dave Allie
46fa186b82 Make extension checks case-insensitive (#273)
## Summary

* Implement new `StringUtils::checkFileExtension` which does case
insensitive checking
* Move all checks over to this
2026-01-07 21:07:23 +11:00
Justin Mitchell
b792b792bf Calibre Web Epub Downloading + Calibre Wireless Device Syncing (#219)
## Summary

Adds support for browsing and downloading books from a Calibre-web
server via OPDS.
How it works
1. Configure server URL in Settings → Calibre Web URL (e.g.,
https://myserver.com:port I use Cloudflare tunnel to make my server
accessible anywhere fwiw)
2. "Calibre Library" will now show on the the home screen
3. Browse the catalog - navigate through categories like "By Newest",
"By Author", "By Series", etc.
4. Download books - select a book and press Confirm to download the EPUB
to your device
Navigation
- Up/Down - Move through entries
- Confirm - Open folder or download book
- Back - Go to parent catalog, or exit to home if at root
- Navigation entries show with > prefix, books show title and author
- Button hints update dynamically ("Open" for folders, "Download" for
books)
Technical details
- Fetches OPDS catalog from {server_url}/opds
- Parses both navigation feeds (catalog links) and acquisition feeds
(downloadable books)
- Maintains navigation history stack for back navigation
- Handles absolute paths in OPDS links correctly (e.g.,
/books/opds/navcatalog/...)
- Downloads EPUBs directly to the SD card root
Note
The server URL should be typed to include https:// if the server
requires it - HTTP→HTTPS redirects may cause SSL errors on ESP32.

## Additional Context

* I also changed the home titles to use uppercase for each word and
added a setting to change the size of the side margins

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2026-01-07 19:58:37 +11:00
Dave Allie
14972b34cb Remove authentication type from hotspot QR code (#235)
## Summary

* Remove the `T` parameter (authentication type), and `P` parameter
(password) for the SoftAP wifi config QR code

## Additional Context

* It is optional according to the spec:
https://www.wi-fi.org/system/files/WPA3%20Specification%20v3.2.pdf#page=25
so should either be omitted or set to nopass
* Fixes https://github.com/daveallie/crosspoint-reader/issues/229
2026-01-04 16:08:32 +11:00
Brendan O'Leary
9e59a5106b Fix race condition with keyboard and Wifi entry (#204)
## Summary

* **What is the goal of this PR?** Fixes a bug -
https://github.com/daveallie/crosspoint-reader/issues/187 - where the
screen would freeze after entering a WiFi password, causing the device
to appear hung.

* **What changes are included?**
- Fixed a race condition in `WifiSelectionActivity::displayTaskLoop()`
that caused rendering of an empty screen when transitioning from the
keyboard subactivity
- Added `vTaskDelay()` when a subactivity is active to prevent CPU
starvation from a tight `continue` loop
- Added a check to skip rendering when in `PASSWORD_ENTRY` state,
allowing the state machine to properly transition to `CONNECTING` before
the display updates

## Additional Context

* **Root cause:** When the keyboard subactivity exited after password
entry, the display task would wake up and attempt to render. However,
the `state` was still `PASSWORD_ENTRY` (before `attemptConnection()`
changed it to `CONNECTING`), and since there was no render case for
`PASSWORD_ENTRY`, the display would show a cleared/empty buffer,
appearing frozen.

* **Performance implications:** The added `vTaskDelay(10)` calls when
subactivity is active or in `PASSWORD_ENTRY` state actually improve
performance by preventing CPU starvation - previously the display task
would spin in a tight loop with `continue` while a subactivity was
present.

* **Testing focus:** Test the full WiFi connection flow:
  1. Enter network selection
  2. Select a network requiring a password
  3. Enter password and press OK
  4. Verify "Connecting..." screen appears
  5. Verify connection completes and prompts to save password
2026-01-02 17:49:16 +11:00
Pavel Liashkov
a922e553ed Prevent device sleep during WiFi file transfer and OTA updates (#203)
## Summary

* **What is the goal of this PR?** Fixes #199 - Device falls asleep
during WiFi file transfer after 10 minutes of inactivity, disconnecting
the web server.

  * **What changes are included?**
    - Add `preventAutoSleep()` virtual method to `Activity` base class
- Modify main loop to reset inactivity timer when `preventAutoSleep()`
returns true
- Override `preventAutoSleep()` in `CrossPointWebServerActivity`
(returns true when web server running)
- Override `preventAutoSleep()` in `OtaUpdateActivity` (returns true
during update check/download)

  ## Additional Context

* The existing `skipLoopDelay()` method controls loop timing (yield vs
delay) for HTTP responsiveness. The new `preventAutoSleep()` method is
semantically separate - it explicitly signals that an activity should
keep the device awake.
* `CrossPointWebServerActivity` uses both methods: `skipLoopDelay()` for
responsive HTTP handling, `preventAutoSleep()` for staying awake.
* `OtaUpdateActivity` only needs `preventAutoSleep()` since the OTA
library handles HTTP internally.
2026-01-02 17:44:17 +11:00
Dave Allie
6e9ba1006a Use sane smaller data types for data in section.bin (#188)
## Summary

* Update EpdFontFamily::Style to be u8 instead of u32 (saving 3 bytes
per word)
* Update layout width/height to be u16 from int
* Update page element count to be u16 from u32
* Update text block element count to be u16 from u32
* Bumped section bin version to version 8
2025-12-31 13:11:36 +11:00
Dave Allie
3dd52f30fa Adjust screen title position 2025-12-30 22:06:57 +11:00
Dave Allie
bf7bffd506 Aleo, Noto Sans, Open Dyslexic fonts (#163)
## Summary

* Swap out Bookerly font due to licensing issues, replace default font
with Aleo
* I did a bunch of searching around for a nice replacement font, and
this trumped several other like Literata, Merriwether, Vollkorn, etc
* Add Noto Sans, and Open Dyslexic as font options
  * They can be selected in the settings screen
* Add font size options (Small, Medium, Large, Extra Large)
  * Adjustable in settings
* Swap out uses of reader font in headings and replaced with slightly
larger Ubuntu font
* Replaced PixelArial14 font as it was difficult to track down, replace
with Space Grotesk
* Remove auto formatting on generated font files
* Massively speeds up formatting step now that there is a lot more CPP
font source
* Include fonts with their licenses in the repo

## Additional Context

Line compression setting will follow

| Font | Small | Medium | Large | X Large |
| --- | --- | --- | --- | --- |
| Aleo |
![IMG_5704](https://github.com/user-attachments/assets/7acb054f-ddef-4080-b3c8-590cfaf13115)
|
![IMG_5705](https://github.com/user-attachments/assets/d4819036-5c89-486e-92c3-86094fa4d89a)
|
![IMG_5706](https://github.com/user-attachments/assets/35caf622-d126-4396-9c3e-f927eba1e1f4)
|
![IMG_5707](https://github.com/user-attachments/assets/af32370a-6244-400f-bea9-5c27db040b5b)
|
| Noto Sans |
![IMG_5708](https://github.com/user-attachments/assets/1f9264a5-c069-4e22-9099-a082bfcaabc5)
|
![IMG_5709](https://github.com/user-attachments/assets/ef6b07fe-8d87-403a-b152-05f50b69b78e)
|
![IMG_5710](https://github.com/user-attachments/assets/112a5d20-262c-4dc0-b67d-980b237e4607)
|
![IMG_5711](https://github.com/user-attachments/assets/d25e0e1d-2ace-450d-96dd-618e4efd4805)
|
| Open Dyslexic |
![IMG_5712](https://github.com/user-attachments/assets/ead64690-f261-4fae-a4a2-0becd1162e2d)
|
![IMG_5713](https://github.com/user-attachments/assets/59d60f7d-5142-4591-96b0-c04e0a4c6436)
|
![IMG_5714](https://github.com/user-attachments/assets/bb6652cd-1790-46a3-93ea-2b8f70d0d36d)
|
![IMG_5715](https://github.com/user-attachments/assets/496e7eb4-c81a-4232-83e9-9ba9148fdea4)
|
2025-12-30 19:21:47 +11:00
Dave Allie
fb5fc32c5d Add exFAT support (#150)
## Summary

* Swap to updated SDCardManager which uses SdFat
* Add exFAT support
  * Swap to using FsFile everywhere
* Use newly exposed `SdMan` macro to get to static instance of
SDCardManager
* Move a bunch of FsHelpers up to SDCardManager
2025-12-30 16:09:30 +11:00
dangson
140d8749a6 Support swapping the functionality of the front buttons (#133)
## Summary

**What is the goal of this PR?** 

Adds a setting to swap the front buttons. The default functionality are:
Back/Confirm/Left/Right. When this setting is enabled they become:
Left/Right/Back/Confirm. This makes it more comfortable to use when
holding in your right hand since your thumb can more easily rest on the
next button. The original firmware has a similar setting.

**What changes are included?**

- Add the new setting.
- Create a mapper to dynamically switch the buttons based on the
setting.
- Use mapper on the various activity screens.
- Update the button hints to reflect the swapped buttons.

## Additional Context

Full disclosure: I used Codex CLI to put this PR together, but did
review it to make sure it makes sense.

Also tested on my device:
https://share.cleanshot.com/k76891NY
2025-12-29 14:59:14 +11:00
Dave Allie
1c33162368 Fix rendering issue with entering keyboard from wifi screen 2025-12-28 21:50:45 +11:00
Dave Allie
bf031fd999 Fix exiting WifiSelectionActivity renderer early 2025-12-28 19:27:00 +11:00
Dave Allie
02350c6a9f Fix underscore on keyboard and standardize activity (#138)
## Summary

* Fix underscore on keyboard
  * Remove special handling of special row characters
* Fix navigating between special row items
* Standardize keyboard activity to use standard loop
  * Fix issue with rendering keyboard non-stop

Fixes https://github.com/daveallie/crosspoint-reader/issues/131
2025-12-28 18:57:06 +11:00
Dave Allie
aff4dc6628 Fix QRCode import attempt 2 2025-12-26 11:33:41 +10:00
Dave Allie
98a39374e8 Fix QRCode import 2025-12-26 11:29:27 +10:00
Jonas Diemer
e8c0fb42d4 Network details QR code (#113)
Using QRCode library from pio to generate the QR code.

Done:
- Display QR code for URL in network mode
- minor fixes of layout
- Display QR for URL in AP mode
- Display QR for AP in AP mode

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-26 12:13:40 +11:00
Brendan O'Leary
e3c1e28b8f Normalize button hints (#130)
## Summary

This creates a `renderer.drawButtonHints` to make all of the "hints"
over buttons to match the home screen.

## Additional Context

* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks, specific areas to
focus on).

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-26 11:54:02 +11:00
Brendan O'Leary
9f4f71fabe Add AP mode option for file transfers (#98)
## Summary

* **What is the goal of this PR?** Adds WiFi Access Point (AP) mode
support for File Transfer, allowing the device to create its own WiFi
network that users can connect to directly - useful when no existing
WiFi network is available. And in my experience is faster when the
device is right next to your laptop (but maybe further from your wifi)

* **What changes are included?**
- New `NetworkModeSelectionActivity` - an interstitial screen asking
users to choose between:
- "Join a Network" - connects to an existing WiFi network (existing
behavior)
- "Create Hotspot" - creates a WiFi access point named
"CrossPoint-Reader"
  - Modified `CrossPointWebServerActivity` to:
    - Launch the network mode selection screen before proceeding
- Support starting an Access Point with mDNS (`crosspoint.local`) and
DNS server for captive portal behavior
    - Display appropriate connection info for both modes
- Modified `CrossPointWebServer` to support starting when WiFi is in AP
mode (not just STA connected mode)

## Additional Context

* **AP Mode Details**: The device creates an open WiFi network named
"CrossPoint-Reader". Once connected, users can access the file transfer
page at `http://crosspoint.local/` or `http://192.168.4.1/`
* **DNS Captive Portal**: A DNS server redirects all domain requests to
the device's IP, enabling captive portal behavior on some devices
* **mDNS**: Hostname resolution via `crosspoint.local` is enabled for
both AP and STA modes
* **No breaking changes**: The "Join a Network" option preserves the
existing WiFi connection flow
* **Memory impact**: Minimal - the AP mode uses roughly the same
resources as STA mode
2025-12-22 17:24:14 +11:00
Dave Allie
b39ce22e54 Cleanup of activities 2025-12-22 00:48:16 +11:00
Dave Allie
77c655fcf5 Give activities names and log when entering and exiting them (#92)
## Summary

* Give activities name and log when entering and exiting them
* Clearer logs when attempting to debug, knowing where users are coming
from/going to helps
2025-12-21 21:17:00 +11:00
Dave Allie
0d32d21d75 Small code cleanup (#83)
## Summary

* Fix cppcheck low violations
* Remove teardown method on parsers, use destructor
* Code cleanup
2025-12-21 15:43:53 +11:00
Dave Allie
9b4dfbd180 Allow any file to be uploaded (#84)
## Summary

- Allow any file to be uploaded
- Removes .epub restriction

## Additional Context

- Fixes #74
2025-12-21 15:43:17 +11:00
Brendan O'Leary
d41d539435 Add connect to Wifi and File Manager Webserver (#41)
## Summary

- **What is the goal of this PR?**  
Implements wireless EPUB file management via a built-in web server,
enabling users to upload, browse, organize, and delete EPUB files from
any device on the same WiFi network without needing a computer cable
connection.

- **What changes are included?**
- **New Web Server**
([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp),
[`CrossPointWebServer.h`](src/CrossPointWebServer.h)):
    - HTTP server on port 80 with a responsive HTML/CSS interface
    - Home page showing device status (version, IP, free memory)
    - File Manager with folder navigation and breadcrumb support
    - EPUB file upload with progress tracking
    - Folder creation and file/folder deletion
    - XSS protection via HTML escaping
- Hidden system folders (`.` prefixed, "System Volume Information",
"XTCache")
  
- **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp),
[`WifiScreen.h`](src/screens/WifiScreen.h)):
    - Network scanning with signal strength indicators
    - Visual indicators for encrypted (`*`) and saved (`+`) networks
- State machine managing: scanning, network selection, password entry,
connecting, save/forget prompts
    - 15-second connection timeout handling
    - Integration with web server (starts on connect, stops on exit)
  
- **WiFi Credential Storage**
([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp),
[`WifiCredentialStore.h`](src/WifiCredentialStore.h)):
    - Persistent storage in `/sd/.crosspoint/wifi.bin`
- XOR obfuscation for stored passwords (basic protection against casual
reading)
    - Up to 8 saved networks with add/remove/update operations
  
- **On-Screen Keyboard**
([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp),
[`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)):
    - Reusable QWERTY keyboard component with shift support
    - Special keys: Shift, Space, Backspace, Done
    - Support for password masking mode
  
- **Settings Screen Integration**
([`SettingsScreen.h`](src/screens/SettingsScreen.h)):
    - Added WiFi action to navigate to the new WiFi screen
  
  - **Documentation** ([`docs/webserver.md`](docs/webserver.md)):
- Comprehensive user guide covering WiFi setup, web interface usage,
file management, troubleshooting, and security notes
    - See this for more screenshots!
- Working "displays the right way in GitHub" on my repo:
https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md

**Video demo**


https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731

## Additional Context

- **Security considerations**: The web server has no
authentication—anyone on the same WiFi network can access files. This is
documented as a limitation, recommending use only on trusted private
networks. Password obfuscation in the credential store is XOR-based, not
cryptographically secure.

- **Memory implications**: The web server and WiFi stack consume
significant memory. The implementation properly cleans up (stops server,
disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to
free resources.

- **Async operations**: Network scanning and connection use async
patterns with FreeRTOS tasks to prevent blocking the UI. The display
task handles rendering on a dedicated thread with mutex protection.

- **Browser compatibility**: The web interface uses standard
HTML5/CSS3/JavaScript and is tested to work with all modern browsers on
desktop and mobile.

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-20 01:05:43 +11:00