crosspoint-reader/src/network/CrossPointWebServer.cpp

823 lines
28 KiB
C++
Raw Normal View History

Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#include "CrossPointWebServer.h"
#include <ArduinoJson.h>
2026-01-21 05:06:07 -08:00
#include <Epub.h>
#include <FsHelpers.h>
#include <SDCardManager.h>
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#include <WiFi.h>
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
#include <esp_task_wdt.h>
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#include <algorithm>
#include "html/FilesPageHtml.generated.h"
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#include "html/HomePageHtml.generated.h"
2026-01-21 05:06:07 -08:00
#include "util/StringUtils.h"
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
namespace {
// Folders/files to hide from the web interface file browser
// Note: Items starting with "." are automatically hidden
const char* HIDDEN_ITEMS[] = {"System Volume Information", "XTCache"};
constexpr size_t HIDDEN_ITEMS_COUNT = sizeof(HIDDEN_ITEMS) / sizeof(HIDDEN_ITEMS[0]);
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Static pointer for WebSocket callback (WebSocketsServer requires C-style callback)
CrossPointWebServer* wsInstance = nullptr;
// WebSocket upload state
FsFile wsUploadFile;
String wsUploadFileName;
String wsUploadPath;
size_t wsUploadSize = 0;
size_t wsUploadReceived = 0;
unsigned long wsUploadStartTime = 0;
bool wsUploadInProgress = false;
2026-01-21 05:06:07 -08:00
// Helper function to clear epub cache after upload
void clearEpubCacheIfNeeded(const String& filePath) {
// Only clear cache for .epub files
if (StringUtils::checkFileExtension(filePath, ".epub")) {
Epub(filePath.c_str(), "/.crosspoint").clearCache();
Serial.printf("[%lu] [WEB] Cleared epub cache for: %s\n", millis(), filePath.c_str());
}
}
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
} // namespace
// File listing page template - now using generated headers:
// - HomePageHtml (from html/HomePage.html)
// - FilesPageHeaderHtml (from html/FilesPageHeader.html)
// - FilesPageFooterHtml (from html/FilesPageFooter.html)
CrossPointWebServer::CrossPointWebServer() {}
CrossPointWebServer::~CrossPointWebServer() { stop(); }
void CrossPointWebServer::begin() {
if (running) {
Serial.printf("[%lu] [WEB] Web server already running\n", millis());
return;
}
Add AP mode option for file transfers (#98) ## Summary * **What is the goal of this PR?** Adds WiFi Access Point (AP) mode support for File Transfer, allowing the device to create its own WiFi network that users can connect to directly - useful when no existing WiFi network is available. And in my experience is faster when the device is right next to your laptop (but maybe further from your wifi) * **What changes are included?** - New `NetworkModeSelectionActivity` - an interstitial screen asking users to choose between: - "Join a Network" - connects to an existing WiFi network (existing behavior) - "Create Hotspot" - creates a WiFi access point named "CrossPoint-Reader" - Modified `CrossPointWebServerActivity` to: - Launch the network mode selection screen before proceeding - Support starting an Access Point with mDNS (`crosspoint.local`) and DNS server for captive portal behavior - Display appropriate connection info for both modes - Modified `CrossPointWebServer` to support starting when WiFi is in AP mode (not just STA connected mode) ## Additional Context * **AP Mode Details**: The device creates an open WiFi network named "CrossPoint-Reader". Once connected, users can access the file transfer page at `http://crosspoint.local/` or `http://192.168.4.1/` * **DNS Captive Portal**: A DNS server redirects all domain requests to the device's IP, enabling captive portal behavior on some devices * **mDNS**: Hostname resolution via `crosspoint.local` is enabled for both AP and STA modes * **No breaking changes**: The "Join a Network" option preserves the existing WiFi connection flow * **Memory impact**: Minimal - the AP mode uses roughly the same resources as STA mode
2025-12-22 01:24:14 -05:00
// Check if we have a valid network connection (either STA connected or AP mode)
const wifi_mode_t wifiMode = WiFi.getMode();
const bool isStaConnected = (wifiMode & WIFI_MODE_STA) && (WiFi.status() == WL_CONNECTED);
const bool isInApMode = (wifiMode & WIFI_MODE_AP) && (WiFi.softAPgetStationNum() >= 0); // AP is running
if (!isStaConnected && !isInApMode) {
Serial.printf("[%lu] [WEB] Cannot start webserver - no valid network (mode=%d, status=%d)\n", millis(), wifiMode,
WiFi.status());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
return;
}
Add AP mode option for file transfers (#98) ## Summary * **What is the goal of this PR?** Adds WiFi Access Point (AP) mode support for File Transfer, allowing the device to create its own WiFi network that users can connect to directly - useful when no existing WiFi network is available. And in my experience is faster when the device is right next to your laptop (but maybe further from your wifi) * **What changes are included?** - New `NetworkModeSelectionActivity` - an interstitial screen asking users to choose between: - "Join a Network" - connects to an existing WiFi network (existing behavior) - "Create Hotspot" - creates a WiFi access point named "CrossPoint-Reader" - Modified `CrossPointWebServerActivity` to: - Launch the network mode selection screen before proceeding - Support starting an Access Point with mDNS (`crosspoint.local`) and DNS server for captive portal behavior - Display appropriate connection info for both modes - Modified `CrossPointWebServer` to support starting when WiFi is in AP mode (not just STA connected mode) ## Additional Context * **AP Mode Details**: The device creates an open WiFi network named "CrossPoint-Reader". Once connected, users can access the file transfer page at `http://crosspoint.local/` or `http://192.168.4.1/` * **DNS Captive Portal**: A DNS server redirects all domain requests to the device's IP, enabling captive portal behavior on some devices * **mDNS**: Hostname resolution via `crosspoint.local` is enabled for both AP and STA modes * **No breaking changes**: The "Join a Network" option preserves the existing WiFi connection flow * **Memory impact**: Minimal - the AP mode uses roughly the same resources as STA mode
2025-12-22 01:24:14 -05:00
// Store AP mode flag for later use (e.g., in handleStatus)
apMode = isInApMode;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [MEM] Free heap before begin: %d bytes\n", millis(), ESP.getFreeHeap());
Add AP mode option for file transfers (#98) ## Summary * **What is the goal of this PR?** Adds WiFi Access Point (AP) mode support for File Transfer, allowing the device to create its own WiFi network that users can connect to directly - useful when no existing WiFi network is available. And in my experience is faster when the device is right next to your laptop (but maybe further from your wifi) * **What changes are included?** - New `NetworkModeSelectionActivity` - an interstitial screen asking users to choose between: - "Join a Network" - connects to an existing WiFi network (existing behavior) - "Create Hotspot" - creates a WiFi access point named "CrossPoint-Reader" - Modified `CrossPointWebServerActivity` to: - Launch the network mode selection screen before proceeding - Support starting an Access Point with mDNS (`crosspoint.local`) and DNS server for captive portal behavior - Display appropriate connection info for both modes - Modified `CrossPointWebServer` to support starting when WiFi is in AP mode (not just STA connected mode) ## Additional Context * **AP Mode Details**: The device creates an open WiFi network named "CrossPoint-Reader". Once connected, users can access the file transfer page at `http://crosspoint.local/` or `http://192.168.4.1/` * **DNS Captive Portal**: A DNS server redirects all domain requests to the device's IP, enabling captive portal behavior on some devices * **mDNS**: Hostname resolution via `crosspoint.local` is enabled for both AP and STA modes * **No breaking changes**: The "Join a Network" option preserves the existing WiFi connection flow * **Memory impact**: Minimal - the AP mode uses roughly the same resources as STA mode
2025-12-22 01:24:14 -05:00
Serial.printf("[%lu] [WEB] Network mode: %s\n", millis(), apMode ? "AP" : "STA");
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Creating web server on port %d...\n", millis(), port);
server.reset(new WebServer(port));
// Disable WiFi sleep to improve responsiveness and prevent 'unreachable' errors.
// This is critical for reliable web server operation on ESP32.
WiFi.setSleep(false);
// Note: WebServer class doesn't have setNoDelay() in the standard ESP32 library.
// We rely on disabling WiFi sleep for responsiveness.
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [MEM] Free heap after WebServer allocation: %d bytes\n", millis(), ESP.getFreeHeap());
if (!server) {
Serial.printf("[%lu] [WEB] Failed to create WebServer!\n", millis());
return;
}
// Setup routes
Serial.printf("[%lu] [WEB] Setting up routes...\n", millis());
server->on("/", HTTP_GET, [this] { handleRoot(); });
server->on("/files", HTTP_GET, [this] { handleFileList(); });
server->on("/api/status", HTTP_GET, [this] { handleStatus(); });
server->on("/api/files", HTTP_GET, [this] { handleFileListData(); });
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Upload endpoint with special handling for multipart form data
server->on("/upload", HTTP_POST, [this] { handleUploadPost(); }, [this] { handleUpload(); });
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Create folder endpoint
server->on("/mkdir", HTTP_POST, [this] { handleCreateFolder(); });
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Delete file/folder endpoint
server->on("/delete", HTTP_POST, [this] { handleDelete(); });
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server->onNotFound([this] { handleNotFound(); });
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [MEM] Free heap after route setup: %d bytes\n", millis(), ESP.getFreeHeap());
server->begin();
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Start WebSocket server for fast binary uploads
Serial.printf("[%lu] [WEB] Starting WebSocket server on port %d...\n", millis(), wsPort);
wsServer.reset(new WebSocketsServer(wsPort));
wsInstance = const_cast<CrossPointWebServer*>(this);
wsServer->begin();
wsServer->onEvent(wsEventCallback);
Serial.printf("[%lu] [WEB] WebSocket server started\n", millis());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
running = true;
Serial.printf("[%lu] [WEB] Web server started on port %d\n", millis(), port);
Add AP mode option for file transfers (#98) ## Summary * **What is the goal of this PR?** Adds WiFi Access Point (AP) mode support for File Transfer, allowing the device to create its own WiFi network that users can connect to directly - useful when no existing WiFi network is available. And in my experience is faster when the device is right next to your laptop (but maybe further from your wifi) * **What changes are included?** - New `NetworkModeSelectionActivity` - an interstitial screen asking users to choose between: - "Join a Network" - connects to an existing WiFi network (existing behavior) - "Create Hotspot" - creates a WiFi access point named "CrossPoint-Reader" - Modified `CrossPointWebServerActivity` to: - Launch the network mode selection screen before proceeding - Support starting an Access Point with mDNS (`crosspoint.local`) and DNS server for captive portal behavior - Display appropriate connection info for both modes - Modified `CrossPointWebServer` to support starting when WiFi is in AP mode (not just STA connected mode) ## Additional Context * **AP Mode Details**: The device creates an open WiFi network named "CrossPoint-Reader". Once connected, users can access the file transfer page at `http://crosspoint.local/` or `http://192.168.4.1/` * **DNS Captive Portal**: A DNS server redirects all domain requests to the device's IP, enabling captive portal behavior on some devices * **mDNS**: Hostname resolution via `crosspoint.local` is enabled for both AP and STA modes * **No breaking changes**: The "Join a Network" option preserves the existing WiFi connection flow * **Memory impact**: Minimal - the AP mode uses roughly the same resources as STA mode
2025-12-22 01:24:14 -05:00
// Show the correct IP based on network mode
const String ipAddr = apMode ? WiFi.softAPIP().toString() : WiFi.localIP().toString();
Serial.printf("[%lu] [WEB] Access at http://%s/\n", millis(), ipAddr.c_str());
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
Serial.printf("[%lu] [WEB] WebSocket at ws://%s:%d/\n", millis(), ipAddr.c_str(), wsPort);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [MEM] Free heap after server.begin(): %d bytes\n", millis(), ESP.getFreeHeap());
}
void CrossPointWebServer::stop() {
if (!running || !server) {
Serial.printf("[%lu] [WEB] stop() called but already stopped (running=%d, server=%p)\n", millis(), running,
server.get());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
return;
}
Serial.printf("[%lu] [WEB] STOP INITIATED - setting running=false first\n", millis());
running = false; // Set this FIRST to prevent handleClient from using server
Serial.printf("[%lu] [WEB] [MEM] Free heap before stop: %d bytes\n", millis(), ESP.getFreeHeap());
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Close any in-progress WebSocket upload
if (wsUploadInProgress && wsUploadFile) {
wsUploadFile.close();
wsUploadInProgress = false;
}
// Stop WebSocket server
if (wsServer) {
Serial.printf("[%lu] [WEB] Stopping WebSocket server...\n", millis());
wsServer->close();
wsServer.reset();
wsInstance = nullptr;
Serial.printf("[%lu] [WEB] WebSocket server stopped\n", millis());
}
// Brief delay to allow any in-flight handleClient() calls to complete
delay(20);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server->stop();
Serial.printf("[%lu] [WEB] [MEM] Free heap after server->stop(): %d bytes\n", millis(), ESP.getFreeHeap());
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Brief delay before deletion
delay(10);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server.reset();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Web server stopped and deleted\n", millis());
Serial.printf("[%lu] [WEB] [MEM] Free heap after delete server: %d bytes\n", millis(), ESP.getFreeHeap());
// Note: Static upload variables (uploadFileName, uploadPath, uploadError) are declared
// later in the file and will be cleared when they go out of scope or on next upload
Serial.printf("[%lu] [WEB] [MEM] Free heap final: %d bytes\n", millis(), ESP.getFreeHeap());
}
void CrossPointWebServer::handleClient() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
static unsigned long lastDebugPrint = 0;
// Check running flag FIRST before accessing server
if (!running) {
return;
}
// Double-check server pointer is valid
if (!server) {
Serial.printf("[%lu] [WEB] WARNING: handleClient called with null server!\n", millis());
return;
}
// Print debug every 10 seconds to confirm handleClient is being called
if (millis() - lastDebugPrint > 10000) {
Serial.printf("[%lu] [WEB] handleClient active, server running on port %d\n", millis(), port);
lastDebugPrint = millis();
}
server->handleClient();
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Handle WebSocket events
if (wsServer) {
wsServer->loop();
}
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
void CrossPointWebServer::handleRoot() const {
server->send(200, "text/html", HomePageHtml);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Served root page\n", millis());
}
void CrossPointWebServer::handleNotFound() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
String message = "404 Not Found\n\n";
message += "URI: " + server->uri() + "\n";
server->send(404, "text/plain", message);
}
void CrossPointWebServer::handleStatus() const {
Add AP mode option for file transfers (#98) ## Summary * **What is the goal of this PR?** Adds WiFi Access Point (AP) mode support for File Transfer, allowing the device to create its own WiFi network that users can connect to directly - useful when no existing WiFi network is available. And in my experience is faster when the device is right next to your laptop (but maybe further from your wifi) * **What changes are included?** - New `NetworkModeSelectionActivity` - an interstitial screen asking users to choose between: - "Join a Network" - connects to an existing WiFi network (existing behavior) - "Create Hotspot" - creates a WiFi access point named "CrossPoint-Reader" - Modified `CrossPointWebServerActivity` to: - Launch the network mode selection screen before proceeding - Support starting an Access Point with mDNS (`crosspoint.local`) and DNS server for captive portal behavior - Display appropriate connection info for both modes - Modified `CrossPointWebServer` to support starting when WiFi is in AP mode (not just STA connected mode) ## Additional Context * **AP Mode Details**: The device creates an open WiFi network named "CrossPoint-Reader". Once connected, users can access the file transfer page at `http://crosspoint.local/` or `http://192.168.4.1/` * **DNS Captive Portal**: A DNS server redirects all domain requests to the device's IP, enabling captive portal behavior on some devices * **mDNS**: Hostname resolution via `crosspoint.local` is enabled for both AP and STA modes * **No breaking changes**: The "Join a Network" option preserves the existing WiFi connection flow * **Memory impact**: Minimal - the AP mode uses roughly the same resources as STA mode
2025-12-22 01:24:14 -05:00
// Get correct IP based on AP vs STA mode
const String ipAddr = apMode ? WiFi.softAPIP().toString() : WiFi.localIP().toString();
JsonDocument doc;
doc["version"] = CROSSPOINT_VERSION;
doc["ip"] = ipAddr;
doc["mode"] = apMode ? "AP" : "STA";
doc["rssi"] = apMode ? 0 : WiFi.RSSI();
doc["freeHeap"] = ESP.getFreeHeap();
doc["uptime"] = millis() / 1000;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
String json;
serializeJson(doc, json);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server->send(200, "application/json", json);
}
void CrossPointWebServer::scanFiles(const char* path, const std::function<void(FileInfo)>& callback) const {
FsFile root = SdMan.open(path);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (!root) {
Serial.printf("[%lu] [WEB] Failed to open directory: %s\n", millis(), path);
return;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
if (!root.isDirectory()) {
Serial.printf("[%lu] [WEB] Not a directory: %s\n", millis(), path);
root.close();
return;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
Serial.printf("[%lu] [WEB] Scanning files in: %s\n", millis(), path);
FsFile file = root.openNextFile();
char name[500];
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
while (file) {
file.getName(name, sizeof(name));
auto fileName = String(name);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Skip hidden items (starting with ".")
bool shouldHide = fileName.startsWith(".");
// Check against explicitly hidden items list
if (!shouldHide) {
for (size_t i = 0; i < HIDDEN_ITEMS_COUNT; i++) {
if (fileName.equals(HIDDEN_ITEMS[i])) {
shouldHide = true;
break;
}
}
}
if (!shouldHide) {
FileInfo info;
info.name = fileName;
info.isDirectory = file.isDirectory();
if (info.isDirectory) {
info.size = 0;
info.isEpub = false;
} else {
info.size = file.size();
info.isEpub = isEpubFile(info.name);
}
callback(info);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
file.close();
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
yield(); // Yield to allow WiFi and other tasks to process during long scans
esp_task_wdt_reset(); // Reset watchdog to prevent timeout on large directories
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
file = root.openNextFile();
}
root.close();
}
bool CrossPointWebServer::isEpubFile(const String& filename) const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
String lower = filename;
lower.toLowerCase();
return lower.endsWith(".epub");
}
void CrossPointWebServer::handleFileList() const { server->send(200, "text/html", FilesPageHtml); }
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
void CrossPointWebServer::handleFileListData() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Get current path from query string (default to root)
String currentPath = "/";
if (server->hasArg("path")) {
currentPath = server->arg("path");
// Ensure path starts with /
if (!currentPath.startsWith("/")) {
currentPath = "/" + currentPath;
}
// Remove trailing slash unless it's root
if (currentPath.length() > 1 && currentPath.endsWith("/")) {
currentPath = currentPath.substring(0, currentPath.length() - 1);
}
}
server->setContentLength(CONTENT_LENGTH_UNKNOWN);
server->send(200, "application/json", "");
server->sendContent("[");
char output[512];
constexpr size_t outputSize = sizeof(output);
bool seenFirst = false;
JsonDocument doc;
scanFiles(currentPath.c_str(), [this, &output, &doc, seenFirst](const FileInfo& info) mutable {
doc.clear();
doc["name"] = info.name;
doc["size"] = info.size;
doc["isDirectory"] = info.isDirectory;
doc["isEpub"] = info.isEpub;
const size_t written = serializeJson(doc, output, outputSize);
if (written >= outputSize) {
// JSON output truncated; skip this entry to avoid sending malformed JSON
Serial.printf("[%lu] [WEB] Skipping file entry with oversized JSON for name: %s\n", millis(), info.name.c_str());
return;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
if (seenFirst) {
server->sendContent(",");
} else {
seenFirst = true;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
server->sendContent(output);
});
server->sendContent("]");
// End of streamed response, empty chunk to signal client
server->sendContent("");
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Served file listing page for path: %s\n", millis(), currentPath.c_str());
}
// Static variables for upload handling
static FsFile uploadFile;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
static String uploadFileName;
static String uploadPath = "/";
static size_t uploadSize = 0;
static bool uploadSuccess = false;
static String uploadError = "";
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Upload write buffer - batches small writes into larger SD card operations
// 4KB is a good balance: large enough to reduce syscall overhead, small enough
// to keep individual write times short and avoid watchdog issues
constexpr size_t UPLOAD_BUFFER_SIZE = 4096; // 4KB buffer
static uint8_t uploadBuffer[UPLOAD_BUFFER_SIZE];
static size_t uploadBufferPos = 0;
// Diagnostic counters for upload performance analysis
static unsigned long uploadStartTime = 0;
static unsigned long totalWriteTime = 0;
static size_t writeCount = 0;
static bool flushUploadBuffer() {
if (uploadBufferPos > 0 && uploadFile) {
esp_task_wdt_reset(); // Reset watchdog before potentially slow SD write
const unsigned long writeStart = millis();
const size_t written = uploadFile.write(uploadBuffer, uploadBufferPos);
totalWriteTime += millis() - writeStart;
writeCount++;
esp_task_wdt_reset(); // Reset watchdog after SD write
if (written != uploadBufferPos) {
Serial.printf("[%lu] [WEB] [UPLOAD] Buffer flush failed: expected %d, wrote %d\n", millis(), uploadBufferPos,
written);
uploadBufferPos = 0;
return false;
}
uploadBufferPos = 0;
}
return true;
}
void CrossPointWebServer::handleUpload() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
static size_t lastLoggedSize = 0;
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Reset watchdog at start of every upload callback - HTTP parsing can be slow
esp_task_wdt_reset();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Safety check: ensure server is still valid
if (!running || !server) {
Serial.printf("[%lu] [WEB] [UPLOAD] ERROR: handleUpload called but server not running!\n", millis());
return;
}
const HTTPUpload& upload = server->upload();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (upload.status == UPLOAD_FILE_START) {
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Reset watchdog - this is the critical 1% crash point
esp_task_wdt_reset();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
uploadFileName = upload.filename;
uploadSize = 0;
uploadSuccess = false;
uploadError = "";
uploadStartTime = millis();
lastLoggedSize = 0;
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
uploadBufferPos = 0;
totalWriteTime = 0;
writeCount = 0;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Get upload path from query parameter (defaults to root if not specified)
// Note: We use query parameter instead of form data because multipart form
// fields aren't available until after file upload completes
if (server->hasArg("path")) {
uploadPath = server->arg("path");
// Ensure path starts with /
if (!uploadPath.startsWith("/")) {
uploadPath = "/" + uploadPath;
}
// Remove trailing slash unless it's root
if (uploadPath.length() > 1 && uploadPath.endsWith("/")) {
uploadPath = uploadPath.substring(0, uploadPath.length() - 1);
}
} else {
uploadPath = "/";
}
Serial.printf("[%lu] [WEB] [UPLOAD] START: %s to path: %s\n", millis(), uploadFileName.c_str(), uploadPath.c_str());
Serial.printf("[%lu] [WEB] [UPLOAD] Free heap: %d bytes\n", millis(), ESP.getFreeHeap());
// Create file path
String filePath = uploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += uploadFileName;
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Check if file already exists - SD operations can be slow
esp_task_wdt_reset();
if (SdMan.exists(filePath.c_str())) {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [UPLOAD] Overwriting existing file: %s\n", millis(), filePath.c_str());
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
esp_task_wdt_reset();
SdMan.remove(filePath.c_str());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Open file for writing - this can be slow due to FAT cluster allocation
esp_task_wdt_reset();
if (!SdMan.openFileForWrite("WEB", filePath, uploadFile)) {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
uploadError = "Failed to create file on SD card";
Serial.printf("[%lu] [WEB] [UPLOAD] FAILED to create file: %s\n", millis(), filePath.c_str());
return;
}
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
esp_task_wdt_reset();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] [UPLOAD] File created successfully: %s\n", millis(), filePath.c_str());
} else if (upload.status == UPLOAD_FILE_WRITE) {
if (uploadFile && uploadError.isEmpty()) {
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Buffer incoming data and flush when buffer is full
// This reduces SD card write operations and improves throughput
const uint8_t* data = upload.buf;
size_t remaining = upload.currentSize;
while (remaining > 0) {
const size_t space = UPLOAD_BUFFER_SIZE - uploadBufferPos;
const size_t toCopy = (remaining < space) ? remaining : space;
memcpy(uploadBuffer + uploadBufferPos, data, toCopy);
uploadBufferPos += toCopy;
data += toCopy;
remaining -= toCopy;
// Flush buffer when full
if (uploadBufferPos >= UPLOAD_BUFFER_SIZE) {
if (!flushUploadBuffer()) {
uploadError = "Failed to write to SD card - disk may be full";
uploadFile.close();
return;
}
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
}
uploadSize += upload.currentSize;
// Log progress every 100KB
if (uploadSize - lastLoggedSize >= 102400) {
const unsigned long elapsed = millis() - uploadStartTime;
const float kbps = (elapsed > 0) ? (uploadSize / 1024.0) / (elapsed / 1000.0) : 0;
Serial.printf("[%lu] [WEB] [UPLOAD] %d bytes (%.1f KB), %.1f KB/s, %d writes\n", millis(), uploadSize,
uploadSize / 1024.0, kbps, writeCount);
lastLoggedSize = uploadSize;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
}
} else if (upload.status == UPLOAD_FILE_END) {
if (uploadFile) {
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// Flush any remaining buffered data
if (!flushUploadBuffer()) {
uploadError = "Failed to write final data to SD card";
}
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
uploadFile.close();
if (uploadError.isEmpty()) {
uploadSuccess = true;
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
const unsigned long elapsed = millis() - uploadStartTime;
const float avgKbps = (elapsed > 0) ? (uploadSize / 1024.0) / (elapsed / 1000.0) : 0;
const float writePercent = (elapsed > 0) ? (totalWriteTime * 100.0 / elapsed) : 0;
Serial.printf("[%lu] [WEB] [UPLOAD] Complete: %s (%d bytes in %lu ms, avg %.1f KB/s)\n", millis(),
uploadFileName.c_str(), uploadSize, elapsed, avgKbps);
Serial.printf("[%lu] [WEB] [UPLOAD] Diagnostics: %d writes, total write time: %lu ms (%.1f%%)\n", millis(),
writeCount, totalWriteTime, writePercent);
2026-01-21 05:06:07 -08:00
// Clear epub cache to prevent stale metadata issues when overwriting files
String filePath = uploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += uploadFileName;
clearEpubCacheIfNeeded(filePath);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
}
} else if (upload.status == UPLOAD_FILE_ABORTED) {
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
uploadBufferPos = 0; // Discard buffered data
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (uploadFile) {
uploadFile.close();
// Try to delete the incomplete file
String filePath = uploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += uploadFileName;
SdMan.remove(filePath.c_str());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
uploadError = "Upload aborted";
Serial.printf("[%lu] [WEB] Upload aborted\n", millis());
}
}
void CrossPointWebServer::handleUploadPost() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (uploadSuccess) {
server->send(200, "text/plain", "File uploaded successfully: " + uploadFileName);
} else {
const String error = uploadError.isEmpty() ? "Unknown error during upload" : uploadError;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server->send(400, "text/plain", error);
}
}
void CrossPointWebServer::handleCreateFolder() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Get folder name from form data
if (!server->hasArg("name")) {
server->send(400, "text/plain", "Missing folder name");
return;
}
const String folderName = server->arg("name");
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Validate folder name
if (folderName.isEmpty()) {
server->send(400, "text/plain", "Folder name cannot be empty");
return;
}
// Get parent path
String parentPath = "/";
if (server->hasArg("path")) {
parentPath = server->arg("path");
if (!parentPath.startsWith("/")) {
parentPath = "/" + parentPath;
}
if (parentPath.length() > 1 && parentPath.endsWith("/")) {
parentPath = parentPath.substring(0, parentPath.length() - 1);
}
}
// Build full folder path
String folderPath = parentPath;
if (!folderPath.endsWith("/")) folderPath += "/";
folderPath += folderName;
Serial.printf("[%lu] [WEB] Creating folder: %s\n", millis(), folderPath.c_str());
// Check if already exists
if (SdMan.exists(folderPath.c_str())) {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
server->send(400, "text/plain", "Folder already exists");
return;
}
// Create the folder
if (SdMan.mkdir(folderPath.c_str())) {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Folder created successfully: %s\n", millis(), folderPath.c_str());
server->send(200, "text/plain", "Folder created: " + folderName);
} else {
Serial.printf("[%lu] [WEB] Failed to create folder: %s\n", millis(), folderPath.c_str());
server->send(500, "text/plain", "Failed to create folder");
}
}
void CrossPointWebServer::handleDelete() const {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Get path from form data
if (!server->hasArg("path")) {
server->send(400, "text/plain", "Missing path");
return;
}
String itemPath = server->arg("path");
const String itemType = server->hasArg("type") ? server->arg("type") : "file";
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Validate path
if (itemPath.isEmpty() || itemPath == "/") {
server->send(400, "text/plain", "Cannot delete root directory");
return;
}
// Ensure path starts with /
if (!itemPath.startsWith("/")) {
itemPath = "/" + itemPath;
}
// Security check: prevent deletion of protected items
const String itemName = itemPath.substring(itemPath.lastIndexOf('/') + 1);
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Check if item starts with a dot (hidden/system file)
if (itemName.startsWith(".")) {
Serial.printf("[%lu] [WEB] Delete rejected - hidden/system item: %s\n", millis(), itemPath.c_str());
server->send(403, "text/plain", "Cannot delete system files");
return;
}
// Check against explicitly protected items
for (size_t i = 0; i < HIDDEN_ITEMS_COUNT; i++) {
if (itemName.equals(HIDDEN_ITEMS[i])) {
Serial.printf("[%lu] [WEB] Delete rejected - protected item: %s\n", millis(), itemPath.c_str());
server->send(403, "text/plain", "Cannot delete protected items");
return;
}
}
// Check if item exists
if (!SdMan.exists(itemPath.c_str())) {
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
Serial.printf("[%lu] [WEB] Delete failed - item not found: %s\n", millis(), itemPath.c_str());
server->send(404, "text/plain", "Item not found");
return;
}
Serial.printf("[%lu] [WEB] Attempting to delete %s: %s\n", millis(), itemType.c_str(), itemPath.c_str());
bool success = false;
if (itemType == "folder") {
// For folders, try to remove (will fail if not empty)
FsFile dir = SdMan.open(itemPath.c_str());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (dir && dir.isDirectory()) {
// Check if folder is empty
FsFile entry = dir.openNextFile();
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
if (entry) {
// Folder is not empty
entry.close();
dir.close();
Serial.printf("[%lu] [WEB] Delete failed - folder not empty: %s\n", millis(), itemPath.c_str());
server->send(400, "text/plain", "Folder is not empty. Delete contents first.");
return;
}
dir.close();
}
success = SdMan.rmdir(itemPath.c_str());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
} else {
// For files, use remove
success = SdMan.remove(itemPath.c_str());
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
}
if (success) {
Serial.printf("[%lu] [WEB] Successfully deleted: %s\n", millis(), itemPath.c_str());
server->send(200, "text/plain", "Deleted successfully");
} else {
Serial.printf("[%lu] [WEB] Failed to delete: %s\n", millis(), itemPath.c_str());
server->send(500, "text/plain", "Failed to delete item");
}
}
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
// WebSocket callback trampoline
void CrossPointWebServer::wsEventCallback(uint8_t num, WStype_t type, uint8_t* payload, size_t length) {
if (wsInstance) {
wsInstance->onWebSocketEvent(num, type, payload, length);
}
}
// WebSocket event handler for fast binary uploads
// Protocol:
// 1. Client sends TEXT message: "START:<filename>:<size>:<path>"
// 2. Client sends BINARY messages with file data chunks
// 3. Server sends TEXT "PROGRESS:<received>:<total>" after each chunk
// 4. Server sends TEXT "DONE" or "ERROR:<message>" when complete
void CrossPointWebServer::onWebSocketEvent(uint8_t num, WStype_t type, uint8_t* payload, size_t length) {
switch (type) {
case WStype_DISCONNECTED:
Serial.printf("[%lu] [WS] Client %u disconnected\n", millis(), num);
// Clean up any in-progress upload
if (wsUploadInProgress && wsUploadFile) {
wsUploadFile.close();
// Delete incomplete file
String filePath = wsUploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += wsUploadFileName;
SdMan.remove(filePath.c_str());
Serial.printf("[%lu] [WS] Deleted incomplete upload: %s\n", millis(), filePath.c_str());
}
wsUploadInProgress = false;
break;
case WStype_CONNECTED: {
Serial.printf("[%lu] [WS] Client %u connected\n", millis(), num);
break;
}
case WStype_TEXT: {
// Parse control messages
String msg = String((char*)payload);
Serial.printf("[%lu] [WS] Text from client %u: %s\n", millis(), num, msg.c_str());
if (msg.startsWith("START:")) {
// Parse: START:<filename>:<size>:<path>
int firstColon = msg.indexOf(':', 6);
int secondColon = msg.indexOf(':', firstColon + 1);
if (firstColon > 0 && secondColon > 0) {
wsUploadFileName = msg.substring(6, firstColon);
wsUploadSize = msg.substring(firstColon + 1, secondColon).toInt();
wsUploadPath = msg.substring(secondColon + 1);
wsUploadReceived = 0;
wsUploadStartTime = millis();
// Ensure path is valid
if (!wsUploadPath.startsWith("/")) wsUploadPath = "/" + wsUploadPath;
if (wsUploadPath.length() > 1 && wsUploadPath.endsWith("/")) {
wsUploadPath = wsUploadPath.substring(0, wsUploadPath.length() - 1);
}
// Build file path
String filePath = wsUploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += wsUploadFileName;
Serial.printf("[%lu] [WS] Starting upload: %s (%d bytes) to %s\n", millis(), wsUploadFileName.c_str(),
wsUploadSize, filePath.c_str());
// Check if file exists and remove it
esp_task_wdt_reset();
if (SdMan.exists(filePath.c_str())) {
SdMan.remove(filePath.c_str());
}
// Open file for writing
esp_task_wdt_reset();
if (!SdMan.openFileForWrite("WS", filePath, wsUploadFile)) {
wsServer->sendTXT(num, "ERROR:Failed to create file");
wsUploadInProgress = false;
return;
}
esp_task_wdt_reset();
wsUploadInProgress = true;
wsServer->sendTXT(num, "READY");
} else {
wsServer->sendTXT(num, "ERROR:Invalid START format");
}
}
break;
}
case WStype_BIN: {
if (!wsUploadInProgress || !wsUploadFile) {
wsServer->sendTXT(num, "ERROR:No upload in progress");
return;
}
// Write binary data directly to file
esp_task_wdt_reset();
size_t written = wsUploadFile.write(payload, length);
esp_task_wdt_reset();
if (written != length) {
wsUploadFile.close();
wsUploadInProgress = false;
wsServer->sendTXT(num, "ERROR:Write failed - disk full?");
return;
}
wsUploadReceived += written;
// Send progress update (every 64KB or at end)
static size_t lastProgressSent = 0;
if (wsUploadReceived - lastProgressSent >= 65536 || wsUploadReceived >= wsUploadSize) {
String progress = "PROGRESS:" + String(wsUploadReceived) + ":" + String(wsUploadSize);
wsServer->sendTXT(num, progress);
lastProgressSent = wsUploadReceived;
}
// Check if upload complete
if (wsUploadReceived >= wsUploadSize) {
wsUploadFile.close();
wsUploadInProgress = false;
unsigned long elapsed = millis() - wsUploadStartTime;
float kbps = (elapsed > 0) ? (wsUploadSize / 1024.0) / (elapsed / 1000.0) : 0;
Serial.printf("[%lu] [WS] Upload complete: %s (%d bytes in %lu ms, %.1f KB/s)\n", millis(),
wsUploadFileName.c_str(), wsUploadSize, elapsed, kbps);
2026-01-21 05:06:07 -08:00
// Clear epub cache to prevent stale metadata issues when overwriting files
String filePath = wsUploadPath;
if (!filePath.endsWith("/")) filePath += "/";
filePath += wsUploadFileName;
clearEpubCacheIfNeeded(filePath);
Turbocharge WiFi uploads with WebSocket + watchdog stability (#364) ## Summary * **What is the goal of this PR?** Fix WiFi file transfer stability issues (especially crashes during uploads) and improve upload speed via WebSocket binary protocol. File transfers now don't really crash as much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s. * **What changes are included?** - **WebSocket upload support** - Adds WebSocket binary protocol for file uploads, achieving faster speeds 335 KB/s vs HTTP multipart) - **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls throughout upload path to prevent watchdog timeouts during: - File creation (FAT allocation can be slow) - SD card write operations - HTTP header parsing - WebSocket chunk processing - **4KB write buffering** - Batches SD card writes to reduce I/O overhead - **WiFi health monitoring** - Detects WiFi disconnection in STA mode and exits gracefully - **Improved handleClient loop** - 500 iterations with periodic watchdog resets and button checks for responsiveness - **Progress bar improvements** - Fixed jumping/inaccurate progress by capping local progress at 95% until server confirms completion - **Exit button responsiveness** - Button now checked inside the handleClient loop every 64 iterations - **Reduced exit delays** - Decreased shutdown delays from ~850ms to ~140ms **Files changed:** - `platformio.ini` - Added WebSockets library dependency - `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering, watchdog resets - `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop, button handling - `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback ## Additional Context - WebSocket uses 4KB chunks with backpressure management to prevent ESP32 buffer overflow - Falls back to HTTP automatically if WebSocket connection fails - The main bottleneck now is SD card write speed (~44% of transfer time), not WiFi - STA mode was more prone to crashes than AP mode due to external network factors; WiFi health monitoring helps detect and handle disconnections gracefully --- ### AI Usage Did you use AI tools to help write this code? _**YES**_ Claude did it ALL, I have no idea what I am doing, but my books transfer fast now. --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-01-14 12:11:28 +00:00
wsServer->sendTXT(num, "DONE");
lastProgressSent = 0;
}
break;
}
default:
break;
}
}