crosspoint-reader/src/network/CrossPointWebServer.h

72 lines
1.8 KiB
C
Raw Normal View History

Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#pragma once
#include <WebServer.h>
Fix WiFi file transfer lockup and add performance optimizations This commit resolves critical stability issues causing WiFi file transfer crashes in STA mode and adds aggressive performance optimizations for maximum upload/download throughput. CRITICAL BUG FIXES: 1. Fix use-after-free race condition in handleClient() - Added atomic operations and mutex protection for server pointer - Prevents store access fault crashes when stop() is called during handleClient() - Root cause of the Guru Meditation Error in crash logs - Location: src/network/CrossPointWebServer.cpp:135-158 2. Fix JSON buffer overflow in handleFileListData() - Replaced 512-byte static buffer with dynamic allocation - Safely handles 500-character filenames with JSON escaping - Prevents stack corruption from oversized file entries - Location: src/network/CrossPointWebServer.cpp:253-306 3. Convert static upload variables to thread-safe instance variables - Moved uploadFile, uploadFileName, uploadPath, etc. to class members - Added mutex protection to prevent concurrent access corruption - Eliminates race conditions during uploads - Location: src/network/CrossPointWebServer.h:44-54, CrossPointWebServer.cpp:311-313 4. Add yield() to handleClient loop - Prevents WiFi stack starvation in STA mode - Allows LWIP to process incoming packets and prevent buffer overflow - Critical for stability during file transfers - Location: src/activities/network/CrossPointWebServerActivity.cpp:304-311 5. Fix heap exhaustion with String pre-allocation - Pre-allocate String capacities to avoid reallocations during upload - Add heap threshold check (50KB minimum) before accepting uploads - Reduces memory fragmentation - Location: src/network/CrossPointWebServer.cpp:323-376 ROBUSTNESS IMPROVEMENTS: 6. Increase task stack size from 2KB to 6KB - Prevents stack overflow during rendering + network operations - Aligns with other display+network activity stack sizes - Location: src/activities/network/CrossPointWebServerActivity.cpp:51 PERFORMANCE OPTIMIZATIONS: 7. Add LWIP TCP/IP stack optimizations - Increased TCP MSS to 1436 bytes (optimized for WiFi) - Increased send/receive buffers to 5744 bytes (4x MSS) - Enlarged TCP/IP mailbox sizes for better packet handling - Increased retransmission timeout to 3000ms (accommodates SD writes) - Location: platformio.ini:31-49 8. Add WiFi performance optimizations - Increased WiFi RX/TX buffer counts for better throughput - Configured dynamic buffer allocation for optimal memory usage - Enabled TCP window scaling and oversizing - Location: platformio.ini:41-49 9. Maximize WiFi TX power - Set WiFi TX power to maximum (19.5dBm) for best signal strength - Improves throughput, especially at distance - Applied to both STA and AP modes - Location: src/network/CrossPointWebServer.cpp:58, src/activities/network/CrossPointWebServerActivity.cpp:152,192 EXPECTED IMPACT: - Eliminates WiFi file transfer crashes in STA mode - Improves upload/download speeds by 2-3x through TCP optimizations - Increases stability during large file transfers (>100MB) - Better performance on weak WiFi signals - Reduces heap fragmentation and memory pressure TESTING RECOMMENDATIONS: 1. Test large file uploads (>50MB) in both STA and AP modes 2. Verify system stability during concurrent uploads 3. Monitor heap usage during file transfers 4. Test with long filenames (400+ characters) 5. Verify performance improvement with speed tests Fixes: WiFi lockup bug causing Guru Meditation Errors during file transfer
2026-01-09 22:03:16 +00:00
#include <atomic>
#include <mutex>
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
#include <vector>
// Structure to hold file information
struct FileInfo {
String name;
size_t size;
bool isEpub;
bool isDirectory;
};
class CrossPointWebServer {
public:
CrossPointWebServer();
~CrossPointWebServer();
// Start the web server (call after WiFi is connected)
void begin();
// Stop the web server
void stop();
// Call this periodically to handle client requests
void handleClient() const;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Check if server is running
Fix WiFi file transfer lockup and add performance optimizations This commit resolves critical stability issues causing WiFi file transfer crashes in STA mode and adds aggressive performance optimizations for maximum upload/download throughput. CRITICAL BUG FIXES: 1. Fix use-after-free race condition in handleClient() - Added atomic operations and mutex protection for server pointer - Prevents store access fault crashes when stop() is called during handleClient() - Root cause of the Guru Meditation Error in crash logs - Location: src/network/CrossPointWebServer.cpp:135-158 2. Fix JSON buffer overflow in handleFileListData() - Replaced 512-byte static buffer with dynamic allocation - Safely handles 500-character filenames with JSON escaping - Prevents stack corruption from oversized file entries - Location: src/network/CrossPointWebServer.cpp:253-306 3. Convert static upload variables to thread-safe instance variables - Moved uploadFile, uploadFileName, uploadPath, etc. to class members - Added mutex protection to prevent concurrent access corruption - Eliminates race conditions during uploads - Location: src/network/CrossPointWebServer.h:44-54, CrossPointWebServer.cpp:311-313 4. Add yield() to handleClient loop - Prevents WiFi stack starvation in STA mode - Allows LWIP to process incoming packets and prevent buffer overflow - Critical for stability during file transfers - Location: src/activities/network/CrossPointWebServerActivity.cpp:304-311 5. Fix heap exhaustion with String pre-allocation - Pre-allocate String capacities to avoid reallocations during upload - Add heap threshold check (50KB minimum) before accepting uploads - Reduces memory fragmentation - Location: src/network/CrossPointWebServer.cpp:323-376 ROBUSTNESS IMPROVEMENTS: 6. Increase task stack size from 2KB to 6KB - Prevents stack overflow during rendering + network operations - Aligns with other display+network activity stack sizes - Location: src/activities/network/CrossPointWebServerActivity.cpp:51 PERFORMANCE OPTIMIZATIONS: 7. Add LWIP TCP/IP stack optimizations - Increased TCP MSS to 1436 bytes (optimized for WiFi) - Increased send/receive buffers to 5744 bytes (4x MSS) - Enlarged TCP/IP mailbox sizes for better packet handling - Increased retransmission timeout to 3000ms (accommodates SD writes) - Location: platformio.ini:31-49 8. Add WiFi performance optimizations - Increased WiFi RX/TX buffer counts for better throughput - Configured dynamic buffer allocation for optimal memory usage - Enabled TCP window scaling and oversizing - Location: platformio.ini:41-49 9. Maximize WiFi TX power - Set WiFi TX power to maximum (19.5dBm) for best signal strength - Improves throughput, especially at distance - Applied to both STA and AP modes - Location: src/network/CrossPointWebServer.cpp:58, src/activities/network/CrossPointWebServerActivity.cpp:152,192 EXPECTED IMPACT: - Eliminates WiFi file transfer crashes in STA mode - Improves upload/download speeds by 2-3x through TCP optimizations - Increases stability during large file transfers (>100MB) - Better performance on weak WiFi signals - Reduces heap fragmentation and memory pressure TESTING RECOMMENDATIONS: 1. Test large file uploads (>50MB) in both STA and AP modes 2. Verify system stability during concurrent uploads 3. Monitor heap usage during file transfers 4. Test with long filenames (400+ characters) 5. Verify performance improvement with speed tests Fixes: WiFi lockup bug causing Guru Meditation Errors during file transfer
2026-01-09 22:03:16 +00:00
bool isRunning() const { return running.load(std::memory_order_acquire); }
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Get the port number
uint16_t getPort() const { return port; }
private:
std::unique_ptr<WebServer> server = nullptr;
Fix WiFi file transfer lockup and add performance optimizations This commit resolves critical stability issues causing WiFi file transfer crashes in STA mode and adds aggressive performance optimizations for maximum upload/download throughput. CRITICAL BUG FIXES: 1. Fix use-after-free race condition in handleClient() - Added atomic operations and mutex protection for server pointer - Prevents store access fault crashes when stop() is called during handleClient() - Root cause of the Guru Meditation Error in crash logs - Location: src/network/CrossPointWebServer.cpp:135-158 2. Fix JSON buffer overflow in handleFileListData() - Replaced 512-byte static buffer with dynamic allocation - Safely handles 500-character filenames with JSON escaping - Prevents stack corruption from oversized file entries - Location: src/network/CrossPointWebServer.cpp:253-306 3. Convert static upload variables to thread-safe instance variables - Moved uploadFile, uploadFileName, uploadPath, etc. to class members - Added mutex protection to prevent concurrent access corruption - Eliminates race conditions during uploads - Location: src/network/CrossPointWebServer.h:44-54, CrossPointWebServer.cpp:311-313 4. Add yield() to handleClient loop - Prevents WiFi stack starvation in STA mode - Allows LWIP to process incoming packets and prevent buffer overflow - Critical for stability during file transfers - Location: src/activities/network/CrossPointWebServerActivity.cpp:304-311 5. Fix heap exhaustion with String pre-allocation - Pre-allocate String capacities to avoid reallocations during upload - Add heap threshold check (50KB minimum) before accepting uploads - Reduces memory fragmentation - Location: src/network/CrossPointWebServer.cpp:323-376 ROBUSTNESS IMPROVEMENTS: 6. Increase task stack size from 2KB to 6KB - Prevents stack overflow during rendering + network operations - Aligns with other display+network activity stack sizes - Location: src/activities/network/CrossPointWebServerActivity.cpp:51 PERFORMANCE OPTIMIZATIONS: 7. Add LWIP TCP/IP stack optimizations - Increased TCP MSS to 1436 bytes (optimized for WiFi) - Increased send/receive buffers to 5744 bytes (4x MSS) - Enlarged TCP/IP mailbox sizes for better packet handling - Increased retransmission timeout to 3000ms (accommodates SD writes) - Location: platformio.ini:31-49 8. Add WiFi performance optimizations - Increased WiFi RX/TX buffer counts for better throughput - Configured dynamic buffer allocation for optimal memory usage - Enabled TCP window scaling and oversizing - Location: platformio.ini:41-49 9. Maximize WiFi TX power - Set WiFi TX power to maximum (19.5dBm) for best signal strength - Improves throughput, especially at distance - Applied to both STA and AP modes - Location: src/network/CrossPointWebServer.cpp:58, src/activities/network/CrossPointWebServerActivity.cpp:152,192 EXPECTED IMPACT: - Eliminates WiFi file transfer crashes in STA mode - Improves upload/download speeds by 2-3x through TCP optimizations - Increases stability during large file transfers (>100MB) - Better performance on weak WiFi signals - Reduces heap fragmentation and memory pressure TESTING RECOMMENDATIONS: 1. Test large file uploads (>50MB) in both STA and AP modes 2. Verify system stability during concurrent uploads 3. Monitor heap usage during file transfers 4. Test with long filenames (400+ characters) 5. Verify performance improvement with speed tests Fixes: WiFi lockup bug causing Guru Meditation Errors during file transfer
2026-01-09 22:03:16 +00:00
std::atomic<bool> running{false};
mutable std::mutex serverMutex; // Protects server pointer access
bool apMode = false; // true when running in AP mode, false for STA mode
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
uint16_t port = 80;
Fix WiFi file transfer lockup and add performance optimizations This commit resolves critical stability issues causing WiFi file transfer crashes in STA mode and adds aggressive performance optimizations for maximum upload/download throughput. CRITICAL BUG FIXES: 1. Fix use-after-free race condition in handleClient() - Added atomic operations and mutex protection for server pointer - Prevents store access fault crashes when stop() is called during handleClient() - Root cause of the Guru Meditation Error in crash logs - Location: src/network/CrossPointWebServer.cpp:135-158 2. Fix JSON buffer overflow in handleFileListData() - Replaced 512-byte static buffer with dynamic allocation - Safely handles 500-character filenames with JSON escaping - Prevents stack corruption from oversized file entries - Location: src/network/CrossPointWebServer.cpp:253-306 3. Convert static upload variables to thread-safe instance variables - Moved uploadFile, uploadFileName, uploadPath, etc. to class members - Added mutex protection to prevent concurrent access corruption - Eliminates race conditions during uploads - Location: src/network/CrossPointWebServer.h:44-54, CrossPointWebServer.cpp:311-313 4. Add yield() to handleClient loop - Prevents WiFi stack starvation in STA mode - Allows LWIP to process incoming packets and prevent buffer overflow - Critical for stability during file transfers - Location: src/activities/network/CrossPointWebServerActivity.cpp:304-311 5. Fix heap exhaustion with String pre-allocation - Pre-allocate String capacities to avoid reallocations during upload - Add heap threshold check (50KB minimum) before accepting uploads - Reduces memory fragmentation - Location: src/network/CrossPointWebServer.cpp:323-376 ROBUSTNESS IMPROVEMENTS: 6. Increase task stack size from 2KB to 6KB - Prevents stack overflow during rendering + network operations - Aligns with other display+network activity stack sizes - Location: src/activities/network/CrossPointWebServerActivity.cpp:51 PERFORMANCE OPTIMIZATIONS: 7. Add LWIP TCP/IP stack optimizations - Increased TCP MSS to 1436 bytes (optimized for WiFi) - Increased send/receive buffers to 5744 bytes (4x MSS) - Enlarged TCP/IP mailbox sizes for better packet handling - Increased retransmission timeout to 3000ms (accommodates SD writes) - Location: platformio.ini:31-49 8. Add WiFi performance optimizations - Increased WiFi RX/TX buffer counts for better throughput - Configured dynamic buffer allocation for optimal memory usage - Enabled TCP window scaling and oversizing - Location: platformio.ini:41-49 9. Maximize WiFi TX power - Set WiFi TX power to maximum (19.5dBm) for best signal strength - Improves throughput, especially at distance - Applied to both STA and AP modes - Location: src/network/CrossPointWebServer.cpp:58, src/activities/network/CrossPointWebServerActivity.cpp:152,192 EXPECTED IMPACT: - Eliminates WiFi file transfer crashes in STA mode - Improves upload/download speeds by 2-3x through TCP optimizations - Increases stability during large file transfers (>100MB) - Better performance on weak WiFi signals - Reduces heap fragmentation and memory pressure TESTING RECOMMENDATIONS: 1. Test large file uploads (>50MB) in both STA and AP modes 2. Verify system stability during concurrent uploads 3. Monitor heap usage during file transfers 4. Test with long filenames (400+ characters) 5. Verify performance improvement with speed tests Fixes: WiFi lockup bug causing Guru Meditation Errors during file transfer
2026-01-09 22:03:16 +00:00
// Upload state (instance variables with mutex protection)
mutable std::mutex uploadMutex;
mutable FsFile uploadFile;
mutable String uploadFileName;
mutable String uploadPath;
mutable size_t uploadSize;
mutable bool uploadSuccess;
mutable String uploadError;
mutable unsigned long lastWriteTime;
mutable unsigned long uploadStartTime;
mutable size_t lastLoggedSize;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// File scanning
void scanFiles(const char* path, const std::function<void(FileInfo)>& callback) const;
String formatFileSize(size_t bytes) const;
bool isEpubFile(const String& filename) const;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
// Request handlers
void handleRoot() const;
void handleNotFound() const;
void handleStatus() const;
void handleFileList() const;
void handleFileListData() const;
void handleUpload() const;
void handleUploadPost() const;
void handleCreateFolder() const;
void handleDelete() const;
Add connect to Wifi and File Manager Webserver (#41) ## Summary - **What is the goal of this PR?** Implements wireless EPUB file management via a built-in web server, enabling users to upload, browse, organize, and delete EPUB files from any device on the same WiFi network without needing a computer cable connection. - **What changes are included?** - **New Web Server** ([`CrossPointWebServer.cpp`](src/CrossPointWebServer.cpp), [`CrossPointWebServer.h`](src/CrossPointWebServer.h)): - HTTP server on port 80 with a responsive HTML/CSS interface - Home page showing device status (version, IP, free memory) - File Manager with folder navigation and breadcrumb support - EPUB file upload with progress tracking - Folder creation and file/folder deletion - XSS protection via HTML escaping - Hidden system folders (`.` prefixed, "System Volume Information", "XTCache") - **WiFi Screen** ([`WifiScreen.cpp`](src/screens/WifiScreen.cpp), [`WifiScreen.h`](src/screens/WifiScreen.h)): - Network scanning with signal strength indicators - Visual indicators for encrypted (`*`) and saved (`+`) networks - State machine managing: scanning, network selection, password entry, connecting, save/forget prompts - 15-second connection timeout handling - Integration with web server (starts on connect, stops on exit) - **WiFi Credential Storage** ([`WifiCredentialStore.cpp`](src/WifiCredentialStore.cpp), [`WifiCredentialStore.h`](src/WifiCredentialStore.h)): - Persistent storage in `/sd/.crosspoint/wifi.bin` - XOR obfuscation for stored passwords (basic protection against casual reading) - Up to 8 saved networks with add/remove/update operations - **On-Screen Keyboard** ([`OnScreenKeyboard.cpp`](src/screens/OnScreenKeyboard.cpp), [`OnScreenKeyboard.h`](src/screens/OnScreenKeyboard.h)): - Reusable QWERTY keyboard component with shift support - Special keys: Shift, Space, Backspace, Done - Support for password masking mode - **Settings Screen Integration** ([`SettingsScreen.h`](src/screens/SettingsScreen.h)): - Added WiFi action to navigate to the new WiFi screen - **Documentation** ([`docs/webserver.md`](docs/webserver.md)): - Comprehensive user guide covering WiFi setup, web interface usage, file management, troubleshooting, and security notes - See this for more screenshots! - Working "displays the right way in GitHub" on my repo: https://github.com/olearycrew/crosspoint-reader/blob/feature/connect-to-wifi/docs/webserver.md **Video demo** https://github.com/user-attachments/assets/283e32dc-2d9f-4ae2-848e-01f41166a731 ## Additional Context - **Security considerations**: The web server has no authentication—anyone on the same WiFi network can access files. This is documented as a limitation, recommending use only on trusted private networks. Password obfuscation in the credential store is XOR-based, not cryptographically secure. - **Memory implications**: The web server and WiFi stack consume significant memory. The implementation properly cleans up (stops server, disconnects WiFi, sets `WIFI_OFF` mode) when exiting the WiFi screen to free resources. - **Async operations**: Network scanning and connection use async patterns with FreeRTOS tasks to prevent blocking the UI. The display task handles rendering on a dedicated thread with mutex protection. - **Browser compatibility**: The web interface uses standard HTML5/CSS3/JavaScript and is tested to work with all modern browsers on desktop and mobile. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
2025-12-19 09:05:43 -05:00
};