cottongin/crosspoint-reader-mod
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Migrate binary settings to json (#920)

Browse Source 
## Summary

* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
  - Settings files are now JSON and can be easily read/edited manually
  - Easier to inspect application state and settings during development
  - JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data

## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
     - CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
     - RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
     - Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
     - HalStorage: Added rename() method for backup operations

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _** YES**_

---------

Co-authored-by: Dave Allie <dave@daveallie.com>
This commit is contained in:
jpirnay
2026-02-22 07:18:25 +01:00
committed by GitHub
parent f62529ad91
commit 6e4d0e534d
16 changed files with 713 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

163
src/CrossPointSettings.cpp
View File

@@ -1,6 +1,7 @@
#include "CrossPointSettings.h"
#include <HalStorage.h>
#include <JsonSettingsIO.h>
#include <Logging.h>
#include <Serialization.h>
@@ -22,28 +23,9 @@ void readAndValidate(FsFile& file, uint8_t& member, const uint8_t maxValue) {
namespace {
constexpr uint8_t SETTINGS_FILE_VERSION = 1;
// SETTINGS_COUNT is now calculated automatically in saveToFile
constexpr char SETTINGS_FILE[] = "/.crosspoint/settings.bin";
// Validate front button mapping to ensure each hardware button is unique.
// If duplicates are detected, reset to the default physical order to prevent invalid mappings.
void validateFrontButtonMapping(CrossPointSettings& settings) {
// Snapshot the logical->hardware mapping so we can compare for duplicates.
const uint8_t mapping[] = {settings.frontButtonBack, settings.frontButtonConfirm, settings.frontButtonLeft,
settings.frontButtonRight};
for (size_t i = 0; i < 4; i++) {
for (size_t j = i + 1; j < 4; j++) {
if (mapping[i] == mapping[j]) {
// Duplicate detected: restore the default physical order (Back, Confirm, Left, Right).
settings.frontButtonBack = CrossPointSettings::FRONT_HW_BACK;
settings.frontButtonConfirm = CrossPointSettings::FRONT_HW_CONFIRM;
settings.frontButtonLeft = CrossPointSettings::FRONT_HW_LEFT;
settings.frontButtonRight = CrossPointSettings::FRONT_HW_RIGHT;
return;
}
}
}
}
constexpr char SETTINGS_FILE_BIN[] = "/.crosspoint/settings.bin";
constexpr char SETTINGS_FILE_JSON[] = "/.crosspoint/settings.json";
constexpr char SETTINGS_FILE_BAK[] = "/.crosspoint/settings.bin.bak";
// Convert legacy front button layout into explicit logical->hardware mapping.
void applyLegacyFrontButtonLayout(CrossPointSettings& settings) {
@@ -77,95 +59,65 @@ void applyLegacyFrontButtonLayout(CrossPointSettings& settings) {
}
} // namespace
class SettingsWriter {
public:
bool is_counting = false;
uint8_t item_count = 0;
template <typename T>
void writeItem(FsFile& file, const T& value) {
if (is_counting) {
item_count++;
} else {
serialization::writePod(file, value);
void CrossPointSettings::validateFrontButtonMapping(CrossPointSettings& settings) {
const uint8_t mapping[] = {settings.frontButtonBack, settings.frontButtonConfirm, settings.frontButtonLeft,
settings.frontButtonRight};
for (size_t i = 0; i < 4; i++) {
for (size_t j = i + 1; j < 4; j++) {
if (mapping[i] == mapping[j]) {
settings.frontButtonBack = FRONT_HW_BACK;
settings.frontButtonConfirm = FRONT_HW_CONFIRM;
settings.frontButtonLeft = FRONT_HW_LEFT;
settings.frontButtonRight = FRONT_HW_RIGHT;
return;
}
}
}
void writeItemString(FsFile& file, const char* value) {
if (is_counting) {
item_count++;
} else {
serialization::writeString(file, std::string(value));
}
}
};
uint8_t CrossPointSettings::writeSettings(FsFile& file, bool count_only) const {
SettingsWriter writer;
writer.is_counting = count_only;
writer.writeItem(file, sleepScreen);
writer.writeItem(file, extraParagraphSpacing);
writer.writeItem(file, shortPwrBtn);
writer.writeItem(file, statusBar);
writer.writeItem(file, orientation);
writer.writeItem(file, frontButtonLayout); // legacy
writer.writeItem(file, sideButtonLayout);
writer.writeItem(file, fontFamily);
writer.writeItem(file, fontSize);
writer.writeItem(file, lineSpacing);
writer.writeItem(file, paragraphAlignment);
writer.writeItem(file, sleepTimeout);
writer.writeItem(file, refreshFrequency);
writer.writeItem(file, screenMargin);
writer.writeItem(file, sleepScreenCoverMode);
writer.writeItemString(file, opdsServerUrl);
writer.writeItem(file, textAntiAliasing);
writer.writeItem(file, hideBatteryPercentage);
writer.writeItem(file, longPressChapterSkip);
writer.writeItem(file, hyphenationEnabled);
writer.writeItemString(file, opdsUsername);
writer.writeItemString(file, opdsPassword);
writer.writeItem(file, sleepScreenCoverFilter);
writer.writeItem(file, uiTheme);
writer.writeItem(file, frontButtonBack);
writer.writeItem(file, frontButtonConfirm);
writer.writeItem(file, frontButtonLeft);
writer.writeItem(file, frontButtonRight);
writer.writeItem(file, fadingFix);
writer.writeItem(file, embeddedStyle);
// New fields need to be added at end for backward compatibility
return writer.item_count;
}
bool CrossPointSettings::saveToFile() const {
// Make sure the directory exists
Storage.mkdir("/.crosspoint");
FsFile outputFile;
if (!Storage.openFileForWrite("CPS", SETTINGS_FILE, outputFile)) {
return false;
}
// First pass: count the items
uint8_t item_count = writeSettings(outputFile, true); // This will just count, not write
// Write header
serialization::writePod(outputFile, SETTINGS_FILE_VERSION);
serialization::writePod(outputFile, static_cast<uint8_t>(item_count));
// Second pass: actually write the settings
writeSettings(outputFile); // This will write the actual data
outputFile.close();
LOG_DBG("CPS", "Settings saved to file");
return true;
return JsonSettingsIO::saveSettings(*this, SETTINGS_FILE_JSON);
}
bool CrossPointSettings::loadFromFile() {
// Try JSON first
if (Storage.exists(SETTINGS_FILE_JSON)) {
String json = Storage.readFile(SETTINGS_FILE_JSON);
if (!json.isEmpty()) {
bool resave = false;
bool result = JsonSettingsIO::loadSettings(*this, json.c_str(), &resave);
if (result && resave) {
if (saveToFile()) {
LOG_DBG("CPS", "Resaved settings to update format");
} else {
LOG_ERR("CPS", "Failed to resave settings after format update");
}
}
return result;
}
}
// Fall back to binary migration
if (Storage.exists(SETTINGS_FILE_BIN)) {
if (loadFromBinaryFile()) {
if (saveToFile()) {
Storage.rename(SETTINGS_FILE_BIN, SETTINGS_FILE_BAK);
LOG_DBG("CPS", "Migrated settings.bin to settings.json");
return true;
} else {
LOG_ERR("CPS", "Failed to save migrated settings to JSON");
return false;
}
}
}
return false;
}
bool CrossPointSettings::loadFromBinaryFile() {
FsFile inputFile;
if (!Storage.openFileForRead("CPS", SETTINGS_FILE, inputFile)) {
if (!Storage.openFileForRead("CPS", SETTINGS_FILE_BIN, inputFile)) {
return false;
}
@@ -180,9 +132,7 @@ bool CrossPointSettings::loadFromFile() {
uint8_t fileSettingsCount = 0;
serialization::readPod(inputFile, fileSettingsCount);
// load settings that exist (support older files with fewer fields)
uint8_t settingsRead = 0;
// Track whether remap fields were present in the settings file.
bool frontButtonMappingRead = false;
do {
readAndValidate(inputFile, sleepScreen, SLEEP_SCREEN_MODE_COUNT);
@@ -195,7 +145,7 @@ bool CrossPointSettings::loadFromFile() {
if (++settingsRead >= fileSettingsCount) break;
readAndValidate(inputFile, orientation, ORIENTATION_COUNT);
if (++settingsRead >= fileSettingsCount) break;
readAndValidate(inputFile, frontButtonLayout, FRONT_BUTTON_LAYOUT_COUNT); // legacy
readAndValidate(inputFile, frontButtonLayout, FRONT_BUTTON_LAYOUT_COUNT);
if (++settingsRead >= fileSettingsCount) break;
readAndValidate(inputFile, sideButtonLayout, SIDE_BUTTON_LAYOUT_COUNT);
if (++settingsRead >= fileSettingsCount) break;
@@ -261,17 +211,16 @@ bool CrossPointSettings::loadFromFile() {
if (++settingsRead >= fileSettingsCount) break;
serialization::readPod(inputFile, embeddedStyle);
if (++settingsRead >= fileSettingsCount) break;
// New fields added at end for backward compatibility
} while (false);
if (frontButtonMappingRead) {
validateFrontButtonMapping(*this);
CrossPointSettings::validateFrontButtonMapping(*this);
} else {
applyLegacyFrontButtonLayout(*this);
}
inputFile.close();
LOG_DBG("CPS", "Settings loaded from file");
LOG_DBG("CPS", "Settings loaded from binary file");
return true;
}