feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
#include "JsonSettingsIO.h"
|
|
|
|
|
|
|
|
|
|
#include <ArduinoJson.h>
|
|
|
|
|
#include <HalStorage.h>
|
|
|
|
|
#include <Logging.h>
|
|
|
|
|
#include <ObfuscationUtils.h>
|
|
|
|
|
|
|
|
|
|
#include <cstring>
|
refactor: Simplify new setting introduction (#1086)
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
2026-03-01 02:54:58 +01:00
|
|
|
#include <string>
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
|
|
|
|
|
#include "CrossPointSettings.h"
|
|
|
|
|
#include "CrossPointState.h"
|
|
|
|
|
#include "KOReaderCredentialStore.h"
|
|
|
|
|
#include "RecentBooksStore.h"
|
refactor: Simplify new setting introduction (#1086)
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
2026-03-01 02:54:58 +01:00
|
|
|
#include "SettingsList.h"
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
#include "WifiCredentialStore.h"
|
|
|
|
|
|
2026-02-25 10:06:38 +00:00
|
|
|
// Convert legacy settings.
|
|
|
|
|
void applyLegacyStatusBarSettings(CrossPointSettings& settings) {
|
|
|
|
|
switch (static_cast<CrossPointSettings::STATUS_BAR_MODE>(settings.statusBar)) {
|
|
|
|
|
case CrossPointSettings::NONE:
|
|
|
|
|
settings.statusBarChapterPageCount = 0;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 0;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::HIDE_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::HIDE_TITLE;
|
|
|
|
|
settings.statusBarBattery = 0;
|
|
|
|
|
break;
|
|
|
|
|
case CrossPointSettings::NO_PROGRESS:
|
|
|
|
|
settings.statusBarChapterPageCount = 0;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 0;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::HIDE_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::CHAPTER_TITLE;
|
|
|
|
|
settings.statusBarBattery = 1;
|
|
|
|
|
break;
|
|
|
|
|
case CrossPointSettings::BOOK_PROGRESS_BAR:
|
|
|
|
|
settings.statusBarChapterPageCount = 1;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 0;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::BOOK_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::CHAPTER_TITLE;
|
|
|
|
|
settings.statusBarBattery = 1;
|
|
|
|
|
break;
|
|
|
|
|
case CrossPointSettings::ONLY_BOOK_PROGRESS_BAR:
|
|
|
|
|
settings.statusBarChapterPageCount = 1;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 0;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::BOOK_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::HIDE_TITLE;
|
|
|
|
|
settings.statusBarBattery = 0;
|
|
|
|
|
break;
|
|
|
|
|
case CrossPointSettings::CHAPTER_PROGRESS_BAR:
|
|
|
|
|
settings.statusBarChapterPageCount = 0;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 1;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::CHAPTER_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::CHAPTER_TITLE;
|
|
|
|
|
settings.statusBarBattery = 1;
|
|
|
|
|
break;
|
|
|
|
|
case CrossPointSettings::FULL:
|
|
|
|
|
default:
|
|
|
|
|
settings.statusBarChapterPageCount = 1;
|
|
|
|
|
settings.statusBarBookProgressPercentage = 1;
|
|
|
|
|
settings.statusBarProgressBar = CrossPointSettings::HIDE_PROGRESS;
|
|
|
|
|
settings.statusBarTitle = CrossPointSettings::CHAPTER_TITLE;
|
|
|
|
|
settings.statusBarBattery = 1;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
// ---- CrossPointState ----
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::saveState(const CrossPointState& s, const char* path) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
doc["openEpubPath"] = s.openEpubPath;
|
|
|
|
|
doc["lastSleepImage"] = s.lastSleepImage;
|
|
|
|
|
doc["readerActivityLoadCount"] = s.readerActivityLoadCount;
|
|
|
|
|
doc["lastSleepFromReader"] = s.lastSleepFromReader;
|
|
|
|
|
|
|
|
|
|
String json;
|
|
|
|
|
serializeJson(doc, json);
|
|
|
|
|
return Storage.writeFile(path, json);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::loadState(CrossPointState& s, const char* json) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
auto error = deserializeJson(doc, json);
|
|
|
|
|
if (error) {
|
|
|
|
|
LOG_ERR("CPS", "JSON parse error: %s", error.c_str());
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
s.openEpubPath = doc["openEpubPath"] | std::string("");
|
|
|
|
|
s.lastSleepImage = doc["lastSleepImage"] | (uint8_t)0;
|
|
|
|
|
s.readerActivityLoadCount = doc["readerActivityLoadCount"] | (uint8_t)0;
|
|
|
|
|
s.lastSleepFromReader = doc["lastSleepFromReader"] | false;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ---- CrossPointSettings ----
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::saveSettings(const CrossPointSettings& s, const char* path) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
|
refactor: Simplify new setting introduction (#1086)
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
2026-03-01 02:54:58 +01:00
|
|
|
for (const auto& info : getSettingsList()) {
|
|
|
|
|
if (!info.key) continue;
|
|
|
|
|
// Dynamic entries (KOReader etc.) are stored in their own files — skip.
|
|
|
|
|
if (!info.valuePtr && !info.stringOffset) continue;
|
|
|
|
|
|
|
|
|
|
if (info.stringOffset) {
|
|
|
|
|
const char* strPtr = (const char*)&s + info.stringOffset;
|
|
|
|
|
if (info.obfuscated) {
|
|
|
|
|
doc[std::string(info.key) + "_obf"] = obfuscation::obfuscateToBase64(strPtr);
|
|
|
|
|
} else {
|
|
|
|
|
doc[info.key] = strPtr;
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
doc[info.key] = s.*(info.valuePtr);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Front button remap — managed by RemapFrontButtons sub-activity, not in SettingsList.
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
doc["frontButtonBack"] = s.frontButtonBack;
|
|
|
|
|
doc["frontButtonConfirm"] = s.frontButtonConfirm;
|
|
|
|
|
doc["frontButtonLeft"] = s.frontButtonLeft;
|
|
|
|
|
doc["frontButtonRight"] = s.frontButtonRight;
|
|
|
|
|
|
mod: Phase 2a - add mod settings, I18n strings, and main.cpp integration
CrossPointSettings: Add mod-specific enums and fields:
- Clock: CLOCK_FORMAT, CLOCK_SIZE, TIMEZONE, clockFormat, clockSize,
timezone, timezoneOffsetHours, autoNtpSync
- Sleep: SLEEP_SCREEN_LETTERBOX_FILL, sleepScreenLetterboxFill
- Reader: preferredPortrait, preferredLandscape
- Indexing: INDEXING_DISPLAY, indexingDisplay
- getTimezonePosixStr() for POSIX TZ string generation
main.cpp: Integrate mod initialization:
- OPDS store loading, boot NTP sync, timezone application
- Clock refresh loop (re-render on minute change)
- RTC time logging on boot
SettingsList.h: Add clock, timezone, and letterbox fill settings
JsonSettingsIO.cpp: Handle int8_t timezoneOffsetHours separately
I18n: Add ~80 mod string keys (english.yaml + regenerated I18nKeys.h)
Made-with: Cursor
2026-03-07 15:14:35 -05:00
|
|
|
// Mod: timezone offset is int8_t, not uint8_t — handled separately
|
|
|
|
|
doc["timezoneOffsetHours"] = s.timezoneOffsetHours;
|
|
|
|
|
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
String json;
|
|
|
|
|
serializeJson(doc, json);
|
|
|
|
|
return Storage.writeFile(path, json);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::loadSettings(CrossPointSettings& s, const char* json, bool* needsResave) {
|
|
|
|
|
if (needsResave) *needsResave = false;
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
auto error = deserializeJson(doc, json);
|
|
|
|
|
if (error) {
|
|
|
|
|
LOG_ERR("CPS", "JSON parse error: %s", error.c_str());
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
auto clamp = [](uint8_t val, uint8_t maxVal, uint8_t def) -> uint8_t { return val < maxVal ? val : def; };
|
|
|
|
|
|
refactor: Simplify new setting introduction (#1086)
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
2026-03-01 02:54:58 +01:00
|
|
|
// Legacy migration: if statusBarChapterPageCount is absent this is a pre-refactor settings file.
|
|
|
|
|
// Populate s with migrated values now so the generic loop below picks them up as defaults and clamps them.
|
|
|
|
|
if (doc["statusBarChapterPageCount"].isNull()) {
|
|
|
|
|
applyLegacyStatusBarSettings(s);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
for (const auto& info : getSettingsList()) {
|
|
|
|
|
if (!info.key) continue;
|
|
|
|
|
// Dynamic entries (KOReader etc.) are stored in their own files — skip.
|
|
|
|
|
if (!info.valuePtr && !info.stringOffset) continue;
|
|
|
|
|
|
|
|
|
|
if (info.stringOffset) {
|
|
|
|
|
const char* strPtr = (const char*)&s + info.stringOffset;
|
|
|
|
|
const std::string fieldDefault = strPtr; // current buffer = struct-initializer default
|
|
|
|
|
std::string val;
|
|
|
|
|
if (info.obfuscated) {
|
|
|
|
|
bool ok = false;
|
|
|
|
|
val = obfuscation::deobfuscateFromBase64(doc[std::string(info.key) + "_obf"] | "", &ok);
|
|
|
|
|
if (!ok || val.empty()) {
|
|
|
|
|
val = doc[info.key] | fieldDefault;
|
|
|
|
|
if (val != fieldDefault && needsResave) *needsResave = true;
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
val = doc[info.key] | fieldDefault;
|
|
|
|
|
}
|
|
|
|
|
char* destPtr = (char*)&s + info.stringOffset;
|
|
|
|
|
if (info.stringMaxLen == 0) {
|
|
|
|
|
LOG_ERR("CPS", "Misconfigured SettingInfo: stringMaxLen is 0 for key '%s'", info.key);
|
|
|
|
|
destPtr[0] = '\0';
|
|
|
|
|
if (needsResave) *needsResave = true;
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
strncpy(destPtr, val.c_str(), info.stringMaxLen - 1);
|
|
|
|
|
destPtr[info.stringMaxLen - 1] = '\0';
|
|
|
|
|
} else {
|
|
|
|
|
const uint8_t fieldDefault = s.*(info.valuePtr); // struct-initializer default, read before we overwrite it
|
|
|
|
|
uint8_t v = doc[info.key] | fieldDefault;
|
|
|
|
|
if (info.type == SettingType::ENUM) {
|
|
|
|
|
v = clamp(v, (uint8_t)info.enumValues.size(), fieldDefault);
|
|
|
|
|
} else if (info.type == SettingType::TOGGLE) {
|
|
|
|
|
v = clamp(v, (uint8_t)2, fieldDefault);
|
|
|
|
|
} else if (info.type == SettingType::VALUE) {
|
|
|
|
|
if (v < info.valueRange.min)
|
|
|
|
|
v = info.valueRange.min;
|
|
|
|
|
else if (v > info.valueRange.max)
|
|
|
|
|
v = info.valueRange.max;
|
|
|
|
|
}
|
|
|
|
|
s.*(info.valuePtr) = v;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Front button remap — managed by RemapFrontButtons sub-activity, not in SettingsList.
|
|
|
|
|
using S = CrossPointSettings;
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
s.frontButtonBack =
|
|
|
|
|
clamp(doc["frontButtonBack"] | (uint8_t)S::FRONT_HW_BACK, S::FRONT_BUTTON_HARDWARE_COUNT, S::FRONT_HW_BACK);
|
|
|
|
|
s.frontButtonConfirm = clamp(doc["frontButtonConfirm"] | (uint8_t)S::FRONT_HW_CONFIRM, S::FRONT_BUTTON_HARDWARE_COUNT,
|
|
|
|
|
S::FRONT_HW_CONFIRM);
|
|
|
|
|
s.frontButtonLeft =
|
|
|
|
|
clamp(doc["frontButtonLeft"] | (uint8_t)S::FRONT_HW_LEFT, S::FRONT_BUTTON_HARDWARE_COUNT, S::FRONT_HW_LEFT);
|
|
|
|
|
s.frontButtonRight =
|
|
|
|
|
clamp(doc["frontButtonRight"] | (uint8_t)S::FRONT_HW_RIGHT, S::FRONT_BUTTON_HARDWARE_COUNT, S::FRONT_HW_RIGHT);
|
|
|
|
|
CrossPointSettings::validateFrontButtonMapping(s);
|
2026-02-25 10:06:38 +00:00
|
|
|
|
mod: Phase 2a - add mod settings, I18n strings, and main.cpp integration
CrossPointSettings: Add mod-specific enums and fields:
- Clock: CLOCK_FORMAT, CLOCK_SIZE, TIMEZONE, clockFormat, clockSize,
timezone, timezoneOffsetHours, autoNtpSync
- Sleep: SLEEP_SCREEN_LETTERBOX_FILL, sleepScreenLetterboxFill
- Reader: preferredPortrait, preferredLandscape
- Indexing: INDEXING_DISPLAY, indexingDisplay
- getTimezonePosixStr() for POSIX TZ string generation
main.cpp: Integrate mod initialization:
- OPDS store loading, boot NTP sync, timezone application
- Clock refresh loop (re-render on minute change)
- RTC time logging on boot
SettingsList.h: Add clock, timezone, and letterbox fill settings
JsonSettingsIO.cpp: Handle int8_t timezoneOffsetHours separately
I18n: Add ~80 mod string keys (english.yaml + regenerated I18nKeys.h)
Made-with: Cursor
2026-03-07 15:14:35 -05:00
|
|
|
// Mod: timezone offset is int8_t, not uint8_t
|
|
|
|
|
s.timezoneOffsetHours = doc["timezoneOffsetHours"] | (int8_t)0;
|
|
|
|
|
if (s.timezoneOffsetHours < -12) s.timezoneOffsetHours = -12;
|
|
|
|
|
if (s.timezoneOffsetHours > 14) s.timezoneOffsetHours = 14;
|
|
|
|
|
|
refactor: Simplify new setting introduction (#1086)
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
2026-03-01 02:54:58 +01:00
|
|
|
LOG_DBG("CPS", "Settings loaded from file");
|
2026-02-25 10:06:38 +00:00
|
|
|
|
feat: Migrate binary settings to json (#920)
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
2026-02-22 07:18:25 +01:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ---- WifiCredentialStore ----
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::saveWifi(const WifiCredentialStore& store, const char* path) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
doc["lastConnectedSsid"] = store.getLastConnectedSsid();
|
|
|
|
|
|
|
|
|
|
JsonArray arr = doc["credentials"].to<JsonArray>();
|
|
|
|
|
for (const auto& cred : store.getCredentials()) {
|
|
|
|
|
JsonObject obj = arr.add<JsonObject>();
|
|
|
|
|
obj["ssid"] = cred.ssid;
|
|
|
|
|
obj["password_obf"] = obfuscation::obfuscateToBase64(cred.password);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
String json;
|
|
|
|
|
serializeJson(doc, json);
|
|
|
|
|
return Storage.writeFile(path, json);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::loadWifi(WifiCredentialStore& store, const char* json, bool* needsResave) {
|
|
|
|
|
if (needsResave) *needsResave = false;
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
auto error = deserializeJson(doc, json);
|
|
|
|
|
if (error) {
|
|
|
|
|
LOG_ERR("WCS", "JSON parse error: %s", error.c_str());
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
store.lastConnectedSsid = doc["lastConnectedSsid"] | std::string("");
|
|
|
|
|
|
|
|
|
|
store.credentials.clear();
|
|
|
|
|
JsonArray arr = doc["credentials"].as<JsonArray>();
|
|
|
|
|
for (JsonObject obj : arr) {
|
|
|
|
|
if (store.credentials.size() >= store.MAX_NETWORKS) break;
|
|
|
|
|
WifiCredential cred;
|
|
|
|
|
cred.ssid = obj["ssid"] | std::string("");
|
|
|
|
|
bool ok = false;
|
|
|
|
|
cred.password = obfuscation::deobfuscateFromBase64(obj["password_obf"] | "", &ok);
|
|
|
|
|
if (!ok || cred.password.empty()) {
|
|
|
|
|
cred.password = obj["password"] | std::string("");
|
|
|
|
|
if (!cred.password.empty() && needsResave) *needsResave = true;
|
|
|
|
|
}
|
|
|
|
|
store.credentials.push_back(cred);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
LOG_DBG("WCS", "Loaded %zu WiFi credentials from file", store.credentials.size());
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ---- RecentBooksStore ----
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::saveRecentBooks(const RecentBooksStore& store, const char* path) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
JsonArray arr = doc["books"].to<JsonArray>();
|
|
|
|
|
for (const auto& book : store.getBooks()) {
|
|
|
|
|
JsonObject obj = arr.add<JsonObject>();
|
|
|
|
|
obj["path"] = book.path;
|
|
|
|
|
obj["title"] = book.title;
|
|
|
|
|
obj["author"] = book.author;
|
|
|
|
|
obj["coverBmpPath"] = book.coverBmpPath;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
String json;
|
|
|
|
|
serializeJson(doc, json);
|
|
|
|
|
return Storage.writeFile(path, json);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
bool JsonSettingsIO::loadRecentBooks(RecentBooksStore& store, const char* json) {
|
|
|
|
|
JsonDocument doc;
|
|
|
|
|
auto error = deserializeJson(doc, json);
|
|
|
|
|
if (error) {
|
|
|
|
|
LOG_ERR("RBS", "JSON parse error: %s", error.c_str());
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
store.recentBooks.clear();
|
|
|
|
|
JsonArray arr = doc["books"].as<JsonArray>();
|
|
|
|
|
for (JsonObject obj : arr) {
|
|
|
|
|
if (store.getCount() >= 10) break;
|
|
|
|
|
RecentBook book;
|
|
|
|
|
book.path = obj["path"] | std::string("");
|
|
|
|
|
book.title = obj["title"] | std::string("");
|
|
|
|
|
book.author = obj["author"] | std::string("");
|
|
|
|
|
book.coverBmpPath = obj["coverBmpPath"] | std::string("");
|
|
|
|
|
store.recentBooks.push_back(book);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
LOG_DBG("RBS", "Recent books loaded from file (%d entries)", store.getCount());
|
|
|
|
|
return true;
|
|
|
|
|
}
|