perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
#pragma once
|
|
|
|
|
|
2026-02-22 21:38:03 +11:00
|
|
|
#include <InflateReader.h>
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
|
2026-03-12 07:05:46 +11:00
|
|
|
#include <vector>
|
|
|
|
|
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
#include "EpdFontData.h"
|
|
|
|
|
|
|
|
|
|
class FontDecompressor {
|
|
|
|
|
public:
|
2026-03-12 07:05:46 +11:00
|
|
|
static constexpr uint16_t MAX_PAGE_GLYPHS = 512;
|
|
|
|
|
|
|
|
|
|
FontDecompressor() = default;
|
|
|
|
|
~FontDecompressor();
|
|
|
|
|
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
bool init();
|
|
|
|
|
void deinit();
|
|
|
|
|
|
|
|
|
|
// Returns pointer to decompressed bitmap data for the given glyph.
|
2026-03-12 07:05:46 +11:00
|
|
|
// Checks the page buffer (from prewarm) first, then falls back to the hot group slot.
|
|
|
|
|
const uint8_t* getBitmap(const EpdFontData* fontData, const EpdGlyph* glyph, uint32_t glyphIndex);
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
|
2026-03-12 07:05:46 +11:00
|
|
|
// Free all cached data (page buffer + hot group).
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
void clearCache();
|
|
|
|
|
|
2026-03-12 07:05:46 +11:00
|
|
|
// Pre-scan UTF-8 text and extract needed glyph bitmaps into a flat page buffer.
|
|
|
|
|
// Each group is decompressed once into a temp buffer; only needed glyphs are kept.
|
|
|
|
|
// Returns the number of glyphs that couldn't be loaded (0 on full success).
|
|
|
|
|
int prewarmCache(const EpdFontData* fontData, const char* utf8Text);
|
|
|
|
|
|
|
|
|
|
struct Stats {
|
|
|
|
|
uint32_t cacheHits = 0;
|
|
|
|
|
uint32_t cacheMisses = 0;
|
|
|
|
|
uint32_t decompressTimeMs = 0;
|
|
|
|
|
uint16_t uniqueGroupsAccessed = 0;
|
|
|
|
|
uint32_t pageBufferBytes = 0; // pageBuffer allocation
|
|
|
|
|
uint32_t pageGlyphsBytes = 0; // pageGlyphs lookup table allocation
|
|
|
|
|
uint32_t hotGroupBytes = 0; // current hot group allocation
|
|
|
|
|
uint32_t peakTempBytes = 0; // largest temp buffer in prewarm
|
|
|
|
|
uint32_t getBitmapTimeUs = 0; // cumulative getBitmap time (micros)
|
|
|
|
|
uint32_t getBitmapCalls = 0; // number of getBitmap calls
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
};
|
2026-03-12 07:05:46 +11:00
|
|
|
void logStats(const char* label = "FDC");
|
|
|
|
|
void resetStats();
|
|
|
|
|
const Stats& getStats() const { return stats; }
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
|
2026-03-12 07:05:46 +11:00
|
|
|
private:
|
|
|
|
|
Stats stats;
|
2026-02-22 21:38:03 +11:00
|
|
|
InflateReader inflateReader;
|
2026-03-12 07:05:46 +11:00
|
|
|
|
|
|
|
|
// Page buffer: flat array of prewarmed glyph bitmaps with sorted lookup
|
|
|
|
|
struct PageGlyphEntry {
|
|
|
|
|
uint32_t glyphIndex;
|
|
|
|
|
uint32_t bufferOffset;
|
|
|
|
|
uint32_t alignedOffset; // byte-aligned offset within its decompressed group (set during prewarm pre-scan)
|
|
|
|
|
};
|
|
|
|
|
uint8_t* pageBuffer = nullptr;
|
|
|
|
|
const EpdFontData* pageFont = nullptr;
|
|
|
|
|
PageGlyphEntry* pageGlyphs = nullptr;
|
|
|
|
|
uint16_t pageGlyphCount = 0;
|
|
|
|
|
|
|
|
|
|
// Hot group: last decompressed group (byte-aligned) for non-prewarmed fallback path.
|
|
|
|
|
// Kept in byte-aligned format; individual glyphs are compacted on demand into hotGlyphBuf.
|
|
|
|
|
const EpdFontData* hotGroupFont = nullptr;
|
|
|
|
|
uint16_t hotGroupIndex = UINT16_MAX;
|
|
|
|
|
std::vector<uint8_t> hotGroup;
|
|
|
|
|
|
|
|
|
|
// Scratch buffer for compacting a single glyph from the hot group.
|
|
|
|
|
// Valid until the next getBitmap() call.
|
|
|
|
|
std::vector<uint8_t> hotGlyphBuf;
|
|
|
|
|
|
|
|
|
|
void freePageBuffer();
|
|
|
|
|
void freeHotGroup();
|
|
|
|
|
uint16_t getGroupIndex(const EpdFontData* fontData, uint32_t glyphIndex);
|
|
|
|
|
uint32_t getAlignedOffset(const EpdFontData* fontData, uint16_t groupIndex, uint32_t glyphIndex);
|
|
|
|
|
bool decompressGroup(const EpdFontData* fontData, uint16_t groupIndex, uint8_t* outBuf, uint32_t outSize);
|
|
|
|
|
static void compactSingleGlyph(const uint8_t* alignedSrc, uint8_t* packedDst, uint8_t width, uint8_t height);
|
|
|
|
|
static int32_t findGlyphIndex(const EpdFontData* fontData, uint32_t codepoint);
|
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary
**What is the goal of this PR?**
Compress reader font bitmaps to reduce flash usage by 30.7%.
**What changes are included?**
- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context
## Flash & RAM
| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |
## Script-Based Grouping (Cold Cache)
Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.
### Render Time
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |
### Memory Usage
| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |
### Cache Effectiveness
| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |
------
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 20:30:15 +11:00
|
|
|
};
|