cottongin/crosspoint-reader-mod
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)

Browse Source 
**What is the goal of this PR?**

Compress reader font bitmaps to reduce flash usage by 30.7%.

**What changes are included?**

- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation

| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |

Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |

| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |

------

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

  Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Adrian Wilkins-Caruana
2026-02-19 20:30:15 +11:00
committed by cottongin
parent 6403ce6309
commit c1d1e98909
73 changed files with 126416 additions and 246010 deletions
Download Patch File Download Diff File Expand all files Collapse all files

119
lib/EpdFont/scripts/fontconvert.py
View File

@@ -15,6 +15,7 @@ parser.add_argument("size", type=int, help="font size to use.")
parser.add_argument("fontstack", action="store", nargs='+', help="list of font files, ordered by descending priority.")
parser.add_argument("--2bit", dest="is2Bit", action="store_true", help="generate 2-bit greyscale bitmap instead of 1-bit black and white.")
parser.add_argument("--additional-intervals", dest="additional_intervals", action="append", help="Additional code point intervals to export as min,max. This argument can be repeated.")
parser.add_argument("--compress", dest="compress", action="store_true", help="Compress glyph bitmaps using DEFLATE with group-based compression.")
args = parser.parse_args()
GlyphProps = namedtuple("GlyphProps", ["width", "height", "advance_x", "left", "top", "data_length", "data_offset", "code_point"])
@@ -270,21 +271,115 @@ for index, glyph in enumerate(all_glyphs):
glyph_data.extend([b for b in packed])
glyph_props.append(props)
compress = args.compress
# Build groups for compression
if compress:
# Script-based grouping: glyphs that co-occur in typical text rendering
# are grouped together for efficient LRU caching on the embedded target.
# Since glyphs are in codepoint order, glyphs in the same Unicode block
# are contiguous in the array and form natural groups.
SCRIPT_GROUP_RANGES = [
(0x0000, 0x007F), # ASCII
(0x0080, 0x00FF), # Latin-1 Supplement
(0x0100, 0x017F), # Latin Extended-A
(0x0300, 0x036F), # Combining Diacritical Marks
(0x0400, 0x04FF), # Cyrillic
(0x2000, 0x206F), # General Punctuation
(0x2070, 0x209F), # Superscripts & Subscripts
(0x20A0, 0x20CF), # Currency Symbols
(0x2190, 0x21FF), # Arrows
(0x2200, 0x22FF), # Math Operators
(0xFFFD, 0xFFFD), # Replacement Character
]
def get_script_group(code_point):
for i, (start, end) in enumerate(SCRIPT_GROUP_RANGES):
if start <= code_point <= end:
return i
return -1
groups = [] # list of (first_glyph_index, glyph_count)
current_group_id = None
group_start = 0
group_count = 0
for i, (props, packed) in enumerate(all_glyphs):
sg = get_script_group(props.code_point)
if sg != current_group_id:
if group_count > 0:
groups.append((group_start, group_count))
current_group_id = sg
group_start = i
group_count = 1
else:
group_count += 1
if group_count > 0:
groups.append((group_start, group_count))
# Compress each group
compressed_groups = [] # list of (compressed_bytes, uncompressed_size, glyph_count, first_glyph_index)
compressed_bitmap_data = []
compressed_offset = 0
# Also build modified glyph props with within-group offsets
modified_glyph_props = list(glyph_props)
for first_idx, count in groups:
# Concatenate bitmap data for this group
group_data = b''
for gi in range(first_idx, first_idx + count):
props, packed = all_glyphs[gi]
# Update glyph's dataOffset to be within-group offset
within_group_offset = len(group_data)
old_props = modified_glyph_props[gi]
modified_glyph_props[gi] = GlyphProps(
width=old_props.width,
height=old_props.height,
advance_x=old_props.advance_x,
left=old_props.left,
top=old_props.top,
data_length=old_props.data_length,
data_offset=within_group_offset,
code_point=old_props.code_point,
)
group_data += packed
# Compress with raw DEFLATE (no zlib/gzip header)
compressor = zlib.compressobj(level=9, wbits=-15)
compressed = compressor.compress(group_data) + compressor.flush()
compressed_groups.append((compressed, len(group_data), count, first_idx))
compressed_bitmap_data.extend(compressed)
compressed_offset += len(compressed)
glyph_props = modified_glyph_props
total_compressed = len(compressed_bitmap_data)
total_uncompressed = len(glyph_data)
print(f"// Compression: {total_uncompressed} -> {total_compressed} bytes ({100*total_compressed/total_uncompressed:.1f}%), {len(groups)} groups", file=sys.stderr)
print(f"""/**
* generated by fontconvert.py
* name: {font_name}
* size: {size}
* mode: {'2-bit' if is2Bit else '1-bit'}
* mode: {'2-bit' if is2Bit else '1-bit'}{' compressed: true' if compress else ''}
* Command used: {' '.join(sys.argv)}
*/
#pragma once
#include "EpdFontData.h"
""")
print(f"static const uint8_t {font_name}Bitmaps[{len(glyph_data)}] = {{")
for c in chunks(glyph_data, 16):
print (" " + " ".join(f"0x{b:02X}," for b in c))
print ("};\n");
if compress:
print(f"static const uint8_t {font_name}Bitmaps[{len(compressed_bitmap_data)}] = {{")
for c in chunks(compressed_bitmap_data, 16):
print (" " + " ".join(f"0x{b:02X}," for b in c))
print ("};\n");
else:
print(f"static const uint8_t {font_name}Bitmaps[{len(glyph_data)}] = {{")
for c in chunks(glyph_data, 16):
print (" " + " ".join(f"0x{b:02X}," for b in c))
print ("};\n");
print(f"static const EpdGlyph {font_name}Glyphs[] = {{")
for i, g in enumerate(glyph_props):
@@ -298,6 +393,14 @@ for i_start, i_end in intervals:
offset += i_end - i_start + 1
print ("};\n");
if compress:
print(f"static const EpdFontGroup {font_name}Groups[] = {{")
compressed_offset = 0
for compressed, uncompressed_size, count, first_idx in compressed_groups:
print(f" {{ {compressed_offset}, {len(compressed)}, {uncompressed_size}, {count}, {first_idx} }},")
compressed_offset += len(compressed)
print("};\n")
print(f"static const EpdFontData {font_name} = {{")
print(f" {font_name}Bitmaps,")
print(f" {font_name}Glyphs,")
@@ -307,4 +410,10 @@ print(f" {norm_ceil(face.size.height)},")
print(f" {norm_ceil(face.size.ascender)},")
print(f" {norm_floor(face.size.descender)},")
print(f" {'true' if is2Bit else 'false'},")
if compress:
print(f" {font_name}Groups,")
print(f" {len(compressed_groups)},")
else:
print(f" nullptr,")
print(f" 0,")
print("};")