0493f300be4a06afa28e6e20b666d4587eae9b49
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7dc518624c |
fix: Use fixed-point fractional x-advance and kerning for better text layout (#1168)
## Summary **What is the goal of this PR?** Hopefully fixes #1182. _Note: I think letterforms got a "heavier" appearance after #1098, which makes this more noticeable. The current version of this PR reverts the change to add `--force-autohint` for Bookerly, which to me seems to bring the font back to a more aesthetic and consistent weight._ #### Problem Character spacing was uneven in certain words. The word "drew" in Bookerly was the clearest example: a visible gap between `d` and `r`, while `e` and `w` appeared tightly condensed. The root cause was twofold: 1. **Integer-only glyph advances.** `advanceX` was stored as a `uint8_t` of whole pixels, sourced from FreeType's hinted `advance.x` (which grid-fits to integers). A glyph whose true advance is 15.56px was stored as 16px -- an error of +0.44px per character that compounds across a line. 2. **Floor-rounded kerning.** Kern adjustments were converted with `math.floor()`, which systematically over-tightened negative kerns. A kern of -0.3px became -1px -- a 0.7px over-correction that visibly closed gaps. Combined, these produced the classic symptom: some pairs too wide, others too tight, with the imbalance varying per word. #### Solution: fixed-point accumulation with 1/16-pixel resolution, for sub-pixel precision during text layout All font metrics now use a "fixed-point 4" format -- 4 fractional bits giving 1/16-pixel (0.0625px) resolution. This is implemented with plain integer arithmetic (shifts and adds), requiring no floating-point on the ESP32. **How it works:** A value like 15.56px is stored as the integer `249`: ``` 249 = 15 * 16 + 9 (where 9/16 = 0.5625, closest to 0.56) ``` Two storage widths share the same 4 fractional bits: | Field | Type | Format | Range | Use | |-------|------|--------|-------|-----| | `advanceX` | `uint16_t` | 12.4 | 0 -- 4095.9375 px | Glyph advance width | | `kernMatrix` | `int8_t` | 4.4 | -8.0 -- +7.9375 px | Kerning adjustment | Because both have 4 fractional bits, they add directly into a single `int32_t` accumulator during layout. The accumulator is only snapped to the nearest whole pixel at the moment each glyph is rendered: ```cpp int32_t xFP = fp4::fromPixel(startX); // pixel to 12.4: startX << 4 for each character: xFP += kernFP; // add 4.4 kern (sign-extends into int32_t) int xPx = fp4::toPixel(xFP); // snap to nearest pixel: (xFP + 8) >> 4 render glyph at xPx; xFP += glyph->advanceX; // add 12.4 advance ``` Fractional remainders carry forward indefinitely. Rounding errors stay below +/- 0.5px and never compound. #### Concrete example: "drew" in Bookerly **Before** (integer advances, floor-rounded kerning): | Char | Advance | Kern | Cursor | Snap | Gap from prev | |------|---------|------|--------|------|---------------| | d | 16 px | -- | 33 | 33 | -- | | r | 12 px | 0 | 49 | 49 | ~2px | | e | 13 px | -1 | 60 | 60 | ~0px | | w | 22 px | -1 | 72 | 72 | ~0px | The d-to-r gap was visibly wider than the tightly packed `rew`. **After** (12.4 advances, 4.4 kerning, fractional accumulation): | Char | Advance (FP) | Kern (FP) | Accumulator | Snap | Ink start | Gap from prev | |------|-------------|-----------|-------------|------|-----------|---------------| | d | 249 (15.56px) | -- | 528 | 33 | 34 | -- | | r | 184 (11.50px) | 0 | 777 | 49 | 49 | 0px | | e | 208 (13.00px) | -8 (-0.50px) | 953 | 60 | 61 | 1px | | w | 356 (22.25px) | -4 (-0.25px) | 1157 | 72 | 72 | 0px | Spacing is now `0, 1, 0` pixels -- nearly uniform. Verified on-device: all 5 copies of "drew" in the test EPUB produce identical spacing, confirming zero accumulator drift. #### Changes **Font conversion (`fontconvert.py`)** - Use `linearHoriAdvance` (FreeType 16.16, unhinted) instead of `advance.x` (26.6, grid-fitted to integers) for glyph advances - Encode kern values as 4.4 fixed-point with `round()` instead of `floor()` - Add `fp4_from_ft16_16()` and `fp4_from_design_units()` helper functions - Add module-level documentation of fixed-point conventions **Font data structures (`EpdFontData.h`)** - `EpdGlyph::advanceX`: `uint8_t` to `uint16_t` (no memory cost due to existing struct padding) - Add `fp4` namespace with `constexpr` helpers: `fromPixel()`, `toPixel()`, `toFloat()` - Document fixed-point conventions **Font API (`EpdFont.h/cpp`, `EpdFontFamily.h/cpp`)** - `getKerning()` return type: `int8_t` to `int` (to avoid truncation of the 4.4 value) **Rendering (`GfxRenderer.cpp`)** - `drawText()`: replace integer cursor with `int32_t` fixed-point accumulator - `drawTextRotated90CW()`: same accumulator treatment for vertical layout - `getTextAdvanceX()`, `getSpaceWidth()`, `getSpaceKernAdjust()`, `getKerning()`: convert from fixed-point to pixel at API boundary **Regenerated all built-in font headers** with new 12.4 advances and 4.4 kern values. #### Memory impact Zero additional RAM. The `advanceX` field grew from `uint8_t` to `uint16_t`, but the `EpdGlyph` struct already had 1 byte of padding at that position, so the struct size is unchanged. The fixed-point accumulator is a single `int32_t` on the stack. #### Test plan - [ ] Verify "drew" spacing in Bookerly at small, medium, and large sizes - [ ] Verify uppercase kerning pairs: AVERY, WAVE, VALUE - [ ] Verify ligature words: coffee, waffle, office - [ ] Verify all built-in fonts render correctly at each size - [ ] Verify rotated text (progress bar percentage) renders correctly - [ ] Verify combining marks (accented characters) still position correctly - [ ] Spot-check a full-length book for any layout regressions --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES, Claude Opus 4.6 helped figure out a non-floating point approach for sub-pixel error accumulation**_ |
||
|
|
cb72916397 |
feat: Latin Extended-B European glyphs (#1167)
## Summary **What is the goal of this PR?** Correction to #1157, which (embarrassingly) failed to actually include the updated font header files. (Maybe we should generate these at build time?) Add Latin Extended-B glyphs for Croatian, Romanian, Pinyin, and European diacritical variants. Fixes #921. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**PARTIALLY, confirmed codepoint ranges with Claude**_ |
||
|
|
5050992bd6 |
feat: Vietnamese glyphs support (#1147)
## Summary * **What is the goal of this PR?** Add Vietnamese glyphs support for the reader's built-in fonts, enabling proper rendering of Vietnamese text in EPUB content. * **What changes are included?** - Added 3 new Unicode intervals to `fontconvert.py` covering Vietnamese characters: - **Latin Extended-B** (Vietnamese subset only): `U+01A0–U+01B0` — Ơ/ơ, Ư/ư - **Vietnamese Extended**: `U+1EA0–U+1EF9` — All precomposed Vietnamese characters with tone marks (Ả, Ấ, Ầ, Ẩ, Ẫ, Ậ, Ắ, …, Ỹ) - Re-generated all 54 built-in font header files (Bookerly, Noto Sans, OpenDyslexic, Ubuntu across all sizes and styles) to include the new Vietnamese glyphs. ## Additional Context * **Scope**: This PR only covers the **reader** fonts. The outer UI still uses the Ubuntu font which does not fully support Vietnamese — UI and i18n will be addressed in a follow-up PR (per discussion in PR #1124). * **Memory impact**: | Metric | Before | After | Delta | |---|---|---|---| | Flash Data (`.rodata`) | 2,971,028 B | 3,290,748 B | **+319,720 B (+10.8%)** | | Total image size | 4,663,235 B | 4,982,955 B | **+319,720 B (+6.9%)** | | Flash usage | 69.1% | 74.0% | **+4.9 pp** | | RAM usage | 29.0% | 29.0% | **No change** | * **Risk**: Low — this is a data-only change (font glyph tables in `.rodata`). No logic changes, no RAM impact. Flash headroom remains comfortable at 74%. --- ### AI Usage Did you use AI tools to help write this code? _**PARTIALLY**_ AI was used to identify the minimal set of Unicode ranges needed for Vietnamese support and to assist with the PR description. --------- Co-authored-by: danoooob <danoooob@example.com> |
||
|
|
0eb8a9346b |
feat: Support for kerning and ligatures (#873)
## Summary **What is the goal of this PR?** Improved typesetting, including [kerning](https://en.wikipedia.org/wiki/Kerning) and [ligatures](https://en.wikipedia.org/wiki/Ligature_(writing)#Latin_alphabet). **What changes are included?** - The script to convert built-in fonts now adds kerning and ligature information to the generated font headers. - Epub page layout calculates proper kerning spaces and makes ligature substitutions according to the selected font.    ## Additional Context - I am not a typography expert. - The implementation has been reworked from the earlier version, so it is no longer necessary to omit Open Dyslexic, and kerning data now covers all fonts, styles, and codepoints for which we include bitmap data. - Claude Opus 4.6 helped with a lot of this. - There's an included test epub document with lots of kerning and ligature examples, shown in the photos. **_After some time to mature, I think this change is in decent shape to merge and get people testing._** After opening this PR I came across #660, which overlaps in adding ligature support. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES, Claude Opus 4.6**_ --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
052f497b9e |
fix: force auto-hinting for Bookerly to fix inconsistent stem widths (#1098)
## Summary Bookerly's native TrueType hinting is effectively a no-op at the sizes used here, causing FreeType to place stems at inconsistent sub-pixel positions. This results in the 'k' stem (8-bit fringe: 0x38=56) falling just below the 2-bit quantization threshold while 'l' and 'h' stems (fringes: 0x4C=76, 0x40=64) land above it --- making 'k' visibly narrower (2.00px vs 2.33px effective width). FreeType's auto-hinter snaps all stems to consistent grid positions, normalizing effective stem width to 2.67px across all glyphs. Adds --force-autohint flag to fontconvert.py and applies it to Bookerly only. NotoSans, OpenDyslexic, and Ubuntu fonts are unaffected. Here is an example of before/after. Take notice of the vertical stems on characters like `l`, `k`, `n`, `i`, etc. The font is Bookerly 12pt regular: **BEFORE**:  **AFTER**:  Claude generated this script to quantitatively determine that this change makes the vertical stems on a variety of characters more consistent for Bookerly _only_. <details> <summary>Python script</summary> ```python #!/usr/bin/env python3 """Compare stem consistency across all font families with and without auto-hinting. Run from repo root: python3 compare_all_fonts.py """ import freetype DPI = 150 CHARS = ["k", "l", "h", "i", "b", "d"] SIZES = [12, 14, 16, 18] FONTS = { "Bookerly": "lib/EpdFont/builtinFonts/source/Bookerly/Bookerly-Regular.ttf", "NotoSans": "lib/EpdFont/builtinFonts/source/NotoSans/NotoSans-Regular.ttf", "OpenDyslexic": "lib/EpdFont/builtinFonts/source/OpenDyslexic/OpenDyslexic-Regular.otf", "Ubuntu": "lib/EpdFont/builtinFonts/source/Ubuntu/Ubuntu-Regular.ttf", } MODES = { "default": freetype.FT_LOAD_RENDER, "autohint": freetype.FT_LOAD_RENDER | freetype.FT_LOAD_FORCE_AUTOHINT, } def q4to2(v): if v >= 12: return 3 elif v >= 8: return 2 elif v >= 4: return 1 else: return 0 def get_stem_eff(face, char, flags): gi = face.get_char_index(ord(char)) if gi == 0: return None face.load_glyph(gi, flags) bm = face.glyph.bitmap w, h = bm.width, bm.rows if w == 0 or h == 0: return None p2 = [] for y in range(h): row = [] for x in range(w): row.append(q4to2(bm.buffer[y * bm.pitch + x] >> 4)) p2.append(row) # Measure leftmost stem in stable middle rows mid_start, mid_end = h // 4, h - h // 4 widths = [] for y in range(mid_start, mid_end): first = next((x for x in range(w) if p2[y][x] > 0), -1) if first < 0: continue last = first for x in range(first, w): if p2[y][x] > 0: last = x else: break eff = sum(p2[y][x] for x in range(first, last + 1)) / 3.0 widths.append(eff) return round(sum(widths) / len(widths), 2) if widths else None def main(): for font_name, font_path in FONTS.items(): try: freetype.Face(font_path) except Exception: print(f"\n {font_name}: SKIPPED (file not found)") continue print(f"\n{'=' * 80}") print(f" {font_name}") print(f"{'=' * 80}") for size in SIZES: print(f"\n {size}pt:") print(f" {'':6s}", end="") for c in CHARS: print(f" '{c}' ", end="") print(" | spread") for mode_name, flags in MODES.items(): face = freetype.Face(font_path) face.set_char_size(size << 6, size << 6, DPI, DPI) vals = [] print(f" {mode_name:6s}", end="") for c in CHARS: v = get_stem_eff(face, c, flags) vals.append(v) print(f" {v:5.2f}" if v else " N/A", end="") valid = [v for v in vals if v is not None] spread = max(valid) - min(valid) if len(valid) >= 2 else 0 marker = " <-- inconsistent" if spread > 0.5 else "" print(f" | {spread:.2f}{marker}") if __name__ == "__main__": main() ``` </details> Here are the results. The table compares how the font-generation `autohint` flag affects the range of widths of various characters. Lower `spread` mean that glyph stroke widths should appear more consistent. ``` Spread = max stem width - min stem width across glyphs (lower = more consistent): ┌──────────────┬──────┬─────────┬──────────┬──────────┐ │ Font │ Size │ Default │ Autohint │ Winner │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ Bookerly │ 12pt │ 1.49 │ 1.12 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 1.39 │ 1.13 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 1.38 │ 1.16 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.90 │ 1.58 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ NotoSans │ 12pt │ 1.16 │ 0.94 │ mixed │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 0.83 │ 1.14 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 1.41 │ 1.51 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.74 │ 1.63 │ mixed │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ OpenDyslexic │ 12pt │ 2.22 │ 1.44 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 2.57 │ 3.29 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 3.13 │ 2.60 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 3.21 │ 3.23 │ ~tied │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ Ubuntu │ 12pt │ 1.25 │ 1.31 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 1.41 │ 1.64 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 2.21 │ 1.71 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.80 │ 1.71 │ autohint │ └──────────────┴──────┴─────────┴──────────┴──────────┘ ``` --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? I used AI to make sure I'm not doing something stupid, since I'm not a typography expert. I made the changes though. |
||
|
|
47aa0dda76 |
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary **What is the goal of this PR?** Compress reader font bitmaps to reduce flash usage by 30.7%. **What changes are included?** - New `EpdFontGroup` struct and extended `EpdFontData` with `groups`/`groupCount` fields - `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group + groups of 8) and compresses each with raw DEFLATE - `FontDecompressor` class with 4-slot LRU cache for on-demand decompression during rendering - `GfxRenderer` transparently routes bitmap access through `getGlyphBitmap()` (compressed or direct flash) - Uses `uzlib` for decompression with minimal heap overhead. - 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated with compression; 5 UI fonts unchanged - Round-trip verification script (`verify_compression.py`) runs as part of font generation ## Additional Context ## Flash & RAM | | baseline | font-compression | Difference | |--|--------|-----------------|------------| | Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B (-30.7%) | | firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) | | RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) | ## Script-Based Grouping (Cold Cache) Comparison of uncompressed baseline vs script-based group compression (4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic, General Punctuation, etc.) instead of sequential groups of 8. ### Render Time | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) | | **Pages** | 37 | 37 | | ### Memory Usage | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB | | **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB | | **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB | | **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB | ### Cache Effectiveness | | Misses/page | Hit rate | |---|---|---| | **Compressed (cold cache)** | 2.1 | 99.85% | ------ ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES**_ Implementation was done by Claude Code (Opus 4.6) based on a plan developed collaboratively. All generated font headers were verified with an automated round-trip decompression test. The firmware was compiled successfully but has not yet been tested on-device. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
a4b9a43ca1 |
docs: add font generation commands to builtin font headers (#547)
## Summary * **What is the goal of this PR?** Simple quality of life, ease maintenance * **What changes are included?** Update fontconvert.py to include the command used to generate each font file in the header comment, making it easier to regenerate fonts when needed. I plan on adding options to this scripts (kerning, and maybe ligatures), thus knowing which command was used, even with already existing options like `--additional-intervals`, is important. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**NO**_ |
||
|
|
c9b5462370 |
feat: Include superscripts and subscripts in fonts (#463)
## Summary * Include superscripts and subscripts in fonts ## Additional Context * Original change came from https://github.com/crosspoint-reader/crosspoint-reader/pull/248 --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? No --------- Co-authored-by: cor <cor@pruijs.dev> |
||
|
|
5fef99c641 |
fix: render U+FFFD replacement character instead of ? (#366)
The current behavior of rendering `?` for an unknown Unicode character can be hard to distinguish from a typo. Use the standard Unicode "replacement character" instead, that's what it's designed for: https://en.wikipedia.org/wiki/Specials_(Unicode_block) I'm making this PR as a draft because I'm not sure I did everything that was needed to change the character set covered by the fonts. Running that script is in its own commit. If this is proper, I'll rebase/squash into one commit and un-draft. Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com> |
||
|
|
f2ca65d752 |
Swap from Aleo to Bookerly for wider glyph support (#172)
## Summary * Swap from Aleo to Bookerly for wider glyph support * Swap from Space Grotesk to a small Noto Sans ## Additional Context * 0.11.0 swapped to Aleo which has a few issues (things like Cyrillic support for eg) |