--- name: TTF Font Investigation overview: Investigate replacing compile-time bitmap fonts with runtime TTF rendering using stb_truetype (the core of lvgl-ttf-esp32), integrated into the existing custom GfxRenderer pipeline for the ESP32-C3 e-ink reader. todos: - id: poc-stb content: "Phase 1: Add stb_truetype.h and build a minimal proof-of-concept that loads a TTF from SD, rasterizes glyphs, and draws them via GfxRenderer" status: pending - id: measure-ram content: "Phase 1: Measure actual RAM consumption and render performance of stb_truetype on ESP32-C3" status: pending - id: spiffs-mmap content: "Phase 3: Test SPIFFS memory-mapping of TTF files using esp_partition_mmap() to avoid loading into RAM" status: pending - id: font-provider content: "Phase 2: Create FontProvider abstraction layer and TtfFontProvider with glyph caching" status: pending - id: renderer-refactor content: "Phase 2: Refactor GfxRenderer to use FontProvider interface instead of direct EpdFontFamily" status: pending - id: settings-integration content: "Phase 4: Update settings to support arbitrary font sizes and custom font selection" status: pending - id: remove-bitmap-fonts content: "Phase 5: Remove compiled bitmap reader fonts, keep only small UI bitmap fonts" status: pending isProject: false --- # TTF Font Rendering Investigation ## Current State The project uses **no LVGL** -- it has a custom `GfxRenderer` that draws directly into an e-ink framebuffer. Fonts are pre-rasterized offline (TTF -> Python FreeType script -> C header bitmaps) and embedded at compile time. **Cost of current approach:** - **~2.7 MB flash** (mod build, Bookerly + NotoSans + Ubuntu) up to **~7 MB** (full build with OpenDyslexic) - **Only 4 discrete sizes** per family (12/14/16/18 pt) -- no runtime scaling - Each size x style (regular/bold/italic/bold-italic) is a separate ~80-200 KB bitmap blob - App partition is only **6.25 MB** -- fonts consume 43-100%+ of available space ## Why lvgl-ttf-esp32 Is Relevant (and What Isn't) The [lvgl-ttf-esp32](https://github.com/huming2207/lvgl-ttf-esp32) repo wraps **stb_truetype** (a single-header C library) with an LVGL font driver. Since this project does not use LVGL, the wrapper is irrelevant, but the **stb_truetype library itself** is exactly what's needed -- a lightweight, zero-dependency TTF rasterizer that runs on ESP32. ## Proposed Architecture ```mermaid flowchart TD subgraph current [Current Pipeline] TTF_Offline["TTF files (offline)"] --> fontconvert["fontconvert.py (FreeType)"] fontconvert --> headers["56 .h files (~2.7-7 MB flash)"] headers --> EpdFont["EpdFont / EpdFontFamily"] EpdFont --> GfxRenderer["GfxRenderer::renderChar()"] end subgraph proposed [Proposed Pipeline] TTF_SD["TTF files on SD card (~100-500 KB each)"] --> stb["stb_truetype.h (runtime)"] stb --> cache["Glyph cache (RAM + SD)"] cache --> TtfFont["TtfFont (new class)"] TtfFont --> FontProvider["FontProvider interface"] FontProvider --> GfxRenderer2["GfxRenderer::renderChar()"] end ``` ### Core Idea 1. **stb_truetype.h** -- add as a single header file in `lib/`. It rasterizes individual glyphs from TTF data on demand. 2. **TTF files on SD card** -- load at runtime from `.crosspoint/fonts/`. A typical TTF family (4 styles) is ~400-800 KB total vs 2.7 MB+ as bitmaps. 3. **Glyph cache** -- since e-ink pages are static, cache rasterized glyphs in RAM (LRU, ~20-50 KB) and optionally persist to SD card to avoid re-rasterizing across page turns. 4. `**FontProvider` abstraction** -- interface over both `EpdFont` (bitmap, for UI fonts) and new `TtfFont` (runtime, for reader fonts), so both can coexist. ## Integration Points These are the key files/interfaces that would need changes: | Component | File | Change | | ----------------- | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Font abstraction | New `lib/FontProvider/` | `FontProvider` interface with `getGlyph()`, `getMetrics()` | | TTF renderer | New `lib/TtfFont/` | Wraps stb_truetype, manages TTF loading + glyph cache | | GfxRenderer | [lib/GfxRenderer/GfxRenderer.h](lib/GfxRenderer/GfxRenderer.h) | Change `fontMap` from `EpdFontFamily` to `FontProvider*`; update `renderChar`, `getTextWidth`, `getSpaceWidth` | | Font registration | [src/main.cpp](src/main.cpp) | Register TTF fonts from SD instead of (or alongside) bitmap fonts | | Settings | [src/CrossPointSettings.cpp](src/CrossPointSettings.cpp) | `getReaderFontId()` supports arbitrary sizes, not just 4 discrete ones | | PlaceholderCover | [lib/PlaceholderCover/PlaceholderCoverGenerator.cpp](lib/PlaceholderCover/PlaceholderCoverGenerator.cpp) | Uses own `renderGlyph()` -- needs similar adaptation | | Text layout | [lib/Epub/Epub/ParsedText.cpp](lib/Epub/Epub/ParsedText.cpp) | Uses `getTextWidth()` / `getSpaceWidth()` for line breaking -- works unchanged if FontProvider is transparent | ## Feasibility Analysis ### Memory (ESP32-C3, ~380 KB RAM) - **stb_truetype itself**: ~15-20 KB code in flash, minimal RAM overhead - **TTF file in memory**: requires the full TTF loaded into RAM for glyph access. Options: - **Memory-mapped from flash (SPIFFS)**: store TTF in SPIFFS (3.4 MB available, currently unused), memory-map via `mmap()` on ESP-IDF -- zero RAM cost - **Partial loading from SD**: read only needed tables on demand (stb_truetype supports custom `stbtt_read()` but the default API expects full file in memory) - **Load into PSRAM**: ESP32-C3 has no PSRAM, so this is not an option - **Glyph cache**: ~50 bytes metadata + bitmap per glyph. At 18pt, a glyph bitmap is ~20x25 pixels = ~63 bytes (1-bit). Caching 256 glyphs = ~30 KB RAM. - **Rasterization temp buffer**: stb_truetype allocates ~10-20 KB temporarily per glyph render (uses `malloc`) **Verdict**: The biggest constraint is holding the TTF file in RAM. A typical Bookerly-Regular.ttf is ~150 KB. With 4 styles loaded, that's ~600 KB -- **too much for 380 KB RAM**. The viable path is using **SPIFFS** to store TTFs and memory-map them, or implementing a chunked reader that loads TTF table data on demand from SD. ### Flash Savings - **Remove**: 2.7-7 MB of bitmap font headers from firmware - **Add**: ~40 KB for stb_truetype + TtfFont code - **Net savings**: **2.6-6.9 MB flash freed** - TTF files move to SD card or SPIFFS (not in firmware) ### Performance - stb_truetype rasterizes a glyph in **~0.5-2 ms** on ESP32 (160 MHz) - A typical page has ~~200-300 glyphs, but with caching, only unique glyphs need rasterizing (~~60-80 per page) - **First page render**: ~60-160 ms extra for cache warmup - **Subsequent pages**: mostly cache hits, negligible overhead - E-ink refresh takes ~300-1000 ms anyway, so TTF rasterization cost is acceptable ### Anti-aliasing for E-ink stb_truetype produces 8-bit alpha bitmaps (256 levels). The current system uses 1-bit or 2-bit glyphs. The adapter would: - **1-bit mode**: threshold the alpha (e.g., alpha > 128 = black) - **2-bit mode**: quantize to 4 levels (0, 85, 170, 255) for e-ink grayscale This should actually produce **better quality** than the offline FreeType conversion since stb_truetype does sub-pixel hinting. ## Recommended Implementation Phases ### Phase 1: Proof of Concept (stb_truetype standalone) - Add stb_truetype.h to the project - Write a minimal test that loads a TTF from SD, rasterizes a few glyphs, and draws them via `GfxRenderer::drawPixel()` - Measure RAM usage and render time - Validate glyph quality on e-ink ### Phase 2: FontProvider Abstraction - Create `FontProvider` interface matching `EpdFontFamily`'s public API - Wrap existing `EpdFontFamily` in a `BitmapFontProvider` - Create `TtfFontProvider` backed by stb_truetype + glyph cache - Refactor `GfxRenderer::fontMap` to use `FontProvider*` ### Phase 3: TTF Storage Strategy - Evaluate SPIFFS memory mapping vs. SD-card chunked loading - Implement the chosen strategy - Handle font discovery (scan SD card for `.ttf` files) ### Phase 4: Settings and UI Integration - Replace discrete font-size enum with a continuous size setting (or finer granularity) - Add "Custom Font" option in settings - Update section cache invalidation when font/size changes ### Phase 5: Remove Bitmap Reader Fonts - Keep bitmap fonts only for UI (Ubuntu 10/12, NotoSans 8) which are small (~62 KB) - Remove Bookerly, NotoSans, OpenDyslexic bitmap headers - Ship TTF files on SD card (or downloadable) ## Key Risk: TTF-in-RAM on ESP32-C3 The critical question is whether TTF file data can be accessed without loading the full file into RAM. Three mitigation strategies: 1. **SPIFFS + mmap**: Store TTFs in the 3.4 MB SPIFFS partition and use ESP-IDF's `esp_partition_mmap()` to map them into the address space. Zero RAM cost, but SPIFFS is read-only after flashing (unless written at runtime). 2. **SD card + custom I/O**: Implement `stbtt_GetFontOffsetForIndex` and glyph extraction using buffered SD reads. stb_truetype's API assumes a contiguous byte array, so this would require a patched or wrapper approach. 3. **Load one style at a time**: Only keep the active style's TTF in RAM (~150 KB). Switch styles by unloading/reloading. Feasible but adds latency on style changes (bold/italic). Strategy 1 (SPIFFS mmap) is the most promising since the SPIFFS partition is already allocated but unused.