crosspoint-reader-mod/.cursor/plans/ttf_font_investigation_61ba7279.plan.md

---
name: TTF Font Investigation
overview: Investigate replacing compile-time bitmap fonts with runtime TTF rendering using stb_truetype (the core of lvgl-ttf-esp32), integrated into the existing custom GfxRenderer pipeline for the ESP32-C3 e-ink reader.
todos:
  - id: poc-stb
    content: "Phase 1: Add stb_truetype.h and build a minimal proof-of-concept that loads a TTF from SD, rasterizes glyphs, and draws them via GfxRenderer"
    status: pending
  - id: measure-ram
    content: "Phase 1: Measure actual RAM consumption and render performance of stb_truetype on ESP32-C3"
    status: pending
  - id: spiffs-mmap
    content: "Phase 3: Test SPIFFS memory-mapping of TTF files using esp_partition_mmap() to avoid loading into RAM"
    status: pending
  - id: font-provider
    content: "Phase 2: Create FontProvider abstraction layer and TtfFontProvider with glyph caching"
    status: pending
  - id: renderer-refactor
    content: "Phase 2: Refactor GfxRenderer to use FontProvider interface instead of direct EpdFontFamily"
    status: pending
  - id: settings-integration
    content: "Phase 4: Update settings to support arbitrary font sizes and custom font selection"
    status: pending
  - id: remove-bitmap-fonts
    content: "Phase 5: Remove compiled bitmap reader fonts, keep only small UI bitmap fonts"
    status: pending
isProject: false
---

# TTF Font Rendering Investigation

## Current State

The project uses **no LVGL** -- it has a custom `GfxRenderer` that draws directly into an e-ink framebuffer. Fonts are pre-rasterized offline (TTF -> Python FreeType script -> C header bitmaps) and embedded at compile time.

**Cost of current approach:**

- **~2.7 MB flash** (mod build, Bookerly + NotoSans + Ubuntu) up to **~7 MB** (full build with OpenDyslexic)
- **Only 4 discrete sizes** per family (12/14/16/18 pt) -- no runtime scaling
- Each size x style (regular/bold/italic/bold-italic) is a separate ~80-200 KB bitmap blob
- App partition is only **6.25 MB** -- fonts consume 43-100%+ of available space

## Why lvgl-ttf-esp32 Is Relevant (and What Isn't)

The [lvgl-ttf-esp32](https://github.com/huming2207/lvgl-ttf-esp32) repo wraps **stb_truetype** (a single-header C library) with an LVGL font driver. Since this project does not use LVGL, the wrapper is irrelevant, but the **stb_truetype library itself** is exactly what's needed -- a lightweight, zero-dependency TTF rasterizer that runs on ESP32.

## Proposed Architecture

```mermaid
flowchart TD
    subgraph current [Current Pipeline]
        TTF_Offline["TTF files (offline)"] --> fontconvert["fontconvert.py (FreeType)"]
        fontconvert --> headers["56 .h files (~2.7-7 MB flash)"]
        headers --> EpdFont["EpdFont / EpdFontFamily"]
        EpdFont --> GfxRenderer["GfxRenderer::renderChar()"]
    end

    subgraph proposed [Proposed Pipeline]
        TTF_SD["TTF files on SD card (~100-500 KB each)"] --> stb["stb_truetype.h (runtime)"]
        stb --> cache["Glyph cache (RAM + SD)"]
        cache --> TtfFont["TtfFont (new class)"]
        TtfFont --> FontProvider["FontProvider interface"]
        FontProvider --> GfxRenderer2["GfxRenderer::renderChar()"]
    end
```



### Core Idea

1. **stb_truetype.h** -- add as a single header file in `lib/`. It rasterizes individual glyphs from TTF data on demand.
2. **TTF files on SD card** -- load at runtime from `.crosspoint/fonts/`. A typical TTF family (4 styles) is ~400-800 KB total vs 2.7 MB+ as bitmaps.
3. **Glyph cache** -- since e-ink pages are static, cache rasterized glyphs in RAM (LRU, ~20-50 KB) and optionally persist to SD card to avoid re-rasterizing across page turns.
4. `**FontProvider` abstraction** -- interface over both `EpdFont` (bitmap, for UI fonts) and new `TtfFont` (runtime, for reader fonts), so both can coexist.

## Integration Points

These are the key files/interfaces that would need changes:


| Component         | File                                                                                                     | Change                                                                                                         |
| ----------------- | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| Font abstraction  | New `lib/FontProvider/`                                                                                  | `FontProvider` interface with `getGlyph()`, `getMetrics()`                                                     |
| TTF renderer      | New `lib/TtfFont/`                                                                                       | Wraps stb_truetype, manages TTF loading + glyph cache                                                          |
| GfxRenderer       | [lib/GfxRenderer/GfxRenderer.h](lib/GfxRenderer/GfxRenderer.h)                                           | Change `fontMap` from `EpdFontFamily` to `FontProvider*`; update `renderChar`, `getTextWidth`, `getSpaceWidth` |
| Font registration | [src/main.cpp](src/main.cpp)                                                                             | Register TTF fonts from SD instead of (or alongside) bitmap fonts                                              |
| Settings          | [src/CrossPointSettings.cpp](src/CrossPointSettings.cpp)                                                 | `getReaderFontId()` supports arbitrary sizes, not just 4 discrete ones                                         |
| PlaceholderCover  | [lib/PlaceholderCover/PlaceholderCoverGenerator.cpp](lib/PlaceholderCover/PlaceholderCoverGenerator.cpp) | Uses own `renderGlyph()` -- needs similar adaptation                                                           |
| Text layout       | [lib/Epub/Epub/ParsedText.cpp](lib/Epub/Epub/ParsedText.cpp)                                             | Uses `getTextWidth()` / `getSpaceWidth()` for line breaking -- works unchanged if FontProvider is transparent  |


## Feasibility Analysis

### Memory (ESP32-C3, ~380 KB RAM)

- **stb_truetype itself**: ~15-20 KB code in flash, minimal RAM overhead
- **TTF file in memory**: requires the full TTF loaded into RAM for glyph access. Options:
  - **Memory-mapped from flash (SPIFFS)**: store TTF in SPIFFS (3.4 MB available, currently unused), memory-map via `mmap()` on ESP-IDF -- zero RAM cost
  - **Partial loading from SD**: read only needed tables on demand (stb_truetype supports custom `stbtt_read()` but the default API expects full file in memory)
  - **Load into PSRAM**: ESP32-C3 has no PSRAM, so this is not an option
- **Glyph cache**: ~50 bytes metadata + bitmap per glyph. At 18pt, a glyph bitmap is ~20x25 pixels = ~63 bytes (1-bit). Caching 256 glyphs = ~30 KB RAM.
- **Rasterization temp buffer**: stb_truetype allocates ~10-20 KB temporarily per glyph render (uses `malloc`)

**Verdict**: The biggest constraint is holding the TTF file in RAM. A typical Bookerly-Regular.ttf is ~150 KB. With 4 styles loaded, that's ~600 KB -- **too much for 380 KB RAM**. The viable path is using **SPIFFS** to store TTFs and memory-map them, or implementing a chunked reader that loads TTF table data on demand from SD.

### Flash Savings

- **Remove**: 2.7-7 MB of bitmap font headers from firmware
- **Add**: ~40 KB for stb_truetype + TtfFont code
- **Net savings**: **2.6-6.9 MB flash freed**
- TTF files move to SD card or SPIFFS (not in firmware)

### Performance

- stb_truetype rasterizes a glyph in **~0.5-2 ms** on ESP32 (160 MHz)
- A typical page has ~~200-300 glyphs, but with caching, only unique glyphs need rasterizing (~~60-80 per page)
- **First page render**: ~60-160 ms extra for cache warmup
- **Subsequent pages**: mostly cache hits, negligible overhead
- E-ink refresh takes ~300-1000 ms anyway, so TTF rasterization cost is acceptable

### Anti-aliasing for E-ink

stb_truetype produces 8-bit alpha bitmaps (256 levels). The current system uses 1-bit or 2-bit glyphs. The adapter would:

- **1-bit mode**: threshold the alpha (e.g., alpha > 128 = black)
- **2-bit mode**: quantize to 4 levels (0, 85, 170, 255) for e-ink grayscale

This should actually produce **better quality** than the offline FreeType conversion since stb_truetype does sub-pixel hinting.

## Recommended Implementation Phases

### Phase 1: Proof of Concept (stb_truetype standalone)

- Add stb_truetype.h to the project
- Write a minimal test that loads a TTF from SD, rasterizes a few glyphs, and draws them via `GfxRenderer::drawPixel()`
- Measure RAM usage and render time
- Validate glyph quality on e-ink

### Phase 2: FontProvider Abstraction

- Create `FontProvider` interface matching `EpdFontFamily`'s public API
- Wrap existing `EpdFontFamily` in a `BitmapFontProvider`
- Create `TtfFontProvider` backed by stb_truetype + glyph cache
- Refactor `GfxRenderer::fontMap` to use `FontProvider*`

### Phase 3: TTF Storage Strategy

- Evaluate SPIFFS memory mapping vs. SD-card chunked loading
- Implement the chosen strategy
- Handle font discovery (scan SD card for `.ttf` files)

### Phase 4: Settings and UI Integration

- Replace discrete font-size enum with a continuous size setting (or finer granularity)
- Add "Custom Font" option in settings
- Update section cache invalidation when font/size changes

### Phase 5: Remove Bitmap Reader Fonts

- Keep bitmap fonts only for UI (Ubuntu 10/12, NotoSans 8) which are small (~62 KB)
- Remove Bookerly, NotoSans, OpenDyslexic bitmap headers
- Ship TTF files on SD card (or downloadable)

## Key Risk: TTF-in-RAM on ESP32-C3

The critical question is whether TTF file data can be accessed without loading the full file into RAM. Three mitigation strategies:

1. **SPIFFS + mmap**: Store TTFs in the 3.4 MB SPIFFS partition and use ESP-IDF's `esp_partition_mmap()` to map them into the address space. Zero RAM cost, but SPIFFS is read-only after flashing (unless written at runtime).
2. **SD card + custom I/O**: Implement `stbtt_GetFontOffsetForIndex` and glyph extraction using buffered SD reads. stb_truetype's API assumes a contiguous byte array, so this would require a patched or wrapper approach.
3. **Load one style at a time**: Only keep the active style's TTF in RAM (~150 KB). Switch styles by unloading/reloading. Feasible but adds latency on style changes (bold/italic).

Strategy 1 (SPIFFS mmap) is the most promising since the SPIFFS partition is already allocated but unused.