## Summary Replaces the picojpeg library with bitbank2/JPEGDEC for JPEG decoding in the EPUB image pipeline. JPEGDEC provides built-in coarse scaling (1/2, 1/4, 1/8), 8-bit grayscale output, and streaming block-based decoding via callbacks. Includes a pre-build patch script for two JPEGDEC changes affecting progressive JPEG support and EIGHT_BIT_GRAYSCALE mode. Closes #912 ## Additional Context # Example progressive jpeg <img src="https://github.com/user-attachments/assets/e63bb4f8-f862-4aa0-a01f-d1ef43a4b27a" width="400" height="800" /> Good performance increase from JPEGDEC over picojpeg cc @bitbank2 thanks ## Baseline JPEG Decode Performance: picojpeg vs JPEGDEC (float in callback) vs JPEGDEC (fixed-point in callback) Tested with `test_jpeg_images.epub` on device (ESP32-C3), first decode (no cache). | Image | Source | Output | picojpeg | JPEGDEC float | JPEGDEC fixed-point | vs picojpeg | vs float | |-------|--------|--------|----------|---------------|---------------------|-------------|----------| | jpeg_format.jpg | 350x250 | 350x250 | 313 ms | 256 ms | **104 ms** | **3.0x** | **2.5x** | | grayscale_test.jpg | 400x600 | 400x600 | 768 ms | 661 ms | **246 ms** | **3.1x** | **2.7x** | | gradient_test.jpg | 400x500 | 400x500 | 707 ms | 597 ms | **247 ms** | **2.9x** | **2.4x** | | centering_test.jpg | 350x400 | 350x400 | 502 ms | 412 ms | **169 ms** | **3.0x** | **2.4x** | | scaling_test.jpg | 1200x1500 | 464x580 | 5487 ms | 1114 ms | **668 ms** | **8.2x** | **1.7x** | | wide_scaling_test.jpg | 1807x736 | 464x188 | 4237 ms | 642 ms | **497 ms** | **8.5x** | **1.3x** | | cache_test_1.jpg | 400x300 | 400x300 | 422 ms | 348 ms | **141 ms** | **3.0x** | **2.5x** | | cache_test_2.jpg | 400x300 | 400x300 | 424 ms | 349 ms | **142 ms** | **3.0x** | **2.5x** | ### Summary - **1:1 scale (fixed-point vs float)**: ~2.5x faster — eliminating software float on the FPU-less ESP32-C3 is the dominant win - **1:1 scale (fixed-point vs picojpeg)**: ~3.0x faster overall - **Downscaled images (vs picojpeg)**: 8-9x faster — JPEGDEC's coarse scaling + fixed-point draw callback - **Downscaled images (fixed-point vs float)**: 1.3-1.7x — less dramatic since JPEG library decode dominates over the draw callback for fewer output pixels - The fixed-point optimization alone (vs float JPEGDEC) saved **~60% of render time** on 1:1 images, confirming that software float emulation was the primary bottleneck in the draw callback - See thread for discussions on quality of progressive images, https://github.com/crosspoint-reader/crosspoint-reader/pull/1136#issuecomment-3952952315 - and the conclusion https://github.com/crosspoint-reader/crosspoint-reader/pull/1136#issuecomment-3959379386 - Proposal to improve quality added at https://github.com/crosspoint-reader/crosspoint-reader/discussions/1179 --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**< PARTIALLY >**_ --------- Co-authored-by: Dave Allie <dave@daveallie.com>
118 lines
4.2 KiB
Python
118 lines
4.2 KiB
Python
"""
|
|
PlatformIO pre-build script: patch JPEGDEC library for progressive JPEG support.
|
|
|
|
Two patches are applied:
|
|
|
|
1. JPEGMakeHuffTables: Skip AC Huffman table construction for progressive JPEGs.
|
|
JPEGDEC 1.8.x fails to open progressive JPEGs because JPEGMakeHuffTables()
|
|
cannot build AC tables with 11+-bit codes (the "slow tables" path is disabled).
|
|
Since progressive decode only uses DC coefficients, AC tables are not needed.
|
|
|
|
2. JPEGDecodeMCU_P: Guard pMCU writes against MCU_SKIP (-8).
|
|
The non-progressive JPEGDecodeMCU checks `iMCU >= 0` before writing to pMCU,
|
|
but JPEGDecodeMCU_P does not. When EIGHT_BIT_GRAYSCALE mode skips chroma
|
|
channels by passing MCU_SKIP, the unguarded write goes to a wild pointer
|
|
(sMCUs[0xFFFFF8]) and crashes.
|
|
|
|
Both patches are applied idempotently so it is safe to run on every build.
|
|
"""
|
|
|
|
Import("env")
|
|
import os
|
|
|
|
def patch_jpegdec(env):
|
|
# Find the JPEGDEC library in libdeps
|
|
libdeps_dir = os.path.join(env["PROJECT_DIR"], ".pio", "libdeps")
|
|
if not os.path.isdir(libdeps_dir):
|
|
return
|
|
for env_dir in os.listdir(libdeps_dir):
|
|
jpeg_inl = os.path.join(libdeps_dir, env_dir, "JPEGDEC", "src", "jpeg.inl")
|
|
if os.path.isfile(jpeg_inl):
|
|
_apply_ac_table_patch(jpeg_inl)
|
|
_apply_mcu_skip_patch(jpeg_inl)
|
|
|
|
def _apply_ac_table_patch(filepath):
|
|
MARKER = "// CrossPoint patch: skip AC tables for progressive JPEG"
|
|
with open(filepath, "r") as f:
|
|
content = f.read()
|
|
|
|
if MARKER in content:
|
|
return # already patched
|
|
|
|
OLD = """\
|
|
}
|
|
// now do AC components (up to 4 tables of 16-bit codes)"""
|
|
|
|
NEW = """\
|
|
}
|
|
""" + MARKER + """
|
|
// Progressive JPEG: only DC coefficients are decoded (first scan), so AC
|
|
// Huffman tables are not needed. Skip building them to avoid failing on
|
|
// 11+-bit AC codes that the optimized table builder cannot handle.
|
|
if (pJPEG->ucMode == 0xc2)
|
|
return 1;
|
|
// now do AC components (up to 4 tables of 16-bit codes)"""
|
|
|
|
if OLD not in content:
|
|
print("WARNING: JPEGDEC AC table patch target not found in %s — library may have been updated" % filepath)
|
|
return
|
|
|
|
content = content.replace(OLD, NEW, 1)
|
|
with open(filepath, "w") as f:
|
|
f.write(content)
|
|
print("Patched JPEGDEC: skip AC tables for progressive JPEG: %s" % filepath)
|
|
|
|
def _apply_mcu_skip_patch(filepath):
|
|
MARKER = "// CrossPoint patch: guard pMCU write for MCU_SKIP"
|
|
with open(filepath, "r") as f:
|
|
content = f.read()
|
|
|
|
if MARKER in content:
|
|
return # already patched
|
|
|
|
# Patch 1: Guard the unconditional pMCU[0] write in JPEGDecodeMCU_P.
|
|
# This is the DC coefficient store that crashes when iMCU = MCU_SKIP (-8).
|
|
OLD_DC = """\
|
|
pMCU[0] = (short)*iDCPredictor; // store in MCU[0]
|
|
}
|
|
// Now get the other 63 AC coefficients"""
|
|
|
|
NEW_DC = """\
|
|
""" + MARKER + """
|
|
if (iMCU >= 0)
|
|
pMCU[0] = (short)*iDCPredictor; // store in MCU[0]
|
|
}
|
|
// Now get the other 63 AC coefficients"""
|
|
|
|
if OLD_DC not in content:
|
|
print("WARNING: JPEGDEC MCU_SKIP patch target not found in %s — library may have been updated" % filepath)
|
|
return
|
|
|
|
content = content.replace(OLD_DC, NEW_DC, 1)
|
|
|
|
# Patch 2: Guard the successive approximation pMCU[0] write.
|
|
# This path is taken on subsequent scans (cApproxBitsHigh != 0), which we
|
|
# don't normally hit (we only decode first scan), but guard it for safety.
|
|
OLD_SA = """\
|
|
pMCU[0] |= iPositive;
|
|
}
|
|
goto mcu_done; // that's it"""
|
|
|
|
NEW_SA = """\
|
|
if (iMCU >= 0)
|
|
pMCU[0] |= iPositive;
|
|
}
|
|
goto mcu_done; // that's it"""
|
|
|
|
if OLD_SA in content:
|
|
content = content.replace(OLD_SA, NEW_SA, 1)
|
|
|
|
with open(filepath, "w") as f:
|
|
f.write(content)
|
|
print("Patched JPEGDEC: guard pMCU writes for MCU_SKIP in JPEGDecodeMCU_P: %s" % filepath)
|
|
|
|
# Apply patches immediately when this pre: script runs, before compilation starts.
|
|
# Previously used env.AddPreAction("buildprog", ...) which deferred patching until
|
|
# the link step — after the library was already compiled from unpatched source.
|
|
patch_jpegdec(env)
|