## Summary
* **What is the goal of this PR?** Address expected hyphenation issue
from
https://github.com/crosspoint-reader/crosspoint-reader/issues/998#issuecomment-3940533510
* Closes#998
* **What changes are included?** Add `„` (U+201E, _Double Low-9
Quotation Mark_), `‚` (U+201A, _Single Low-9 Quotation Mark_) and `‹`
(U+2039, _Single Left-pointing Angle Quotation Mark_) exceptions, other
quote types were handled correctly.
**Before**
<img width="480" height="155" alt="hyph3"
src="https://github.com/user-attachments/assets/e06b4071-2c8c-4814-965d-96fbe302a450"
/>
**After**
<img width="480" height="154" alt="hyph3-fix"
src="https://github.com/user-attachments/assets/4f7f5406-e200-451c-8bee-3f410cc84bbe"
/>
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* This PR fixes decomposed diacritic handling end-to-end:
- Hyphenation: normalize common Latin base+combining sequences to
precomposed codepoints before Liang pattern matching, so decomposed
words hyphenate correctly
- Rendering: correct combining-mark placement logic so non-spacing marks
are attached to the preceding base glyph in normal and rotated text
rendering paths, with corresponding text-bounds consistency updates.
- Hyphenation around non breaking space variants have been fixed (and
extended)
- Hyphenation of terms that already included of hyphens were fixed to
include Liang pattern application (eg "US-Satellitensystem" was
*exclusively* broken at the existing hyphen)
## Additional Context
* Before
<img width="800" height="480" alt="2"
src="https://github.com/user-attachments/assets/b9c515c4-ab75-45cc-8b52-f4d86bce519d"
/>
* After
<img width="480" height="800" alt="fix1"
src="https://github.com/user-attachments/assets/4999f6a8-f51c-4c0a-b144-f153f77ddb57"
/>
<img width="800" height="480" alt="fix2"
src="https://github.com/user-attachments/assets/7355126b-80c7-441f-b390-4e0897ee3fb6"
/>
* Note 1: the hyphenation fix is not a 100% bullet proof implementation.
It adds composition of *common* base+combining sequences (e.g. O +
U+0308 -> Ö) during codepoint collection. A complete solution would
require implementing proper Unicode normalization (at least NFC,
possibly NFKC in specific cases) before hyphenation and rendering,
instead of hand-mapping a few combining marks. That was beyond the scope
of this fix.
* Note 2: the render fix should be universal and not limited to the
constraints outlined above: it properly x-centers the compund glyph over
the previous one, and it uses at least 1pt of visual distance in y.
Before:
<img width="478" height="167" alt="Image"
src="https://github.com/user-attachments/assets/f8db60d5-35b1-4477-96d0-5003b4e4a2a1"
/>
After:
<img width="479" height="180" alt="Image"
src="https://github.com/user-attachments/assets/1b48ef97-3a77-475a-8522-23f4aca8e904"
/>
* This should resolve the issues described in #998
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
* Add additional punctuation marks to the list of characters that can be
immediately followed by a line break even where there is no explicit
space
## Additional Context
* Huge appreciation to @osteotek for his amazing work on hyphenation.
Reading on the device is so much better now.
* I am getting bad line breaks when ellipses (…) are between words and
book file does not explicitly include some kind of breaking space.
* Per
[discussion](https://github.com/crosspoint-reader/crosspoint-reader/pull/305#issuecomment-3765411406),
several new characters are added in this PR to the `isExplicitHyphen`
list to allow line breaks immediately after them:
Character | Unicode | Usage | Why include it?
-- | -- | -- | --
Solidus (Slash) | U+002F | / | Essential for breaking URLs and "and/or"
constructs.
Backslash | U+005C | \ | Critical for technical text, file paths, and
coding documentation.
Underscore | U+005F | _ | Prevents "runaway" line lengths in usernames
or code snippets.
Middle Dot | U+00B7 | · | Acts as a semantic separator in dictionaries
or stylistic lists.
Ellipsis | U+2026 | … | Prevents justification failure when dialogue
lacks following spaces.
Midline Horizontal Ellipsis | U+22EF | ⋯ | Useful for mathematical
sequences and technical notation.
### Example:
This shows an example of what line breaking looks like *with* this PR.
Note the line break after "matter…" (which would not previously have
been allowed). It's particularly important here because the book
includes non-breaking spaces in "Mr. Aldrich" and "Mr. Rockefeller."
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **PARTIALLY**
## Summary
* Adds (optional) Hyphenation for English, French, German, Russian
languages
## Additional Context
* Included hyphenation dictionaries add approximately 280kb to the flash
usage (German alone takes 200kb)
* Trie encoded dictionaries are adopted from hypher project
(https://github.com/typst/hypher)
* Soft hyphens (and other explicit hyphens) take precedence over
dict-based hyphenation. Overall, the hyphenation rules are quite
aggressive, as I believe it makes more sense on our smaller screen.
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
