5f5561b684
## Summary * This PR fixes decomposed diacritic handling end-to-end: - Hyphenation: normalize common Latin base+combining sequences to precomposed codepoints before Liang pattern matching, so decomposed words hyphenate correctly - Rendering: correct combining-mark placement logic so non-spacing marks are attached to the preceding base glyph in normal and rotated text rendering paths, with corresponding text-bounds consistency updates. - Hyphenation around non breaking space variants have been fixed (and extended) - Hyphenation of terms that already included of hyphens were fixed to include Liang pattern application (eg "US-Satellitensystem" was *exclusively* broken at the existing hyphen) ## Additional Context * Before <img width="800" height="480" alt="2" src="https://github.com/user-attachments/assets/b9c515c4-ab75-45cc-8b52-f4d86bce519d" /> * After <img width="480" height="800" alt="fix1" src="https://github.com/user-attachments/assets/4999f6a8-f51c-4c0a-b144-f153f77ddb57" /> <img width="800" height="480" alt="fix2" src="https://github.com/user-attachments/assets/7355126b-80c7-441f-b390-4e0897ee3fb6" /> * Note 1: the hyphenation fix is not a 100% bullet proof implementation. It adds composition of *common* base+combining sequences (e.g. O + U+0308 -> Ö) during codepoint collection. A complete solution would require implementing proper Unicode normalization (at least NFC, possibly NFKC in specific cases) before hyphenation and rendering, instead of hand-mapping a few combining marks. That was beyond the scope of this fix. * Note 2: the render fix should be universal and not limited to the constraints outlined above: it properly x-centers the compund glyph over the previous one, and it uses at least 1pt of visual distance in y. Before: <img width="478" height="167" alt="Image" src="https://github.com/user-attachments/assets/f8db60d5-35b1-4477-96d0-5003b4e4a2a1" /> After: <img width="479" height="180" alt="Image" src="https://github.com/user-attachments/assets/1b48ef97-3a77-475a-8522-23f4aca8e904" /> * This should resolve the issues described in #998 --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**PARTIALLY**_
20 lines
858 B
C++
20 lines
858 B
C++
#pragma once
|
|
|
|
#include <cstdint>
|
|
#include <string>
|
|
#define REPLACEMENT_GLYPH 0xFFFD
|
|
|
|
uint32_t utf8NextCodepoint(const unsigned char** string);
|
|
// Remove the last UTF-8 codepoint from a std::string and return the new size.
|
|
size_t utf8RemoveLastChar(std::string& str);
|
|
// Truncate string by removing N UTF-8 codepoints from the end.
|
|
void utf8TruncateChars(std::string& str, size_t numChars);
|
|
|
|
// Returns true for Unicode combining diacritical marks that should not advance the cursor.
|
|
inline bool utf8IsCombiningMark(const uint32_t cp) {
|
|
return (cp >= 0x0300 && cp <= 0x036F) // Combining Diacritical Marks
|
|
|| (cp >= 0x1DC0 && cp <= 0x1DFF) // Combining Diacritical Marks Supplement
|
|
|| (cp >= 0x20D0 && cp <= 0x20FF) // Combining Diacritical Marks for Symbols
|
|
|| (cp >= 0xFE20 && cp <= 0xFE2F); // Combining Half Marks
|
|
}
|