Implement proper UTF-8 grapheme cluster support for modern terminals

This commit addresses critical UTF-8 display and input issues, particularly
with emoji variations and complex Unicode sequences that caused rendering
problems in modern terminals like Ghostty.

Key improvements:
- Enhanced utf8.c with grapheme cluster detection using utf8proc
- Fixed input field handling to preserve emoji variation selectors
- Improved text measurement and cursor positioning for complex Unicode
- Added smart paste processing for multi-codepoint characters
- Bypassed problematic TRANSLIT when not needed for UTF-8 content
- Enhanced GUI entry and readline to handle grapheme clusters properly

The implementation maintains full backward compatibility while providing
proper support for modern Unicode standards, fixing display corruption
that occurred with emoji sequences containing variation selectors.

Tested with emoji sequences like: 💕💋😘🥰💞❣️💓♥️♥️💓❣️🥰🥰😘💋💋
This commit is contained in:
kofany 2025-09-10 17:44:46 +02:00
commit 28a793d659
9 changed files with 564 additions and 81 deletions

6
.gitignore vendored
View file

@ -89,3 +89,9 @@ subprojects/*
Irssi-Dist
setup.cfg
*.egg-info
.specstory/
.cursorindexingignore
.cursorignore
.kombai/
.claude/
inst/