Repository navigation
Conversation
|
I do expect this to be valuable, but we haven't had the bandwidth to review this yet. |
a2818f7 to
c8e0497
Compare
…er#811) LLM Contributions: Unknown for linebender#620, c-emoji. I used them for review, bootstrapping the new crate to follow https://linebender.org/wiki/formatting-scheme/, and getting/implementing the technique of sticking the properties into our existing property infrastructure, plus rebasing/reordering the series. Adaptation of linebender#620 by @fundon. This uses a state machine based on @chansen's [c-emoji](https://github.com/chansen/c-emoji) to implement detection of Emoji sequences, as described in [tr-51](https://www.unicode.org/reports/tr51/tr51-29.html). This is Unicode 17 Emoji, as opposed to Unicode 18 (releasing tomorrow...), but I don't think the difference has a huge impact for us. The state machine lives in a new crate `parley-emoji`, so as to follow c-emoji's license. We were unable to get its code relicensed under our MIT/Apache 2.0 license. With this, we are *not* completely compliant with Unicode Emoji, but this gives us a significantly better jumping off point. Things I've chosen to defer: - Better testing. I have local changes which generate a suitable subset of Noto Color Emoji, but I think end-to-end testing needs more care. - Also not yet tested using the upstream Unicode data. - Splitting a single grapheme cluster into its 'text presentation/emoji presentation' preferred sections. This can happen in three cases that I'm aware of: - Prepend character then Emoji, all within one grapheme cluster - Non-Emoji modifier base, then Fitzgerald skin tone modifier - ZWJ sequence containing VS15 text variation selectors. - Having the Emoji font in the fallback chain, even if text presentation is preferred. - Thinking through how the 'reject' case should be handled - Doing the proper rendering of malformed tag sequences, which is required by spec, but which as far as I know, no system implements. - [#font-tech > Appearance of unsupported emoji sequences](https://xi.zulipchat.com/#narrow/channel/296640-font-tech/topic/Appearance.20of.20unsupported.20emoji.20sequences/with/623471132). - Verifying that the handling of RI sequences is correct - in particular, the way that three regional indicators in a row merge into grapheme clusters can lead to jank. - Fix the editing behaviour to use this properly (see also linebender#693) - Documenting our level of compliance with the Unicode Emoji spec. - Handling legacy modifier sequences with an intervening VS16. There's a TODO in the code for this. - The 'emoji presentation locale' thing. I don't believe any of those are blocking - this is still a better state than before. Fixes linebender#744. Supersedes linebender#620, linebender#788, linebender#692. cc @kane50613 (wrt linebender#745). **Changelog** > ### Fixed > > - Parley's detection of Emoji is now more robust. --------- Co-authored-by: Fangdun Tsai <fundon@pindash.io> Co-authored-by: Tom Churchman <thomas@churchman.nl>
c8e0497 to
ce001ed
Compare
Each variation sequence now resolves independently: an exact cmap format 14 mapping short-circuits the color-table heuristic for that sequence alone, and a font matches the requested presentation only when every sequence in the cluster is satisfied. A variant mapping no longer claims coverage for repeated occurrences of the same base character. Fonts are now bucketed by presentation before coverage is compared, so a zero-coverage mismatched font no longer blocks a later matching one. The exact-mapping early stop was redundant with the completeness check below it.
|
I've rebased from the latest main branch. |
|
An initial AI-review is positive for correctness, but notes a 3-5% performance regression for non-emoji text. Suggested causes and fixes:
|
fe0cb01 to
4960f41
Compare
`CharCluster::fill` already classifies presentation selectors per character, so the flag comes for free and replaces a scan of every cluster in `select_font`.
|
The latest changes seem to have brought performance ~back to parity, although my LLM does have a note that:
|
|
I've asked opus to squeeze it to the extreme. He agrees this PR is 2% slower than main branch, and mentioned one possible architectural change:
Should I still aim for the extreme 🤔 |
LLM Contributions: Claude fact check + generates, Codex review.
Now font selection only checks for "who has the first char?". Variation selectors like
U+FE0Fare ignored, same forU+FE0E. It also happened in the downstream kane50613/takumi#1218.Per candidate font, an exact cmap format 14 match for the whole sequence wins first. Otherwise
U+FE0Fprefers fonts with color tables andU+FE0Efonts without. When no font matches the requested presentation, the base character still renders from a mismatched font instead of tofu.To check whether the font has color tables, I've mirrored ColorTableLookup from Blink.
#692 seems to be solving the same bug, but only by moving the emoji generic family to the front, which overrides user-selected fonts that have their own emoji glyphs. Checking each font's capability avoids that.Suppressed by #811
#620 is the segmentation layer above this and complementary.
Changelog
fontique::Charmap::map_variant, withMapVariantre-exported fromread-fonts.FontInfo::has_color_tables().QueryFont.has_color_tables.QueryFontstruct literals must provide the newhas_color_tablesfield.