Bad ukar ligature in স্কুল

[simoncozens]

As reported in google/fonts#9875

While typing a common Bengali word স্কুল (School), I saw the Noto Sans Bengali font rendered 'ল' incorrectly after 'স্কু'.

The letter 'ল' (U+09B2) gets combined with the vowel sign 'ু' (u-kar, U+09C1) forms an incorrect ligature. 'ল' appears as 'ল ফলা' (lô fôla) and sits on top of the 'ু' vowel sign, making the word illegible.

To Reproduce

Set the font to Noto Sans Bengali (as Browser Font/Web font) / visit: https://fonts.google.com/noto/specimen/Noto+Sans+Bengali/tester

Type the Bengali word for "School": স + ্ + ক + ু / ু / ৃ + ল,

Observe the rendering of the final part of the word.

Expected behavior

The consonant cluster 'স্কু' followed by 'ল' should render correctly,

[simoncozens]

OK, this is super weird:

$ hb-shape master_ttf/NotoSansBengali-Regular.ttf 'স্কুল'
[s-beng.half=0+745|baphala-beng.alt4=2@-580,125+0|ka-beng.part=2@-263,98+0|headline-beng.250=2@-250,0+0|uMatra-beng=2@-295,-5+0|la-beng.blws=4@-689,-214+0]

Notice here that la-beng has taken below form la-beng.blws. I wonder how that happened, let's trace the shaping:

$ hb-shape master_ttf/NotoSansBengali-Regular.ttf 'স্কুল' -V
...
trace: end lookup 193 feature 'rclt'    buffer: [s-beng.half=0|baphala-beng.alt4=2|ka-beng.part=2|headline-beng.250=2|uMatra-beng=2|la-beng=4]
trace: end table GSUB script tag 'bng2' buffer: [s-beng.half=0|baphala-beng.alt4=2|ka-beng.part=2|headline-beng.250=2|uMatra-beng=2|la-beng=4]

Harfbuzz bug! @behdad

[simoncozens]

Bug in Harfbuzz was fixed somewhere between 11.5 and 12.1.

[simoncozens]

OK, so the bug is in BNG_pres_onTheFly_last_biconsonant. There is an explicit rule to make the la a below-base form in this case:

    sub s-beng.half baphala-beng.alt4 ka-beng.part la-beng' by la-beng.blws;

I don't know what the rule is for; it looks wrong to me because there's nothing there (halant etc.) which should make the la into a below-base form. So it should either be (a) removed, if it's wrong or (b) blocked by an ukar in the mark filtering set. @joanaranito, any thoughts?

[lianghai]

There’s:

bengali/sources/features/NotoSansBengali.fea

Line 802 in 3e68be0

sub s-beng.half k-beng.half' by baphala-beng.alt4 ka-beng.part;

But also:

bengali/sources/features/NotoSansBengali.fea

Line 859 in 3e68be0

sub [b-beng.half2 l-beng.half3 ss-beng.half4 s-beng.half n-beng.headline] ka-beng' by baphala-beng.alt4 ka-beng.part;

Both under “pres”. The former is probably what caused the assumption that baphala-beng.alt4 ka-beng.part is an alternative of k-beng.half.

Such purely graphic manipulation appears rather fragile, but what confuses me the most is why such a substitution can happen across the syllable boundary in “pres”. I see the same result with Core Text:

% hb-shape --shapers=coretext NotoSansBengali-v3.000/NotoSansBengali/googlefonts/variable-ttf/NotoSansBengali\[wdth\,wght\].ttf "স্কল"
[uni09B809CD.half=0+165|baphalabeng.alt4=2@0,125+317|uni0995.part=2@0,98+13|headlinebeng.250=2+-439|uni09B2.blws=3@0,-214+689]

[joanaranito]

Hi Simon, you're right. It is odd that the pres substitution is happening across syllable boundary, but I'll have a look and get back to you in a few minutes.

[joanaranito]

@simoncozens this is the order of substitutions at the moment:

sa-beng halant-beng ka-beng halant-beng la-beng = s-beng.half k-beng.half la-beng

s-beng.half k-beng.half la-beng = s-beng.half baphala-beng ka-beng.part la-beng

s-beng.half baphala-beng ka-beng.part la-beng = s-beng.half baphala-beng ka-beng.part la-beng.blws

--

The issue is that স্ক uses the same sa and ka parts as স্ক্ল, and because the syllable boundaries are not being respected, the la in স্কল is also being replaced by la-beng.blws and attaching to the conjunct.

If we change the order of substitution to replace the la-beng.blws before we change the half-form, and then replace the half-form only in context of the la-beng.blws, the issue gets fixed:

sa-beng halant-beng ka-beng halant-beng la-beng = s-beng.half k-beng.half la-beng

s-beng.half k-beng.half la-beng = s-beng.half k-beng.half la-beng.blws

s-beng.half k-beng.half la-beng.blws = s-beng.half baphala-beng ka-beng.part la-beng.blws

--

I can do a review of all similar cases, and then open a pull request with the fix.

EDIT: fixed markdown issues

[behdad]

So, HarfBuzz bug or not?

[behdad]

It is odd that the pres substitution is happening across syllable boundary

It's intentionally done that way in HarfBuzz. pref et al are done withing syllable, but pres et al across syllables.

[joanaranito]

@behdad Then it's a font bug, not HarfBuzz :)

I've fixed it, just doing final reviews before opening a PR

[joanaranito]

@behdad actually, one question:

To fix another issue (some clashes between uMatra and the following syllable/cluster, related to issue #30 ), I contextually substituted the uMatra by a deeper version of uMatra in rclt. I was using rclt and not blws, because I thought blws only worked within syllable, and so the context beyond the syllable would be ignored.

But now that you mentioned pres et al work across syllables, I tried moving the lookup to blws (since uMatra is a below-base mark, and all other uMatra substitutions are in blws), and the substitution stopped working. Do pres et all work for substitutions across syllables, but not for context across syllables?

EDIT: Just to reiterate that in any case, the particular issue @simoncozens found really is a font bug, my question above is out of curiosity :)

[lianghai]

@behdad,

So, HarfBuzz bug or not?

It is odd that the pres substitution is happening across syllable boundary

It's intentionally done that way in HarfBuzz. pref et al are done withing syllable, but pres et al across syllables.

This implementation of the “pres et al” (mandatory presentation forms) features doesn’t feel right. Has HarfBuzz always been behaving in this way or is this new?

See Microsoft’s spec, even though it’s known to be full of mistakes:

Transformations discussed in this document do not cross syllable boundaries.

See also n8willis/opentype-shaping-documents:

After the syllables have been identified, each of the subsequent shaping stages occurs on a per-syllable basis.

A preliminary test of NotoSansBengali-v3.000/NotoSansBengali/googlefonts/variable-ttf/NotoSansBengali[wdth,wght].ttf with the string “স্কল”, shaped by the native engines of Windows 11 and macOS 15:

[behdad]

Lemme check the code and get back to you.

[behdad]

Lemme check the code and get back to you.

I was wrong. pres et al are also applied per-syllable. I apologize for the confusion.