`Sentence case` doesn't work when it begins by non-letter characters

[Thomas-D-C]

Bug description

Sentence case doesn't work when it begins by non-letter characters (spaces or punctuation).

Steps to reproduce

1. Type ant select (text between parenthesis).
2. Convert it to Sentence case. You can't do it, since it's identical.
3. Select text between parenthesis, inside parenthesis.
4. Convert it to Sentence case.
5. You can see that it works.

Actual behavior

(text between parenthesis) makes (text between parenthesis) (Lowercase).

Expected behavior

(text between parenthesis) would make (Text between parenthesis) (Sentence case inside parenthesis).

Reproducibility on test website

• Issue is reproducible on https://rugk.github.io/unicodify/ for text input fields.
• Issue is reproducible on https://rugk.github.io/unicodify/ for textarea fields.
• Issue is reproducible on https://rugk.github.io/unicodify/ for ContentEditable Divs.
• Issue is not reproducible on https://rugk.github.io/unicodify/

System

Browser and version: Firefox 150.0.2
Add-on version: 0.8

Possible solution

I don't remember where to find the API.
But if I'm right: It decomposes (text between parenthesis) in: (, text, , between, , parenthesis, ). Then, the Add-on chooses to apply Capitalize Each Word only to the first item (and Lowercase to the others).

So, words always are 1 item out of 2, never 2 consecutive items.
So, you could apply Capitalize Each Word to the first 2 items, to fix this bug, without taking the risk of capitalizing 2 words.

[tdulcet]

The API is Intl.Segmenter, which automatically splits the input into sentences, words or graphemes. In your example for sentence case, it considers the entire input as a single sentence (at least for the en locale):

The extension then takes the first character of each sentence and capitalizes it.

However, we could further split each sentence into words (as "Capitalize Each Word" does) and then capitalize just the first word:

This fixes your example (at least when using the en locale), so hopefully it would not break any expected behavior. I will make this change in the next version.

[rugk]

This fixes your example, so hopefully it would not break any expected behavior

It does not, strictly speaking, because ( would still be the first word though? The discussion here would be whether that is actually a sentence "beginning". But yeah I guess I would be fine with both. Maybe implement the "word-based" algorithm as a fallback when the "sentence-based" one does not change anything? This way, we prevent "breaking existing behavior"?

[tdulcet]

It does not, strictly speaking, because ( would still be the first word though?

The word splitter has that additional isWordLike property for each segment, which at least for the en locale is false for (, but then true for text (see screenshot above). My proposed implementation would iterate the segments and then capitalize the first word where isWordLike is true.

The segmenter API is somewhat of a black box, so it is unclear how it might behave with other inputs or different locales, which is what I meant by the change potentially breaking existing behavior. However, it does seem like an improvement.

[Thomas-D-C]

Maybe implement the "word-based" algorithm as a fallback when the "sentence-based" one does not change anything?

Warning: It happens when the sentence is already in Sentence case, and it does not when the sentence is not in Lowercase.
The only good solution is to make the desired result directly. :-)

Thank you for doing that. :-) :-)
Is this a step towards #100?

[rugk]

Huh complex topic. But yeah there must be a spec somewhere for the segmented api to explain this behavior or so.