fix(fts): split ICU tokens on punctuation

[Xuanwo]

ICU became the default FTS tokenizer, but it can keep punctuation inside word segments where the simple tokenizer would split on every non-alphanumeric character. This makes ICU preserve the multilingual word segmentation behavior while applying the same intra-segment delimiter rule as the simple tokenizer.

This restores expected matching for terms such as foo_bar and also aligns punctuation cases like apostrophes, hyphens, and dots with the simple tokenizer.

[codecov]

Codecov Report

❌ Patch coverage is 97.87234% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-tokenizer/src/icu.rs	97.87%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!