Incompatibility with `tree-sitter-markdown==0.5.0`

[GaetanLepage]

textual version: 3.4.0.

When updating tree-sitter-markdown to its latest 0.5.0 version, the following textual tests fail:

=========================== short test summary info ============================
FAILED tests/text_area/test_languages.py::test_setting_builtin_language_via_attribute - ValueError: Incompatible Language version 15. Must be between 13 and 14
FAILED tests/text_area/test_languages.py::test_setting_builtin_language_via_constructor - ValueError: Incompatible Language version 15. Must be between 13 and 14
======= 2 failed, 2849 passed, 1 skipped, 4 xfailed, 1 warning in 16.57s =======

Is this something that textual should adapt too? Is it safe to skip those tests?

Context: bumping tree-sitter-markdown in nixpkgs.
NixOS/nixpkgs#411181

[github-actions]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[TomJGooding]

I don't think the Python bindings for tree-sitter even supports ABI 15 yet. The latest tree-sitter-markdown on PyPi is v0.3.2.

[dtomvan]

They have merged this PR to their branch. After bumping it up to there (so that it works with tree-sitter 0.15) I get a couple of test failures, all related to this line:

=========================== short test summary info ============================
FAILED tests/test_widget.py::test_widget_construct - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_languages.py::test_setting_language_to_none - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_languages.py::test_update_highlight_query - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_languages.py::test_register_language - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_languages.py::test_setting_unknown_language - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_setting_themes.py::test_setting_unknown_theme_raises_exception - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_setting_themes.py::test_default_theme - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_setting_themes.py::test_setting_builtin_themes - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
FAILED tests/text_area/test_setting_themes.py::test_registering_and_setting_theme - AttributeError: 'tree_sitter.Query' object has no attribute 'captures'
= 9 failed, 2840 passed, 1 skipped, 4 xfailed, 10 warnings in 62.22s (0:01:02) =

I guess that file needs to be refactored to work with QueryCursor's instead.

[TomJGooding]

It looks there's definitely some major changes in ABI 15 which will need addressing.

However, the tree-sitter package on PyPI hasn't had a release since Jan 2025, so the whole Python ecosystem for tree-sitter is lagging behind. For example, the latest version of tree-sitter-markdown on PyPi is v0.3.2, so several versions behind v0.5.0 referenced in this issue.

I assume NixOS have taken it upon themselves to create their own versions of these tree-sitter packages?

[gepbird]

I assume NixOS have taken it upon themselves to create their own versions of these tree-sitter packages?

We are fetching tree-sitter-markdown and py-tree-sitter repos from GitHub due to them being outdated on PyPI

[TomJGooding]

So you've broken all Python packages that depend on tree-sitter in NixOS? Unless they also fetch from GitHub rather than PyPI releases? Sorry but I really don't understand how this beneits anyone.

[gepbird]

Good point, probably we should roll it back if it doesn't break other packages. Tracking this at upstream: tree-sitter-grammars/tree-sitter-markdown#193

[dtomvan]

We've done this because otherwise they wouldn't work with the tree-sitter package in NixOS. If we didn't bump the py-tree-sitter to the git version, every package using it would inherently be broken because it fails to build against TS 0.15 (ValueError: Incompatible Language version 15. Must be between 13 and 14). At least this way only packages which suffer from the breaking changes fail to build, which probably is a small subset (note that those failing builds are largely unrelated to the change at hand, they've been broken for longer)

[dtomvan]

Related discussion: tree-sitter/py-tree-sitter#368

[TomJGooding]

Ugh, I think I see the problem. py-tree-sitter apparently uses the tree-sitter-rust package in some of its tests. The problem is that the latest release on PyPI isn't actually compatible, which I suspect is why you're seeing this "Incompatible Language version 15" failure in your build. Related issue here: tree-sitter/tree-sitter-rust#273

[leona-ya]

py-tree-sitter 0.25.0 was released btw

[TomJGooding]

py-tree-sitter 0.25.0 was released btw

Not yet according to PyPI? https://pypi.org/project/tree-sitter/#history