How to package Python bindings for Tree-sitter parsers?

[Peter Oliver]

TL;DR: Should Rust bindings for Tree-sitter parsers be packaged independently (using the usual Python packaging process), or should they be a subpackage of the main Tree-sitter parser package (effectively giving us Python bindings for free)?  Do we even want Python bindings for Tree-sitter parsers in the first place?

Here’s the long version.  Tree-sitter is a library for writing parsers for source code, for use in, for example, syntax highlighters and text editors. Nearly 500 different parsers are available separately (https://github.com/tree-sitter/tree-sitter/wiki/List-of-parsers).  However, we only have one of them packaged for Fedora so far.

Spec files for Tree-sitter parsers will all be very similar to each other, and could probably be generated by simple-minded RPM macros (a draft of which are at https://src.fedoraproject.org/rpms/tree-sitter/pull-request/1#request_diff).

Official bindings are available for using the parsers from a number of languages, including Python.  The Tree-sitter project has tooling to automatically generate these bindings, which are committed to the Git repository of each parser.  On release, the bindings are uploaded to the native repository for each language (i.e., pypi.org for Python).

This gives us a choice:

- We could treat Python bindings for Tree-sitter as any other Python project, with the souce fetched from pypi.org.  If someone wanted them, they would need to package them in the usual Python way.

- We could package Python bindings for Tree-sitter as a subpackage of the package that builds the parser as a C library, with the source fetched from github.com.  We would have Python bindings for every Tree-sitter parser packaged for Fedora.  The Tree-sitter-specific RPM macros used in these packages may need tweaking in future to keep up with any changes to the Python packaging guidelines.

Which sounds preferable?

By the way, I have asked this question on the packaging list (https://lists.fedoraproject.org/archives/list/[e-mail] .org/thread/X3B2MYPPSWBGWRFD7K62NNLIMIL3O56F/), but it didn’t generate much discussion.  I’ve also asked the equivalent question to the Rust SIG.

[Miro Hrončok]

On 26. 11. 24 14:52, Peter Oliver via python-devel wrote:
> TL;DR: Should Rust bindings for Tree-sitter parsers be packaged independently 
> (using the usual Python packaging process), or should they be a subpackage of 
> the main Tree-sitter parser package (effectively giving us Python bindings for 
> free)?  Do we even want Python bindings for Tree-sitter parsers in the first 
> place?
>
> Here’s the long version.  Tree-sitter is a library for writing parsers for 
> source code, for use in, for example, syntax highlighters and text editors. 
> Nearly 500 different parsers are available separately (https://github.com/tree- 
> sitter/tree-sitter/wiki/List-of-parsers).  However, we only have one of them 
> packaged for Fedora so far.
>
> Spec files for Tree-sitter parsers will all be very similar to each other, and 
> could probably be generated by simple-minded RPM macros (a draft of which are 
> at https://src.fedoraproject.org/rpms/tree-sitter/pull-request/1#request_diff).
>
> Official bindings are available for using the parsers from a number of 
> languages, including Python.  The Tree-sitter project has tooling to 
> automatically generate these bindings, which are committed to the Git 
> repository of each parser.  On release, the bindings are uploaded to the native 
> repository for each language (i.e., pypi.org for Python).
>
> This gives us a choice:
> We could treat Python bindings for Tree-sitter as any other Python project,
>
> with the souce fetched from pypi.org.  If someone wanted them, they would need 
> to package them in the usual Python way.
> We could package Python bindings for Tree-sitter as a subpackage of the
>
> package that builds the parser as a C library, with the source fetched from 
> github.com.  We would have Python bindings for every Tree-sitter parser 
> packaged for Fedora.  The Tree-sitter-specific RPM macros used in these 
> packages may need tweaking in future to keep up with any changes to the Python 
> packaging guidelines.

Or we could not package them at all unless something we care about depends on them.