Rust bindings for Python (pyo3 versions <0.19, cpython) broken with Python 3.12
by Fabio Valentini
Hello Pythonistas and Rustaceans,
TL;DR: Only PyO3 v0.19.2 (and later) will ever properly support Python
3.12. Port your Python projects to v0.19 **NOW**.
Older versions of PyO3 (especially pyo3 v0.15, v0.16, v0.17, and
v0.18) are *not* compatible with Python 3.12 due to some ABI changes
in unicode strings and behavioural changes wrt/ "immortal" objects.
This also affects all current versions of the "cpython" Rust bindings,
with no timeline for Python 3.12 support.
As far as I can tell, extensions that use pyo3 < v0.19 or the
"cpython" bindings can (and likely will) not work as expected on
Python 3.12 if they use the affected APIs (either by producing garbage
data in strings that are passed over the FFI boundary, or by
crashing).
There are applications in Fedora that still rely on *ancient* versions
of PyO3, potentially affected by this:
- cpython: mercurial
- pyo3 v0.15: fapolicy-analyzer, python-bcrypt, python-cryptography
- pyo3 v0.16: python-y-py
- pyo3 v0.17: unused compat packages, will be retired
- pyo3 v0.18: matrix-synapse
I *stongly* recommend to move all of these packages to pyo3 v0.19 in
Rawhide as soon as possible. I will try to submit pull requests with
the required changes for affected packages (except mercurial, since
there's no version of the "cpython" crate that supports Python 3.12 in
sight).
There's already a few packages that depend on pyo3 v0.19, which I will
rebuild in rawhide for pyo3 v0.19.2, which has much better support for
Python 3.12 than v0.19.0 and v0.19.1 (breezy, python-rpds-py, orjson)
unless there are any objections.
As soon as no packages depend on the compat packages for old versions
of pyo3 any longer, I will retire them from Rawhide (and F39,
depending on the timing), since they will never work with Python 3.12
and nothing should use them.
I've added <package>-maintainers(a)fedoraproject.org for all these
packages to the CC of this message.
Fabio
Rust SIG / PyO3 maintainer in Fedora
5 months, 2 weeks
Upcoming bindgen / bindgen-cli update from v0.63 to v0.68
by Fabio Valentini
Hi all,
I've prepared an update for the "bindgen" Rust crate and the
associated CLI interface, bindgen-cli, from version 0.63.0 to the
latest version, 0.68.1. New releases of bindgen releases are, for most
purposes, backwards compatible.
Due to the large number of dependent packages (list included below),
patching all of them across all branches of Fedora and in EPEL 9 is
not feasible, which is why I am going to include a compat package for
the current bindgen version in the update.
Because of how dependency resolution for Rust crates works, no changes
to use the compat package are necessary - i.e. packages currently
depend on something like (bindgen >= 0.63 with bindgen < 0.64), so
they will just pull in the compat package instead of the latest
version transparently.
This way, dependent packages can be upgraded to the latest bindgen
version without needing to patch / port all of them immediately.
However, packages definitely *SHOULD* migrate to the new version
quickly, since mixing and matching different versions can lead to
confusing errors (caused by dlopening libclang.so multiple times), and
should be avoided. I will port Rust SIG packages to the new version
ASAP, and will send PRs for other packages as time permits.
Fabio
===
Packages currently depending on bindgen v0.63:
- clamav
- fapolicy-analyzer
- rust-aom-sys
- rust-devicemapper-sys
- rust-drm-fourcc
- rust-input-sys
- rust-libblkid-rs-sys
- rust-libbpf-sys
- rust-libcryptsetup-rs-sys
- rust-libheif-sys
- rust-libspa-sys
- rust-libsqlite3-sys
- rust-loopdev
- rust-nettle-sys
- rust-onig_sys
- rust-openssl-sys
- rust-pipewire-sys
- rust-psa-crypto-sys
- rust-tss-esapi-sys
- rust-userfaultfd-sys
- rust-zstd-sys
- stratisd
There's also still some packages currently depending on bindgen v0.59:
- rust-cryptoki-sys
- rust-drm-sys
- rust-tree-sitter
These should be migrated to a newer version of bindgen ASAP. I'll try
to open PRs for them.
7 months, 1 week