= Proposed Self Contained Change: Packaging Rust applications/libraries = https://fedoraproject.org/wiki/Changes/Packaging_Rust_applications_and_libra...
Change owner(s): * Igor Gnatenko <ignatenkobrain@fedoraproject.org > (on behalf of Rust SIG)
Add required tools/instructions for packaging applications/libraries written in Rust. Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.
== Detailed Description ==
During initial research of SIG about packaging we identified that inability to specify version range dependencies (1.0 <= foo < 2.0) in RPM is main blocker. This problem hits almost every other language ecosystem (esp. NodeJS), but it is not very noticable due to having not more than 2 versions. While packaging some applications we discovered need of having 3 or more versions of same crate.
The most of the work already has been done and users can consume applications without needing to do anything from Rust/Playground COPR repository https://copr.fedorainfracloud.org/coprs/g/rust/playground/ .
== Scope ==
* Proposal owners: Create tool for automatic creation of rpm-spec-file from crate on crates.io, create RPM macro for easy packaging, write packaging guidelines.
* Other developers: RPM developers to add support for expressing version range dependencies.
* Release engineering: https://pagure.io/releng/issue/6889
* List of deliverables: N/A (not a System Wide Change)
* Policies and guidelines: Packaging Guidelines needs to be written for packaging Rust applications/libraries.
* Trademark approval: N/A (not needed for this Change)
I'd like to see this change land in F28!
Anyway, I think there is a showstopper: * Fedora Packaging guidelines demand packages bundle no dependencies (e.g. libraries, or in Rust's case crates). This is a good choice in my opinion. * Rust currently has no ABI definition compatible over different versions of the rust compiler. As a result, every time a new compiler is released it will also require *ALL* rust packages to be rebuild. See https://github.com/rust-lang/rfcs/issues/600 for details and discussion. * To get a sane update strategy, it is critical that not too much packages are involved in rebuilds. Consider a CVE in openssl: Luckily we can rebuild openssl (plus its subpackages) and ship it. No need to rebuild every application that uses openssl or uses another package which uses openssl.
As far as I understand, these 3 points are conflicting right now. Either Rust gets an ABI or we need to give up one of the other two principles, which I highly recommend not to do.
Disclaimer: I'm no Rust expert, nor am I an ABI expert, so I might be completely wrong about this.
On 08/21/2017 04:20 PM, Christian Stadelmann wrote:
I'd like to see this change land in F28!
Anyway, I think there is a showstopper:
- Fedora Packaging guidelines demand packages bundle no dependencies (e.g. libraries, or in Rust's case crates). This is a good choice in my opinion.
- Rust currently has no ABI definition compatible over different versions of the rust compiler. As a result, every time a new compiler is released it will also require *ALL* rust packages to be rebuild. See https://github.com/rust-lang/rfcs/issues/600 for details and discussion.
- To get a sane update strategy, it is critical that not too much packages are involved in rebuilds. Consider a CVE in openssl: Luckily we can rebuild openssl (plus its subpackages) and ship it. No need to rebuild every application that uses openssl or uses another package which uses openssl.
As far as I understand, these 3 points are conflicting right now. Either Rust gets an ABI or we need to give up one of the other two principles, which I highly recommend not to do.
I think ABI compatibility is achieved by shipping source code only in the -devel packages (similar to what Perl does).
The rebuild issue is still there, but it's no different from certain Java packaging practices, and what C++ (with template metaprogramming), Ocaml, Go, and so on currently do.
Thanks, Florian
On 08/21/2017 04:20 PM, Christian Stadelmann wrote:
I think ABI compatibility is achieved by shipping source code only in the -devel packages (similar to what Perl does).
No, you are writing about API stability. Source code may provide API stability, but no ABI stability.
ABI stability is about making sure that any public (exported) symbol is generated the same way on every compilation with any compiler version. This affects e.g. the order of fields in structs or the way how parameters are laid out in RAM and registers before a function call happens. This cannot be done by source code but requires compiler work.
With perl (and other interpreters such as python), API stability (usually) induces ABI stability because the interpreter of any perl package is the same, so it only needs internal ABI stability.
Java is compiled to some intermediate code with the interpreter providing backwards compatibiliy for previous versions.
In C++, ABI breaks only happen when the compiler gets updated, which only happens on rawhide or pre-alpha testing phase and is followed by a rebuild. Still you may ship any update to any C++ package without breaking ABI or the need to recompile all dependend packages. Boost is an exception here.
I have no idea about Ocaml and Go.
Please note: In Rust, fixing any bug in the standard library will require _every_ rust package to be recompiled. This is quite different from C++ or Java or perl.
On Mon, Aug 21, 2017 at 12:23 PM, Christian Stadelmann genodeftest@fedoraproject.org wrote:
On 08/21/2017 04:20 PM, Christian Stadelmann wrote:
I think ABI compatibility is achieved by shipping source code only in the -devel packages (similar to what Perl does).
No, you are writing about API stability. Source code may provide API stability, but no ABI stability.
ABI stability is about making sure that any public (exported) symbol is generated the same way on every compilation with any compiler version. This affects e.g. the order of fields in structs or the way how parameters are laid out in RAM and registers before a function call happens. This cannot be done by source code but requires compiler work.
With perl (and other interpreters such as python), API stability (usually) induces ABI stability because the interpreter of any perl package is the same, so it only needs internal ABI stability.
Java is compiled to some intermediate code with the interpreter providing backwards compatibiliy for previous versions.
In C++, ABI breaks only happen when the compiler gets updated, which only happens on rawhide or pre-alpha testing phase and is followed by a rebuild. Still you may ship any update to any C++ package without breaking ABI or the need to recompile all dependend packages. Boost is an exception here.
I have no idea about Ocaml and Go.
The vast majority of rust packages will be piles of source code, just like Go. This reduces the scope of rebuilding for rust core changes to only applications, which right now is no worse than what Go does.
Eventually, when we get a stabilized ABI from Rust, we'll transition to dynamic link libraries and handle it like we do for most ecosystems that are compiled.
On Mon, Aug 21, 2017 at 2:25 PM, Christian Stadelmann genodeftest@fedoraproject.org wrote:
So for now, there will be no shared rust libraries?
No pure-Rust ones that don't export a C library interface (those are stable), yes.
On 08/21/2017 07:20 AM, Christian Stadelmann wrote:
I'd like to see this change land in F28!
Anyway, I think there is a showstopper:
- Fedora Packaging guidelines demand packages bundle no dependencies (e.g. libraries, or in Rust's case crates). This is a good choice in my opinion.
- Rust currently has no ABI definition compatible over different versions of the rust compiler. As a result, every time a new compiler is released it will also require *ALL* rust packages to be rebuild. See https://github.com/rust-lang/rfcs/issues/600 for details and discussion.
- To get a sane update strategy, it is critical that not too much packages are involved in rebuilds. Consider a CVE in openssl: Luckily we can rebuild openssl (plus its subpackages) and ship it. No need to rebuild every application that uses openssl or uses another package which uses openssl.
As far as I understand, these 3 points are conflicting right now. Either Rust gets an ABI or we need to give up one of the other two principles, which I highly recommend not to do.
Disclaimer: I'm no Rust expert, nor am I an ABI expert, so I might be completely wrong about this.
These package guidelines are exactly meant to solve the bundling question, so we can package crates separately. So it's more of a problem of static or dynamic linking.
Since there's no Rust ABI, we can't support shared libraries for Rust interfaces. So all crates will ship source-based devel packages, and applications statically link these.
The only Rust shared libraries we'll have at all are those from the compiler and standard library, which are explicitly filtered from Provides. Their only use for other packages is in transient build scripts and plugins, which don't get installed.
For FFI to libraries like openssl, it's still expected to use dynamic linking. And in the other direction, if a Rust library provides a C ABI (a "cdylib" crate type), this is dynamic too.
-
Christian Stadelmann -
Florian Weimer -
Jan Kurik -
Josh Stone -
Neal Gompa