Re: musings on rust packaging [was Re: F38 proposal: RPM Sequoia (System-Wide Change proposal)]
On Wed, 2022-11-30 at 18:26 +0000, Simon Farnsworth via devel wrote:
On Wednesday, 30 November 2022 17:47:16 GMT Fabio Valentini wrote:
> On Wed, Nov 30, 2022 at 6:21 PM Daniel Alley <dalley(a)redhat.com> wrote:
> > I feel like there is insufficient recognition of the extent to which C
> > libraries do "bundling". Not "bundling" in the sense of
vendoring a
> > whole library, but in the sense of including one-off implementations of
> > basic data structures, configuration parsers, hashing algorithms, etc. I
> > would love to hear anyone argue that 100 different variations of
> > "sha256.c" across 100 different packages better follows the spirit of
the
> > "no bundling" guidelines than a vendored crate named
"sha256" with 100x
> > as many eyes on it, and a higher likelihood to actually be updated if a
> > problem is found.
>
> >
> >
> > Many of the tiny, "sprawling" Rust dependencies are like this - not
all of
> > them of course, but many.
>
> But ... none of these "tiny" Rust crates are dynamically linked in
> Fedora anyway - because Rust doesn't really support that?
> So I fail to see your point there, unless you meant to say "C projects
> don't 'bundle', they just often 'copy' some code into their
projects"?
>
I think the point he's making is that developers don't write common
functionality from scratch, in general. We reuse code from elsewhere.
It's just that in C, I'll copy-and-paste code from the web into my library or
application, not necessarily even bothering with a full "vendoring", whereas
in Rust, I'll use the crc crate (say), or the base64 crate, or other simple
utility crate.
The result is that I have N implementations of common functionality, each with
its own unique quirks and security risks, in my C binaries; but my C binaries
have only a small number of dependencies.
In Rust, however, I'm directly reusing the small utility crate, and while I
may use `cargo vendor` to import the crate's source into my tree, I'm unlikely
to edit it. The result is that a Rust binary has a larger number of
dependencies than the equivalent C program, because I'm depending on a crate
instead of copy-and-pasting the code and "hiding" it from the packager.
This is a challenge for Fedora: how do we cope with a world where instead of
having a few tens of dependencies, and a lot of copy-and-paste code, we have
hundreds of dependencies, but no copy-and-paste code?
One answer is to say that Rust is bad for encouraging developers to depend on
small crates instead of copying-and-pasting "small" utilities around. Another
(which you're doing a great job of) is to package up all the dependencies, so
that we represent the true dependency tree in RPM. Yet another would be to
manually decide which dependencies get bundled, and which don't - doing the
same thing as the C world does to keep its dependency count down.
This is a bit of a misleading characterization.
- the amount of copying in C programs is overblown in my opinion (no
data provided to back this up though)
- dependency minimization for C programs/libraries is assumed but no
data provided to back it up.
- the dilemma is how to manage rust programs not to decide what to
bundle or not, that's basically decided upstream
Although there are certainly instances of copying/paste in C code, the
vast majority of reuse comes through linking to utility libraries
dynamically, which makes distro-level maintenance much easier because
you have only once place to fix/rebuild/distribute when there are
serious issues.
The problems with Rust crates (or go modules, or any other
"vendoring"), is that not only you have to go and find each place where
a problematic crate was vendored; you also have to figure out, often
under pressure, if the crate can simply be summarily updated or if you
need a backport because the vendoring application can't cope with the
semantic changes that happened in the problematic crate's new version.
Multiply this by N packages using M different versions of the
problematic crate.
Although vendored crates can be tracked (this i much better than
copy/pasting), with additional tooling, the distribution remains on the
hook for solving the same problem in N packages, without easy
coordination. Some upstream may be quick and do the work for you, some
may not care, disappear etc... or are simply too slow for the urgency
the distribution has leading potentially to diverging solutions ...
Simo.
--
Simo Sorce
RHEL Crypto Team
Red Hat, Inc