F34 Change Proposal: Rust Crate Packages For Release Branches (System-Wide Change)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...

== Summary ==

This Change proposal aims to enable shipping Rust crate packages
(<code>rust-$CRATE_NAME</code>) on release branches of fedora.
Currently, they are only available for rawhide, which makes building
Rust packages for release branches difficult.

== Owner ==

* Name: [[User:decathorpe| Fabio Valentini]]
* Email: <decathorpe(a)gmail.com&gt;


== Detailed Description ==

Following the general upwards trend in the Rust language's popularity,
more and more applications and services in fedora are written in Rust.
This includes some CoreOS services, PARSEC, some nice command line
tools (e.g. ripgrep, bat, fedora-update-feedback, ...), and parts of
the GNOME stack.
However, because rust crate packages are currently only available on
rawhide, packaging rust applications for release branches is difficult
and involves more steps than usual.

This Change proposal aims to bring Rust packaging in line with the
normal packaging workflows in fedora.

In particular, the following additional steps will become obsolete:

* use koji side tags for *every package build* on release branches
* manual tagging and untagging of koji buildroot contents

Instead, rust packages can be built like any other package in fedora.


== Benefit to Fedora ==

This Change lowers the bar for contributing to the Rust stack in
fedora, because it will no longer be a special case that involves
additional steps.

It will also make it possible to build Rust packages for release
branches locally in mock without the need for custom mock buildroot
configurations and / or third-party repositories.

== Scope ==

* Proposal owner(s):
** one-off change: submit PR to revert the special-case handling for
<code>rust-*</code> packages in the mass branching releng script
** ongoing effort: help package maintainers with merging changes from
rawhide (where appropriate) and creating compat packages (when
necessary) - this is made easier by the strong SemVer compatibility
guarantees of Rust crates

* Other developers:

Initially, there is no impact on other developers.
However, as soon as fedora 34 is branched, building Rust applications
on that release branch will be easier than without this change.
This will require packagers to merge rawhide updates into release
branches when appropriate (again, bringing Rust packaging in line with
the rest of fedora).

I also expect there to be reduced load on koji due less side tags
being in use concurrently, which will benefit all package maintainers
in fedora.

* Release engineering: [https://pagure.io/releng/issue/9753 #9753]

Release engineering will need to remove special-case handling of
<code>rust-*</code> packages from their mass branching script before
f34 is branched off of rawhide.

* Policies and guidelines:

The Rust packaging Guidelines will need small adaptations.

They are already outdated, so Change owner(s) will update them to the
current state of Rust packaging regardless.

* Trademark approval: N/A (not needed for this Change)

* Alignment with Objectives:

The fedora IoT Edition promotes PARSEC, which is comprised of Rust
packages - its package maintainers will benefit from being able to
update and build those packages faster and more easily for release
branches.

== Upgrade/compatibility impact ==

There will be no impact on upgrades from previous fedora releases,
since Rust crate packages will only be available for those users for
the first time.

If for some reason a user installed <code>rust-*-devel</code> packages
manually after downloading them from the rawhide repositories, they
will be gracefully upgraded.

== How To Test ==

Users should be able to build Rust applications for fedora 34 without
workarounds or special steps (both in mock locally and in koji - both
scratch and non-scratch builds).

== User Experience ==

Users should not notice this change. However, I expect some
application updates to be available for release branches faster, since
it will be easier for package maintainers to create them.

== Dependencies ==

N/A (this only affects the Rust package stack and has no external dependencies)

== Contingency Plan ==

* Contingency mechanism: untag and block <code>rust-*</code> packages
in the f34 tag in koji (help from releng / a koji admin required),
revert mass branching script changes before f35 branch point
* Contingency deadline: Final Freeze (removing packages from koji will
no longer be possible after this point)
* Blocks release? No (the initial Change is small and does not
negatively affect release process)
* Blocks product? N/A

== Documentation ==

The Packaging Guidelines for Rust will be updated to reflect this Change.

== Release Notes ==

TBD


-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Neal Gompa]

On Mon, Sep 14, 2020 at 10:11 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>
>  == Summary ==
>
>  This Change proposal aims to enable shipping Rust crate packages
>  (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
>  Currently, they are only available for rawhide, which makes building
>  Rust packages for release branches difficult.
>
>  == Owner ==
>
>  * Name: [[User:decathorpe| Fabio Valentini]]
>  * Email: <decathorpe(a)gmail.com&gt;
>
>
>  == Detailed Description ==
>
>  Following the general upwards trend in the Rust language's popularity,
>  more and more applications and services in fedora are written in Rust.
>  This includes some CoreOS services, PARSEC, some nice command line
>  tools (e.g. ripgrep, bat, fedora-update-feedback, ...), and parts of
>  the GNOME stack.
>  However, because rust crate packages are currently only available on
>  rawhide, packaging rust applications for release branches is difficult
>  and involves more steps than usual.
>
>  This Change proposal aims to bring Rust packaging in line with the
>  normal packaging workflows in fedora.
>
>  In particular, the following additional steps will become obsolete:
>
>  * use koji side tags for *every package build* on release branches
>  * manual tagging and untagging of koji buildroot contents
>
>  Instead, rust packages can be built like any other package in fedora.
>
>
>  == Benefit to Fedora ==
>
>  This Change lowers the bar for contributing to the Rust stack in
>  fedora, because it will no longer be a special case that involves
>  additional steps.
>
>  It will also make it possible to build Rust packages for release
>  branches locally in mock without the need for custom mock buildroot
>  configurations and / or third-party repositories.
>
>  == Scope ==
>
>  * Proposal owner(s):
>  ** one-off change: submit PR to revert the special-case handling for
>  <code>rust-*</code> packages in the mass branching releng script
>  ** ongoing effort: help package maintainers with merging changes from
>  rawhide (where appropriate) and creating compat packages (when
>  necessary) - this is made easier by the strong SemVer compatibility
>  guarantees of Rust crates
>
>  * Other developers:
>
>  Initially, there is no impact on other developers.
>  However, as soon as fedora 34 is branched, building Rust applications
>  on that release branch will be easier than without this change.
>  This will require packagers to merge rawhide updates into release
>  branches when appropriate (again, bringing Rust packaging in line with
>  the rest of fedora).
>
>  I also expect there to be reduced load on koji due less side tags
>  being in use concurrently, which will benefit all package maintainers
>  in fedora.
>
>  * Release engineering: [https://pagure.io/releng/issue/9753 #9753]
>
>  Release engineering will need to remove special-case handling of
>  <code>rust-*</code> packages from their mass branching script before
>  f34 is branched off of rawhide.
>
>  * Policies and guidelines:
>
>  The Rust packaging Guidelines will need small adaptations.
>
>  They are already outdated, so Change owner(s) will update them to the
>  current state of Rust packaging regardless.
>
>  * Trademark approval: N/A (not needed for this Change)
>
>  * Alignment with Objectives:
>
>  The fedora IoT Edition promotes PARSEC, which is comprised of Rust
>  packages - its package maintainers will benefit from being able to
>  update and build those packages faster and more easily for release
>  branches.
>
>  == Upgrade/compatibility impact ==
>
>  There will be no impact on upgrades from previous fedora releases,
>  since Rust crate packages will only be available for those users for
>  the first time.
>
>  If for some reason a user installed <code>rust-*-devel</code> packages
>  manually after downloading them from the rawhide repositories, they
>  will be gracefully upgraded.
>
>  == How To Test ==
>
>  Users should be able to build Rust applications for fedora 34 without
>  workarounds or special steps (both in mock locally and in koji - both
>  scratch and non-scratch builds).
>
>  == User Experience ==
>
>  Users should not notice this change. However, I expect some
>  application updates to be available for release branches faster, since
>  it will be easier for package maintainers to create them.
>
>  == Dependencies ==
>
>  N/A (this only affects the Rust package stack and has no external dependencies)
>
>  == Contingency Plan ==
>
>  * Contingency mechanism: untag and block <code>rust-*</code> packages
>  in the f34 tag in koji (help from releng / a koji admin required),
>  revert mass branching script changes before f35 branch point
>  * Contingency deadline: Final Freeze (removing packages from koji will
>  no longer be possible after this point)
>  * Blocks release? No (the initial Change is small and does not
>  negatively affect release process)
>  * Blocks product? N/A
>
>  == Documentation ==
>
>  The Packaging Guidelines for Rust will be updated to reflect this Change.
>
>  == Release Notes ==
>
>  TBD
>  
+100000000000000000

This will make life so much easier for getting Rust stuff packaged in Fedora.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Miro Hrončok]

On 14. 09. 20 16:10, Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>  
>  == Summary ==
>  
>  This Change proposal aims to enable shipping Rust crate packages
>  (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
>  Currently, they are only available for rawhide, which makes building
>  Rust packages for release branches difficult. 
Yes please!

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Michel Alexandre Salim]

On Mon, 2020-09-14 at 10:10 -0400, Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>  
>  * Other developers:
>  
>  Initially, there is no impact on other developers.
>  However, as soon as fedora 34 is branched, building Rust applications
>  on that release branch will be easier than without this change.
>  This will require packagers to merge rawhide updates into release
>  branches when appropriate (again, bringing Rust packaging in line
>  with
>  the rest of fedora).
>   
This, so much. Being able to make major changes in Rawhide without
worrying about losing the ability to make minor changes in released
branches if necessary (because now the specs can diverge) will be
great.

-- 
Michel Alexandre Salim
profile: https://keyoxide.org/[e-mail]
chat via email: https://delta.chat/
GPG key: 5DCE 2E7E 9C3B 1CFF D335 C1D7 8B22 9D2F 7CCC 04F2

[Zbigniew Jędrzejewski-Szmek]

On Mon, Sep 14, 2020 at 10:10:30AM -0400, Ben Cotton wrote:
> https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>  
>  == Summary ==
>  
>  This Change proposal aims to enable shipping Rust crate packages
>  (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
>  Currently, they are only available for rawhide, which makes building
>  Rust packages for release branches difficult. 
Should the update policy for rust-devel packages be relaxes in stable
releases to allow updates to follow rawhide, at least if there are
no major breaking changes?

Current update policy states that major version changes should be avoided
in after stable release. But rust upstreams tend to move quickly, so having
~1 year old packages in stable Fedora might not be too useful to build
newest upstreams. It might make sense to allow the rust-*-devel packages
to be updated more aggressively.

Zbyszek



> == Owner ==
> 
> * Name: [[User:decathorpe| Fabio Valentini]]
> * Email: <decathorpe(a)gmail.com&gt;
> 
> 
> == Detailed Description ==
> 
> Following the general upwards trend in the Rust language's popularity,
> more and more applications and services in fedora are written in Rust.
> This includes some CoreOS services, PARSEC, some nice command line
> tools (e.g. ripgrep, bat, fedora-update-feedback, ...), and parts of
> the GNOME stack.
> However, because rust crate packages are currently only available on
> rawhide, packaging rust applications for release branches is difficult
> and involves more steps than usual.
> 
> This Change proposal aims to bring Rust packaging in line with the
> normal packaging workflows in fedora.
> 
> In particular, the following additional steps will become obsolete:
> 
> * use koji side tags for *every package build* on release branches
> * manual tagging and untagging of koji buildroot contents
> 
> Instead, rust packages can be built like any other package in fedora.
> 
> 
> == Benefit to Fedora ==
> 
> This Change lowers the bar for contributing to the Rust stack in
> fedora, because it will no longer be a special case that involves
> additional steps.
> 
> It will also make it possible to build Rust packages for release
> branches locally in mock without the need for custom mock buildroot
> configurations and / or third-party repositories.
> 
> == Scope ==
> 
> * Proposal owner(s):
> ** one-off change: submit PR to revert the special-case handling for
> <code>rust-*</code> packages in the mass branching releng script
> ** ongoing effort: help package maintainers with merging changes from
> rawhide (where appropriate) and creating compat packages (when
> necessary) - this is made easier by the strong SemVer compatibility
> guarantees of Rust crates
> 
> * Other developers:
> 
> Initially, there is no impact on other developers.
> However, as soon as fedora 34 is branched, building Rust applications
> on that release branch will be easier than without this change.
> This will require packagers to merge rawhide updates into release
> branches when appropriate (again, bringing Rust packaging in line with
> the rest of fedora).
> 
> I also expect there to be reduced load on koji due less side tags
> being in use concurrently, which will benefit all package maintainers
> in fedora.
> 
> * Release engineering: [https://pagure.io/releng/issue/9753 #9753]
> 
> Release engineering will need to remove special-case handling of
> <code>rust-*</code> packages from their mass branching script before
> f34 is branched off of rawhide.
> 
> * Policies and guidelines:
> 
> The Rust packaging Guidelines will need small adaptations.
> 
> They are already outdated, so Change owner(s) will update them to the
> current state of Rust packaging regardless.
> 
> * Trademark approval: N/A (not needed for this Change)
> 
> * Alignment with Objectives:
> 
> The fedora IoT Edition promotes PARSEC, which is comprised of Rust
> packages - its package maintainers will benefit from being able to
> update and build those packages faster and more easily for release
> branches.
> 
> == Upgrade/compatibility impact ==
> 
> There will be no impact on upgrades from previous fedora releases,
> since Rust crate packages will only be available for those users for
> the first time.
> 
> If for some reason a user installed <code>rust-*-devel</code> packages
> manually after downloading them from the rawhide repositories, they
> will be gracefully upgraded.
> 
> == How To Test ==
> 
> Users should be able to build Rust applications for fedora 34 without
> workarounds or special steps (both in mock locally and in koji - both
> scratch and non-scratch builds).
> 
> == User Experience ==
> 
> Users should not notice this change. However, I expect some
> application updates to be available for release branches faster, since
> it will be easier for package maintainers to create them.
> 
> == Dependencies ==
> 
> N/A (this only affects the Rust package stack and has no external dependencies)
> 
> == Contingency Plan ==
> 
> * Contingency mechanism: untag and block <code>rust-*</code> packages
> in the f34 tag in koji (help from releng / a koji admin required),
> revert mass branching script changes before f35 branch point
> * Contingency deadline: Final Freeze (removing packages from koji will
> no longer be possible after this point)
> * Blocks release? No (the initial Change is small and does not
> negatively affect release process)
> * Blocks product? N/A
> 
> == Documentation ==
> 
> The Packaging Guidelines for Rust will be updated to reflect this Change.
> 
> == Release Notes ==
> 
> TBD
> 
> 
> -- 
> Ben Cotton
> He / Him / His
> Senior Program Manager, Fedora & CentOS Stream
> Red Hat
> TZ=America/Indiana/Indianapolis
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Neal Gompa]

On Mon, Sep 14, 2020 at 2:34 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Mon, Sep 14, 2020 at 10:10:30AM -0400, Ben Cotton wrote:
>  > https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>  >
>  > == Summary ==
>  >
>  > This Change proposal aims to enable shipping Rust crate packages
>  > (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
>  > Currently, they are only available for rawhide, which makes building
>  > Rust packages for release branches difficult.
>
>  Should the update policy for rust-devel packages be relaxes in stable
>  releases to allow updates to follow rawhide, at least if there are
>  no major breaking changes?
>
>  Current update policy states that major version changes should be avoided
>  in after stable release. But rust upstreams tend to move quickly, so having
>  ~1 year old packages in stable Fedora might not be too useful to build
>  newest upstreams. It might make sense to allow the rust-*-devel packages
>  to be updated more aggressively.
>  
This is probably a good idea.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Fabio Valentini]

On Mon, Sep 14, 2020 at 8:38 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote:
>  On Mon, Sep 14, 2020 at 2:34 PM Zbigniew Jędrzejewski-Szmek
>  <zbyszek(a)in.waw.pl&gt; wrote:
>  >
>  > On Mon, Sep 14, 2020 at 10:10:30AM -0400, Ben Cotton wrote:
>  > >
> https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
>  > >
>  > > == Summary ==
>  > >
>  > > This Change proposal aims to enable shipping Rust crate packages
>  > > (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
>  > > Currently, they are only available for rawhide, which makes building
>  > > Rust packages for release branches difficult.
>  >
>  > Should the update policy for rust-devel packages be relaxes in stable
>  > releases to allow updates to follow rawhide, at least if there are
>  > no major breaking changes?
>  >
>  > Current update policy states that major version changes should be avoided
>  > in after stable release. But rust upstreams tend to move quickly, so having
>  > ~1 year old packages in stable Fedora might not be too useful to build
>  > newest upstreams. It might make sense to allow the rust-*-devel packages
>  > to be updated more aggressively.
>  >
>
>  This is probably a good idea. 
I agree. The only thing to keep in mind for semver-incompatible -devel
package updates is to check dependent packages, and either patch them
to use the new version (like in rawhide), or create compat packages if
necessary - so no broken dependencies are generated in the stable
branches.

Fabio

[Josh Stone]

On 9/14/20 11:40 AM, Fabio Valentini wrote:
>  On Mon, Sep 14, 2020 at 8:38 PM Neal Gompa <ngompa13(a)gmail.com&gt;
> wrote:
> >
> > On Mon, Sep 14, 2020 at 2:34 PM Zbigniew Jędrzejewski-Szmek
> > <zbyszek(a)in.waw.pl&gt; wrote:
> >>
> >> On Mon, Sep 14, 2020 at 10:10:30AM -0400, Ben Cotton wrote:
> >>>
> https://fedoraproject.org/wiki/Changes/Rust_Crate_Packages_For_Release_Br...
> >>>
> >>> == Summary ==
> >>>
> >>> This Change proposal aims to enable shipping Rust crate packages
> >>> (<code>rust-$CRATE_NAME</code>) on release branches of fedora.
> >>> Currently, they are only available for rawhide, which makes building
> >>> Rust packages for release branches difficult.
> >>
> >> Should the update policy for rust-devel packages be relaxes in stable
> >> releases to allow updates to follow rawhide, at least if there are
> >> no major breaking changes?
> >>
> >> Current update policy states that major version changes should be avoided
> >> in after stable release. But rust upstreams tend to move quickly, so having
> >> ~1 year old packages in stable Fedora might not be too useful to build
> >> newest upstreams. It might make sense to allow the rust-*-devel packages
> >> to be updated more aggressively.
> >>
> >
> > This is probably a good idea.
>  
>  I agree. The only thing to keep in mind for semver-incompatible -devel
>  package updates is to check dependent packages, and either patch them
>  to use the new version (like in rawhide), or create compat packages if
>  necessary - so no broken dependencies are generated in the stable
>  branches. 
Are we going to recommend that these devel packages are still not really
for end users? Otherwise, checking dependent packages is not sufficient.

[Fabio Valentini]

On Mon, Sep 14, 2020 at 9:02 PM Josh Stone <jistone(a)redhat.com&gt; wrote:
>  On 9/14/20 11:40 AM, Fabio Valentini wrote:
>  > I agree. The only thing to keep in mind for semver-incompatible -devel
>  > package updates is to check dependent packages, and either patch them
>  > to use the new version (like in rawhide), or create compat packages if
>  > necessary - so no broken dependencies are generated in the stable
>  > branches.
>
>  Are we going to recommend that these devel packages are still not really
>  for end users? Otherwise, checking dependent packages is not sufficient. 
Is there - or will there be - any use case for manually installing
rust-*-devel packages outside of mock?
If not, then I don't see a reason why this would change.

Fabio

[Zbigniew Jędrzejewski-Szmek]

On Mon, Sep 14, 2020 at 09:15:27PM +0200, Fabio Valentini wrote:
>  On Mon, Sep 14, 2020 at 9:02 PM Josh Stone <jistone(a)redhat.com&gt;
> wrote:
>  >
>  > On 9/14/20 11:40 AM, Fabio Valentini wrote:
>  > > I agree. The only thing to keep in mind for semver-incompatible -devel
>  > > package updates is to check dependent packages, and either patch them
>  > > to use the new version (like in rawhide), or create compat packages if
>  > > necessary - so no broken dependencies are generated in the stable
>  > > branches.
>  >
>  > Are we going to recommend that these devel packages are still not really
>  > for end users? Otherwise, checking dependent packages is not sufficient. 
I don't think those packages make sense for users. As a user I would use cargo
and download packages from the web.

>  Is there - or will there be - any use case for manually installing
>  rust-*-devel packages outside of mock?
>  If not, then I don't see a reason why this would change. 
I guess as a developer of a package I could build this package in the
normal system, not in mock. 'fedpkg local' is very quick and it makes debugging
(both of the code and of the spec file) simpler.

Zbyszek

[Josh Stone]

On 9/14/20 12:18 PM, Zbigniew Jędrzejewski-Szmek wrote:
>  On Mon, Sep 14, 2020 at 09:15:27PM +0200, Fabio Valentini wrote:
> > On Mon, Sep 14, 2020 at 9:02 PM Josh Stone <jistone(a)redhat.com&gt; wrote:
> >>
> >> On 9/14/20 11:40 AM, Fabio Valentini wrote:
> >>> I agree. The only thing to keep in mind for semver-incompatible -devel
> >>> package updates is to check dependent packages, and either patch them
> >>> to use the new version (like in rawhide), or create compat packages if
> >>> necessary - so no broken dependencies are generated in the stable
> >>> branches.
> >>
> >> Are we going to recommend that these devel packages are still not really
> >> for end users? Otherwise, checking dependent packages is not sufficient.
>  
>  I don't think those packages make sense for users. As a user I would use cargo
>  and download packages from the web. 
I agree, but I think we still need to set that expectation.

[Zbigniew Jędrzejewski-Szmek]

On Mon, Sep 14, 2020 at 12:53:57PM -0700, Josh Stone wrote:
>  On 9/14/20 12:18 PM, Zbigniew Jędrzejewski-Szmek wrote:
>  > On Mon, Sep 14, 2020 at 09:15:27PM +0200, Fabio Valentini wrote:
>  >> On Mon, Sep 14, 2020 at 9:02 PM Josh Stone <jistone(a)redhat.com&gt; wrote:
>  >>>
>  >>> On 9/14/20 11:40 AM, Fabio Valentini wrote:
>  >>>> I agree. The only thing to keep in mind for semver-incompatible -devel
>  >>>> package updates is to check dependent packages, and either patch them
>  >>>> to use the new version (like in rawhide), or create compat packages if
>  >>>> necessary - so no broken dependencies are generated in the stable
>  >>>> branches.
>  >>>
>  >>> Are we going to recommend that these devel packages are still not really
>  >>> for end users? Otherwise, checking dependent packages is not sufficient.
>  > 
>  > I don't think those packages make sense for users. As a user I would use cargo
>  > and download packages from the web.
>  
>  I agree, but I think we still need to set that expectation. 
rust packages are usually generated with rust2rpm. We could emit something in the
description template. This wouldn't help with existing packages (at least until
they are regenerated), but would help with new ones.

Zbyszek

[Josh Stone]

On 9/14/20 1:14 PM, Zbigniew Jędrzejewski-Szmek wrote:
>  On Mon, Sep 14, 2020 at 12:53:57PM -0700, Josh Stone wrote:
> > On 9/14/20 12:18 PM, Zbigniew Jędrzejewski-Szmek wrote:
> >> On Mon, Sep 14, 2020 at 09:15:27PM +0200, Fabio Valentini wrote:
> >>> On Mon, Sep 14, 2020 at 9:02 PM Josh Stone <jistone(a)redhat.com&gt; wrote:
> >>>>
> >>>> On 9/14/20 11:40 AM, Fabio Valentini wrote:
> >>>>> I agree. The only thing to keep in mind for semver-incompatible
> -devel
> >>>>> package updates is to check dependent packages, and either patch
> them
> >>>>> to use the new version (like in rawhide), or create compat packages
> if
> >>>>> necessary - so no broken dependencies are generated in the stable
> >>>>> branches.
> >>>>
> >>>> Are we going to recommend that these devel packages are still not really
> >>>> for end users? Otherwise, checking dependent packages is not sufficient.
> >>
> >> I don't think those packages make sense for users. As a user I would use
> cargo
> >> and download packages from the web.
> >
> > I agree, but I think we still need to set that expectation.
>  
>  rust packages are usually generated with rust2rpm. We could emit something in the
>  description template. This wouldn't help with existing packages (at least until
>  they are regenerated), but would help with new ones. 
Ok, well the template already does make a statement like that:

  This package contains library source intended for building
  other packages which use "%{crate}" crate.

Maybe that's good enough.

[Luca BRUNO]

On Mon, 14 Sep 2020 10:10:30 -0400
Ben Cotton <bcotton(a)redhat.com&gt; wrote:

>  Following the general upwards trend in the Rust language's
> popularity,
>  more and more applications and services in fedora are written in Rust.
>  This includes some CoreOS services, PARSEC, some nice command line
>  tools (e.g. ripgrep, bat, fedora-update-feedback, ...), and parts of
>  the GNOME stack. 
As an upstream developer for most of those CoreOS projects, I wonder
whether this proposal is going to make it even more complex to
package applications on release branches, due to the network effect of
dependencies.

>  ** ongoing effort: help package maintainers with merging changes
> from
>  rawhide (where appropriate) and creating compat packages (when
>  necessary) - this is made easier by the strong SemVer compatibility
>  guarantees of Rust crates 
In particular, it sounds like Fedora will try to:
 * package most of crates.io libraries, now *one more time per each
   branch*
 * align all applications to use the same dependency versions
   (possibly across release branches?)

Taking one of those projects as an example, Afterburn has (more or less)
monthly releases[0], a bot keeping dependencies fresh[1], and ~200
crates in its full set of transitive dependencies[2] (only a subset of which
relevant for Fedora).

[0] https://github.com/coreos/afterburn/releases
[1] https://github.com/coreos/afterburn/issues?q=label%3Adependencies+is%3Acl...
[2] https://github.com/coreos/afterburn/blob/master/Cargo.lock

Now, these dependencies when packaged have to live with additional
constraints from other applications, which may still be on older
non-semver-compatible versions.
Effectively this means that Fedora downgrades[4] those dependencies
until all other packaged applications converge on a shared version
(which is not guaranteed to happen, depending on relative velocity of
different upstream projects).

[4]
https://src.fedoraproject.org/rpms/rust-afterburn/blob/master/f/afterburn...

The resulting composition has
 1) never been tested by upstream CI.
 2) may have re-introduced bugs patched by newer library versions
    (and seen as fixed by upstream CI). 

I'm admittedly not a knowledgeable packager so I don't have a good
alternative proposal to offer, but I wonder whether there is space for a
different path where Fedora:
 * doesn't have to re-package all of crates.io (especially not for each
   branch).
 * doesn't try to align all applications on a single
   (major+minor) version of a library crate.
 * tries to stay closer to upstream-specified dependencies, handling
   those crates relative to the specific application.
 * only package library crates whose logic needs to be functionally
   patched (affecting multiple applications), and selectively override
   only those in manifest/lockfile patches.
 * takes a more relaxed approach on having multiple (major+minor)
   crate versions co-existing, as long as required by applications.

To be explicit: I'm aware of the cons of vendoring and I'm not asking
for that, but for exploring a model with an "application first" approach
and possibly less toiling on packaging efforts.

Ciao, Luca

[Fabio Valentini]

On Thu, Sep 24, 2020 at 11:51 AM Luca BRUNO <lucab(a)lucabruno.net&gt; wrote:
>  On Mon, 14 Sep 2020 10:10:30 -0400
>  Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>
>  > Following the general upwards trend in the Rust language's popularity,
>  > more and more applications and services in fedora are written in Rust.
>  > This includes some CoreOS services, PARSEC, some nice command line
>  > tools (e.g. ripgrep, bat, fedora-update-feedback, ...), and parts of
>  > the GNOME stack. 
Hi Luca,
Thanks for your detailed feedback, I've responded to your questions below.

>  As an upstream developer for most of those CoreOS projects, I wonder
>  whether this proposal is going to make it even more complex to
>  package applications on release branches, due to the network effect of
>  dependencies. 
I hope that this won't be the case, and I'll also invest time to
prevent this outcome, if at all possible.

>  > ** ongoing effort: help package maintainers with merging changes
> from
>  > rawhide (where appropriate) and creating compat packages (when
>  > necessary) - this is made easier by the strong SemVer compatibility
>  > guarantees of Rust crates
>
>  In particular, it sounds like Fedora will try to:
>   * package most of crates.io libraries, now *one more time per each
>     branch* 
No. This is definitely out of scope.
I don't see a reason to package things that nothing in fedora depends
on, for example.
crates.io ships 47,039 crates today, but only a fraction of those are
actually packaged and / or required for fedora (today, 1225).

>   * align all applications to use the same dependency versions
>     (possibly across release branches?) 
This is also out of scope, because I don't think it would be possible.
The proposal text should make it pretty clear that release branches
should mostly follow rawhide in terms of source-only package updates,
and - if necessary - compat package creations. Binary packages already
follow the Update Policy, so nothing changes for those.

Essentially, all I want to change is that package dependencies won't
be broken as freely by pushing semver-incompatible version updates in
rawhide, at least not without creating the necessary compat packages
for older versions. My hope is that this will actually make it
*easier* to package Rust applications on both rawhide *and* on release
branches.

>  Taking one of those projects as an example, Afterburn has (more or
> less)
>  monthly releases[0], a bot keeping dependencies fresh[1], and ~200
>  crates in its full set of transitive dependencies[2] (only a subset of which
>  relevant for Fedora).
>
>  [0] https://github.com/coreos/afterburn/releases
>  [1] https://github.com/coreos/afterburn/issues?q=label%3Adependencies+is%3Acl...
>  [2] https://github.com/coreos/afterburn/blob/master/Cargo.lock
>
>  Now, these dependencies when packaged have to live with additional
>  constraints from other applications, which may still be on older
>  non-semver-compatible versions.
>  Effectively this means that Fedora downgrades[4] those dependencies
>  until all other packaged applications converge on a shared version
>  (which is not guaranteed to happen, depending on relative velocity of
>  different upstream projects). 
I hope that dependency downgrades will happen less often, not more
often. I don't see a need for all dependent packages to converge on a
shared version, either, since I don't share the disdain for compat
packages some others do :D - so long as they're used in moderation,
and removed when no longer needed.

>  [4]
> https://src.fedoraproject.org/rpms/rust-afterburn/blob/master/f/afterburn...
>
>  The resulting composition has
>   1) never been tested by upstream CI.
>   2) may have re-introduced bugs patched by newer library versions
>      (and seen as fixed by upstream CI). 
I agree, this is not a good situation, and one that I hope this
proposal and the resulting changes will remedy, instead of making it
worse.

>  I'm admittedly not a knowledgeable packager so I don't have a
> good
>  alternative proposal to offer, but I wonder whether there is space for a
>  different path where Fedora:
>   * doesn't have to re-package all of crates.io (especially not for each
>     branch). 
There's no need to re-package things for each branch. Most changes can
be "git merge"d.
This is how package updates are handled in fedora, Rust has only been
a weird special case in this regard ...

>   * doesn't try to align all applications on a single
>     (major+minor) version of a library crate. 
As explained above, this isn't the intention of this proposal.

>   * tries to stay closer to upstream-specified dependencies, handling
>     those crates relative to the specific application. 
I hope that this is actually the outcome of the proposed change.

>   * only package library crates whose logic needs to be functionally
>     patched (affecting multiple applications), and selectively override
>     only those in manifest/lockfile patches. 
This is actually what I'm aiming for. If there are no *code changes*
necessary when updating a dependency, then I don't see a reason to not
do that, and submit the change to the upstream project as well. For
updated where code changes are necessary, that's not always feasible
without upstream doing that first, so in this case, compat packages
for the old library version come in.

>   * takes a more relaxed approach on having multiple (major+minor)
>     crate versions co-existing, as long as required by applications. 
This is actually what I'm aiming for :)

>  To be explicit: I'm aware of the cons of vendoring and I'm
> not asking
>  for that, but for exploring a model with an "application first" approach
>  and possibly less toiling on packaging efforts. 
I agree. I ship two Rust applications as fedora packages (admittedly,
with smaller dependency trees), but it's still painful to do for
release branches.
Are your dependencies all maintained by the Rust SIG? Then I'm aiming
for a future where you don't have to care about them at all when
submitting updates for your applications, because all your
dependencies will already be up-to-date, and you'll only have to care
about *new* dependencies.

I hope that I was able to explain some things better than in the
initial proposal.

Thanks,
Fabio