New Go Packaging Guidelines landed in rawhide (koji) today

[Nicolas Mailhot]

Hi,

Fedora’s new Go packaging macros landed in rawhide (koji) today.

The corresponding Fedora Go packaging conventions are therefore
EFFECTIVE for new rawhide builds. For the first time in Fedora’s
history, we will be able to perform Go package builds conforming to an
approved Fedora Packaging Guideline.

Packaging documentation:
https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
and approval: https://pagure.io/packaging-committee/issue/382
The go-rpm-templates package provides more complete info.

F31 change page:
https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
and approval: https://pagure.io/fesco/issue/2120

While the guidelines will feel familiar to anyone who created a Fedora
Go packages in the last two years, they DO include a backwards-
incompatible change. Making GOPATH manipulation robust required moving
the corresponding logic to %prep with a new %goprep macro.

Therefore, existing specs are expected to fail without the addition of
the %goprep call.

This is of course not the end of the road, just a key step.

It opens the way to a mass cleanup and refresh of the Fedora Go stack.
https://pagure.io/packaging-committee/issue/901

A preview of this refresh is available here:
https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/

Enormous thanks to
– Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
updating and cleaning-up all those packages, and to
– Elliott Sales de Andrade (Qulogic), that picked up maintenance of
golist and fixed many of its long-standing bugs and limitations.

Many thanks to the mock, rpm and redhat-rpm-config maintainers,
that integrated the changes, we built upon (Igor Gnatenko, Florian
Festi, Miroslav Suchý, Panu Matilainen)

The macro set supports Go DynamicBuildRequires
https://fedoraproject.org/wiki/Changes/DynamicBuildRequires

They will be usable in mock as soon as rpm 4.15 lands
https://fedoraproject.org/wiki/Changes/RPM-4.15

Use in koji or copr will have to wait for the corresponding refresh
buldsystem-side. So this part of the change is a technology preview for
now.

Best regards,

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot via devel"
> <devel(a)lists.fedoraproject.org&gt;
>  To: golang(a)lists.fedoraproject.org
>  Cc: devel(a)lists.fedoraproject.org, announce(a)lists.fedoraproject.org, "Nicolas
> Mailhot" <nicolas.mailhot(a)laposte.net&gt;
>  Sent: Saturday, June 8, 2019 3:45:20 PM
>  Subject: New Go Packaging Guidelines landed in rawhide (koji) today
>  
>  Hi,
>  
>  Fedora’s new Go packaging macros landed in rawhide (koji) today.
>   
I thought that we have agreed on Go SIG meeting with eclipseo to do this in side tag along
with golang rebase(to avoid 2 rebuilds), so there is no observable breakage(if any would
occur) for package builds and all packages "just" pop in using new macros and
following new guidelines. Currently following packages are FTBFS due to this change
https://apps.fedoraproject.org/koschei/affected-by/go-srpm-macros?epoch1=...
(thanks to qulogic for this query). And still we will have to do the side tag rebuild(for
this change). We could have done that in one pass without causing any observable breakage
for anyone.

>  The corresponding Fedora Go packaging conventions are therefore
>  EFFECTIVE for new rawhide builds. For the first time in Fedora’s
>  history, we will be able to perform Go package builds conforming to an
>  approved Fedora Packaging Guideline.
>  
>  Packaging documentation:
>  https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
>  and approval: https://pagure.io/packaging-committee/issue/382
>  The go-rpm-templates package provides more complete info.
>  
>  F31 change page:
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  and approval: https://pagure.io/fesco/issue/2120 
It seems that this change has been accepted as Self Contained Change but IMHO it is System
Wide Change as it seems to affect (nearly) all Go based packages in distribution(and will
require work/attention of people that are not change owners, actually not accounted for in
change proposal). For past several releases I have been doing rebase of Go compiler
change(yet to be filed for F31) that is IMHO comparable(maybe a bit smaller) in scope and
they were always deemed by FESCO as System Wide Changes. This really leaves me confused.
Could someone from FESCO clarify? 

>  
>  While the guidelines will feel familiar to anyone who created a Fedora
>  Go packages in the last two years, they DO include a backwards-
>  incompatible change. Making GOPATH manipulation robust required moving
>  the corresponding logic to %prep with a new %goprep macro.
>  
>  Therefore, existing specs are expected to fail without the addition of
>  the %goprep call. 
When this has been discussed, new macros have been presented to me as backwards
compatible(i.e. current packages will work and build as is, although requiring refresh to
adopt new features), so it has not been concern for me. Other issue that I have is that
you have not communicated this change(landing the new macro package) prior it happening
here or elsewhere. I'm a Go SIG member(and maintainer of the previous macros packages,
which by the way are still out there) and I have not been aware of this pending change
landing now.

>  
>  This is of course not the end of the road, just a key step. 
I would much appreciate if you would communicate a bit more before landing such a big
changes(go-rpm-package) in future, it would made possible to avoid some breakages, collect
feedback and improve overall packagers experience. I think that communication is not easy,
at least not for me, and I don't think that it is my strong skill, but we
shouldn't resign on it as it is IMO crucial part of the Fedora community. Is there
something that can I do to improve the information flow from my side?

JC

> 
> It opens the way to a mass cleanup and refresh of the Fedora Go stack.
> https://pagure.io/packaging-committee/issue/901
> 
> A preview of this refresh is available here:
> https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/
> 
> Enormous thanks to
> – Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
> updating and cleaning-up all those packages, and to
> – Elliott Sales de Andrade (Qulogic), that picked up maintenance of
> golist and fixed many of its long-standing bugs and limitations.
> 
> Many thanks to the mock, rpm and redhat-rpm-config maintainers,
> that integrated the changes, we built upon (Igor Gnatenko, Florian
> Festi, Miroslav Suchý, Panu Matilainen)
> 
> The macro set supports Go DynamicBuildRequires
> https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
> 
> They will be usable in mock as soon as rpm 4.15 lands
> https://fedoraproject.org/wiki/Changes/RPM-4.15
> 
> Use in koji or copr will have to wait for the corresponding refresh
> buldsystem-side. So this part of the change is a technology preview for
> now.
> 
> Best regards,
> 
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Nicolas Mailhot]

Le 2019-06-12 10:39, Jakub Cajka a écrit :

> > Fedora’s new Go packaging macros landed in rawhide (koji) today.
> > 
>  
>  I thought that we have agreed on Go SIG meeting with eclipseo to do
>  this in side tag along with golang rebase(to avoid 2 rebuilds), 
https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#...

“ This proposal consists of:
     Packaging the new Go macros: go-rpm-macros
     Getting the Guidelines approved by the FPC
     Updating all Go libraries with the new macros
     Mass-rebuilding all the Go package in a side tag ”

You complain that step 3 and 4 are separate, but that’s how it was 
planed from the start up and approved in the change page.

You're conflating merging two mass Go package rebuilds (one for the new 
Go compiler, and another for the new, and first, Go packaging 
guidelines) with merging step 3 and 4 (which would have had other 
drawbacks, that were never discussed, because that's not how we planed 
things).

And BTW it was already so in
https://pagure.io/GoSIG/go-sig/issue/20 6 months ago (though this page 
is obsolete, you made us rewrite the plan in so many formats over time 
I've lost track or what is up to date or not. The change page is up to 
date, it’s the most recent rewrite)

Sincerely,

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot"
> <nicolas.mailhot(a)laposte.net&gt;
>  To: "Jakub Cajka" <jcajka(a)redhat.com&gt;
>  Cc: golang(a)lists.fedoraproject.org, "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Wednesday, June 12, 2019 11:23:34 AM
>  Subject: Re: New Go Packaging Guidelines landed in rawhide (koji) today
>  
>  Le 2019-06-12 10:39, Jakub Cajka a écrit :
>  
>  >> Fedora’s new Go packaging macros landed in rawhide (koji) today.
>  >> 
>  > 
>  > I thought that we have agreed on Go SIG meeting with eclipseo to do
>  > this in side tag along with golang rebase(to avoid 2 rebuilds),
>  
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#...
>  
>  “ This proposal consists of:
>       Packaging the new Go macros: go-rpm-macros
>       Getting the Guidelines approved by the FPC
>       Updating all Go libraries with the new macros
>       Mass-rebuilding all the Go package in a side tag ”
>  
>  You complain that step 3 and 4 are separate, but that’s how it was
>  planed from the start up and approved in the change page.
>  
>  You're conflating merging two mass Go package rebuilds (one for the new
>  Go compiler, and another for the new, and first, Go packaging
>  guidelines) with merging step 3 and 4 (which would have had other
>  drawbacks, that were never discussed, because that's not how we planed
>  things).
>  
>  And BTW it was already so in
>  https://pagure.io/GoSIG/go-sig/issue/20 6 months ago (though this page
>  is obsolete, you made us rewrite the plan in so many formats over time
>  I've lost track or what is up to date or not. The change page is up to
>  date, it’s the most recent rewrite) 
I guess that we have not agreed on the SIG meeting then. I don't complain and this is
not in any ways personal, keep it on mind please. The change proposal predates that SIG
discussion as other bit predates the change proposal. I'm pointing out that we could
have avoided any breakage if we did few thing slightly differently. Currently by your
actions there are several FTBFS packages, it is not really a serious issues(as I'm
sure that eclipso will fix up all the packages that need it in time for Fedora 31, kudos
for committing for that work :)) but it is unnecessary and avoidable inconvenience that
I(and I guess others too) will have to account for(spend some time on). In my case
preparing for Go rebase(I do scratch rebuilds). One of the points been also possibility to
fit in the Go rebase in to that side tag, but after further discussion with eclipseo on
Wednesday it will make more sense to use regular mass-rebuild for that(as I usually do)
assuming that the side tag rebuild will conclude 1-2w prior to it(so I will be able to
observe dist git in coherent shape).

JC

> 
> Sincerely,
> 
> --
> Nicolas Mailhot
>

[Robin Lee]

On Sat, Jun 8, 2019 at 10:35 PM Nicolas Mailhot via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  Hi,
>
>  Fedora’s new Go packaging macros landed in rawhide (koji) today.
>
>  The corresponding Fedora Go packaging conventions are therefore
>  EFFECTIVE for new rawhide builds. For the first time in Fedora’s
>  history, we will be able to perform Go package builds conforming to an
>  approved Fedora Packaging Guideline.
>
>  Packaging documentation:
>  https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
>  and approval: https://pagure.io/packaging-committee/issue/382
>  The go-rpm-templates package provides more complete info.
>
>  F31 change page:
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  and approval: https://pagure.io/fesco/issue/2120
>
>  While the guidelines will feel familiar to anyone who created a Fedora
>  Go packages in the last two years, they DO include a backwards-
>  incompatible change. Making GOPATH manipulation robust required moving
>  the corresponding logic to %prep with a new %goprep macro.
>
>  Therefore, existing specs are expected to fail without the addition of
>  the %goprep call.
>
>  This is of course not the end of the road, just a key step.
>
>  It opens the way to a mass cleanup and refresh of the Fedora Go stack.
>  https://pagure.io/packaging-committee/issue/901
>
>  A preview of this refresh is available here:
>  https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/
>
>  Enormous thanks to
>  – Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
>  updating and cleaning-up all those packages, and to
>  – Elliott Sales de Andrade (Qulogic), that picked up maintenance of
>  golist and fixed many of its long-standing bugs and limitations.
>
>  Many thanks to the mock, rpm and redhat-rpm-config maintainers,
>  that integrated the changes, we built upon (Igor Gnatenko, Florian
>  Festi, Miroslav Suchý, Panu Matilainen)
>
>  The macro set supports Go DynamicBuildRequires
>  https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
>
>  They will be usable in mock as soon as rpm 4.15 lands
>  https://fedoraproject.org/wiki/Changes/RPM-4.15
>
>  Use in koji or copr will have to wait for the corresponding refresh
>  buldsystem-side. So this part of the change is a technology preview for
>  now.
>
>  Best regards,
>
>  --
>  Nicolas Mailhot 
What should I do at this moment as a packager that maintaining some
Go packages?
Should I fix my packages and build against f31-go in Koji?
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Hi,

>  What should I do at this moment as a packager that maintaining some
>  Go packages?
>  Should I fix my packages and build against f31-go in Koji? 
Yes, sure, if you can that would be appreciated. The vast majority of
packages is easy to clean up (just adapt the templates in go-rpm-
templates or use go2rpm), it's just there is an awful lot of them, so
Robert-André Mauchin (eclipseo) can not do all of them in a single
pass.

If you hit one of the cases where a project needs weird stuff, or if
you do not understand something, there is help available here and in
the #fedora-golang channel.

Please just state here (or tell eclispeo directly) what you will
convert, so you do not end up doing the same work in parallel.

Quite often, eclipseo will have prepared things in his copr, just not
built them in koji yet.
https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/

So taking up from where he prepared, and checking the result builds in
koji and works for you, will accelerate things.

@eclipseo: please correct if I wrote something that does not make
things easier your side

Regards,

-- 
Nicolas Mailhot