New Go Packaging Guidelines landed in rawhide (koji) today

[Nicolas Mailhot]

Hi,

Fedora’s new Go packaging macros landed in rawhide (koji) today.

The corresponding Fedora Go packaging conventions are therefore
EFFECTIVE for new rawhide builds. For the first time in Fedora’s
history, we will be able to perform Go package builds conforming to an
approved Fedora Packaging Guideline.

Packaging documentation:
https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
and approval: https://pagure.io/packaging-committee/issue/382
The go-rpm-templates package provides more complete info.

F31 change page:
https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
and approval: https://pagure.io/fesco/issue/2120

While the guidelines will feel familiar to anyone who created a Fedora
Go packages in the last two years, they DO include a backwards-
incompatible change. Making GOPATH manipulation robust required moving
the corresponding logic to %prep with a new %goprep macro.

Therefore, existing specs are expected to fail without the addition of
the %goprep call.

This is of course not the end of the road, just a key step.

It opens the way to a mass cleanup and refresh of the Fedora Go stack.
https://pagure.io/packaging-committee/issue/901

A preview of this refresh is available here:
https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/

Enormous thanks to
– Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
updating and cleaning-up all those packages, and to
– Elliott Sales de Andrade (Qulogic), that picked up maintenance of
golist and fixed many of its long-standing bugs and limitations.

Many thanks to the mock, rpm and redhat-rpm-config maintainers,
that integrated the changes, we built upon (Igor Gnatenko, Florian
Festi, Miroslav Suchý, Panu Matilainen)

The macro set supports Go DynamicBuildRequires
https://fedoraproject.org/wiki/Changes/DynamicBuildRequires

They will be usable in mock as soon as rpm 4.15 lands
https://fedoraproject.org/wiki/Changes/RPM-4.15

Use in koji or copr will have to wait for the corresponding refresh
buldsystem-side. So this part of the change is a technology preview for
now.

Best regards,

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot via devel"
> <devel(a)lists.fedoraproject.org&gt;
>  To: golang(a)lists.fedoraproject.org
>  Cc: devel(a)lists.fedoraproject.org, announce(a)lists.fedoraproject.org, "Nicolas
> Mailhot" <nicolas.mailhot(a)laposte.net&gt;
>  Sent: Saturday, June 8, 2019 3:45:20 PM
>  Subject: New Go Packaging Guidelines landed in rawhide (koji) today
>  
>  Hi,
>  
>  Fedora’s new Go packaging macros landed in rawhide (koji) today.
>   
I thought that we have agreed on Go SIG meeting with eclipseo to do this in side tag along
with golang rebase(to avoid 2 rebuilds), so there is no observable breakage(if any would
occur) for package builds and all packages "just" pop in using new macros and
following new guidelines. Currently following packages are FTBFS due to this change
https://apps.fedoraproject.org/koschei/affected-by/go-srpm-macros?epoch1=...
(thanks to qulogic for this query). And still we will have to do the side tag rebuild(for
this change). We could have done that in one pass without causing any observable breakage
for anyone.

>  The corresponding Fedora Go packaging conventions are therefore
>  EFFECTIVE for new rawhide builds. For the first time in Fedora’s
>  history, we will be able to perform Go package builds conforming to an
>  approved Fedora Packaging Guideline.
>  
>  Packaging documentation:
>  https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
>  and approval: https://pagure.io/packaging-committee/issue/382
>  The go-rpm-templates package provides more complete info.
>  
>  F31 change page:
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  and approval: https://pagure.io/fesco/issue/2120 
It seems that this change has been accepted as Self Contained Change but IMHO it is System
Wide Change as it seems to affect (nearly) all Go based packages in distribution(and will
require work/attention of people that are not change owners, actually not accounted for in
change proposal). For past several releases I have been doing rebase of Go compiler
change(yet to be filed for F31) that is IMHO comparable(maybe a bit smaller) in scope and
they were always deemed by FESCO as System Wide Changes. This really leaves me confused.
Could someone from FESCO clarify? 

>  
>  While the guidelines will feel familiar to anyone who created a Fedora
>  Go packages in the last two years, they DO include a backwards-
>  incompatible change. Making GOPATH manipulation robust required moving
>  the corresponding logic to %prep with a new %goprep macro.
>  
>  Therefore, existing specs are expected to fail without the addition of
>  the %goprep call. 
When this has been discussed, new macros have been presented to me as backwards
compatible(i.e. current packages will work and build as is, although requiring refresh to
adopt new features), so it has not been concern for me. Other issue that I have is that
you have not communicated this change(landing the new macro package) prior it happening
here or elsewhere. I'm a Go SIG member(and maintainer of the previous macros packages,
which by the way are still out there) and I have not been aware of this pending change
landing now.

>  
>  This is of course not the end of the road, just a key step. 
I would much appreciate if you would communicate a bit more before landing such a big
changes(go-rpm-package) in future, it would made possible to avoid some breakages, collect
feedback and improve overall packagers experience. I think that communication is not easy,
at least not for me, and I don't think that it is my strong skill, but we
shouldn't resign on it as it is IMO crucial part of the Fedora community. Is there
something that can I do to improve the information flow from my side?

JC

> 
> It opens the way to a mass cleanup and refresh of the Fedora Go stack.
> https://pagure.io/packaging-committee/issue/901
> 
> A preview of this refresh is available here:
> https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/
> 
> Enormous thanks to
> – Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
> updating and cleaning-up all those packages, and to
> – Elliott Sales de Andrade (Qulogic), that picked up maintenance of
> golist and fixed many of its long-standing bugs and limitations.
> 
> Many thanks to the mock, rpm and redhat-rpm-config maintainers,
> that integrated the changes, we built upon (Igor Gnatenko, Florian
> Festi, Miroslav Suchý, Panu Matilainen)
> 
> The macro set supports Go DynamicBuildRequires
> https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
> 
> They will be usable in mock as soon as rpm 4.15 lands
> https://fedoraproject.org/wiki/Changes/RPM-4.15
> 
> Use in koji or copr will have to wait for the corresponding refresh
> buldsystem-side. So this part of the change is a technology preview for
> now.
> 
> Best regards,
> 
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Nicolas Mailhot]

Le 2019-06-12 10:39, Jakub Cajka a écrit :

> > Fedora’s new Go packaging macros landed in rawhide (koji) today.
> > 
>  
>  I thought that we have agreed on Go SIG meeting with eclipseo to do
>  this in side tag along with golang rebase(to avoid 2 rebuilds), 
https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#...

“ This proposal consists of:
     Packaging the new Go macros: go-rpm-macros
     Getting the Guidelines approved by the FPC
     Updating all Go libraries with the new macros
     Mass-rebuilding all the Go package in a side tag ”

You complain that step 3 and 4 are separate, but that’s how it was 
planed from the start up and approved in the change page.

You're conflating merging two mass Go package rebuilds (one for the new 
Go compiler, and another for the new, and first, Go packaging 
guidelines) with merging step 3 and 4 (which would have had other 
drawbacks, that were never discussed, because that's not how we planed 
things).

And BTW it was already so in
https://pagure.io/GoSIG/go-sig/issue/20 6 months ago (though this page 
is obsolete, you made us rewrite the plan in so many formats over time 
I've lost track or what is up to date or not. The change page is up to 
date, it’s the most recent rewrite)

Sincerely,

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot"
> <nicolas.mailhot(a)laposte.net&gt;
>  To: "Jakub Cajka" <jcajka(a)redhat.com&gt;
>  Cc: golang(a)lists.fedoraproject.org, "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Wednesday, June 12, 2019 11:23:34 AM
>  Subject: Re: New Go Packaging Guidelines landed in rawhide (koji) today
>  
>  Le 2019-06-12 10:39, Jakub Cajka a écrit :
>  
>  >> Fedora’s new Go packaging macros landed in rawhide (koji) today.
>  >> 
>  > 
>  > I thought that we have agreed on Go SIG meeting with eclipseo to do
>  > this in side tag along with golang rebase(to avoid 2 rebuilds),
>  
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#...
>  
>  “ This proposal consists of:
>       Packaging the new Go macros: go-rpm-macros
>       Getting the Guidelines approved by the FPC
>       Updating all Go libraries with the new macros
>       Mass-rebuilding all the Go package in a side tag ”
>  
>  You complain that step 3 and 4 are separate, but that’s how it was
>  planed from the start up and approved in the change page.
>  
>  You're conflating merging two mass Go package rebuilds (one for the new
>  Go compiler, and another for the new, and first, Go packaging
>  guidelines) with merging step 3 and 4 (which would have had other
>  drawbacks, that were never discussed, because that's not how we planed
>  things).
>  
>  And BTW it was already so in
>  https://pagure.io/GoSIG/go-sig/issue/20 6 months ago (though this page
>  is obsolete, you made us rewrite the plan in so many formats over time
>  I've lost track or what is up to date or not. The change page is up to
>  date, it’s the most recent rewrite) 
I guess that we have not agreed on the SIG meeting then. I don't complain and this is
not in any ways personal, keep it on mind please. The change proposal predates that SIG
discussion as other bit predates the change proposal. I'm pointing out that we could
have avoided any breakage if we did few thing slightly differently. Currently by your
actions there are several FTBFS packages, it is not really a serious issues(as I'm
sure that eclipso will fix up all the packages that need it in time for Fedora 31, kudos
for committing for that work :)) but it is unnecessary and avoidable inconvenience that
I(and I guess others too) will have to account for(spend some time on). In my case
preparing for Go rebase(I do scratch rebuilds). One of the points been also possibility to
fit in the Go rebase in to that side tag, but after further discussion with eclipseo on
Wednesday it will make more sense to use regular mass-rebuild for that(as I usually do)
assuming that the side tag rebuild will conclude 1-2w prior to it(so I will be able to
observe dist git in coherent shape).

JC

> 
> Sincerely,
> 
> --
> Nicolas Mailhot
>

[Zbigniew Jędrzejewski-Szmek]

On Wed, Jun 12, 2019 at 04:39:07AM -0400, Jakub Cajka wrote:
>  > F31 change page:
>  > https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  > and approval: https://pagure.io/fesco/issue/2120
>
>  It seems that this change has been accepted as Self Contained Change
>  but IMHO it is System Wide Change as it seems to affect (nearly) all
>  Go based packages in distribution(and will require work/attention of
>  people that are not change owners, actually not accounted for in
>  change proposal). For past several releases I have been doing rebase
>  of Go compiler change(yet to be filed for F31) that is IMHO
>  comparable(maybe a bit smaller) in scope and they were always deemed
>  by FESCO as System Wide Changes. This really leaves me
>  confused. Could someone from FESCO clarify? 
https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes says
>  Examples include [...] a coordinated effort within a SIG with
>  limited impact outside the SIG's functional area 
So in this case, even though the change affects so many packages, it
falls into the "self contained change" category.

That said, the difference between "system-wide" and "self-contained"
boils down to two things: some additional data required in the change
page, and filing the change a bit earlier. In this case the additional
data is mostly there in the change page, and the change was filed early,
so even if we were to change the Change to "system-wide", the effect
would be cosmetic.

Zbyszek

[Jakub Cajka]

----- Original Message -----
>  From: "Zbigniew Jędrzejewski-Szmek"
> <zbyszek(a)in.waw.pl&gt;
>  To: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Cc: golang(a)lists.fedoraproject.org
>  Sent: Wednesday, June 12, 2019 4:20:40 PM
>  Subject: Re: New Go Packaging Guidelines landed in rawhide (koji) today
>  
>  On Wed, Jun 12, 2019 at 04:39:07AM -0400, Jakub Cajka wrote:
>  > > F31 change page:
>  > > https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  > > and approval: https://pagure.io/fesco/issue/2120
>  >
>  > It seems that this change has been accepted as Self Contained Change
>  > but IMHO it is System Wide Change as it seems to affect (nearly) all
>  > Go based packages in distribution(and will require work/attention of
>  > people that are not change owners, actually not accounted for in
>  > change proposal). For past several releases I have been doing rebase
>  > of Go compiler change(yet to be filed for F31) that is IMHO
>  > comparable(maybe a bit smaller) in scope and they were always deemed
>  > by FESCO as System Wide Changes. This really leaves me
>  > confused. Could someone from FESCO clarify?
>  
>  https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes says
>  > Examples include [...] a coordinated effort within a SIG with
>  > limited impact outside the SIG's functional area
>  
>  So in this case, even though the change affects so many packages, it
>  falls into the "self contained change" category.
>  
>  That said, the difference between "system-wide" and "self-contained"
>  boils down to two things: some additional data required in the change
>  page, and filing the change a bit earlier. In this case the additional
>  data is mostly there in the change page, and the change was filed early,
>  so even if we were to change the Change to "system-wide", the effect
>  would be cosmetic.
>  
>  Zbyszek 
Thank you for clarification :). IMO it would be great if that has been recorded in
https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes, this approach
can't be really inferred from it(at least I don't see it there).

With that popped on my mind that may be calling it "early/late window change"
would fit better along with recording the complexity of the change, rather then system
wide, contained change. But this is really just my brain storm.

Unfortunately no early changes for me as Golang release are aligned late in to the Fedora
devel cycle.

JC

> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Robin Lee]

On Sat, Jun 8, 2019 at 10:35 PM Nicolas Mailhot via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  Hi,
>
>  Fedora’s new Go packaging macros landed in rawhide (koji) today.
>
>  The corresponding Fedora Go packaging conventions are therefore
>  EFFECTIVE for new rawhide builds. For the first time in Fedora’s
>  history, we will be able to perform Go package builds conforming to an
>  approved Fedora Packaging Guideline.
>
>  Packaging documentation:
>  https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/
>  and approval: https://pagure.io/packaging-committee/issue/382
>  The go-rpm-templates package provides more complete info.
>
>  F31 change page:
>  https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines
>  and approval: https://pagure.io/fesco/issue/2120
>
>  While the guidelines will feel familiar to anyone who created a Fedora
>  Go packages in the last two years, they DO include a backwards-
>  incompatible change. Making GOPATH manipulation robust required moving
>  the corresponding logic to %prep with a new %goprep macro.
>
>  Therefore, existing specs are expected to fail without the addition of
>  the %goprep call.
>
>  This is of course not the end of the road, just a key step.
>
>  It opens the way to a mass cleanup and refresh of the Fedora Go stack.
>  https://pagure.io/packaging-committee/issue/901
>
>  A preview of this refresh is available here:
>  https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/
>
>  Enormous thanks to
>  – Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
>  updating and cleaning-up all those packages, and to
>  – Elliott Sales de Andrade (Qulogic), that picked up maintenance of
>  golist and fixed many of its long-standing bugs and limitations.
>
>  Many thanks to the mock, rpm and redhat-rpm-config maintainers,
>  that integrated the changes, we built upon (Igor Gnatenko, Florian
>  Festi, Miroslav Suchý, Panu Matilainen)
>
>  The macro set supports Go DynamicBuildRequires
>  https://fedoraproject.org/wiki/Changes/DynamicBuildRequires
>
>  They will be usable in mock as soon as rpm 4.15 lands
>  https://fedoraproject.org/wiki/Changes/RPM-4.15
>
>  Use in koji or copr will have to wait for the corresponding refresh
>  buldsystem-side. So this part of the change is a technology preview for
>  now.
>
>  Best regards,
>
>  --
>  Nicolas Mailhot 
What should I do at this moment as a packager that maintaining some
Go packages?
Should I fix my packages and build against f31-go in Koji?
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Hi,

>  What should I do at this moment as a packager that maintaining some
>  Go packages?
>  Should I fix my packages and build against f31-go in Koji? 
Yes, sure, if you can that would be appreciated. The vast majority of
packages is easy to clean up (just adapt the templates in go-rpm-
templates or use go2rpm), it's just there is an awful lot of them, so
Robert-André Mauchin (eclipseo) can not do all of them in a single
pass.

If you hit one of the cases where a project needs weird stuff, or if
you do not understand something, there is help available here and in
the #fedora-golang channel.

Please just state here (or tell eclispeo directly) what you will
convert, so you do not end up doing the same work in parallel.

Quite often, eclipseo will have prepared things in his copr, just not
built them in koji yet.
https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/

So taking up from where he prepared, and checking the result builds in
koji and works for you, will accelerate things.

@eclipseo: please correct if I wrote something that does not make
things easier your side

Regards,

-- 
Nicolas Mailhot

[Fabio Valentini]

On Sat, Jun 29, 2019 at 10:15 AM Nicolas Mailhot via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  Hi,
>
>  > What should I do at this moment as a packager that maintaining some
>  > Go packages?
>  > Should I fix my packages and build against f31-go in Koji?
>
>  Yes, sure, if you can that would be appreciated. The vast majority of
>  packages is easy to clean up (just adapt the templates in go-rpm-
>  templates or use go2rpm), it's just there is an awful lot of them, so
>  Robert-André Mauchin (eclipseo) can not do all of them in a single
>  pass. 
It's great to see that all this stuff is finally coming together.

I've tried to fix some of my broken packages locally, but I had
trouble getting go2rpm to run from the sources -
because AFAICT, go2rpm isn't packaged for fedora yet. Is there
anything blocking a package or has just nobody worked on it yet?

Fabio

> If you hit one of the cases where a project needs weird stuff, or if
> you do not understand something, there is help available here and in
> the #fedora-golang channel.
>
> Please just state here (or tell eclispeo directly) what you will
> convert, so you do not end up doing the same work in parallel.
>
> Quite often, eclipseo will have prepared things in his copr, just not
> built them in koji yet.
> https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/
>
> So taking up from where he prepared, and checking the result builds in
> koji and works for you, will accelerate things.
>
> @eclipseo: please correct if I wrote something that does not make
> things easier your side
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Robert-André Mauchin]

On Saturday, 29 June 2019 09:26:20 CEST Nicolas Mailhot via devel wrote:
>  Hi,
>  
>  
>  > What should I do at this moment as a packager that maintaining some
>  > Go packages?
>  > Should I fix my packages and build against f31-go in Koji?
>  
>  
>  Yes, sure, if you can that would be appreciated. The vast majority of
>  packages is easy to clean up (just adapt the templates in go-rpm-
>  templates or use go2rpm), it's just there is an awful lot of them, so
>  Robert-André Mauchin (eclipseo) can not do all of them in a single
>  pass.
>  
DO NOT TOUCH ANY PACKAGES BEFORE I FINISH WITH F31-GO.
Otherwise I'll need to rebase my git and it  will be a PITA. All packages are 
being updated anyway.
As for your Deepin packages, Robin, I have updated according to your work on 
COPR.