PSA: builds using forge macros with tags broken on rawhide

[Fabio Valentini]

Just a heads-up for anyone experiencing build failures like me.

Recently the forge macros were ported to use distprefix on f30. An
unrelated change followed and a new build was pushed to rawhide (maybe
without realising that it would also push the forge macro changes).
This change was introduced with redhat-rpm-config-121-1.fc30.

It looks like the forge macros weren't tested to be backwards
compatible, because packages that build successfully now fail to build
on rawhide (see, for example, build [0]).

The reason: The %{gosource} macro now expands to a different string on
rawhide. So, sources uploaded on <f30 won't be found by koji's
buildSRPMfromSCM on rawhide, and vice versa - see the failed SRPM
build [1].

The difference is the inclusion (or lack thereof) of a leading "v"
from the tag in the file name. Up to f29, it will include the leading
"v" in the Source tag (and file download, and source file entry, and
lookaside cache file), and on rawhide, it won't.

!!! This will break koji builds of all packages that use the forge macros
!!! (with a tag, not a commit) in rawhide.
!!! Koschei builds are already starting to fail because of this change.

Since backporting the change won't work due to distprefix not being
available, I think the leading "v" in the version part of the Source
tag has to be reintroduced.

But I won't upload two differently named source archives to the
lookaside cache just for building the same version of a package on
different releases of fedora.

I reported this issue at [2].

Fabio

[0]: https://koji.fedoraproject.org/koji/taskinfo?taskID=30225034
[1]: https://kojipkgs.fedoraproject.org//work/tasks/5035/30225035/build.log
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1639025

[Nicolas Mailhot]

Le dimanche 14 octobre 2018 à 15:37 +0200, Fabio Valentini a écrit :
>  
>  It looks like the forge macros weren't tested to be backwards
>  compatible, because packages that build successfully now fail to build
>  on rawhide (see, for example, build [0]). 
Fabio,

You can’t assume the archive name returned by forge macros will be
stable  over time.

First, because GitHub, GitLab, etc are not themselves stable over time.
They change their URLs and the files those URL generate regularly.
Sometimes they change due to issues filled by Fedora people (as happened
to GitLab recently after Neal Gompa pointed problems in their old URLs).

Second, because at any time they allow access to the same files through
multiple URLs, and the one Fedora chooses to use may change over time,
due to better understanding of the (undocumented) way forge work, or the
old URL not working for some projects, and so on.

The forge macros abstract all this complexity away, so you do not need
to read the wiki and rewrite your spec file every time Fedora a hosting
service changes its rules or Fedora changes its preferred URL pattern.
The drawback being, the resulting archive filename *may* change from the
one in the lookaside cache over time. You can not both adapt
transparently to changes, and not-change filenames impacted by those
changes.

In the specific case you hit the old code required that you set a tag
and the new code will work even without a vversion tag, and recognizes
that the tag you set is really a version tag. So, net win for everyone
except packagers that had to workaround via a tag the fact the older
code was not smart enough.

The macro still computes for your spec a correct archive URL and a
correct archive name. Archive name which BTW is better than the previous
one since it removes the v GitHub loves to inject in random places, and
matches the archive name one would get today in clicking on GitHub’s
download link (no idea if the old URL you used before still works and
even if it did how long GitHub will keep it working now it switched to
newer better API for its own service).

Also, the next version of the forge macro with is being proposed in
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35

will give you more flexibility and isolate archive-specific metadata
block so you can build a download for a tag unrelated to the package
version if that's what you want.

So, the problem you hit is annoying but it's only a spectool away. And
we should probably sync older Fedora releases to the same level of forge
macros once the new redhat-rpm-config gets some use in devel.

Now, if you want to avoid this kind of problem in the future:

— lobby hosting services to commit to and document a stable set of APIs
to retrieve project archives (not just an URL set in stone, an URL that
produces the same filename with the same topdir inside). That's
basically what Neal did for Gitlab and why I have good hopes that GitLab
archive and url naming will be stable now

— lobby git upstream to create a first-class release tag, with the same
syntax for every hosting service and project, so I can kill the huge
pile of heuristics that tries to guess what a specific project will use.
An heuristic as everyone knows is something that works perfectly most of
the time and fails miserably the rest of the time. And it is definitely
*not* stable over time.

Alternatively, you can hardcode an url and archive name in your spec
files, the result will be stable within the spec, but probably not work
anymore whenever you need to bump the project version or commit.

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le dimanche 14 octobre 2018 à 18:13 +0200, Nicolas Mailhot a écrit :
>  Le dimanche 14 octobre 2018 à 15:37 +0200, Fabio Valentini a écrit :
>  > It looks like the forge macros weren't tested to be backwards
>  > compatible, because packages that build successfully now fail to
>  > build
>  > on rawhide (see, for example, build [0]).
>  
>  Fabio,
>  
>  You can’t assume the archive name returned by forge macros will be
>  stable  over time. 
And, BTW, I did lobby the lookaside maintainer last year to stop
considering a source unit is a checksummed archive, but to consider two
archives which content checksums the same as identical (checksum content
not enveloppe, since enveloppes are not stable in our brand new web2
world).

That went nowhere but feel free to explain him the problem better than
me.

-- 
Nicolas Mailhot

[Fabio Valentini]

On Sun, Oct 14, 2018 at 6:14 PM Nicolas Mailhot
<nicolas.mailhot(a)laposte.net&gt; wrote:
>  Le dimanche 14 octobre 2018 à 15:37 +0200, Fabio Valentini a écrit :
>  >
>  > It looks like the forge macros weren't tested to be backwards
>  > compatible, because packages that build successfully now fail to build
>  > on rawhide (see, for example, build [0]).
>
>  Fabio,
>
>  You can’t assume the archive name returned by forge macros will be
>  stable over time. 
Why not? I would surely hope that the file names are stable.
Otherwise packages might just randomly break in rawhide. Oh, wait ...

>  First, because GitHub, GitLab, etc are not themselves stable over
> time.
>  They change their URLs and the files those URL generate regularly.
>  Sometimes they change due to issues filled by Fedora people (as happened
>  to GitLab recently after Neal Gompa pointed problems in their old URLs). 
You know that GitHub has supported the same thing for a long time?
The URL
"https://github.com/project/repo/archive/$REF/whatever-I-feel-like-1.0.tar.gz"
will produce an archive of ref $REF (be it a tag, commit, or branch)
with the file name "whatever-I-feel-like-1.0.tar.gz". The forge macros
just don't make use of that feature, but use the old "#anchor hack",
which not even the SourceURL packaging guidelines mention anymore.

>  Second, because at any time they allow access to the same files
> through
>  multiple URLs, and the one Fedora chooses to use may change over time,
>  due to better understanding of the (undocumented) way forge work, or the
>  old URL not working for some projects, and so on.
>
>  The forge macros abstract all this complexity away, so you do not need
>  to read the wiki and rewrite your spec file every time Fedora a hosting
>  service changes its rules or Fedora changes its preferred URL pattern.
>  The drawback being, the resulting archive filename *may* change from the
>  one in the lookaside cache over time. You can not both adapt
>  transparently to changes, and not-change filenames impacted by those
>  changes. 
Of course you can - if you map "unstable URLs" to the expected file names.
I thought that's what the "abstraction" is for.

>  In the specific case you hit the old code required that you set a
> tag
>  and the new code will work even without a vversion tag, and recognizes
>  that the tag you set is really a version tag. So, net win for everyone
>  except packagers that had to workaround via a tag the fact the older
>  code was not smart enough.
>
>  The macro still computes for your spec a correct archive URL and a
>  correct archive name. Archive name which BTW is better than the previous
>  one since it removes the v GitHub loves to inject in random places, and
>  matches the archive name one would get today in clicking on GitHub’s
>  download link (no idea if the old URL you used before still works and
>  even if it did how long GitHub will keep it working now it switched to
>  newer better API for its own service). 
The URL used for the download is _exactly_ the same as before this
change, just the "#repo-vVersion.tar.gz" anchor is different now than
it was before - so that's entirely in  macros.forge's hands and has
nothing to do with GitHub.

>  Also, the next version of the forge macro with is being proposed in
>  https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
>
>  will give you more flexibility and isolate archive-specific metadata
>  block so you can build a download for a tag unrelated to the package
>  version if that's what you want.
>
>  So, the problem you hit is annoying but it's only a spectool away. And
>  we should probably sync older Fedora releases to the same level of forge
>  macros once the new redhat-rpm-config gets some use in devel. 
It might be easy, but do I want to waste time on this? No.

>  Now, if you want to avoid this kind of problem in the future:
>
>  — lobby hosting services to commit to and document a stable set of APIs
>  to retrieve project archives (not just an URL set in stone, an URL that
>  produces the same filename with the same topdir inside). That's
>  basically what Neal did for Gitlab and why I have good hopes that GitLab
>  archive and url naming will be stable now 
You know that github supports the same thing already, right? (Except
the "v" that's inserted before tags that don't match SemVer.)

>  — lobby git upstream to create a first-class release tag, with the
> same
>  syntax for every hosting service and project, so I can kill the huge
>  pile of heuristics that tries to guess what a specific project will use.
>  An heuristic as everyone knows is something that works perfectly most of
>  the time and fails miserably the rest of the time. And it is definitely
>  *not* stable over time. 
Sure, upstream projects can always do stupid things ... but I don't
think that "stupid" should be the base-line here.

>  Alternatively, you can hardcode an url and archive name in your spec
>  files, the result will be stable within the spec, but probably not work
>  anymore whenever you need to bump the project version or commit. 
I've been doing exactly that for all my ~50 github based projects for
two years now, following this scheme:

for stable releases:
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{version}/%{name}-%{version}.tar.gz
- %{autosetup}

for snapshots (setting commit myself and computing shortcommit automatically):
- URL: https://github.com/project/%{name}
- Source0: %{url}/archive/%{commit}/%{name}-%{shortcommit}.tar.gz
- %autosetup -n %{name}-%{commit}

And this never broke. Not once.

Which is why the SourceURL packaging guidelines recommend using
exactly these schemes.
(They already do so for github for a long time, and I updated them
some time ago for the new and better URLs that gitlab supports now.)

But I've been trying to tell you this when the original forge macros
were introduced, and that has led nowhere either.

So, I'll probably just gradually move back to using the Source URLs
for github/gitlab which are _actually documented_ in the Guidelines
and produce _stable_ results.

Fabio

> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Le dimanche 14 octobre 2018 à 18:56 +0200, Fabio Valentini a écrit :
>  
>  You know that GitHub has supported the same thing for a long time?
>  The URL "
>  https://github.com/project/repo/archive/$REF/whatever-I-feel-like-1.0.tar.gz
>  "
>  will produce an archive of ref $REF (be it a tag, commit, or branch)
>  with the file name "whatever-I-feel-like-1.0.tar.gz". The forge macros
>  just don't make use of that feature,  
Actually, they do, that's the exact change you're complaining about
here.

And whatever-I-feel-like-1.0.tar.gz is useless because the topdir inside
whatever-I-feel-like-1.0.tar.gz is definitively not whatever-I-feel-
like-1.0, so you need to compute the actual topdir GitHub will generate
and while you're at it the filename better match it or much %setup
hilarity and packager unhappiness will ensue.

And besides the code handle other things that GiHub that have changed
behaviour over time.

>  but use the old "#anchor hack",
>  which not even the SourceURL packaging guidelines mention anymore. 
When the code was written it matched the guidelines whenever they
produced working URLs (some guidelines didn’t). The guidelines and the
code have been improved and fixed since. One reason the guidelines have
been fixed is that the forge macro showed they were erroneous.

>  You know that github supports the same thing already, right? (Except
>  the "v" that's inserted before tags that don't match SemVer.) 
Ha ha ha. “except” v. And v does not have any simple rule, it is GiHub
heuristic pile land, you have projects with some of their releases that
uses v and others not, and GitHub can not even make up its mind in how
to use it consistently for the same project release (it's in the url but
not in the archive). I so wish GitHub would just give up on the whole v
thing. Or do it all the time. But not sometimes yes, other times no.

Since Google uses v in googlecode, and other hosting services do not,
and GitHub tries to mirror everyone, their setup is doomed to fail as
long as they do not standardise release tags git-upstream-side. By they
still try to heuristic the problem away.

>  > — lobby git upstream to create a first-class release tag, with
> the
>  > same
>  > syntax for every hosting service and project, so I can kill the huge
>  > pile of heuristics that tries to guess what a specific project will
>  > use.
>  > An heuristic as everyone knows is something that works perfectly
>  > most of
>  > the time and fails miserably the rest of the time. And it is
>  > definitely
>  > *not* stable over time.
>  
>  Sure, upstream projects can always do stupid things ... but I don't
>  think that "stupid" should be the base-line here.
>  
>  > Alternatively, you can hardcode an url and archive name in your spec
>  > files, the result will be stable within the spec, but probably not
>  > work
>  > anymore whenever you need to bump the project version or commit.
>  
>  I've been doing exactly that for all my ~50 github based projects for
>  two years now, following this scheme:
>  
>  for stable releases:
>  - URL: https://github.com/project/%{name}
>  - Source0: %{url}/archive/%{version}/%{name}-%{version}.tar.gz
>  - %{autosetup}
>  
>  for snapshots (setting commit myself and computing shortcommit
>  automatically):
>  - URL: https://github.com/project/%{name}
>  - Source0: %{url}/archive/%{commit}/%{name}-%{shortcommit}.tar.gz
>  - %autosetup -n %{name}-%{commit}
>  
>  And this never broke. Not once.
>  
>  Which is why the SourceURL packaging guidelines recommend using
>  exactly these schemes.
>  (They already do so for github for a long time, and I updated them
>  some time ago for the new and better URLs that gitlab supports now.)
>  
>  But I've been trying to tell you this when the original forge macros
>  were introduced, and that has led nowhere either.
>  
>  So, I'll probably just gradually move back to using the Source URLs
>  for github/gitlab which are _actually documented_ in the Guidelines
>  and produce _stable_ results.
>  
>  Fabio
>  
>  > Regards,
>  > 
>  > --
>  > Nicolas Mailhot
>  > _______________________________________________
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  > List Guidelines: 
>  > https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives: 
>  > https://lists.fedoraproject.org/archives/list/[e-mail]
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: 
>  https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives: 
>  https://lists.fedoraproject.org/archives/list/[e-mail] 
-- 
Nicolas Mailhot

[Fabio Valentini]

On Sun, Oct 14, 2018 at 7:21 PM Nicolas Mailhot
<nicolas.mailhot(a)laposte.net&gt; wrote:
>  Le dimanche 14 octobre 2018 à 18:56 +0200, Fabio Valentini a écrit :
>  >
>  > You know that GitHub has supported the same thing for a long time?
>  > The URL "
>  > https://github.com/project/repo/archive/$REF/whatever-I-feel-like-1.0.tar.gz
>  > "
>  > will produce an archive of ref $REF (be it a tag, commit, or branch)
>  > with the file name "whatever-I-feel-like-1.0.tar.gz". The forge macros
>  > just don't make use of that feature,
>
>  Actually, they do, that's the exact change you're complaining about
>  here.
>
>  And whatever-I-feel-like-1.0.tar.gz is useless because the topdir inside
>  whatever-I-feel-like-1.0.tar.gz is definitively not whatever-I-feel-
>  like-1.0, so you need to compute the actual topdir GitHub will generate
>  and while you're at it the filename better match it or much %setup
>  hilarity and packager unhappiness will ensue.
>
>  And besides the code handle other things that GiHub that have changed
>  behaviour over time.
>
>  > but use the old "#anchor hack",
>  > which not even the SourceURL packaging guidelines mention anymore.
>
>  When the code was written it matched the guidelines whenever they
>  produced working URLs (some guidelines didn’t). The guidelines and the
>  code have been improved and fixed since. One reason the guidelines have
>  been fixed is that the forge macro showed they were erroneous.
>
>  > You know that github supports the same thing already, right? (Except
>  > the "v" that's inserted before tags that don't match SemVer.)
>
>  Ha ha ha. “except” v. And v does not have any simple rule, it is GiHub
>  heuristic pile land, you have projects with some of their releases that
>  uses v and others not, and GitHub can not even make up its mind in how
>  to use it consistently for the same project release (it's in the url but
>  not in the archive). I so wish GitHub would just give up on the whole v
>  thing. Or do it all the time. But not sometimes yes, other times no.
>
>  Since Google uses v in googlecode, and other hosting services do not,
>  and GitHub tries to mirror everyone, their setup is doomed to fail as
>  long as they do not standardise release tags git-upstream-side. By they
>  still try to heuristic the problem away. 
Ok. Let me backtrack. Since I don't assume that you broke a pile of
packages on purpose, let's focus on how to fix this.

Because right now, package builds prepared on fedora 27-29 will result
in failing koji builds for rawhide - and nobody should have to install
rawhide to be able to build packages.

Fabio

> > > — lobby git upstream to create a first-class release tag, with the
> > > same
> > > syntax for every hosting service and project, so I can kill the huge
> > > pile of heuristics that tries to guess what a specific project will
> > > use.
> > > An heuristic as everyone knows is something that works perfectly
> > > most of
> > > the time and fails miserably the rest of the time. And it is
> > > definitely
> > > *not* stable over time.
> >
> > Sure, upstream projects can always do stupid things ... but I don't
> > think that "stupid" should be the base-line here.
> >
> > > Alternatively, you can hardcode an url and archive name in your spec
> > > files, the result will be stable within the spec, but probably not
> > > work
> > > anymore whenever you need to bump the project version or commit.
> >
> > I've been doing exactly that for all my ~50 github based projects for
> > two years now, following this scheme:
> >
> > for stable releases:
> > - URL: https://github.com/project/%{name}
> > - Source0: %{url}/archive/%{version}/%{name}-%{version}.tar.gz
> > - %{autosetup}
> >
> > for snapshots (setting commit myself and computing shortcommit
> > automatically):
> > - URL: https://github.com/project/%{name}
> > - Source0: %{url}/archive/%{commit}/%{name}-%{shortcommit}.tar.gz
> > - %autosetup -n %{name}-%{commit}
> >
> > And this never broke. Not once.
> >
> > Which is why the SourceURL packaging guidelines recommend using
> > exactly these schemes.
> > (They already do so for github for a long time, and I updated them
> > some time ago for the new and better URLs that gitlab supports now.)
> >
> > But I've been trying to tell you this when the original forge macros
> > were introduced, and that has led nowhere either.
> >
> > So, I'll probably just gradually move back to using the Source URLs
> > for github/gitlab which are _actually documented_ in the Guidelines
> > and produce _stable_ results.
> >
> > Fabio
> >
> > > Regards,
> > >
> > > --
> > > Nicolas Mailhot
> > > _______________________________________________
> > > devel mailing list -- devel(a)lists.fedoraproject.org
> > > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > > List Guidelines:
> > > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> > >
https://lists.fedoraproject.org/archives/list/[e-mail]
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines:
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Le 2018-10-15 09:09, Fabio Valentini a écrit :

>  Because right now, package builds prepared on fedora 27-29 will
> result
>  in failing koji builds for rawhide - and nobody should have to install
>  rawhide to be able to build packages. 
Well basically you can
1. use a rawhide vm on container of whatever to prepare your rawhide 
srpms (but, as you noted, not cheap to setup)
2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory 
that works – not tested)
3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide 
behaviour (only takes changing dist definition in 
/etc/macros.d/macros.dist to %{?distprefix}.fcxx
4. use a local backport of the code. You basicaly just need to insert 
the following at line 251 or 252 of the macros.forge rawhide file
   -- Workaround releases where distprefix is not used by default
   local dist = rpm.expand("%{?dist}")
   local edistprefix = rpm.expand(distprefix)
   if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~= 
edistprefix) then
     rpmmacros.explicitset("dist", "%{?distprefix}" .. dist,verbose)
   end
5. ask to accelerate the backport to stable. I can prepare the backport 
PR, but applying the PR is out of my hands
6. ask redhat-rpm-config maintainers to process 
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35 
since I have a backport already prepared and tested for this one
7. all of the above

I don’t know which solution best matches your workflow

And normally, you do stuff in and for rawhide first, and then backport, 
but this is kind of a special case since the rawhide bit does not 
usually extend to the srpm part, and Go packaging in Fedora is so 
immature every release is pretty much a devel release from Go's POW, so 
I understand where you're coming from.


Regards,

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot"
> <nicolas.mailhot(a)laposte.net&gt;
>  To: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Cc: "Fabio Valentini" <nicolas.mailhot(a)laposte.net&gt;
>  Sent: Monday, October 15, 2018 11:56:45 AM
>  Subject: Re: PSA: builds using forge macros with tags broken on rawhide
>  
>  Le 2018-10-15 09:09, Fabio Valentini a écrit :
>  
>  > Because right now, package builds prepared on fedora 27-29 will result
>  > in failing koji builds for rawhide - and nobody should have to install
>  > rawhide to be able to build packages.
>  
>  Well basically you can
>  1. use a rawhide vm on container of whatever to prepare your rawhide
>  srpms (but, as you noted, not cheap to setup)
>  2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory
>  that works – not tested)
>  3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide
>  behaviour (only takes changing dist definition in
>  /etc/macros.d/macros.dist to %{?distprefix}.fcxx
>  4. use a local backport of the code. You basicaly just need to insert
>  the following at line 251 or 252 of the macros.forge rawhide file
>     -- Workaround releases where distprefix is not used by default
>     local dist = rpm.expand("%{?dist}")
>     local edistprefix = rpm.expand(distprefix)
>     if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~=
>  edistprefix) then
>       rpmmacros.explicitset("dist", "%{?distprefix}" .. dist,verbose)
>     end
>  5. ask to accelerate the backport to stable. I can prepare the backport
>  PR, but applying the PR is out of my hands
>  6. ask redhat-rpm-config maintainers to process
>  https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
>  since I have a backport already prepared and tested for this one
>  7. all of the above 
And what about backing up the breaking change in Rawhide? At least until there is a
backward compatible way of doing that(or it is backported in to the stable releases)?

To be honest we should not introduce deliberately backwards incompatible changes without
at least publicly announcing them way ahead(and in general avoid them). This creates
really bad experience for all and waste lot of time of the packagers.

JC

> 
> I don’t know which solution best matches your workflow
> 
> And normally, you do stuff in and for rawhide first, and then backport,
> but this is kind of a special case since the rawhide bit does not
> usually extend to the srpm part, and Go packaging in Fedora is so
> immature every release is pretty much a devel release from Go's POW, so
> I understand where you're coming from.
> 
> 
> Regards,
> 
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Nicolas Mailhot]

Le vendredi 19 octobre 2018 à 04:58 -0400, Jakub Cajka a écrit :
>  
>  And what about backing up the breaking change in Rawhide? At least
>  until there is a backward compatible way of doing that(or it is
>  backported in to the stable releases)? 
We're talking about redhat-rpm-config there. No one wants to backport
redhat-rpm-config changes to stable directly, without some proof period
in devel first (and that decision is pretty much in FPC’s hands anyway).

The “fastest” path I see is several people telling FPC the change is ok
now and the backport should not be delayed any more. And maybe we are
all being overly cautious and the caution delay does more harm than good
.

I will prepare a backport PR.

Whether it's applied or not (and when) is something else. The change
that hit devel recently was queued in the review pipe two months ago.

And there's also PR35, that will add further changes, and has been
cooking quite a lot of time already. Though it will be mostly new
features, now that the forge URL fixes have hit devel.

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le vendredi 19 octobre 2018 à 11:48 +0200, Nicolas Mailhot a écrit :
>  Le vendredi 19 octobre 2018 à 04:58 -0400, Jakub Cajka a écrit :
>  > And what about backing up the breaking change in Rawhide? At least
>  > until there is a backward compatible way of doing that(or it is
>  > backported in to the stable releases)?
>  
>  We're talking about redhat-rpm-config there. No one wants to backport
>  redhat-rpm-config changes to stable directly, without some proof
>  period
>  in devel first (and that decision is pretty much in FPC’s hands
>  anyway).
>  
>  The “fastest” path I see is several people telling FPC the change is
>  ok
>  now and the backport should not be delayed any more. And maybe we are
>  all being overly cautious and the caution delay does more harm than
>  good
>  
>  I will prepare a backport PR. 
Anyway are the backport PRs

https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/44
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/43
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/42

Regards,

-- 
Nicolas Mailhot

[Fabio Valentini]

On Fri, Oct 19, 2018, 10:58 Jakub Cajka <jcajka(a)redhat.com&gt; wrote:

>  ----- Original Message -----
>  > From: "Nicolas Mailhot" <nicolas.mailhot(a)laposte.net&gt;
>  > To: "Development discussions related to Fedora" <
>  devel(a)lists.fedoraproject.org&gt;
>  > Cc: "Fabio Valentini" <nicolas.mailhot(a)laposte.net&gt;
>  > Sent: Monday, October 15, 2018 11:56:45 AM
>  > Subject: Re: PSA: builds using forge macros with tags broken on rawhide
>  >
>  > Le 2018-10-15 09:09, Fabio Valentini a écrit :
>  >
>  > > Because right now, package builds prepared on fedora 27-29 will result
>  > > in failing koji builds for rawhide - and nobody should have to install
>  > > rawhide to be able to build packages.
>  >
>  > Well basically you can
>  > 1. use a rawhide vm on container of whatever to prepare your rawhide
>  > srpms (but, as you noted, not cheap to setup)
>  > 2. use mock --shell to get a cheap rawhide buildroot srpm env (in theory
>  > that works – not tested)
>  > 3. use a local rebuild of the rawhide redhat-rpm-config to match rawhide
>  > behaviour (only takes changing dist definition in
>  > /etc/macros.d/macros.dist to %{?distprefix}.fcxx
>  > 4. use a local backport of the code. You basicaly just need to insert
>  > the following at line 251 or 252 of the macros.forge rawhide file
>  >    -- Workaround releases where distprefix is not used by default
>  >    local dist = rpm.expand("%{?dist}")
>  >    local edistprefix = rpm.expand(distprefix)
>  >    if (edistprefix ~= "") and (string.sub(dist, 1, #edistprefix) ~=
>  > edistprefix) then
>  >      rpmmacros.explicitset("dist", "%{?distprefix}" ..
> dist,verbose)
>  >    end
>  > 5. ask to accelerate the backport to stable. I can prepare the backport
>  > PR, but applying the PR is out of my hands
>  > 6. ask redhat-rpm-config maintainers to process
>  > https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/35
>  > since I have a backport already prepared and tested for this one
>  > 7. all of the above
>
>  And what about backing up the breaking change in Rawhide? At least until
>  there is a backward compatible way of doing that(or it is backported in to
>  the stable releases)?
>
>  To be honest we should not introduce deliberately backwards incompatible
>  changes without at least publicly announcing them way ahead(and in general
>  avoid them). This creates really bad experience for all and waste lot of
>  time of the packagers.
>  
By the way, with the latest update to redhat-rpm-config in rawhide
(122-1.fc30), *all* packages using the new go macros are now broken. With
that, I now have about 40 broken golang packages.

I have neither the time nor the patience to fix them.

Fabio


>  JC
>
>  >
>  > I don’t know which solution best matches your workflow
>  >
>  > And normally, you do stuff in and for rawhide first, and then backport,
>  > but this is kind of a special case since the rawhide bit does not
>  > usually extend to the srpm part, and Go packaging in Fedora is so
>  > immature every release is pretty much a devel release from Go's POW, so
>  > I understand where you're coming from.
>  >
>  >
>  > Regards,
>  >
>  > --
>  > Nicolas Mailhot
>  > _______________________________________________
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
>  >
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  >
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Robert-André Mauchin]

Yeah all "gosetup -q" (which is gofed default) are broken because of your
commit:

294   # Convenience macro to relay computed arguments to %setup
295 - %forgesetup(a:b:cDn:Tq) %setup %{?forgesetupargs} %{-a} %{-b} %{-c} %{-D} %{-n}
%{-T} %{-q}
296 + # Optional parameters:
297 + #   -a          process all sources in one go, instead of using separate -z calls
298 + #   -z <number> read %{?forgesetupargs<number>}
299 + #   -v          be verbose
300 + %forgesetup(az:v) %{lua:
301 + local fedora = require "fedora.common"
302 + if (rpm.expand("%{-z}") == "") and
(rpm.expand("%{-a}") ~= "") then
303 +   for _,s in pairs(fedora.getsuffixes("forgesetupargs")) do
304 +     print(rpm.expand("%setup %{!-v:-q} %{?forgesetupargs" .. s            
        .. "}\\n"))
305 +   end
306 + else
307 +   print(  rpm.expand("%setup %{!-v:-q} %{?forgesetupargs" ..
rpm.expand("%{-z*}") .. "}\\n"))
308 + end
309 + }
310   
311   # Convenience macro to relay computed arguments to %autosetup
312 - %forgeautosetup(a:b:cDn:TvNS:p:) %autosetup %{?forgesetupargs} %{-a} %{-b} %{-c}
%{-D} %{-n} %{-T} %{-v} %{-N} %{-S} %{-p}
313 + # Parameters relayed to %autosetup: -v -N -S -p
314 + # Optional parameters:
315 + #   -z <number> read %{?forgesetupargs<number>}
316 + %forgeautosetup(z:vNS:p:) %{lua:
317 + print(rpm.expand("%autosetup %{-v} %{-N} %{?-S} %{?-p} %{?forgesetupargs"
.. rpm.expand("%{-z*}") .. "}\\n"))
318 + }
319 + 
320 + # List files matching inclusion globs, excluding files matching exclusion blogs
321 + # Parameters:
322 + #  -i "<globs>" include shell globs (also takes all other macro
arguments)
323 + #  -x "<globs>" exclude shell globs
324 + %listfiles(i:x:) %{expand:
325 + while IFS= read -r -d $'\\n' finc ; do
326 +   printf "%s\\n" %{?-x*} \\
327 +     | xargs -i realpath --relative-base=. '{}' \\
328 +     | grep "${finc}" >/dev/null || echo "${finc}"
329 + done <<< $(printf "%s\\n" %{?-i*} %* | xargs -i realpath
--relative-base=. '{}' | sort -u)
330 + }

Please revert this, we should be able to use whatever flag supported by %setup and
 %autosetup, not a small subset.  How do you even pass the basic -n flags now?

With you mods, we have to edit hundreds of specs to remove 
the -q flags. So much for keeping the API stable.

[Nicolas Mailhot]

Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit :
>  Yeah all "gosetup -q" (which is gofed default) are broken
> because of
>  your commit: 
Well I know no such a thing, there was never any gosetup macro in the
macro set, and I think I told you a year ago I would not define a
gosetup macro just to avoid typing forgesetup, unless it actually added
some processing over the forgesetup macro. That would obfuscate specs
for no good reason and increase gratuituously the maintenance surface
(as we see *now*).

And I doubt -q is your problem, since (*precisely for backwards
compatible reasons) it is still accepted by forgesetup (even though it's
ignored, because it’s the default behaviour now).

Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
was already its default behaviour. So you're forcing somewhere a -q that
I don't think was ever needed. I will add the -q to the macro definition
if that makes you feel better.

So, no biggie. Easy to fix. That's why such changes hit -devel before
anyone dreams of queuing them to stable.

What package exactly installs your gosetup macro in /usr/lib/rpm so I
can see what other things it tries to do? I see no macro file in the
Fedora gofed package manifests.

If you could PR your changes to the official Fedora packages that
coordinate Fedora go macros (the go-macros repository on github, that
will be rehosted in pagure as soon as the @rh contributors ok the move)
instead of doing it in other places, that would be a tad easier to
coordinate.

>  Please revert this, we should be able to use whatever flag supported
>  by %setup and
>   %autosetup, not a small subset.  How do you even pass the basic -n
>  flags now? 
So basically, you have a macro, which sole purpose is to pass
precomputed -n values to *setup, and you want to use it with manual -n
flags. Why? Could you tell us what exactly is the point of

1. wrapping a Fedora macro in another name just because you do not like
the official macro name in Fedora
2. overriding the only thing this macro does over autosetup
3. and then complaining all the argument passing to autosetup does not
work as you wish it does

So all this complexity, because what you really want is to use autosetup
directly. Then just do. What's the problem exactly with typing autosetup
in your spec? No one stops you from doing it.

One reason I removed the -n in forgeautosetup and forgesetup precisely
to stop packagers confusing themselves the way you are confusing
yourself.

The other being -n is incompatible with the processing of multiple
archives that many people asked me to add to the macros. Which is
finally implemented after months of work. And which just hit rawhide
after more than a month of review.

So, do you have actual spec files in Fedora with this use of the redhat-
rpm-config macros? If that is the case, I can put those flags in the
macros so you can continue shooting yourself in the foot using undefined
known-broken use patterns. If not, I'd rather remove the possibility
altogether before someone harms himself.

>  With you mods, we have to edit hundreds of specs to remove 
>  the -q flags. 
And that concretely, if why pre-generating specs instead of making the
effort to include default processing in the macro code Fedora ships is
wrong.

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le lundi 22 octobre 2018 à 10:00 +0200, Nicolas Mailhot a écrit :
>  Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
> :
>  > Yeah all "gosetup -q" (which is gofed default) are broken because of
>  > your commit:
>  
>  Well I know no such a thing, there was never any gosetup macro in the
>  macro set, and I think I told you a year ago I would not define a
>  gosetup macro just to avoid typing forgesetup, unless it actually
>  added
>  some processing over the forgesetup macro. That would obfuscate specs
>  for no good reason and increase gratuituously the maintenance surface
>  (as we see *now*).
>  
>  And I doubt -q is your problem, since (*precisely for backwards
>  compatible reasons) it is still accepted by forgesetup (even though
>  it's
>  ignored, because it’s the default behaviour now).
>  
>  Oh, I see, I forgot to add a phantom -q to forgeautosetup as it
>  already
>  was already its default behaviour. So you're forcing somewhere a -q
>  that
>  I don't think was ever needed. I will add the -q to the macro
>  definition
>  if that makes you feel better.
>
>  So, no biggie. Easy to fix. That's why such changes hit -devel before
>  anyone dreams of queuing them to stable. 
And it's here:

https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/45



-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le lundi 22 octobre 2018 à 10:00 +0200, Nicolas Mailhot a écrit :
>  Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
> :
>  > Yeah all "gosetup -q" (which is gofed default) are broken because of
>  > your commit:
>  
>  Well I know no such a thing, there was never any gosetup macro in the
>  macro set, and I think I told you a year ago I would not define a
>  gosetup macro just to avoid typing forgesetup, unless it actually
>  added
>  some processing over the forgesetup macro. That would obfuscate specs
>  for no good reason and increase gratuituously the maintenance surface
>  (as we see *now*). 
However, you *can* PR your own definition of gosetup to the Fedora go
macros package if you absolutely want it to exist. I will be happy to
support the PR so this macro has a maintainer in Fedora.

I'm not the Fedora maintainer of this package (even though, at this
point, I wrote most of it in line numbers). But I will be happy to
support your request.

Regards,
-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot"
> <nicolas.mailhot(a)laposte.net&gt;
>  To: "Robert-André Mauchin" <nicolas.mailhot(a)laposte.net&gt;,
> "Development discussions related to Fedora"
>  <devel(a)lists.fedoraproject.org&gt;
>  Cc: "Nicolas Mailhot" <nicolas.mailhot(a)laposte.net&gt;
>  Sent: Monday, October 22, 2018 10:00:40 AM
>  Subject: Re: PSA: builds using forge macros with tags broken on rawhide
>  
>  Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit :
>  > Yeah all "gosetup -q" (which is gofed default) are broken because of
>  > your commit:
>  
>  Well I know no such a thing, there was never any gosetup macro in the
>  macro set, and I think I told you a year ago I would not define a
>  gosetup macro just to avoid typing forgesetup, unless it actually added
>  some processing over the forgesetup macro. That would obfuscate specs
>  for no good reason and increase gratuituously the maintenance surface
>  (as we see *now*).
>  
>  And I doubt -q is your problem, since (*precisely for backwards
>  compatible reasons) it is still accepted by forgesetup (even though it's
>  ignored, because it’s the default behaviour now).
>  
>  Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
>  was already its default behaviour. So you're forcing somewhere a -q that
>  I don't think was ever needed. I will add the -q to the macro definition
>  if that makes you feel better.
>  
>  So, no biggie. Easy to fix. That's why such changes hit -devel before
>  anyone dreams of queuing them to stable. 
I think that it would be reasonable to test the changes against the Fedora package base
before even pushing the change in to the rawhide. This kind of unnecessary breakage is
easily avoidable if you would spent sometime on testing this by scratch rebuilding the
Fedora packages locally, prior pushing this.

Could we avoid any such kind of regressions in the future?

JC

> 
> What package exactly installs your gosetup macro in /usr/lib/rpm so I
> can see what other things it tries to do? I see no macro file in the
> Fedora gofed package manifests.
> 
> If you could PR your changes to the official Fedora packages that
> coordinate Fedora go macros (the go-macros repository on github, that
> will be rehosted in pagure as soon as the @rh contributors ok the move)
> instead of doing it in other places, that would be a tad easier to
> coordinate.
> 
> > Please revert this, we should be able to use whatever flag supported
> > by %setup and
> >  %autosetup, not a small subset.  How do you even pass the basic -n
> > flags now?
> 
> So basically, you have a macro, which sole purpose is to pass
> precomputed -n values to *setup, and you want to use it with manual -n
> flags. Why? Could you tell us what exactly is the point of
> 
> 1. wrapping a Fedora macro in another name just because you do not like
> the official macro name in Fedora
> 2. overriding the only thing this macro does over autosetup
> 3. and then complaining all the argument passing to autosetup does not
> work as you wish it does
> 
> So all this complexity, because what you really want is to use autosetup
> directly. Then just do. What's the problem exactly with typing autosetup
> in your spec? No one stops you from doing it.
> 
> One reason I removed the -n in forgeautosetup and forgesetup precisely
> to stop packagers confusing themselves the way you are confusing
> yourself.
> 
> The other being -n is incompatible with the processing of multiple
> archives that many people asked me to add to the macros. Which is
> finally implemented after months of work. And which just hit rawhide
> after more than a month of review.
> 
> So, do you have actual spec files in Fedora with this use of the redhat-
> rpm-config macros? If that is the case, I can put those flags in the
> macros so you can continue shooting yourself in the foot using undefined
> known-broken use patterns. If not, I'd rather remove the possibility
> altogether before someone harms himself.
> 
> > With you mods, we have to edit hundreds of specs to remove
> > the -q flags.
> 
> And that concretely, if why pre-generating specs instead of making the
> effort to include default processing in the macro code Fedora ships is
> wrong.
> 
> Regards,
> 
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Nicolas Mailhot]

Le 2018-10-22 13:09, Jakub Cajka a écrit :

>  I think that it would be reasonable to test the changes against the
>  Fedora package base before even pushing the change in to the rawhide.
>  This kind of unnecessary breakage is easily avoidable if you would
>  spent sometime on testing this by scratch rebuilding the Fedora
>  packages locally, prior pushing this. 
There's certainly many things that could be done better if someone took 
the time to do it. But, everyone's time is limited including mine. I 
already rebuild hundreds of Go packages regularly to QA the changes. It 
takes days. Those 40 packages were not part of the set.

So, if we want more things to be done with the existing manpower, we 
need to fix all the thousands of little things that waste the available 
contributor time:
  * rehost things cleanly on pagure
  * clean up the maze of repurposed Go infra specs so any change or fix 
can be QAed and pushed quickly
  * contribute the fixes we want or need to the Fedora packages that ship 
the corresponding code, instead of 'saving' time by doing messy 
ill-thought workarounds in a private spec file generator

And then when things are clearly in one place this place can be used to 
sync the reviews, QA runs and Koji/copr mass rebuilds. Which is clearly 
not the case today.

Call that purism if you like

-- 
Nicolas Mailhot

[Jakub Cajka]

----- Original Message -----
>  From: "Nicolas Mailhot"
> <nicolas.mailhot(a)laposte.net&gt;
>  To: "Jakub Cajka" <jcajka(a)redhat.com&gt;
>  Cc: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Monday, October 22, 2018 1:55:02 PM
>  Subject: Re: PSA: builds using forge macros with tags broken on rawhide
>  
>  Le 2018-10-22 13:09, Jakub Cajka a écrit :
>  
>  > I think that it would be reasonable to test the changes against the
>  > Fedora package base before even pushing the change in to the rawhide.
>  > This kind of unnecessary breakage is easily avoidable if you would
>  > spent sometime on testing this by scratch rebuilding the Fedora
>  > packages locally, prior pushing this.
>  
>  There's certainly many things that could be done better if someone took
>  the time to do it. But, everyone's time is limited including mine. I
>  already rebuild hundreds of Go packages regularly to QA the changes. It
>  takes days. Those 40 packages were not part of the set. 
In my experience this doesn't require that much time as this is fully scriptable from
getting all the dependent(Go based packages) and rebuilding them locally(even using
something dumb as mockchain, it takes under 1 day to complete for Go, without needing any
babysitting(in VM on i7/ssd laptop)). One day(of computation power) seems like reasonable
trade-off for not wasting time of significant number of (Go) packagers by breaking macros.
Or AFAIK it should be possible to do such change in the koji side tag(ideally with formal
change proposal) not affecting the main tag.

>  
>  So, if we want more things to be done with the existing manpower, we
>  need to fix all the thousands of little things that waste the available
>  contributor time:
>    * rehost things cleanly on pagure
>    * clean up the maze of repurposed Go infra specs so any change or fix
>  can be QAed and pushed quickly
>    * contribute the fixes we want or need to the Fedora packages that ship
>  the corresponding code, instead of 'saving' time by doing messy
>  ill-thought workarounds in a private spec file generator 
I'm sure that we all want to advance Fedora, but could you be more specific? What
needs to be done in your opinion? Would you mind opening issues in Go SIG for those
specific changes(or better PRs in respective packages)?

JC

> 
> And then when things are clearly in one place this place can be used to
> sync the reviews, QA runs and Koji/copr mass rebuilds. Which is clearly
> not the case today.
> 
> Call that purism if you like
> 
> --
> Nicolas Mailhot
>

[Jan Chaloupka]

Nicolas,


On 10/22/2018 10:00 AM, Nicolas Mailhot wrote:
>  Le lundi 22 octobre 2018 à 00:31 +0200, Robert-André Mauchin a écrit
> :
> > Yeah all "gosetup -q" (which is gofed default) are broken because of
> > your commit:
>  Well I know no such a thing, there was never any gosetup macro in the
>  macro set, and I think I told you a year ago I would not define a
>  gosetup macro just to avoid typing forgesetup, unless it actually added
>  some processing over the forgesetup macro. That would obfuscate specs
>  for no good reason and increase gratuituously the maintenance surface
>  (as we see *now*). 
Yet, you were aware of that. And, it was me who told you I am fine with 
most of the go macros you implemented, including their names. We both 
spent many hours improving the implementation, extending golist binary I 
provided. And the %gosetup was the only macro I wanted to keep so all 
the regular go macros are in %goVERB form. We have %gometa, %gosetup, 
%gobuild, %goinstall, %gotest on purpose so they reflect:

- meta: after some effort even meta can be interpreted as a verb, i.e. 
meta the macros
- setup: extract the tarball and prepare working dir
- build: set gopath, build go binaries
- install: install resources under proper directories
- test: test installed go packages

Do you ignore that fact on purpose? Since after discussion we had so 
many times you still seem to put more emphasis on your own and only your 
opinion. I though we made a compromise and that we agreed on that. I.e. 
I will respect all the macro names you choose up to the only one and you 
will respect the gosetup one. After having merged your changes you are 
sending me a message that even if we come to a common agreement you 
still don't care as long as you reach your own goals. I would really 
like to think I am wrong in this opinion. Though, you actions say otherwise.

>  And I doubt -q is your problem, since (*precisely for backwards
>  compatible reasons) it is still accepted by forgesetup (even though it's
>  ignored, because it’s the default behaviour now).
>
>  Oh, I see, I forgot to add a phantom -q to forgeautosetup as it already
>  was already its default behaviour. So you're forcing somewhere a -q that
>  I don't think was ever needed. I will add the -q to the macro definition
>  if that makes you feel better. 
You were the one that implemented the forge macros (pardon me if I am 
wrong, I am unsure if it was all of it of most of it). And I love those 
macros since they save so much time and make the spec file a lot 
simpler. So its natural to be protective if anyone wants to change them 
they way that does not align with your world view. Yet, go macros 
started to rely on the forge macros and the change you made is not 
backward compatible (given we started using the macros in older versions 
of Fedora as well). So please, make the %forgeautosetup compatible with 
the %gosetup again.

Once the %gosetup macro is functional again, we can discuss its removal 
in favor of forgesetup as part of global announcement so all go 
packagers have time to migrate to new macros.

For future reference, please don't change the forge macros in any way 
that makes the go macros incompatible or broken until we have official 
Go packaging guidelines and set of macros the go-sig community agrees on.

Thank you Nicolas


>  So, no biggie. Easy to fix. That's why such changes hit -devel
> before
>  anyone dreams of queuing them to stable.
>
>  What package exactly installs your gosetup macro in /usr/lib/rpm so I
>  can see what other things it tries to do? I see no macro file in the
>  Fedora gofed package manifests.
>
>  If you could PR your changes to the official Fedora packages that
>  coordinate Fedora go macros (the go-macros repository on github, that
>  will be rehosted in pagure as soon as the @rh contributors ok the move)
>  instead of doing it in other places, that would be a tad easier to
>  coordinate.
>
> > Please revert this, we should be able to use whatever flag supported
> > by %setup and
> >   %autosetup, not a small subset.  How do you even pass the basic -n
> > flags now?
>  So basically, you have a macro, which sole purpose is to pass
>  precomputed -n values to *setup, and you want to use it with manual -n
>  flags. Why? Could you tell us what exactly is the point of
>
>  1. wrapping a Fedora macro in another name just because you do not like
>  the official macro name in Fedora
>  2. overriding the only thing this macro does over autosetup
>  3. and then complaining all the argument passing to autosetup does not
>  work as you wish it does
>
>  So all this complexity, because what you really want is to use autosetup
>  directly. Then just do. What's the problem exactly with typing autosetup
>  in your spec? No one stops you from doing it.
>
>  One reason I removed the -n in forgeautosetup and forgesetup precisely
>  to stop packagers confusing themselves the way you are confusing
>  yourself.
>
>  The other being -n is incompatible with the processing of multiple
>  archives that many people asked me to add to the macros. Which is
>  finally implemented after months of work. And which just hit rawhide
>  after more than a month of review.
>
>  So, do you have actual spec files in Fedora with this use of the redhat-
>  rpm-config macros? If that is the case, I can put those flags in the
>  macros so you can continue shooting yourself in the foot using undefined
>  known-broken use patterns. If not, I'd rather remove the possibility
>  altogether before someone harms himself.
>
> > With you mods, we have to edit hundreds of specs to remove
> > the -q flags.
>  And that concretely, if why pre-generating specs instead of making the
>  effort to include default processing in the macro code Fedora ships is
>  wrong.
>
>  Regards,
>

[Nicolas Mailhot]

Le 2018-10-23 12:59, Jan Chaloupka a écrit :

Jan

>  So please, make the %forgeautosetup
>  compatible with the %gosetup again.
>  
>  Once the %gosetup macro is functional again, we can discuss its
>  removal in favor of forgesetup as part of global announcement so all
>  go packagers have time to migrate to new macros.
>  
>  For future reference, please don't change the forge macros in any way
>  that makes the go macros incompatible or broken until we have official
>  Go packaging guidelines and set of macros the go-sig community agrees
>  on.
>  
>  Thank you Nicolas
>  
>  
> > So, no biggie. Easy to fix. That's why such changes hit -devel before
> > anyone dreams of queuing them to stable.
> > 
> > What package exactly installs your gosetup macro in /usr/lib/rpm so I
> > can see what other things it tries to do? I see no macro file in the
> > Fedora gofed package manifests.
> > 
> > If you could PR your changes to the official Fedora packages that
> > coordinate Fedora go macros (the go-macros repository on github, that
> > will be rehosted in pagure as soon as the @rh contributors ok the 
> > move)
> > instead of doing it in other places, that would be a tad easier to
> > coordinate.
> > 
> >> Please revert this, we should be able to use whatever flag supported
> >> by %setup and
> >>   %autosetup, not a small subset.  How do you even pass the basic -n
> >> flags now?
> > So basically, you have a macro, which sole purpose is to pass
> > precomputed -n values to *setup, and you want to use it with manual -n
> > flags. Why? Could you tell us what exactly is the point of
> > 
> > 1. wrapping a Fedora macro in another name just because you do not 
> > like
> > the official macro name in Fedora
> > 2. overriding the only thing this macro does over autosetup
> > 3. and then complaining all the argument passing to autosetup does not
> > work as you wish it does
> > 
> > So all this complexity, because what you really want is to use 
> > autosetup
> > directly. Then just do. What's the problem exactly with typing 
> > autosetup
> > in your spec? No one stops you from doing it.
> > 
> > One reason I removed the -n in forgeautosetup and forgesetup precisely
> > to stop packagers confusing themselves the way you are confusing
> > yourself.
> > 
> > The other being -n is incompatible with the processing of multiple
> > archives that many people asked me to add to the macros. Which is
> > finally implemented after months of work. And which just hit rawhide
> > after more than a month of review.
> > 
> > So, do you have actual spec files in Fedora with this use of the 
> > redhat-
> > rpm-config macros? If that is the case, I can put those flags in the
> > macros so you can continue shooting yourself in the foot using 
> > undefined
> > known-broken use patterns. If not, I'd rather remove the possibility
> > altogether before someone harms himself.
> > 
> >> With you mods, we have to edit hundreds of specs to remove
> >> the -q flags.
> > And that concretely, if why pre-generating specs instead of making the
> > effort to include default processing in the macro code Fedora ships is
> > wrong.
> > 
> > Regards,
> > 
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail] 
-- 
Nicolas Mailhot

[Nicolas Mailhot]

Jan,

I do not recall this all the same way, but no matter, that was months 
ago, a long development tunnel away.

The thing which is failing today is a brittle and dangerous rpm macro 
aliasing code that was added against my best technical advice. Sound 
advice as we see now. And I had honestly forgotten about it. It serves 
absolutely no technical purpose, it is buried inside an unrelated macro, 
and it is not documented anywhere¹.

All the blame games in the world won't change where the technical 
failure is.

So what do you want me to do now? Fix this aliasing code? That requires 
rewriting it. The way it was written it can not keep working reliably 
with or without redhat-rpm-config changes.

Rewrite it just so gosetup calls can be removed from Fedora specs? I'd 
rather write the script to remove the problem calls directly.

Rewrite it so removal can be postponed indefinitely via endless vetos by 
you or Jakub? That makes me the effective maintainer of code I have no 
use for and disagree with.

I'm the one doing the work here. I'd appreciate if it was taken in 
account sometimes.

Regards,

¹ Neither on https://fedoraproject.org/wiki/More_Go_packaging, nor on 
https://fedoraproject.org/wiki/PackagingDrafts/Go, nor in the macro 
files themselves.

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Anyway, since no one answers when I list the possible fixes and ask to
choose between them, I fixed the Fedora spec files myself.

Feel free to reintroduce %gosetup calls if you really want them, once
the %gosetup implementation has been fixed.

-- 
Nicolas Mailhot

[Robert-André Mauchin]

On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  Anyway, since no one answers when I list the possible fixes and ask
> to
>  choose between them, I fixed the Fedora spec files myself.
>  
>  Feel free to reintroduce %gosetup calls if you really want them, once
>  the %gosetup implementation has been fixed.
>  
>  -- 
>  Nicolas Mailhot 

Thanks for doing but you messed up the bumping of all the %changelog entries.

[Robert-André Mauchin]

On mardi 23 octobre 2018 23:47:05 CEST you wrote:
>  On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  > Anyway, since no one answers when I list the possible fixes and ask to
>  > choose between them, I fixed the Fedora spec files myself.
>  > 
>  > Feel free to reintroduce %gosetup calls if you really want them, once
>  > the %gosetup implementation has been fixed.
>  
>  Thanks for doing but you messed up the bumping of all the %changelog
>  entries. 
Used this Fish script to fix my SPEC:

for f in (grep --include=\*.spec -rnwl './' -e "redhat-rpm-config-123")

    sed -i -e '/\* Tue Oct 23 2018 Nicolas Mailhot
<nim(a)fedoraproject\.org&gt;/{n;d}&#39; $f
    set version (rpmspec -q --srpm --qf "%{version}-%{release}" $f | sed
's|\.fc2[0-9]||')
    sed -i "s|\* Tue Oct 23 2018 Nicolas Mailhot <nim(a)fedoraproject\.org&gt;|\*
Tue Oct 23 2018 Nicolas Mailhot <nim(a)fedoraproject\.org&gt; - $version|" $f
    pushd (dirname $f)
    fedpkg ci -m "Fix changelog entry"
    popd
end

Wish you would fix the rest.

[Nicolas Mailhot]

Le mercredi 24 octobre 2018 à 00:52 +0200, Robert-André Mauchin a
écrit :
>  On mardi 23 octobre 2018 23:47:05 CEST you wrote:
>  > On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  > > Anyway, since no one answers when I list the possible fixes and
>  > > ask to
>  > > choose between them, I fixed the Fedora spec files myself.
>  > > 
>  > > Feel free to reintroduce %gosetup calls if you really want them,
>  > > once
>  > > the %gosetup implementation has been fixed.
>  > 
>  > Thanks for doing but you messed up the bumping of all the %changelog
>  > entries.
>  
>  Used this Fish script to fix my SPEC:
>   
This does not need any fixing, it may not be the changelog format you're
used to, but it's one of the changelog styles rpm understands, which is
used by packages in Fedora, and which was documented at one point in
guidelines (no idea where the corresponding paragraph is today with the
doc.fedoraproject changes)

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le mercredi 24 octobre 2018 à 08:42 +0200, Nicolas Mailhot a écrit :
>  Le mercredi 24 octobre 2018 à 00:52 +0200, Robert-André Mauchin a
>  écrit :
>  > On mardi 23 octobre 2018 23:47:05 CEST you wrote:
>  > > On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  > > > Anyway, since no one answers when I list the possible fixes and
>  > > > ask to
>  > > > choose between them, I fixed the Fedora spec files myself.
>  > > > 
>  > > > Feel free to reintroduce %gosetup calls if you really want them,
>  > > > once
>  > > > the %gosetup implementation has been fixed.
>  > > 
>  > > Thanks for doing but you messed up the bumping of all the
>  > > %changelog
>  > > entries.
>  > 
>  > Used this Fish script to fix my SPEC:
>  > 
>  
>  This does not need any fixing, it may not be the changelog format
>  you're
>  used to, but it's one of the changelog styles rpm understands, which
>  is
>  used by packages in Fedora, and which was documented at one point in
>  guidelines (no idea where the corresponding paragraph is today with
>  the
>  doc.fedoraproject changes) 
Concretely the 

* <date> <name> <mail>
- version-release
- changes

format is a tad more flexible and allows publishing several fixes in a
row without duplicating date lines

* <date> <name> <mail>
- version-release
- changes
- version-release
-
changes

and so on

And, our infra does not care if a changelog uses mixed style or not,
since mass rebuilds will drop changelog entries in a single format in
all Fedora specs, so you have many mixed style changelogs in the
distribution today.

(of course it's your specs, you can rewrite changelog entries any way
that makes you feel better)

Regards,

-- 
Nicolas Mailhot

[Bob Mauchin]

On Wed, 24 Oct 2018, 09:46 Nicolas Mailhot, <nicolas.mailhot(a)laposte.net
>  
wrote:

>  Le mercredi 24 octobre 2018 à 08:42 +0200, Nicolas Mailhot a écrit :
>  > Le mercredi 24 octobre 2018 à 00:52 +0200, Robert-André Mauchin a
>  > écrit :
>  > > On mardi 23 octobre 2018 23:47:05 CEST you wrote:
>  > > > On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  > > > > Anyway, since no one answers when I list the possible fixes and
>  > > > > ask to
>  > > > > choose between them, I fixed the Fedora spec files myself.
>  > > >
> >  
>  > > > > Feel free to
> reintroduce %gosetup calls if you really want them,
>  > > > > once
>  > > > > the %gosetup implementation has been fixed.
>  > >
> >  
>  > > > Thanks for doing but
> you messed up the bumping of all the
>  > > > %changelog
>  > > > entries.
>  >
> >  
>  > > Used this Fish script to fix my SPEC:
>  >
> >  
> >  
>  > This does not need any fixing,
> it may not be the changelog format
>  > you're
>  > used to, but it's one of the changelog styles rpm understands, which
>  > is
>  > used by packages in Fedora, and which was documented at one point in
>  > guidelines (no idea where the corresponding paragraph is today with
>  > the
>  > doc.fedoraproject changes)
>
>  Concretely the
>
>  * <date> <name> <mail
> >  
>  -
> version-release
>  - changes
>
>  format is a tad more flexible and allows publishing several fixes in a
>  row without duplicating date lines
>
>  * <date> <name> <mail
> >  
>  -
> version-release
>  - changes
>  - version-release
>  -
>  changes
>
>  and so on
>
>  And, our infra does not care if a changelog uses mixed style or not,
>  since mass rebuilds will drop changelog entries in a single format in
>  all Fedora specs, so you have many mixed style changelogs in the
>  distribution today.
>
>  (of course it's your specs, you can rewrite changelog entries any way
>  that makes you feel better)
>
>  Regards,
>
>  --
>  Nicolas Mailhot
>  
I'm talking about dropping the git hash from the release part of the entry.

>

[Nicolas Mailhot]

Le 2018-10-24 14:50, Bob Mauchin a écrit :

>  I'm talking about dropping the git hash from the release part of
> the
>  entry. 
Ah, interesting, that's a side-effect of distprefix being pulled in by 
dist in now, and mass rebuild changelog scripts typically null dist in 
release entries… So, probably not possible any longer to do mass 
rebuilds without full dist in changelogs (unless you want to enter 
hack-land).

Not sure if going full dist is a problem, mass rebuilds are pretty much 
release specific anyway.

I had missed this part, sorry about that.

However, I'm pretty sure mass improving changelog entries is not in the 
scope of
https://fedoraproject.org/wiki/Who_is_allowed_to_modify_which_packages

Fixing failing builds clearly was.

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le mardi 23 octobre 2018 à 23:47 +0200, Robert-André Mauchin a écrit :
>  On mardi 23 octobre 2018 22:43:16 CEST Nicolas Mailhot wrote:
>  > Anyway, since no one answers when I list the possible fixes and ask
>  > to
>  > choose between them, I fixed the Fedora spec files myself.
>  > 
>  > Feel free to reintroduce %gosetup calls if you really want them,
>  > once
>  > the %gosetup implementation has been fixed.
>  > 
>  Thanks for doing but you messed up the bumping of all the %changelog
>  entries. 
How exactly please?

-- 
Nicolas Mailhot