What would it take to drop release and changelog from our spec files? (and do we want to?)

[Pierre-Yves Chibon]

Good Morning Everyone,

This is not a new idea, it has been presented at flock last year and spoken
about on this very list this fall, so I'd like to push it a little further.

Do we want to drop release and changelog from our spec file?
If we do, how would this work?

The release field would need to be set by koji ignoring whatever is in the spec
file. How do we want to do this?
  - Based on dates?
  - Using an always increasing integer?
  - Using the number of successful builds since the last time the version field changed?
  - Another idea?

The third option looks like to be the one closest to our current behavior.


Another question regarding the release field is: how would we deal with
pre-releases and other unsortable versions?
The current doc relies on <extraver> etc. in the release field as per:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_un...
Would using the tilde to specify pre-releases in the version field be sufficient
(as described in
https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_ve...
I.e., how can we cater for snapshot releases (e.g. based off a specific git
commit)?


With the changelog it becomes a little bit more tricky.
We currently have 3 changelogs in Fedora with 3 different target audience (this
is how I understand them):
  - One for the files in the git repository, meant to be "consumed" by our
    fellow packagers, not meant to leave the Fedora infrastructure
  - One in the spec file describing the changes applied to it. This one is meant
    to be accessible to sysadmins who want to know/check what changed in a package
  - One in bodhi, meant for end-user consumption and which should give some
    explanation as to why the package was updated or where information about the
    update can be found

So we need to, somehow, merge two changelogs into one while realizing that some
information in one may not be desirable in the other (for example the world
famous commit message: "oops I've forgot to upload the sources" does not
need to
appear in the RPM's changelog).
Would it be easier to merge the git changelog with the spec changelog or the
spec changelog with the bodhi notes?

For the former one easy way to achieve this is to consider all the commits since
the last successful build and have a magic keyword to either include or exclude
a commit message in the changelog.
For the latter, we discussed the idea of using annotated git tags this fall.

Both have their pros and cons, do people have some experience with either and
could share their feedback?
Is there a different approach, e.g. by using towncrier[1] or something
comparable, to track changes outside the spec file?


To give some context to this discussion, the CPE team is moving toward quarterly
planning and for the first months of 2020 a small team of people in the CPE team
is willing to do the work to make this happen, but there are two requirements:
  - The work needs to be scoped and defined by January 20th 2020 (so we can
    evaluate whether or not we have the knowledge, resources and time to do it)
  - The work needs to be achievable before March 31st 2020 (cf the quarterly
    objectives we're working towards)

Thus our call for input to accept or reject the idea and if the former
scope/define the system.


Thanks in advance for your help,

Pierre


[1] https://github.com/hawkowl/towncrier

[Iñaki Ucar]

On Fri, 10 Jan 2020 at 17:45, Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
>  With the changelog it becomes a little bit more tricky.
>  We currently have 3 changelogs in Fedora with 3 different target audience (this
>  is how I understand them):
>    - One for the files in the git repository, meant to be "consumed" by our
>      fellow packagers, not meant to leave the Fedora infrastructure
>    - One in the spec file describing the changes applied to it. This one is meant
>      to be accessible to sysadmins who want to know/check what changed in a package
>    - One in bodhi, meant for end-user consumption and which should give some
>      explanation as to why the package was updated or where information about the
>      update can be found
>
>  So we need to, somehow, merge two changelogs into one while realizing that some
>  information in one may not be desirable in the other (for example the world
>  famous commit message: "oops I've forgot to upload the sources" does not
> need to
>  appear in the RPM's changelog).
>  Would it be easier to merge the git changelog with the spec changelog or the
>  spec changelog with the bodhi notes?
>
>  For the former one easy way to achieve this is to consider all the commits since
>  the last successful build and have a magic keyword to either include or exclude
>  a commit message in the changelog.
>  For the latter, we discussed the idea of using annotated git tags this fall. 
Most of the time, I end up copying the spec changelog in the commit
message and I don't change the update template, so the bodhi changelog
is also the same. The spec changelog is a pain for me, so I'd vote for
git commit messages + tags (unnanotated; otherwise, I don't see much
benefit).

-- 
Iñaki Úcar

[Miroslav Suchý]

Dne 10. 01. 20 v 18:21 Iñaki Ucar napsal(a):
>  Most of the time, I end up copying the spec changelog in the commit
>  message and I don't change the update template,  
+1
Thou, occasionaly I *delete* some of those commits as they are unnessary in changelog.
E.g.:

* typo fix
* revert of ....
* previous fix was not complete...


-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[David Kaufmann]

On Mon, Jan 13, 2020 at 12:20:15PM +0100, Miroslav Suchý wrote:
>  Dne 10. 01. 20 v 18:21 Iñaki Ucar napsal(a):
>  > Most of the time, I end up copying the spec changelog in the commit
>  > message and I don't change the update template, 
>  
>  +1
>  Thou, occasionaly I *delete* some of those commits as they are unnessary in changelog.
> E.g.:
>  
>  * typo fix
>  * revert of ....
>  * previous fix was not complete... 
It seems I'm using git quite differently.

My spec file changelog describes what the spec changes did in regard to
functionality or version changes of the tool.

My git commit messages usually try to summarize, what happened to the
(dist-)git-repo - mostly stuff that can be seen in `git show --stat`,
although sometimes I describe what I changed inside a file.

The field for bodhi I usually copy from the changelog - but to be honest
I only fill it, because it's there - I don't even really know what it is
used for, except being shown on the update page.

On the other side when I'm looking for something, because something
broke or whatever, usually I go to the spec-file on
src.fedoraproject.org[1] first, and if I can't find the reason I'm looking
for in the first few seconds I take a look at the changelog, which for
convenience is right there.

Didn't even know about `rpm -q xx --changelog` - quite nice, but as I
prefer to look at the spec file instructions first anyway I'll probably
still check there. (Usually I look for a specific Patch file or
configure-flag, so the directly checking the spec is faster)

For me this is a plus and one of the reasons I do enjoy packaging for
Fedora more than I enjoy packaging for Debian - one single spec file vs.
many files, and each single one of those has a different format.

But I also do understand that separating the changelog from the spec
file might have its benefits and that duplication of the messages for
people using the git log message as changelog messages is annoying.

All the best,
Astra

[Randy Barlow]

On Tue, 2020-01-14 at 00:30 +0100, David Kaufmann wrote:
>  The field for bodhi I usually copy from the changelog - but to be
>  honest I only fill it, because it's there - I don't even really know
>  what it is used for, except being shown on the update page. 
Users can read the update text with the updateinfo dnf subcommand. For
example:

$ dnf updateinfo --info FEDORA-2019-ed226e6112
Last metadata expiration check: 0:02:22 ago on Wed 15 Jan 2020 01:45:20 PM EST.
===============================================================================
  xorg-x11-server-1.20.6-1.fc30
===============================================================================
  Update ID: FEDORA-2019-ed226e6112
       Type: bugfix
    Updated: 2020-01-14 19:37:24
       Bugs: 1752211 - crash of /usr/libexec/Xorg
           : 1761241 - [abrt] xorg-x11-server-Xwayland: present_wnmd_abort_vblank():
Xwayland killed by SIGABRT
Description: xserver 1.20.6
   Severity: None

[Fabio Valentini]

On Fri, Jan 10, 2020, 17:37 Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:

>  Good Morning Everyone,
>
>  This is not a new idea, it has been presented at flock last year and spoken
>  about on this very list this fall, so I'd like to push it a little further.
>
>  Do we want to drop release and changelog from our spec file?
>  If we do, how would this work?
>
>  The release field would need to be set by koji ignoring whatever is in the
>  spec
>  file. How do we want to do this?
>    - Based on dates?
>    - Using an always increasing integer?
>    - Using the number of successful builds since the last time the version
>  field changed?
>    - Another idea?
>  
What about "number of commits since last version update" (possibly tagged
in git)? That should encompass the possibilities you listed above, is
well-defined, and should be most like the current behavior.

The number of successful builds doesn't sound like a well-defined / stable
number to me, since it relies on information outside of git.


>  The third option looks like to be the one closest to our current
> behavior.
>
>
>  Another question regarding the release field is: how would we deal with
>  pre-releases and other unsortable versions?
>  The current doc relies on <extraver> etc. in the release field as per:
>
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_un...
>  Would using the tilde to specify pre-releases in the version field be
>  sufficient
>  (as described in
>
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_ve...
>  )?
>  I.e., how can we cater for snapshot releases (e.g. based off a specific git
>  commit)?
>  
The tilde notation is not enough for general snapshots, though it works
well for tagged alpha / beta / rc releases (because they are real releases,
not snapshots).

For snapshot builds, I think something like %releaseprefix of 0." (cf.
%distprefix) could be used to indicate a prerelease in the .spec file, and
incorporated in the automatically generated Release tag.


>  With the changelog it becomes a little bit more tricky.
>  We currently have 3 changelogs in Fedora with 3 different target audience
>  (this
>  is how I understand them):
>    - One for the files in the git repository, meant to be "consumed" by our
>      fellow packagers, not meant to leave the Fedora infrastructure
>    - One in the spec file describing the changes applied to it. This one is
>  meant
>      to be accessible to sysadmins who want to know/check what changed in a
>  package
>    - One in bodhi, meant for end-user consumption and which should give some
>      explanation as to why the package was updated or where information
>  about the
>      update can be found
>  
I think it would be good to merge git commit messages and the .spec
changelog. This is also how many projects handle this on GitHub. Using
[skip-changelog] in commit messages or something or something similar to
indicate that it should not be mentioned in generated release notes is also
common practice (just like [skip-ci]).

So I think this is very much doable, since there's actually a lot of
precedent for doing this. Also, the two audiences for git commit messages
and RPM changelogs are probably more similar than the audience for bodhi
updates (where I usually put more user-facing information).


Fabio


>  So we need to, somehow, merge two changelogs into one while realizing
> that
>  some
>  information in one may not be desirable in the other (for example the world
>  famous commit message: "oops I've forgot to upload the sources" does not
>  need to
>  appear in the RPM's changelog).
>  Would it be easier to merge the git changelog with the spec changelog or
>  the
>  spec changelog with the bodhi notes?
>
>  For the former one easy way to achieve this is to consider all the commits
>  since
>  the last successful build and have a magic keyword to either include or
>  exclude
>  a commit message in the changelog.
>  For the latter, we discussed the idea of using annotated git tags this
>  fall.
>
>  Both have their pros and cons, do people have some experience with either
>  and
>  could share their feedback?
>  Is there a different approach, e.g. by using towncrier[1] or something
>  comparable, to track changes outside the spec file?
>
>
>  To give some context to this discussion, the CPE team is moving toward
>  quarterly
>  planning and for the first months of 2020 a small team of people in the
>  CPE team
>  is willing to do the work to make this happen, but there are two
>  requirements:
>    - The work needs to be scoped and defined by January 20th 2020 (so we can
>      evaluate whether or not we have the knowledge, resources and time to
>  do it)
>    - The work needs to be achievable before March 31st 2020 (cf the
>  quarterly
>      objectives we're working towards)
>
>  Thus our call for input to accept or reject the idea and if the former
>  scope/define the system.
>
>
>  Thanks in advance for your help,
>
>  Pierre
>
>
>  [1] https://github.com/hawkowl/towncrier
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Vít Ondruch]

Dne 10. 01. 20 v 18:33 Fabio Valentini napsal(a):
>  On Fri, Jan 10, 2020, 17:37 Pierre-Yves Chibon
> <pingou(a)pingoured.fr
>  <mailto:[e-mail]>> wrote:
>
>      Good Morning Everyone,
>
>      This is not a new idea, it has been presented at flock last year
>      and spoken
>      about on this very list this fall, so I'd like to push it a little
>      further.
>
>      Do we want to drop release and changelog from our spec file?
>      If we do, how would this work?
>
>      The release field would need to be set by koji ignoring whatever
>      is in the spec
>      file. How do we want to do this?
>        - Based on dates?
>        - Using an always increasing integer?
>        - Using the number of successful builds since the last time the
>      version field changed?
>        - Another idea?
>
>
>  What about "number of commits since last version update" (possibly
>  tagged in git)? That should encompass the possibilities you listed
>  above, is well-defined, and should be most like the current behavior. 

That won't work. This assumes that all subpackages have the same version
as the main package, but that might not be true (it is definitely not
true for Ruby neither for Perl AFAIK). If nothing else, there must be
way to override/hint the automation (unless the automation is smart
enough to detect such scenarios, which would be cool).


Vít

[Fabio Valentini]

On Fri, Jan 10, 2020 at 6:52 PM Vít Ondruch <vondruch(a)redhat.com&gt; wrote:
>  Dne 10. 01. 20 v 18:33 Fabio Valentini napsal(a):
>
>  On Fri, Jan 10, 2020, 17:37 Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
> >
> > Good Morning Everyone,
> >
> > This is not a new idea, it has been presented at flock last year and spoken
> > about on this very list this fall, so I'd like to push it a little further.
> >
> > Do we want to drop release and changelog from our spec file?
> > If we do, how would this work?
> >
> > The release field would need to be set by koji ignoring whatever is in the spec
> > file. How do we want to do this?
> >   - Based on dates?
> >   - Using an always increasing integer?
> >   - Using the number of successful builds since the last time the version field
> changed?
> >   - Another idea?
>
>
>  What about "number of commits since last version update" (possibly tagged in
> git)? That should encompass the possibilities you listed above, is well-defined, and
> should be most like the current behavior. 
(snip)

>  That won't work. This assumes that all subpackages have the same
> version as the main package, but that might not be true (it is definitely not true for
> Ruby neither for Perl AFAIK). 
It won't work in all cases, true. But in the predominant case where
all produced packages have the same version, it works fine.

>  If nothing else, there must be way to override/hint the automation
> (unless the automation is smart enough to detect such scenarios, which would be cool).
For example: For the handful of special cases like ruby it should be
easy to define a macro to tell the Release tag generator "hey, count
up releases from point X instead" (which could just be a manually
specified git ref).

Fabio

> Vít
>
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Neal Gompa]

On Fri, Jan 10, 2020 at 12:52 PM Vít Ondruch <vondruch(a)redhat.com&gt; wrote:
>  Dne 10. 01. 20 v 18:33 Fabio Valentini napsal(a):
>
>  On Fri, Jan 10, 2020, 17:37 Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
> >
> > Good Morning Everyone,
> >
> > This is not a new idea, it has been presented at flock last year and spoken
> > about on this very list this fall, so I'd like to push it a little further.
> >
> > Do we want to drop release and changelog from our spec file?
> > If we do, how would this work?
> >
> > The release field would need to be set by koji ignoring whatever is in the spec
> > file. How do we want to do this?
> >   - Based on dates?
> >   - Using an always increasing integer?
> >   - Using the number of successful builds since the last time the version field
> changed?
> >   - Another idea?
>
>
>  What about "number of commits since last version update" (possibly tagged in
> git)? That should encompass the possibilities you listed above, is well-defined, and
> should be most like the current behavior.
>
>
>  That won't work. This assumes that all subpackages have the same version as the main
> package, but that might not be true (it is definitely not true for Ruby neither for Perl
> AFAIK). If nothing else, there must be way to override/hint the automation (unless the
> automation is smart enough to detect such scenarios, which would be cool).
>  
Yes it will, because the source version is predictable. Version
updates would work off the source RPM version, and that isn't affected
by games like that. The query would be something like the following:

$ rpmspec -q --srpm --qf "%{VERSION}-%{RELEASE}\n" foo.spec

That will *always* do the right thing.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Petr Pisar]

On Fri, Jan 10, 2020 at 10:14:39PM -0500, Neal Gompa wrote:
>  On Fri, Jan 10, 2020 at 12:52 PM Vít Ondruch
> <vondruch(a)redhat.com&gt; wrote:
>  > Dne 10. 01. 20 v 18:33 Fabio Valentini napsal(a):
>  > > What about "number of commits since last version update" (possibly
>  > > tagged in git)? That should encompass the possibilities you listed
>  > > above, is well-defined, and should be most like the current behavior.
>  >
>  > That won't work. This assumes that all subpackages have the same version
>  > as the main package, but that might not be true (it is definitely not true
>  > for Ruby neither for Perl AFAIK). If nothing else, there must be way to
>  > override/hint the automation (unless the automation is smart enough to
>  > detect such scenarios, which would be cool).
>  >
>  Yes it will, because the source version is predictable. Version updates
>  would work off the source RPM version, and that isn't affected by games like
>  that.
>  
No, it won't as we have competing %{version}-%{release} strings among multiple
packages. E.g. perl source bundles an Encode module. And we know the module
is updated frequenly on CPAN. Thus we build perl-Encode-V1-R1 from perl.spec,
then we build perl-Encode-V1-R2 from CPAN perl-Encode.spec so that R2 >= R1, then we
rebuild perl.spec again with disabled perl-Encode subpackage.

With unpredictable releases we would have to fake version of the bundled
module so that it is always lower than a version of the standalone package.
Not impossible but, a nuissance.

-- Petr

[Miroslav Suchý]

Dne 13. 01. 20 v 10:46 Petr Pisar napsal(a):
>  No, it won't as we have competing %{version}-%{release} strings
> among multiple
>  packages. E.g. perl source bundles an Encode module. And we know the module
>  is updated frequenly on CPAN. Thus we build perl-Encode-V1-R1 from perl.spec,
>  then we build perl-Encode-V1-R2 from CPAN perl-Encode.spec so that R2 >= R1, then we
>  rebuild perl.spec again with disabled perl-Encode subpackage. 
Wow!
Why it is not in separate package then? It can have the same Source0, but if subpackage
have different release cadence,
then I see no reason why it is a subpackage.

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Nicolas Mailhot]

Le 2020-01-13 11:28, Miroslav Suchý a écrit :
>  Dne 13. 01. 20 v 10:46 Petr Pisar napsal(a):
> > No, it won't as we have competing %{version}-%{release} strings among 
> > multiple
> > packages. E.g. perl source bundles an Encode module. And we know the 
> > module
> > is updated frequenly on CPAN. Thus we build perl-Encode-V1-R1 from 
> > perl.spec,
> > then we build perl-Encode-V1-R2 from CPAN perl-Encode.spec so that R2 
> > >= R1, then we
> > rebuild perl.spec again with disabled perl-Encode subpackage.
>  
>  Wow!
>  Why it is not in separate package then? It can have the same Source0,
>  but if subpackage have different release cadence,
>  then I see no reason why it is a subpackage. 
Personaly, I tend to consider that something that gets definitely 
bundled inside another project, and is no longer released separately, 
surrenders its right for separate versioning, and inherits this 
versioning from the bundler, whatever upstream feels about it (if they 
want to keep versioning separate they can provide a separate release).

Of course if it *is* still released separately packaging the bundled 
version instead of the real upstream is bad practice.

Regards,

-- 
Nicolas Mailhot

[Petr Pisar]

On Mon, Jan 13, 2020 at 11:28:41AM +0100, Miroslav Suchý wrote:
>  Dne 13. 01. 20 v 10:46 Petr Pisar napsal(a):
>  > No, it won't as we have competing %{version}-%{release} strings among multiple
>  > packages. E.g. perl source bundles an Encode module. And we know the module
>  > is updated frequenly on CPAN. Thus we build perl-Encode-V1-R1 from perl.spec,
>  > then we build perl-Encode-V1-R2 from CPAN perl-Encode.spec so that R2 >= R1, then
> we
>  > rebuild perl.spec again with disabled perl-Encode subpackage.
>  
>  Wow!
>  Why it is not in separate package then? It can have the same Source0, but if
>  subpackage have different release cadence, then I see no reason why it is
>  a subpackage.
>   
The subpackage is part of a standard library (because it is bundled) and other
packages including the standalone one need it for building or for running.
Therefore we keep the subpackage available for the short time when
bootstrapping a new perl.

(Theoretically if we disabled tests and reimplemented build scripts of the
standalone packages, wou could remove the subpackages completely. But we
consider that more labour intensive, fragile and risky than the currenct
approach.)

-- Petr

[Richard Shaw]

Just catching up on this thread...

How about an incremental step? (I don't know how difficult it would be to
implement however)...

What about separating the change log to a separate file in dist-git?
Something like the traditional "ChangeLog"?

I like to keep my spec files in sync between releases unless there's a good
reason for them to diverge but sometimes there are changes to a specific
release that aren't really a part of the others.

That way I can keep the spec files in sync (git merge master, et. all) but
without merging changelogs that don't actually apply to that release (and
dealing with merge conflicts)?

Thanks,
Richard

[Pierre-Yves Chibon]

On Mon, Jan 13, 2020 at 08:19:36AM -0600, Richard Shaw wrote:
>     Just catching up on this thread...
>     How about an incremental step? (I don't know how difficult it would be to
>     implement however)... 
This is what Vit was suggesting which seems like a very nice idea :)

>     What about separating the change log to a separate file in
> dist-git?
>     Something like the traditional "ChangeLog"?
>     I like to keep my spec files in sync between releases unless there's a
>     good reason for them to diverge but sometimes there are changes to a
>     specific release that aren't really a part of the others. 
>     That way I can keep the spec files in sync (git merge master, et. all) but
>     without merging changelogs that don't actually apply to that release (and
>     dealing with merge conflicts)? 
The conflicts won't go away, it'll be moved from the .spec file to the ChangeLog
file.


Pierre

[David Cantrell]

On Mon, Jan 13, 2020 at 08:19:36AM -0600, Richard Shaw wrote:
> Just catching up on this thread...
>
> How about an incremental step? (I don't know how difficult it would be to
> implement however)...
>
> What about separating the change log to a separate file in dist-git?
> Something like the traditional "ChangeLog"?
>
> I like to keep my spec files in sync between releases unless there's a good
> reason for them to diverge but sometimes there are changes to a specific
> release that aren't really a part of the others.
>
> That way I can keep the spec files in sync (git merge master, et. all) but
> without merging changelogs that don't actually apply to that release (and
> dealing with merge conflicts)? 
I do a variant of this with rpminspect and some other packages.  I
automatically generate 'changelog' when I make a new upstream release.  This
and the tarball then Source0 and Source1.  The spec file has:

     %changelog
     %include %{SOURCE1}

I don't upload the changelog to the lookaside cache, but just commit it in
dist-git.  I'm trying to strictly release from upstream and push builds in to
dist-git, but I'm doing this changelog structure so that if there is
downstream maintenance needed them commits to the changelog can be separate
from spec and patch changes which makes cherry-picking across branches
theoretically easier.

There is the problem of Release, which I would like to have in a separate file
to that I can also %include.  Haven't tried that yet mostly because I haven't
had an excuse to.

Thanks,

-- 
David Cantrell <dcantrell(a)redhat.com&gt;
Red Hat, Inc. | Boston, MA | EST5EDT

[Vít Ondruch]

Dne 11. 01. 20 v 4:14 Neal Gompa napsal(a):
>  On Fri, Jan 10, 2020 at 12:52 PM Vít Ondruch
> <vondruch(a)redhat.com&gt; wrote:
> >
> > Dne 10. 01. 20 v 18:33 Fabio Valentini napsal(a):
> >
> > On Fri, Jan 10, 2020, 17:37 Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
> >> Good Morning Everyone,
> >>
> >> This is not a new idea, it has been presented at flock last year and spoken
> >> about on this very list this fall, so I'd like to push it a little further.
> >>
> >> Do we want to drop release and changelog from our spec file?
> >> If we do, how would this work?
> >>
> >> The release field would need to be set by koji ignoring whatever is in the spec
> >> file. How do we want to do this?
> >>   - Based on dates?
> >>   - Using an always increasing integer?
> >>   - Using the number of successful builds since the last time the version field
> changed?
> >>   - Another idea?
> >
> > What about "number of commits since last version update" (possibly tagged
> in git)? That should encompass the possibilities you listed above, is well-defined, and
> should be most like the current behavior.
> >
> >
> > That won't work. This assumes that all subpackages have the same version as the
> main package, but that might not be true (it is definitely not true for Ruby neither for
> Perl AFAIK). If nothing else, there must be way to override/hint the automation (unless
> the automation is smart enough to detect such scenarios, which would be cool).
> >
>  Yes it will, because the source version is predictable. Version
>  updates would work off the source RPM version, and that isn't affected
>  by games like that. The query would be something like the following:
>
>  $ rpmspec -q --srpm --qf "%{VERSION}-%{RELEASE}\n" foo.spec
>
>  That will *always* do the right thing.
>
>  
I might not get your point, but working on Ruby 2.7, I cannot reset its
release to -1, because Ruby 2.6 ships
rubygem-net-telnet-0.2.0-124.fc32.noarch.rpm [1], while the version of
this subpackage has not changed for Ruby 2.7. If I did the reset, I
would end up with rubygem-net-telnet-0.2.0-1.fc32.noarch.rpm which would
never override the package coming from Ruby 2.6.

I don't even want to discuss the case Petr described in the follow up,
because that is even more complex situation ...


Vít



[1] https://koji.fedoraproject.org/koji/buildinfo?buildID=1397861

[Nicolas Mailhot]

Le vendredi 10 janvier 2020 à 17:36 +0100, Pierre-Yves Chibon a écrit :
>  Good Morning Everyone,
>  
>  This is not a new idea, it has been presented at flock last year and
>  spoken
>  about on this very list this fall, so I'd like to push it a little
>  further.
>  
>  Do we want to drop release and changelog from our spec file?
>  If we do, how would this work? 
Dropping changelog is easy. Since we have a clean separation of spec
repo (src.fedoraproject.org) and project repo (pagure, gitlab or
elsewhere) the spec should just be assembled from all the
src.fedoraproject.org commit messages not present in the previous
generated changelog

(that won't work for thinks ike rehat-rpm-config because it does not
separate the project files in a separate repository but it’s high time
it behaved likea normal project, the non separation is a major PITA to
deal with)

Droping releases is much harder to design for because we don’t have a
linear build history, there are branches that split and then re-merge
at system release time (sometimes, with excursions in copr or another
repo), none of the proposed solutions would accomodate those workflows.

Regards,

-- 
Nicolas Mailhot

[Fabio Valentini]

On Fri, Jan 10, 2020 at 6:36 PM Nicolas Mailhot via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  Le vendredi 10 janvier 2020 à 17:36 +0100, Pierre-Yves Chibon a écrit :
>  > Good Morning Everyone,
>  >
>  > This is not a new idea, it has been presented at flock last year and
>  > spoken
>  > about on this very list this fall, so I'd like to push it a little
>  > further.
>  >
>  > Do we want to drop release and changelog from our spec file?
>  > If we do, how would this work?
>
>  Dropping changelog is easy. Since we have a clean separation of spec
>  repo (src.fedoraproject.org) and project repo (pagure, gitlab or
>  elsewhere) the spec should just be assembled from all the
>  src.fedoraproject.org commit messages not present in the previous
>  generated changelog
>
>  (that won't work for thinks ike rehat-rpm-config because it does not
>  separate the project files in a separate repository but it’s high time
>  it behaved likea normal project, the non separation is a major PITA to
>  deal with) 
(snip)

>  Droping releases is much harder to design for because we don’t have
> a
>  linear build history, there are branches that split and then re-merge
>  at system release time (sometimes, with excursions in copr or another
>  repo), none of the proposed solutions would accomodate those workflows. 
You can never expect our tooling to do "magic" (TM) and work "just
right", no matter which Versions and Releases and Epochs of packages
are available from third-party repos and coprs. This has nothing to do
with proposed auto-generated Release tag, and it's definitely not a
new problem. We've basically ignored consistency with third-party
repos until now - and rightly so, IMO - because that's what "dnf
distro-sync" is for. (Even upgrade path from fedora N to N+1 doesn't
have to be "clean" anymore, because system-upgrade operates in
"distro-sync" mode by default now ...)

Fabio

> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Le vendredi 10 janvier 2020 à 20:53 +0100, Fabio Valentini a écrit :
>  
>  You can never expect our tooling to do "magic" (TM) and work "just
>  right", no matter which Versions and Releases and Epochs of packages
>  are available from third-party repos and coprs.  
Yes, sure, but the current way we manage releases accomodate those
worklows.

For example:
1. I hit a fontconfig bug while preparing the new font packaging
guidelines,
2. Akira kindly fixed the problem upstream,
https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/185
3. it was trivial to build a package matching the Fedora fontconfig
with just the fix added in copr, without breaking the release thread
https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/build/1126851/

(it exists just for me in my copre because general availability was not
required to advance on the guidelines proposal; general availability
would have required a push to rawhide and a support commitment by
Akira)

And, it will all converge once FPC finishes its review, Akira releases
fontconfig upstream, and the result is rebuilt for Fedora. Had I waited
for Akira to wrap up an upstream release and build the result rawhide-
side the FPC submission would have been pushed back for months (and
then it would have delayed other Fedora changes, it's a cascading
effect).

The non-linear progression permitted by current manual release setting
allows parallelizing work and getting things done a lot faster within
the project. I don’t see how to manage this with the autogeneration
proposal

Regards,

-- 
Nicolas Mailhot

[Pierre-Yves Chibon]

On Fri, Jan 10, 2020 at 09:18:35PM +0100, Nicolas Mailhot via devel wrote:
>  Le vendredi 10 janvier 2020 à 20:53 +0100, Fabio Valentini a écrit :
>  > 
>  > You can never expect our tooling to do "magic" (TM) and work "just
>  > right", no matter which Versions and Releases and Epochs of packages
>  > are available from third-party repos and coprs. 
>  
>  Yes, sure, but the current way we manage releases accomodate those
>  worklows.
>  
>  For example:
>  1. I hit a fontconfig bug while preparing the new font packaging
>  guidelines,
>  2. Akira kindly fixed the problem upstream,
>  https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/185
>  3. it was trivial to build a package matching the Fedora fontconfig
>  with just the fix added in copr, without breaking the release thread
>  https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/build/1126851/
>  
>  (it exists just for me in my copre because general availability was not
>  required to advance on the guidelines proposal; general availability
>  would have required a push to rawhide and a support commitment by
>  Akira)
>  
>  And, it will all converge once FPC finishes its review, Akira releases
>  fontconfig upstream, and the result is rebuilt for Fedora. Had I waited
>  for Akira to wrap up an upstream release and build the result rawhide-
>  side the FPC submission would have been pushed back for months (and
>  then it would have delayed other Fedora changes, it's a cascading
>  effect).
>  
>  The non-linear progression permitted by current manual release setting
>  allows parallelizing work and getting things done a lot faster within
>  the project. I don’t see how to manage this with the autogeneration
>  proposal 
I don't quite see how it conflicts with it either.
You may end up having foo-1.0-42 in copr and foo-1.0-32 in koji which would lead
to a foo-1.0-32 (or -39) at the next build, but I'm not seeing how it's a
problem per say.
Could you give a more concrete example?


Thanks,
Pierre

[Nicolas Mailhot]

Le 2020-01-13 12:07, Pierre-Yves Chibon a écrit :

Hi,

>  I don't quite see how it conflicts with it either.
>  You may end up having foo-1.0-42 in copr and foo-1.0-32 in koji which 
>  would lead
>  to a foo-1.0-32 (or -39) at the next build, but I'm not seeing how it's 
>  a
>  problem per say. 
Correct handling of release streams means my private copr build is 
higher than the current koji build, but lower than the next koji build. 
Therefore whenever Akira decided to do a new official build it will 
replace my own (in dnf and mock), and I know I need to check if my 
private patch is still relevant, or if problem that justified the patch 
is solved in the koji build

Such a packaging workflow (builds that diverge in separate repos and 
then re-converge in koji) is very common.

Regards,

-- 
Nicolas Mailhot

[Jun Aruga]

When removing the changelog in a spec file, users (not package
maintainers) using this command are disappointed?

```
$ rpm -q --changelog python3
* Tue Oct 15 2019 Miro Hrončok <mhroncok(a)redhat.com&gt; - 3.7.5-1
- Update to 3.7.5
...
```

-- 
Jun | He - His - Him

[Miro Hrončok]

On 13. 01. 20 12:40, Jun Aruga wrote:
>  When removing the changelog in a spec file, users (not package
>  maintainers) using this command are disappointed?
>  
>  ```
>  $ rpm -q --changelog python3
>  * Tue Oct 15 2019 Miro Hrončok <mhroncok(a)redhat.com&gt; - 3.7.5-1
>  - Update to 3.7.5
>  ...
>  ``` 
No, this would still work. It would be injected into the rpm during buildtime, 
from a more sophisticated source than the spec file.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Panu Matilainen]

On 1/10/20 7:35 PM, Nicolas Mailhot via devel wrote:
>  
>  Dropping changelog is easy. Since we have a clean separation of spec
>  repo (src.fedoraproject.org) and project repo (pagure, gitlab or
>  elsewhere) the spec should just be assembled from all the
>  src.fedoraproject.org commit messages not present in the previous
>  generated changelog
>  
>  (that won't work for thinks ike rehat-rpm-config because it does not
>  separate the project files in a separate repository but it’s high time
>  it behaved likea normal project, the non separation is a major PITA to
>  deal with) 
You keep saying that, but maybe you were not involved with 
redhat-rpm-config back when it was that way. It was the most hideous 
piece of package I had ever worked with, because the model of external 
tarballs is just absurd and does not work at all with what it is and 
does. We've been there, done that and the current model is the first 
time that redhat-rpm-config is actually nice to maintain.

	- Panu -

[Nicolas Mailhot]

Le 2020-01-13 11:01, Panu Matilainen a écrit :

>  You keep saying that, but maybe you were not involved with
>  redhat-rpm-config back when it was that way. It was the most hideous
>  piece of package I had ever worked with, because the model of external
>  tarballs is just absurd and does not work at all with what it is and
>  does. We've been there, done that and the current model is the first
>  time that redhat-rpm-config is actually nice to maintain. 
I’ve also been there and done that and contributed to redhat-rpm-config 
and other macro packages that do not use the redhat-rpm-config model, 
and redhat-rpm-config is the most annoying to work with, with no 
directory structure to speak of, file-by-file special casing in the spec 
file, need to mangle file names because the way redhat-rpm-config is set 
up breaks pagure assumptions, and probably others I forget. This 
one-of-a-kind-ism is horrible except for people that only contribute to 
redhat-rpm-config and are so used to its quirks they don’t feel them 
anymore.

It is trivial nowadays to point a spec to pagure (or gitlab, or github) 
with the forge macros, and get an archive spectool understands that 
matches the version declared in the src.fedora.org spec, with a correct 
directory structure that can be used to simplify the spec logic

Regards,

-- 
Nicolas Mailhot

[Florian Weimer]

* Nicolas Mailhot via devel:

>  Le vendredi 10 janvier 2020 à 17:36 +0100, Pierre-Yves Chibon a écrit
> :
> > Good Morning Everyone,
> > 
> > This is not a new idea, it has been presented at flock last year and
> > spoken
> > about on this very list this fall, so I'd like to push it a little
> > further.
> > 
> > Do we want to drop release and changelog from our spec file?
> > If we do, how would this work?
>
>  Dropping changelog is easy. Since we have a clean separation of spec
>  repo (src.fedoraproject.org) and project repo (pagure, gitlab or
>  elsewhere) the spec should just be assembled from all the
>  src.fedoraproject.org commit messages not present in the previous
>  generated changelog 
That way, you cannot fix typos, add missing CVE IDs, and the like.  It's
a significant functionality change.

It may be possible to auto-generate a %changelog section listing new
commits since the last %changelog update.  Combined with a tool that
people can run before editing the %changelog section manually, to make
its contents explicit, I expect that to work.

>  (that won't work for thinks ike rehat-rpm-config because it does
> not
>  separate the project files in a separate repository but it’s high time
>  it behaved likea normal project, the non separation is a major PITA to
>  deal with) 
I have trouble matching this claim to my experience working on
redhat-rpm-config.  Why is it painful to use Git as it was designed?

I don't think it's an improvement if contributors have to figure out how
to generate a new upstream tarball for each change.

Thanks,
Florian

[Nicolas Mailhot]

Le 2020-01-13 12:52, Florian Weimer a écrit :

>  I have trouble matching this claim to my experience working on
>  redhat-rpm-config.  Why is it painful to use Git as it was designed? 
Because redhat-rpm-config is not "using Git as it was designed". It’s 
using git as a centralized flat and linear SCM (à la SVN/CVS) without 
putting the project info in the repository but in a static spec mapping.

In sane macro project that use separate sources you can do things like 
"install rpm/macros.d/* to %{_rpmconfigdir}/macros.d/" (because if you 
accepted a PR that puts things in rpm/macros.d and tagged the result you 
*want* this result deployed in %{_rpmconfigdir}/macros.d/, right?). In 
redhat-rpm-config that does not work. Every single file is a different 
source with custom spec handling.

Apart from being inefficient this custom handling badly collides 
whenever two PRs try to add new files. With a real detached source 
project, with a real tree managed by git, all the eventual tree 
collisions would be managed by git. With everything-is-a-separate-source 
files collide (at the source number level) even when they have different 
filenames, and will be deployed in different places.

Regards,

-- 
Nicolas Mailhot

[Panu Matilainen]

On 1/13/20 2:44 PM, Nicolas Mailhot via devel wrote:
>  Le 2020-01-13 12:52, Florian Weimer a écrit :
>  
> > I have trouble matching this claim to my experience working on
> > redhat-rpm-config.  Why is it painful to use Git as it was designed?
>  
>  Because redhat-rpm-config is not "using Git as it was designed". It’s 
>  using git as a centralized flat and linear SCM (à la SVN/CVS) without 
>  putting the project info in the repository but in a static spec mapping.
>  
>  In sane macro project that use separate sources you can do things like 
>  "install rpm/macros.d/* to %{_rpmconfigdir}/macros.d/" (because if you 
>  accepted a PR that puts things in rpm/macros.d and tagged the result you 
>  *want* this result deployed in %{_rpmconfigdir}/macros.d/, right?). In 
>  redhat-rpm-config that does not work. Every single file is a different 
>  source with custom spec handling.
>  
>  Apart from being inefficient this custom handling badly collides 
>  whenever two PRs try to add new files. With a real detached source 
>  project, with a real tree managed by git, all the eventual tree 
>  collisions would be managed by git. With everything-is-a-separate-source 
>  files collide (at the source number level) even when they have different 
>  filenames, and will be deployed in different places. 
Well yes, nothing is perfect. These are such minor issues compared to 
the patches-on-patches-of-patches horror that was before that it fails 
to register on my annoyance scale at all, it's nothing but a nice 
tradeoff for being able to use git as deity meant it for this content.

	- Panu -

[Neal Gompa]

On Mon, Jan 13, 2020 at 8:07 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  On 1/13/20 2:44 PM, Nicolas Mailhot via devel wrote:
>  > Le 2020-01-13 12:52, Florian Weimer a écrit :
>  >
>  >> I have trouble matching this claim to my experience working on
>  >> redhat-rpm-config.  Why is it painful to use Git as it was designed?
>  >
>  > Because redhat-rpm-config is not "using Git as it was designed". It’s
>  > using git as a centralized flat and linear SCM (à la SVN/CVS) without
>  > putting the project info in the repository but in a static spec mapping.
>  >
>  > In sane macro project that use separate sources you can do things like
>  > "install rpm/macros.d/* to %{_rpmconfigdir}/macros.d/" (because if you
>  > accepted a PR that puts things in rpm/macros.d and tagged the result you
>  > *want* this result deployed in %{_rpmconfigdir}/macros.d/, right?). In
>  > redhat-rpm-config that does not work. Every single file is a different
>  > source with custom spec handling.
>  >
>  > Apart from being inefficient this custom handling badly collides
>  > whenever two PRs try to add new files. With a real detached source
>  > project, with a real tree managed by git, all the eventual tree
>  > collisions would be managed by git. With everything-is-a-separate-source
>  > files collide (at the source number level) even when they have different
>  > filenames, and will be deployed in different places.
>
>  Well yes, nothing is perfect. These are such minor issues compared to
>  the patches-on-patches-of-patches horror that was before that it fails
>  to register on my annoyance scale at all, it's nothing but a nice
>  tradeoff for being able to use git as deity meant it for this content.
>  
It's also important to note that splitting is only useful if there are
multiple disparate downstreams. In this case, there isn't. Only Fedora
(and its downstream RHEL) use redhat-rpm-config. Contrast that to
Mageia's rpm-setup and OpenMandriva's rpm-openmandriva-setup, which
are managed this way *because* it is the upstream for multiple
disparate distributions (rpm-setup is the upstream for Mageia and
PCLinuxOS, while rpm-omv-setup is the upstream for OpenMandriva and
ROSA).



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Richard W.M. Jones]

On Fri, Jan 10, 2020 at 05:36:46PM +0100, Pierre-Yves Chibon wrote:
>  Good Morning Everyone,
>  
>  This is not a new idea, it has been presented at flock last year and spoken
>  about on this very list this fall, so I'd like to push it a little further.
>  
>  Do we want to drop release and changelog from our spec file?
>  If we do, how would this work? 
OpenSUSE proved years and years ago that dropping %changelog is
possible, easy and desirable.  We should do that IMHO.

I wouldn't personally bother touching the Release header.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

[Michael Catanzaro]

On Fri, Jan 10, 2020 at 9:46 pm, Richard W.M. Jones <rjones(a)redhat.com&gt; 
wrote:
>  OpenSUSE proved years and years ago that dropping %changelog is
>  possible, easy and desirable.  We should do that IMHO. 
They still have %changelog at the bottom of each spec file, but as the 
last line of the file. The actual changelog is stored as a separate 
.changes file. That's a *lot* better than what Fedora does now, because 
it makes it way easier to scroll through the spec file. But getting rid 
of the changelog entirely would be even nicer. :)

Michael

[Neal Gompa]

On Fri, Jan 10, 2020 at 8:20 PM Michael Catanzaro <mcatanzaro(a)gnome.org&gt; wrote:
>  On Fri, Jan 10, 2020 at 9:46 pm, Richard W.M. Jones <rjones(a)redhat.com&gt;
>  wrote:
>  > OpenSUSE proved years and years ago that dropping %changelog is
>  > possible, easy and desirable.  We should do that IMHO.
>
>  They still have %changelog at the bottom of each spec file, but as the
>  last line of the file. The actual changelog is stored as a separate
>  .changes file. That's a *lot* better than what Fedora does now, because
>  it makes it way easier to scroll through the spec file. But getting rid
>  of the changelog entirely would be even nicer. :)
>  
openSUSE needs the changes file because the integrated VCS in their
build system is awful. History is not able to be preserved across
submissions to projects, so the only complete artifact of history is
that file. If you want an example of a distribution that completely
eliminated the management of an RPM changelog, look at Mageia (and its
ancestor, Mandriva). Their Dist-SVN system dynamically generated
changelogs from the SVN log.

However, they had two properties that we don't have with Git, and we
need some analogues for that:

* Dist-SVN is still SVN, and that means the log data can be edited
without editing the revision itself. This means the message could be
changed after the fact and the next build would incorporate it.
* Dist-SVN relied on tags of successful builds to track the
checkpoints for changelog generation and did not use branches in any
meaningful capacity, so the history was always fully linear.

I think any successful equivalent of that for Dist-Git will require us
to emulate these two properties somehow. My thinking is that annotated
tags would be the right way to allow for changelogs to be set without
the complexity of figuring out commit filtering. The annotation must
be editable. If not, this won't work. We have to have a way to let
people clean up changelogs. I think we can enforce linear tracking
within a branch for Koji to generate Release values, but it's going to
be tricky, given the contortions to the history that Git allows.



--
真実はいつも一つ！/ Always, there's only one truth!

[Przemek Klosowski]

On 1/10/20 8:14 PM, Michael Catanzaro wrote:
>  On Fri, Jan 10, 2020 at 9:46 pm, Richard W.M. Jones 
>  <rjones(a)redhat.com&gt; wrote:
> > OpenSUSE proved years and years ago that dropping %changelog is
> > possible, easy and desirable.  We should do that IMHO.
>
>  They still have %changelog at the bottom of each spec file, but as the 
>  last line of the file. The actual changelog is stored as a separate 
>  .changes file. That's a *lot* better than what Fedora does now, 
>  because it makes it way easier to scroll through the spec file. But 
>  getting rid of the changelog entirely would be even nicer. :)  
changelogs often include CVE information, especially useful when the 
fixes are backported rather than included as part of the regular 
update/release process.

How could the CVE info be available in the absence of changelogs?

[Neal Gompa]

On Mon, Jan 13, 2020 at 2:02 PM Przemek Klosowski via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  On 1/10/20 8:14 PM, Michael Catanzaro wrote:
>  > On Fri, Jan 10, 2020 at 9:46 pm, Richard W.M. Jones
>  > <rjones(a)redhat.com&gt; wrote:
>  >> OpenSUSE proved years and years ago that dropping %changelog is
>  >> possible, easy and desirable.  We should do that IMHO.
>  >
>  > They still have %changelog at the bottom of each spec file, but as the
>  > last line of the file. The actual changelog is stored as a separate
>  > .changes file. That's a *lot* better than what Fedora does now,
>  > because it makes it way easier to scroll through the spec file. But
>  > getting rid of the changelog entirely would be even nicer. :)
>
>  changelogs often include CVE information, especially useful when the
>  fixes are backported rather than included as part of the regular
>  update/release process.
>
>  How could the CVE info be available in the absence of changelogs? 
In Fedora, this information has always been available as part of
updateinfo, just like with RHEL. Only CentOS seems to still not have
updateinfo published for advisories and including security
information.

That said, the information could *still* be in changelogs if the
packager deems so.




--
真実はいつも一つ！/ Always, there's only one truth!

[Przemek Klosowski]

On 1/13/20 2:47 PM, Neal Gompa wrote:
> > changelogs often include CVE information, especially useful when the
> > fixes are backported rather than included as part of the regular
> > update/release process.
> >
> > How could the CVE info be available in the absence of changelogs?
>  In Fedora, this information has always been available as part of
>  updateinfo, just like with RHEL. Only CentOS seems to still not have
>  updateinfo published for advisories and including security
>  information. 
You're right, the updateinfo capability in dnf is awesome!! Thanks for 
bringing it up here, I missed the --cve option. I think specifically

dnf updateinfo list --cve=CVE-2015-2080

should list the packages that address this particular CVE, which would 
be better than grepping changelog for CVEs, except that it didn't work 
for me right now somehow. I found very little info about it, e.g. on 
Oracle pages:

https://docs.oracle.com/en/operating-systems/oracle-linux/8/software-mana...

Is there a better description somewhere, maybe with some examples?


>  That said, the information could *still* be in changelogs if the
>  packager deems so. 
I am all for automating all this---the CVE-in-changelog looked like a 
manual effort on the part of some packagers, so if there's an automatic 
workflow that takes care of it in the updateinfo records, I am all for 
it and won't miss the changelogs.

[Neal Gompa]

On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
>  The release field would need to be set by koji ignoring whatever is in the spec
>  file. How do we want to do this?
>    - Based on dates?
>    - Using an always increasing integer?
>    - Using the number of successful builds since the last time the version field changed?
>    - Another idea?
>
>  The third option looks like to be the one closest to our current behavior.
>  
I always envisioned that we'd use a variant of the third option.

The options I've thought of:

* <commit-at-version>%{dist}.<build-at-version>
* <commit-at-version>.<build-at-version>%{?dist}
* %{dist}.<commit-at-version>.<build-at-version>

The first option aligns with our current guidelines suggesting that
such bumps go at the end after the DistTag. I personally have felt
those were ugly, but there you go. The second option is actually what
I use at $DAYJOB for our environment, and I've been pretty pleased
with how that works in practice (our buildsystem builds for all target
distros from the same commit at once, so the EVRs are rather strictly
matched). However, it might not be workable with the whole "Koji and
manual build per target" thing. The third option is how openSUSE Leap
is currently done, primarily to make Leap packages lower than SUSE
Linux Enterprise packages (openSUSE Leap can "upgrade" to SLE). I'm
not a fan of this scheme as it feels rather "weird" to me.

Naturally, there is also an option that would involve enhancing RPM:
extending EVR comparisons to factor in the DistTag as a formal
property (e.g. EVRD). ALT Linux recently started doing this. I will
admit there is some appeal to this, as it makes the distro versioning
aspect a property unto itself. This would require making Koji move
from NVR to NVRD for its uniqueness constraint, which would be an
"interesting" change to implement. This would eliminate the %dist
macro entirely, and remove boilerplate entirely from the Release
field. This would make all the questions and games about sequencing
the values in the Release field to be completely about the unique
build versioning of the package. In practice, I think this is just a
fancier variant of option two above.

>  Another question regarding the release field is: how would we deal with
>  pre-releases and other unsortable versions?
>  The current doc relies on <extraver> etc. in the release field as per:
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_un...
>  Would using the tilde to specify pre-releases in the version field be sufficient
>  (as described in
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_ve...
>  I.e., how can we cater for snapshot releases (e.g. based off a specific git
>  commit)?
>  
Pre-releases can use tilde versioning, and starting with Fedora 31, we
can also use carat versions for post-releases.

Carat versioning draft from tibbs from long ago:
https://fedoraproject.org/wiki/User:Tibbs/TildeCaretVersioning


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Zbigniew Jędrzejewski-Szmek]

On Fri, Jan 10, 2020 at 09:54:59PM -0500, Neal Gompa wrote:
>  On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon
> <pingou(a)pingoured.fr&gt; wrote:
>  >
>  >
>  > The release field would need to be set by koji ignoring whatever is in the spec
>  > file. 
Yes, please!!! This is a relatively small step that will make so
many other things much easier. This is long overdue.

I think this should be done with two principles:

- that this automatization will coexist for a while with manual
  release tags and changelogs that we have now, so they need to be
  compatible.  Initially this could be opt-in, and once we get a
  feeling that things works nicely, this could be made opt-out or even
  mandatory.

- only support simple versioning (i.e. don't bother supporting
  pre-release or post-release versions in the dist tag. Packages that
  need that would either need to use tile/caret versioning or not be
  supported for automatic tagging.)

>  > How do we want to do this?
>  >   - Based on dates?
>  >   - Using an always increasing integer?
>  >   - Using the number of successful builds since the last time the version field
> changed?
>  >   - Another idea?
>  >
>  > The third option looks like to be the one closest to our current behavior.
>  >
>  
>  I always envisioned that we'd use a variant of the third option.
>  
>  The options I've thought of:
>  
>  * <commit-at-version>%{dist}.<build-at-version>
>  * <commit-at-version>.<build-at-version>%{?dist}
>  * %{dist}.<commit-at-version>.<build-at-version> 
(I understand that <commit-at-version> is basically %version.)

In the first variant, builds from the same %version with different
dist tags would always compare different, so a build from a later
Fedora release would compare higher.

If builds are tagged automatically, the automatic build-number tag
cannot be meanigfully compared between different Fedora releases.
In the second variant, builds from the same %version would compare
by the build number, which seems useless.

>  The first option aligns with our current guidelines suggesting that
>  such bumps go at the end after the DistTag. I personally have felt
>  those were ugly, but there you go. The second option is actually what
>  I use at $DAYJOB for our environment, and I've been pretty pleased
>  with how that works in practice (our buildsystem builds for all target
>  distros from the same commit at once, so the EVRs are rather strictly
>  matched). However, it might not be workable with the whole "Koji and
>  manual build per target" thing. 
Exactly. I don't think this makes sense in
Fedora.

>  The third option is how openSUSE Leap
>  is currently done, primarily to make Leap packages lower than SUSE
>  Linux Enterprise packages (openSUSE Leap can "upgrade" to SLE). I'm
>  not a fan of this scheme as it feels rather "weird" to me. 
Agreed.

>  Naturally, there is also an option that would involve enhancing RPM:
>  extending EVR comparisons to factor in the DistTag as a formal
>  property (e.g. EVRD). ALT Linux recently started doing this. I will
>  admit there is some appeal to this, as it makes the distro versioning
>  aspect a property unto itself. This would require making Koji move
>  from NVR to NVRD for its uniqueness constraint, which would be an
>  "interesting" change to implement. This would eliminate the %dist
>  macro entirely, and remove boilerplate entirely from the Release
>  field. This would make all the questions and games about sequencing
>  the values in the Release field to be completely about the unique
>  build versioning of the package. In practice, I think this is just a
>  fancier variant of option two above. 
I don't think we should go down this route, unless we know for sure
that we can't make things work otherwise. I hope we could phase-in
this automatic tagging in koji, and this will be much easier if we are
able to do this without changes to rpm and the version comparison
logic (which is also embedded in countless other places).

>  > Another question regarding the release field is: how would we
> deal with
>  > pre-releases and other unsortable versions?
>  > The current doc relies on <extraver> etc. in the release field as per:
>  >
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_un...
>  > Would using the tilde to specify pre-releases in the version field be sufficient
>  > (as described in
>  >
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_ve...
>  > I.e., how can we cater for snapshot releases (e.g. based off a specific git
>  > commit)?
>  >
>  
>  Pre-releases can use tilde versioning, and starting with Fedora 31, we
>  can also use carat versions for post-releases.
>  
>  Carat versioning draft from tibbs from long ago:
>  https://fedoraproject.org/wiki/User:Tibbs/TildeCaretVersioning 
I would limit automatic tagging for packages which do "sane"
versioning, i.e. don't embed any pre-release or post-release data in
disttag. FPC should finally approve tile+caret versioning guidelines [1,2].
Right now F30 doesn't support carets, but we could proceed anyway. By
the time this is ready, F30 should have support and F29 will be EOL.

[1] https://pagure.io/packaging-committee/issue/904
[2] https://pagure.io/packaging-committee/pull-request/908

Zbyszek

[Neal Gompa]

On Sat, Jan 11, 2020 at 12:38 PM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Fri, Jan 10, 2020 at 09:54:59PM -0500, Neal Gompa wrote:
>  > On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt;
> wrote:
>  > >
>  > >
>  > > The release field would need to be set by koji ignoring whatever is in the
> spec
>  > > file.
>
>  Yes, please!!! This is a relatively small step that will make so
>  many other things much easier. This is long overdue.
>
>  I think this should be done with two principles:
>
>  - that this automatization will coexist for a while with manual
>    release tags and changelogs that we have now, so they need to be
>    compatible.  Initially this could be opt-in, and once we get a
>    feeling that things works nicely, this could be made opt-out or even
>    mandatory.
>
>  - only support simple versioning (i.e. don't bother supporting
>    pre-release or post-release versions in the dist tag. Packages that
>    need that would either need to use tile/caret versioning or not be
>    supported for automatic tagging.)
>  
I think we have to make sure there's no option in which automatic
tagging is not supportable. Otherwise we can basically never
transition fully over.

>  > > How do we want to do this?
>  > >   - Based on dates?
>  > >   - Using an always increasing integer?
>  > >   - Using the number of successful builds since the last time the version field
> changed?
>  > >   - Another idea?
>  > >
>  > > The third option looks like to be the one closest to our current behavior.
>  > >
>  >
>  > I always envisioned that we'd use a variant of the third option.
>  >
>  > The options I've thought of:
>  >
>  > * <commit-at-version>%{dist}.<build-at-version>
>  > * <commit-at-version>.<build-at-version>%{?dist}
>  > * %{dist}.<commit-at-version>.<build-at-version>
>
>  (I understand that <commit-at-version> is basically %version.)
>  
To clarify, <commit-at-version> is a counter of the number of commits
since %version was bump, starting at 1. <build-at-version> is a
counter for the number of builds with that
%version-<commit-at-version>, starting at 1.

>  In the first variant, builds from the same %version with different
>  dist tags would always compare different, so a build from a later
>  Fedora release would compare higher.
>
>  If builds are tagged automatically, the automatic build-number tag
>  cannot be meanigfully compared between different Fedora releases.
>  In the second variant, builds from the same %version would compare
>  by the build number, which seems useless.
>
>  > The first option aligns with our current guidelines suggesting that
>  > such bumps go at the end after the DistTag. I personally have felt
>  > those were ugly, but there you go. The second option is actually what
>  > I use at $DAYJOB for our environment, and I've been pretty pleased
>  > with how that works in practice (our buildsystem builds for all target
>  > distros from the same commit at once, so the EVRs are rather strictly
>  > matched). However, it might not be workable with the whole "Koji and
>  > manual build per target" thing.
>  Exactly. I don't think this makes sense in Fedora.
>  
The only reason I mentioned it is because since we distro-sync between
releases, it doesn't actually matter as much as it used to.

>  > The third option is how openSUSE Leap
>  > is currently done, primarily to make Leap packages lower than SUSE
>  > Linux Enterprise packages (openSUSE Leap can "upgrade" to SLE). I'm
>  > not a fan of this scheme as it feels rather "weird" to me.
>  Agreed.
>
>  > Naturally, there is also an option that would involve enhancing RPM:
>  > extending EVR comparisons to factor in the DistTag as a formal
>  > property (e.g. EVRD). ALT Linux recently started doing this. I will
>  > admit there is some appeal to this, as it makes the distro versioning
>  > aspect a property unto itself. This would require making Koji move
>  > from NVR to NVRD for its uniqueness constraint, which would be an
>  > "interesting" change to implement. This would eliminate the %dist
>  > macro entirely, and remove boilerplate entirely from the Release
>  > field. This would make all the questions and games about sequencing
>  > the values in the Release field to be completely about the unique
>  > build versioning of the package. In practice, I think this is just a
>  > fancier variant of option two above.
>
>  I don't think we should go down this route, unless we know for sure
>  that we can't make things work otherwise. I hope we could phase-in
>  this automatic tagging in koji, and this will be much easier if we are
>  able to do this without changes to rpm and the version comparison
>  logic (which is also embedded in countless other places).
>  
Another way to go here would be to do EVDR instead of EVRD.
This more closely mirrors the intent that we have in Fedora for newer
distribution releases to always be considered "upgrades" from older
ones.

In this case, it would make sense to enhance RPM and leverage DistTag
for version comparison.

Unlike the equivalent variant with the Release field (like how
openSUSE does), we wouldn't have to take a hit with lots of things
looking like it is "downgrading" for a distro upgrade.

>  > > Another question regarding the release field is: how would
> we deal with
>  > > pre-releases and other unsortable versions?
>  > > The current doc relies on <extraver> etc. in the release field as per:
>  > >
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_un...
>  > > Would using the tilde to specify pre-releases in the version field be
> sufficient
>  > > (as described in
>  > >
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_ve...
>  > > I.e., how can we cater for snapshot releases (e.g. based off a specific git
>  > > commit)?
>  > >
>  >
>  > Pre-releases can use tilde versioning, and starting with Fedora 31, we
>  > can also use carat versions for post-releases.
>  >
>  > Carat versioning draft from tibbs from long ago:
>  > https://fedoraproject.org/wiki/User:Tibbs/TildeCaretVersioning
>
>  I would limit automatic tagging for packages which do "sane"
>  versioning, i.e. don't embed any pre-release or post-release data in
>  disttag. FPC should finally approve tile+caret versioning guidelines [1,2].
>  Right now F30 doesn't support carets, but we could proceed anyway. By
>  the time this is ready, F30 should have support and F29 will be EOL.
>
>  [1] https://pagure.io/packaging-committee/issue/904
>  [2] https://pagure.io/packaging-committee/pull-request/908
>  
And carat versions are being backported to EL8:
https://bugzilla.redhat.com/show_bug.cgi?id=1654901







--
真実はいつも一つ！/ Always, there's only one truth!

[Nicolas Mailhot]

Le samedi 11 janvier 2020 à 13:09 -0500, Neal Gompa a écrit :
>  
>  The only reason I mentioned it is because since we distro-sync
>  between
>  releases, it doesn't actually matter as much as it used to. 
rawhide does not distro-sync (and some may say that rawhide does not
matter, but early problem detection by rawhide users is one big reason
distro sync works so well for other releases)

-- 
Nicolas Mailhot

[Neal Gompa]

On Sun, Jan 12, 2020 at 4:03 AM Nicolas Mailhot via devel
<devel(a)lists.fedoraproject.org&gt; wrote:
>  Le samedi 11 janvier 2020 à 13:09 -0500, Neal Gompa a écrit :
>  >
>  > The only reason I mentioned it is because since we distro-sync
>  > between
>  > releases, it doesn't actually matter as much as it used to.
>
>  rawhide does not distro-sync (and some may say that rawhide does not
>  matter, but early problem detection by rawhide users is one big reason
>  distro sync works so well for other releases)
>  
It is strongly recommended that you should be using 'dnf distro-sync'
for Rawhide if you want to use it as a daily driver. That handles all
the cases that occur in a rolling branch effectively.



--
真実はいつも一つ！/ Always, there's only one truth!

[Kevin Fenzi]

On Sun, Jan 12, 2020 at 04:38:55AM -0500, Neal Gompa wrote:
>  On Sun, Jan 12, 2020 at 4:03 AM Nicolas Mailhot via devel
>  <devel(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > Le samedi 11 janvier 2020 à 13:09 -0500, Neal Gompa a écrit :
>  > >
>  > > The only reason I mentioned it is because since we distro-sync
>  > > between
>  > > releases, it doesn't actually matter as much as it used to.
>  >
>  > rawhide does not distro-sync (and some may say that rawhide does not
>  > matter, but early problem detection by rawhide users is one big reason
>  > distro sync works so well for other releases) 
It sometimes does. Sometimes it does not.

>  It is strongly recommended that you should be using 'dnf
> distro-sync'
>  for Rawhide if you want to use it as a daily driver. That handles all
>  the cases that occur in a rolling branch effectively. 
It does not. 

For example, right now on my laptop: 

Error: 
 Problem: problem with installed package awscli-1.16.309-1.fc32.noarch
  - package awscli-1.16.309-1.fc32.noarch requires python3.8dist(botocore) = 1.13.45, but
none of the providers can be installed
  - python3-botocore-1.13.45-2.fc32.noarch does not belong to a distupgrade repository
  - awscli-1.16.309-1.fc32.noarch does not belong to a distupgrade repository
(try to add '--skip-broken' to skip uninstallable packages)

(--skip-broken doesn't do anything here, same exact output)

distro-sync is only going to work if the repo(s) are all consistent for
you to sync to. :( Which I think we should get to, but it's not the case
yet. 

kevin

[Miroslav Suchý]

Dne 11. 01. 20 v 3:54 Neal Gompa napsal(a):
>  * %{dist}.<commit-at-version>.<build-at-version> 
-1
Packages with commitish in release version are usually developers snapshot.
We already have few packages with such release in Fedora, but I would dislike to make this
standard.

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Neal Gompa]

On Wed, Jan 15, 2020 at 7:10 AM Miroslav Suchý <msuchy(a)redhat.com&gt; wrote:
>  Dne 11. 01. 20 v 3:54 Neal Gompa napsal(a):
>  > * %{dist}.<commit-at-version>.<build-at-version>
>
>  -1
>  Packages with commitish in release version are usually developers snapshot.
>  We already have few packages with such release in Fedora, but I would dislike to make
> this standard. 
That is not putting the hash in there. That is a counter. It's poorly labeled.

Basically, an NEVRA would look like this:

foo-0:1.0.0-fc32.1.1.noarch

"commit-at-version" is a counter, starting from 1 that indicates the
number of commits in the build system of the version of a package. So
version 1.1 would start at 1 initially, and every subsequent commit
where you _don't_ change the version, this number goes up.

"build-at-version" is a counter, starting from 1 that indicates the
number of builds in the build system of the commit at that version of
a package. So at version 1.1 with commit-at-version 1 would initially
have build-at-version 1 for the first build, but on the fourth
rebuild, it would be 5.

Given that, here's what the options I presented would look like the
following Release fields:

* <commit-at-version>%{dist}.<build-at-version>: 1.fc32.5
* <commit-at-version>.<build-at-version>%{?dist}: 1.5.fc32
* %{dist}.<commit-at-version>.<build-at-version>: fc32.1.5


--
真実はいつも一つ！/ Always, there's only one truth!

[Pierre-Yves Chibon]

On Fri, Jan 10, 2020 at 09:54:59PM -0500, Neal Gompa wrote:
>  On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon
> <pingou(a)pingoured.fr&gt; wrote:
>  >
>  >
>  > The release field would need to be set by koji ignoring whatever is in the spec
>  > file. How do we want to do this?
>  >   - Based on dates?
>  >   - Using an always increasing integer?
>  >   - Using the number of successful builds since the last time the version field
> changed?
>  >   - Another idea?
>  >
>  > The third option looks like to be the one closest to our current behavior.
>  >
>  
>  I always envisioned that we'd use a variant of the third option.
>  
>  The options I've thought of:
>  
>  * <commit-at-version>%{dist}.<build-at-version>
>  * <commit-at-version>.<build-at-version>%{?dist}
>  * %{dist}.<commit-at-version>.<build-at-version> 
I've been thinking a bit about this and been wondering any reason why not to do
simply?
   <build-at-version>%{dist}

This would basically mimic what we are currently doing by hand, it would be the
less changes to our current way of working (making opting-in smoother).


Thanks,
Pierre

[Neal Gompa]

On Tue, Jan 28, 2020 at 7:27 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
>  On Fri, Jan 10, 2020 at 09:54:59PM -0500, Neal Gompa wrote:
>  > On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt;
> wrote:
>  > >
>  > >
>  > > The release field would need to be set by koji ignoring whatever is in the
> spec
>  > > file. How do we want to do this?
>  > >   - Based on dates?
>  > >   - Using an always increasing integer?
>  > >   - Using the number of successful builds since the last time the version field
> changed?
>  > >   - Another idea?
>  > >
>  > > The third option looks like to be the one closest to our current behavior.
>  > >
>  >
>  > I always envisioned that we'd use a variant of the third option.
>  >
>  > The options I've thought of:
>  >
>  > * <commit-at-version>%{dist}.<build-at-version>
>  > * <commit-at-version>.<build-at-version>%{?dist}
>  > * %{dist}.<commit-at-version>.<build-at-version>
>
>  I've been thinking a bit about this and been wondering any reason why not to do
>  simply?
>     <build-at-version>%{dist}
>
>  This would basically mimic what we are currently doing by hand, it would be the
>  less changes to our current way of working (making opting-in smoother).
>  
If we're not doing automatic builds, sure. I've been going on two big
assumptions:

1. We're going to do automatic building
2. We need *some* kind of stable leading portion of release for
packagers to use for specified dependencies, especially
Obsoletes+Provides combos.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Dan Čermák]

Neal Gompa <ngompa13(a)gmail.com&gt; writes:

>  On Tue, Jan 28, 2020 at 7:27 AM Pierre-Yves Chibon
> <pingou(a)pingoured.fr&gt; wrote:
> >
> > On Fri, Jan 10, 2020 at 09:54:59PM -0500, Neal Gompa wrote:
> > > On Fri, Jan 10, 2020 at 11:37 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt;
> wrote:
> > > >
> > > >
> > > > The release field would need to be set by koji ignoring whatever is in the
> spec
> > > > file. How do we want to do this?
> > > >   - Based on dates?
> > > >   - Using an always increasing integer?
> > > >   - Using the number of successful builds since the last time the version
> field changed?
> > > >   - Another idea?
> > > >
> > > > The third option looks like to be the one closest to our current behavior.
> > > >
> > >
> > > I always envisioned that we'd use a variant of the third option.
> > >
> > > The options I've thought of:
> > >
> > > * <commit-at-version>%{dist}.<build-at-version>
> > > * <commit-at-version>.<build-at-version>%{?dist}
> > > * %{dist}.<commit-at-version>.<build-at-version>
> >
> > I've been thinking a bit about this and been wondering any reason why not to do
> > simply?
> >    <build-at-version>%{dist}
> >
> > This would basically mimic what we are currently doing by hand, it would be the
> > less changes to our current way of working (making opting-in smoother).
> >
>
>  If we're not doing automatic builds, sure. I've been going on two big
>  assumptions:
>
>  1. We're going to do automatic building
>  2. We need *some* kind of stable leading portion of release for
>  packagers to use for specified dependencies, especially
>  Obsoletes+Provides combos. 
How is openSUSE taking care of 2 then? OBS bumps the release on each
rebuild and that can result in crazily high release numbers. I assume
they got a mechanism for Obsoletes+Provides and we could use that too?

[Marius Schwarz]

Am 10.01.20 um 17:36 schrieb Pierre-Yves Chibon:
>  Good Morning Everyone,
>
>  This is not a new idea, it has been presented at flock last year and spoken
>  about on this very list this fall, so I'd like to push it a little further.
>
>  Do we want to drop release and changelog from our spec file? 
Vote: no.

The correct releases and changelogs in the rpms are important to check
for security patches made. This need of any admin will override
any argument for a removal, as it's the most important source on a
working system to gather it's security state.

Best regards,
Marius

[Neal Gompa]

On Sun, Jan 12, 2020 at 4:20 PM Marius Schwarz <fedoradev(a)cloud-foo.de&gt; wrote:
>  Am 10.01.20 um 17:36 schrieb Pierre-Yves Chibon:
>  > Good Morning Everyone,
>  >
>  > This is not a new idea, it has been presented at flock last year and spoken
>  > about on this very list this fall, so I'd like to push it a little further.
>  >
>  > Do we want to drop release and changelog from our spec file?
>  Vote: no.
>
>  The correct releases and changelogs in the rpms are important to check
>  for security patches made. This need of any admin will override
>  any argument for a removal, as it's the most important source on a
>  working system to gather it's security state.
>  
The idea is to remove the manual management of them, not the existence
of them altogether. I agree with you in that we should not remove them
entirely for many of the same reasons.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Miro Hrončok]

On 12. 01. 20 22:19, Marius Schwarz wrote:
>  Am 10.01.20 um 17:36 schrieb Pierre-Yves Chibon:
> > Good Morning Everyone,
> >
> > This is not a new idea, it has been presented at flock last year and spoken
> > about on this very list this fall, so I'd like to push it a little further.
> >
> > Do we want to drop release and changelog from our spec file?
>  Vote: no.
>  
>  The correct releases and changelogs in the rpms are important to check
>  for security patches made. This need of any admin will override
>  any argument for a removal, as it's the most important source on a
>  working system to gather it's security state. 
It would stay in the RPM, we would just populate it differently and it would no 
longer be hardcoded in the spec file in our infrastructure.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Joe Doss]

On 1/12/20 3:19 PM, Marius Schwarz wrote:
>  Am 10.01.20 um 17:36 schrieb Pierre-Yves Chibon:
> > Good Morning Everyone,
> >
> > This is not a new idea, it has been presented at flock last year and 
spoken
> > about on this very list this fall, so I'd like to push it a
> little 
further.
> >
> > Do we want to drop release and changelog from our spec file?
>  Vote: no.
>
>  The correct releases and changelogs in the rpms are important to check
>  for security patches made. This need of any admin will override
>  any argument for a removal, as it's the most important source on a
>  working system to gather it's security state. 
Finally the reply I was looking for! As someone who relies the changelog
of the RPM for security reasons this whole thread has me worried.

On 1/12/20 3:38 PM, Miro Hrončok wrote:
>  It would stay in the RPM, we would just populate it differently and
> it
>  would no longer be hardcoded in the spec file in our infrastructure. 
How will it be populated? Will it ensure that the information that is
important for security minding end users is still available? Sorry in
advance if I missed the details of how it would still be managed and
included for end users to consume?

Thanks!
Joe


-- 
Joe Doss
joe(a)solidadmin.com

[Stephen John Smoogen]

On Mon, 13 Jan 2020 at 15:23, Joe Doss <joe(a)solidadmin.com&gt; wrote:
>  On 1/12/20 3:19 PM, Marius Schwarz wrote:
>  > Am 10.01.20 um 17:36 schrieb Pierre-Yves Chibon:
>  >> Good Morning Everyone,
>  >>
>  >> This is not a new idea, it has been presented at flock last year and
>  spoken
>  >> about on this very list this fall, so I'd like to push it a little
>  further.
>  >>
>  >> Do we want to drop release and changelog from our spec file?
>  > Vote: no.
>  >
>  > The correct releases and changelogs in the rpms are important to check
>  > for security patches made. This need of any admin will override
>  > any argument for a removal, as it's the most important source on a
>  > working system to gather it's security state.
>
>  Finally the reply I was looking for! As someone who relies the changelog
>  of the RPM for security reasons this whole thread has me worried.
>
>  On 1/12/20 3:38 PM, Miro Hrončok wrote:
>  > It would stay in the RPM, we would just populate it differently and it
>  > would no longer be hardcoded in the spec file in our infrastructure.
>
>  How will it be populated? Will it ensure that the information that is
>  important for security minding end users is still available? Sorry in
>  advance if I missed the details of how it would still be managed and
>  included for end users to consume? 
The CVE information there is mostly on the whim of the packager. Some
packagers do put items in there and others forget (aka I have
forgotten to do so a couple of times).

How will the new way be populated? That is what part of this thread is
trying to get to. It has not been decided or implemented but would
hopefully be part of getting the changelog out



-- 
Stephen J Smoogen.

[Miro Hrončok]

On 10. 01. 20 17:36, Pierre-Yves Chibon wrote:
>  Is there a different approach, e.g. by using towncrier[1] or
> something
>  comparable, to track changes outside the spec file? 
Is the idea of using annotated git tags abandoned altogether?

We could even create a tool that would "prefill" a template with all commit 
messages since the latest annotated tag.

So you would do:


- commit, commit, commit (optional pushes)
- fedpkg release
- edit the message, inspect the e:v-r, be able to abort if I don't like it
- fedpkg push - pushes the tags and branch
- fedpkg build


And when somebody attempts to do a untagged build, it would fail, similarly to 
what it does now when you try to build unpushed changes.



Example template:


     # You are about to tag foobar 1.1-1
     # Here are the commit messages since 1.0-6
     #
     # This is the 1st commit message:

     Update to 1.1

     # This is the commit message #2:

     Damn sources

     # This is the commit message #3:

     Fix the tests

     # This is the commit message #4:

     Typo

     # Please amend the commit messages to a %changelog entry. Lines starting
     # with '#' will be ignored, and an empty message aborts the release.
     # If you like to change the release number, abort the process and follow
     # <link to docs>
     #
     # Author:    Miro Hrončok <mhroncok(a)redhat.com&gt;
     # Date:      Sun Jan 12 22:54:32 2020 +0100
     #
     # Use --author and/or --date to change the above.


-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Pierre-Yves Chibon]

On Sun, Jan 12, 2020 at 10:56:00PM +0100, Miro Hrončok wrote:
>  On 10. 01. 20 17:36, Pierre-Yves Chibon wrote:
>  > Is there a different approach, e.g. by using towncrier[1] or something
>  > comparable, to track changes outside the spec file?
>  
>  Is the idea of using annotated git tags abandoned altogether? 
It was the sentence just above the section you quoted here ;-)

For memory:
--
So we need to, somehow, merge two changelogs into one while realizing that some
information in one may not be desirable in the other (for example the world
famous commit message: "oops I've forgot to upload the sources" does not
need to
appear in the RPM's changelog).
Would it be easier to merge the git changelog with the spec changelog or the
spec changelog with the bodhi notes?

For the former one easy way to achieve this is to consider all the commits since
the last successful build and have a magic keyword to either include or exclude
a commit message in the changelog.
For the latter, we discussed the idea of using annotated git tags this fall.
--

>  We could even create a tool that would "prefill" a template
> with all commit
>  messages since the latest annotated tag.
>  
>  So you would do:
>  
>  
>  - commit, commit, commit (optional pushes)
>  - fedpkg release
>  - edit the message, inspect the e:v-r, be able to abort if I don't like it
>  - fedpkg push - pushes the tags and branch
>  - fedpkg build
>  
>  
>  And when somebody attempts to do a untagged build, it would fail, similarly
>  to what it does now when you try to build unpushed changes. 
This is definitely an option, it might even be the simplest to implement.
One observation about it though is that we would still have three changelogs
then, git commit, git tags and bodhi update.
So it is easier to implement as we rely on the packager to do the review of what
goes into the RPM changelog.
We still have Neal's question: do we let packager edit these tags? (Considering
we allow editing old changelog entries, I guess we could).

One question, if the release is set in koji and the changelog in git tag.
If I tag foo 1.0 with a first changelog entry, then koji builds 1.0-1 with that
changelog.
I find out that there is a mistake in the changelog, so I edit the tag, won't
koji build 1.0-2? Which would then have two repeating changelog entries one for
-1 with the error and one for -2 with the fix.
Does it matter?

Note that this question may be applicable regardless of the solution we adopt
(git tag, git commits or something else?)


Pierre

[Miro Hrončok]

On 13. 01. 20 12:28, Pierre-Yves Chibon wrote:
>  On Sun, Jan 12, 2020 at 10:56:00PM +0100, Miro Hrončok wrote:
> > On 10. 01. 20 17:36, Pierre-Yves Chibon wrote:
> >> Is there a different approach, e.g. by using towncrier[1] or something
> >> comparable, to track changes outside the spec file?
> >
> > Is the idea of using annotated git tags abandoned altogether?
>  
>  It was the sentence just above the section you quoted here ;-) 
"For the latter, we discussed the idea of using annotated git tags this fall."

Sorry about that, I must have missed it.

> > We could even create a tool that would "prefill" a
> template with all commit
> > messages since the latest annotated tag.
> >
> > So you would do:
> >
> >
> > - commit, commit, commit (optional pushes)
> > - fedpkg release
> > - edit the message, inspect the e:v-r, be able to abort if I don't like it
> > - fedpkg push - pushes the tags and branch
> > - fedpkg build
> >
> >
> > And when somebody attempts to do a untagged build, it would fail, similarly
> > to what it does now when you try to build unpushed changes.
>  
>  This is definitely an option, it might even be the simplest to implement.
>  One observation about it though is that we would still have three changelogs
>  then, git commit, git tags and bodhi update.
>  So it is easier to implement as we rely on the packager to do the review of what
>  goes into the RPM changelog.
>  We still have Neal's question: do we let packager edit these tags? (Considering
>  we allow editing old changelog entries, I guess we could). 
If we build from the tag hash in Koji, we cannot.
If we build from the commit hash and let Koji "fetch" the changelog from 
existing tags, we can.

We still need to construct the changelog entries from old tags, to the second 
option makes sense.

>  One question, if the release is set in koji and the changelog in git
> tag.
>  If I tag foo 1.0 with a first changelog entry, then koji builds 1.0-1 with that
>  changelog.
>  I find out that there is a mistake in the changelog, so I edit the tag, won't
>  koji build 1.0-2? Which would then have two repeating changelog entries one for
>  -1 with the error and one for -2 with the fix.
>  Does it matter? 
Currently, you cannot change already built existing changelog entry without a 
bump and rebuild. This would be the same:

1. You change the changelog entry of 1.0-1.
2. You attempt build.
3. Koji tells you 1.0-1 was already build.
4. You add another entry.

It doesn't matter that much. Obviously, the ability to edit tags puts some 
constrains on how the release is counted:

- if it is counted simply from number of tags, you cannot ever remove them, just 
edit them to "empty changelog"
- locally, it counts all tags, and in Koji, only pushed tags, this may produce 
confusion in corner cases - even if we only count pushed tags locally, somebody 
else might have pushed another one in the meantime
- Hence, we might count the release number from commits, always

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Randy Barlow]

On Mon, 2020-01-13 at 12:28 +0100, Pierre-Yves Chibon wrote:
>  If I tag foo 1.0 with a first changelog entry, then koji builds 1.0-1
>
>  with that changelog. 
I think it would be better if the release were part of the git tag,
instead of automatically generating it. Not all packages use an integer
for the release, and the release also encodes the distro version
(though there is a proposal in the thread to change that too)
currently. Some packages do this due to a policy for packages that
don't have releases from upstream. For example:

$ rpm -q dino
dino-0.0-0.15.20191216.git.11c18cd.fc30.x86_64

Dino doesn't make releases upstream, so the release field encodes the
commit and commit date, along with an incremental "integer", which is
really a float because we want it to sort less than 1, in case Dino
ever makes a 0.0 release.

Now, we could change the policy too to get around this, but I think
"git tag" is a natural way for me to indicate "I want to build and
release this commit to this Fedora release". I can encode the full NEVR
in the git tag, and I can use the tag body to encode the end user
release notes for Bodhi and for the changelog. This also allows the git
commit messages to be logs for packagers to read (I do like the
proposal of a fedpkg feature to prefill the git tag message body with
the git commits since last release as a starting point though!)

[Nicolas Mailhot]

Le 2020-01-13 16:20, Randy Barlow a écrit :

>  Now, we could change the policy too to get around this, but I think
>  "git tag" is a natural way for me to indicate "I want to build and
>  release this commit to this Fedora release". 
Please do not try to infer packaged commit from src.fedoraproject.org 
commit tags.

As a general principle, inventing complex string structures that require 
complex one-of-a-king parsers (with heuristics that always fail one way 
or another) does not work. rpm is not perl (for a long time). Keep a 
separate variable/field/tag for separate info, do not multiplex them.

Practically, we already have a standard way to set target commit in the 
distribution, %{commit} and several hundred distro packages depend on 
this mecanism.

Auto-release, if it exists, should correspond to spec release info and 
nothing more, with all the packaged project info read from the spec. Or 
else we’ll get madness like a tag that says something is at 
version/tag/commit X and the tagged spec that declares it is Y.

Regards,

-- 
Nicolas Mailhot

[Vít Ondruch]

Dne 12. 01. 20 v 22:56 Miro Hrončok napsal(a):
>  On 10. 01. 20 17:36, Pierre-Yves Chibon wrote:
> > Is there a different approach, e.g. by using towncrier[1] or something
> > comparable, to track changes outside the spec file?
>
>  Is the idea of using annotated git tags abandoned altogether?
>
>  We could even create a tool that would "prefill" a template with all
>  commit messages since the latest annotated tag.
>
>  So you would do:
>
>
>  - commit, commit, commit (optional pushes)
>  - fedpkg release
>  - edit the message, inspect the e:v-r, be able to abort if I don't
>  like it
>  - fedpkg push - pushes the tags and branch
>  - fedpkg build
>
>
>  And when somebody attempts to do a untagged build, it would fail,
>  similarly to what it does now when you try to build unpushed changes.
>
>
>
>  Example template:
>
>
>      # You are about to tag foobar 1.1-1
>      # Here are the commit messages since 1.0-6
>      #
>      # This is the 1st commit message:
>
>      Update to 1.1
>
>      # This is the commit message #2:
>
>      Damn sources
>
>      # This is the commit message #3:
>
>      Fix the tests
>
>      # This is the commit message #4:
>
>      Typo
>
>      # Please amend the commit messages to a %changelog entry. Lines
>  starting
>      # with '#' will be ignored, and an empty message aborts the release.
>      # If you like to change the release number, abort the process and
>  follow
>      # <link to docs>
>      #
>      # Author:    Miro Hrončok <mhroncok(a)redhat.com&gt;
>      # Date:      Sun Jan 12 22:54:32 2020 +0100
>      #
>      # Use --author and/or --date to change the above.
>
>  
While I like the annotated tag proposal, I would really appreciate if
the first step was replacing the:


~~~

%changelog

* Tue Jan 07 2020 Vít Ondruch <vondruch(a)redhat.com&gt; - 2.7.0-1
- Upgrade to Ruby 2.7.0.
- Drop useless %%{rubygems_default_filter}.
- Fix checksec 2.0+ compatibility.

* Tue Jun 25 2019 Vít Ondruch <vondruch(a)redhat.com&gt; - 2.6.3-121
- Properly support %%prerelease in %%gemspec_ macros.

... snip ...

~~~


in .spec file by


~~~

%changelog

%include changelog

~~~


where the `changelog` file would be either available with the original
changelog content in repository or if it was not available, it would be
provided by build infra. The *provided by infra* is the most important
thing here. This IMO would allow us incrementally improve the situation.

Then we could discuss how to provide the content of the `changelog`
file, while maintainers could still maintain old changelogs, or they
could split the changelog into separate `changelog` file and maintain it
there, or leave it for infrastructure (simple collecting git commits) or
using annotated tags.


Vít

[Vít Ondruch]

Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
>  Dne 12. 01. 20 v 22:56 Miro Hrončok napsal(a):
> > On 10. 01. 20 17:36, Pierre-Yves Chibon wrote:
> >> Is there a different approach, e.g. by using towncrier[1] or something
> >> comparable, to track changes outside the spec file?
> > Is the idea of using annotated git tags abandoned altogether?
> >
> > We could even create a tool that would "prefill" a template with all
> > commit messages since the latest annotated tag.
> >
> > So you would do:
> >
> >
> > - commit, commit, commit (optional pushes)
> > - fedpkg release
> > - edit the message, inspect the e:v-r, be able to abort if I don't
> > like it
> > - fedpkg push - pushes the tags and branch
> > - fedpkg build
> >
> >
> > And when somebody attempts to do a untagged build, it would fail,
> > similarly to what it does now when you try to build unpushed changes.
> >
> >
> >
> > Example template:
> >
> >
> >     # You are about to tag foobar 1.1-1
> >     # Here are the commit messages since 1.0-6
> >     #
> >     # This is the 1st commit message:
> >
> >     Update to 1.1
> >
> >     # This is the commit message #2:
> >
> >     Damn sources
> >
> >     # This is the commit message #3:
> >
> >     Fix the tests
> >
> >     # This is the commit message #4:
> >
> >     Typo
> >
> >     # Please amend the commit messages to a %changelog entry. Lines
> > starting
> >     # with '#' will be ignored, and an empty message aborts the release.
> >     # If you like to change the release number, abort the process and
> > follow
> >     # <link to docs>
> >     #
> >     # Author:    Miro Hrončok <mhroncok(a)redhat.com&gt;
> >     # Date:      Sun Jan 12 22:54:32 2020 +0100
> >     #
> >     # Use --author and/or --date to change the above.
> >
> >
>  While I like the annotated tag proposal, I would really appreciate if
>  the first step was replacing the:
>
>
>  ~~~
>
>  %changelog
>
>  * Tue Jan 07 2020 Vít Ondruch <vondruch(a)redhat.com&gt; - 2.7.0-1
>  - Upgrade to Ruby 2.7.0.
>  - Drop useless %%{rubygems_default_filter}.
>  - Fix checksec 2.0+ compatibility.
>
>  * Tue Jun 25 2019 Vít Ondruch <vondruch(a)redhat.com&gt; - 2.6.3-121
>  - Properly support %%prerelease in %%gemspec_ macros.
>
>  ... snip ...
>
>  ~~~
>
>
>  in .spec file by
>
>
>  ~~~
>
>  %changelog
>
>  %include changelog
>
>  ~~~
>
>
>  where the `changelog` file would be either available with the original
>  changelog content in repository or if it was not available, it would be
>  provided by build infra. The *provided by infra* is the most important
>  thing here. This IMO would allow us incrementally improve the situation.
>
>  Then we could discuss how to provide the content of the `changelog`
>  file, while maintainers could still maintain old changelogs, or they
>  could split the changelog into separate `changelog` file and maintain it
>  there, or leave it for infrastructure (simple collecting git commits) or
>  using annotated tags. 

Actually, I took the liberty and filled this guideline proposal change
[1], which just suggests keeping changelogs in `changelog` file, outside
of the .spec file.


Vít


[1] https://pagure.io/packaging-committee/pull-request/942

[Felix Schwarz]

Am 13.01.20 um 14:05 schrieb Vít Ondruch:
>  While I like the annotated tag proposal, I would really appreciate
> if
>  the first step was replacing the: 
(..:)
>  %changelog
>  
>  %include changelog
>  
>  ~~~
>  
>  where the `changelog` file would be either available with the original
>  changelog content in repository or if it was not available, it would be
>  provided by build infra. The *provided by infra* is the most important
>  thing here. This IMO would allow us incrementally improve the situation.
>  
>  Then we could discuss how to provide the content of the `changelog`
>  file, while maintainers could still maintain old changelogs, or they
>  could split the changelog into separate `changelog` file and maintain it
>  there, or leave it for infrastructure (simple collecting git commits) or
>  using annotated tags. 
I like that proposal. As a volunteer packager with only a few packages this
appproach is very easy to understand and I can opt out easily if I dislike the
automation (I'd like not to maintain a changelog manually but I also don't
want to have it auto-generated based on all commit messages).

If we look at the modularity disaster I think we should first start with a
very simple approach which is easy to understand but also provides an opt-out.

Felix

[Miroslav Suchý]

Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
>  %changelog
>  
>  %include changelog 
+1

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Panu Matilainen]

On 1/15/20 2:13 PM, Miroslav Suchý wrote:
>  Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
> > %changelog
> >
> > %include changelog
>  
>  +1
>   
As I pointed out in 
https://pagure.io/packaging-committee/pull-request/942, %include is 
nasty because it breaks the stand-alone attribute of specs. There are 
umphteen dupes and variants of bugs about systemd.spec's use of %include 
breaking this and that use pattern that aren't systemd's fault, it's 
just that %include isn't as useful as it initially seems.

	- Panu -

[Vít Ondruch]

Dne 15. 01. 20 v 13:33 Panu Matilainen napsal(a):
>  On 1/15/20 2:13 PM, Miroslav Suchý wrote:
> > Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
> >> %changelog
> >>
> >> %include changelog
> >
> > +1
> >
>
>  As I pointed out in
>  https://pagure.io/packaging-committee/pull-request/942, %include is
>  nasty because it breaks the stand-alone attribute of specs. There are
>  umphteen dupes and variants of bugs about systemd.spec's use of
>  %include breaking this and that use pattern that aren't systemd's
>  fault, it's just that %include isn't as useful as it initially seems. 

Would it be helpful to open RPM upstream ticket to discuss when the
missing included file is problematic and whether there should be other
graceful variant of include? May be the %include should always behave
gracefully, because (S)RPM build is going to always fail due to missing
files specified by Source directive.


Vít


>      - Panu -
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]

[Panu Matilainen]

On 1/15/20 3:33 PM, Vít Ondruch wrote:
>  
>  Dne 15. 01. 20 v 13:33 Panu Matilainen napsal(a):
> > On 1/15/20 2:13 PM, Miroslav Suchý wrote:
> >> Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
> >>> %changelog
> >>>
> >>> %include changelog
> >>
> >> +1
> >>
> >
> > As I pointed out in
> > https://pagure.io/packaging-committee/pull-request/942, %include is
> > nasty because it breaks the stand-alone attribute of specs. There are
> > umphteen dupes and variants of bugs about systemd.spec's use of
> > %include breaking this and that use pattern that aren't systemd's
> > fault, it's just that %include isn't as useful as it initially seems.
>  
>  
>  Would it be helpful to open RPM upstream ticket to discuss when the
>  missing included file is problematic and whether there should be other
>  graceful variant of include? May be the %include should always behave
>  gracefully, because (S)RPM build is going to always fail due to missing
>  files specified by Source directive.
>   
This was actually discussed upstream not too long ago, just can't find 
the reference offhand.

Basically an %include can never be allowed to fail as it can contain 
anything at all, including mandatory parts of the spec. In theory you 
can have a spec consisting of nothing but one or more %include directives.

(S)RPM builds are not an issue, it's spec queries, in particular for 
build-dependencies but also for other data. Rpm has special logic to 
allow missing sources and patches for query purposes, because they're 
only needed for building. Something changelog-specific could also safely 
ignore the missing file as %changelog is always an optional part of the 
spec.

	- Panu -

[David Cantrell]

On Wed, Jan 15, 2020 at 04:28:56PM +0200, Panu Matilainen wrote:
> On 1/15/20 3:33 PM, Vít Ondruch wrote:
> >
> >Dne 15. 01. 20 v 13:33 Panu Matilainen napsal(a):
> >>On 1/15/20 2:13 PM, Miroslav Suchý wrote:
> >>>Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
> >>>>%changelog
> >>>>
> >>>>%include changelog
> >>>
> >>>+1
> >>>
> >>
> >>As I pointed out in
> >>https://pagure.io/packaging-committee/pull-request/942, %include is
> >>nasty because it breaks the stand-alone attribute of specs. There are
> >>umphteen dupes and variants of bugs about systemd.spec's use of
> >>%include breaking this and that use pattern that aren't systemd's
> >>fault, it's just that %include isn't as useful as it initially seems.
> >
> >
> >Would it be helpful to open RPM upstream ticket to discuss when the
> >missing included file is problematic and whether there should be other
> >graceful variant of include? May be the %include should always behave
> >gracefully, because (S)RPM build is going to always fail due to missing
> >files specified by Source directive.
> >
> This was actually discussed upstream not too long ago, just can't find 
> the reference offhand.
>
> Basically an %include can never be allowed to fail as it can contain 
> anything at all, including mandatory parts of the spec. In theory you 
> can have a spec consisting of nothing but one or more %include 
> directives.
>
> (S)RPM builds are not an issue, it's spec queries, in particular for 
> build-dependencies but also for other data. Rpm has special logic to 
> allow missing sources and patches for query purposes, because they're 
> only needed for building. Something changelog-specific could also 
> safely ignore the missing file as %changelog is always an optional 
> part of the spec. 
I ran in to some issues when using %include for the changelog and it may be
related to this.  I made my changelog the Source1 file for the package and
did:

     %changelog
     %include %{SOURCE1}

It works for the packages I'm using it in.

Thanks,

-- 
David Cantrell <dcantrell(a)redhat.com&gt;
Red Hat, Inc. | Boston, MA | EST5EDT

[Panu Matilainen]

On 1/20/20 7:03 PM, David Cantrell wrote:
>  On Wed, Jan 15, 2020 at 04:28:56PM +0200, Panu Matilainen wrote:
> > On 1/15/20 3:33 PM, Vít Ondruch wrote:
> >>
> >> Dne 15. 01. 20 v 13:33 Panu Matilainen napsal(a):
> >>> On 1/15/20 2:13 PM, Miroslav Suchý wrote:
> >>>> Dne 13. 01. 20 v 14:05 Vít Ondruch napsal(a):
> >>>>> %changelog
> >>>>>
> >>>>> %include changelog
> >>>>
> >>>> +1
> >>>>
> >>>
> >>> As I pointed out in
> >>> https://pagure.io/packaging-committee/pull-request/942, %include is
> >>> nasty because it breaks the stand-alone attribute of specs. There are
> >>> umphteen dupes and variants of bugs about systemd.spec's use of
> >>> %include breaking this and that use pattern that aren't systemd's
> >>> fault, it's just that %include isn't as useful as it initially
> seems.
> >>
> >>
> >> Would it be helpful to open RPM upstream ticket to discuss when the
> >> missing included file is problematic and whether there should be other
> >> graceful variant of include? May be the %include should always behave
> >> gracefully, because (S)RPM build is going to always fail due to missing
> >> files specified by Source directive.
> >>
> > This was actually discussed upstream not too long ago, just can't find 
> > the reference offhand.
> >
> > Basically an %include can never be allowed to fail as it can contain 
> > anything at all, including mandatory parts of the spec. In theory you 
> > can have a spec consisting of nothing but one or more %include 
> > directives.
> >
> > (S)RPM builds are not an issue, it's spec queries, in particular for 
> > build-dependencies but also for other data. Rpm has special logic to 
> > allow missing sources and patches for query purposes, because they're 
> > only needed for building. Something changelog-specific could also 
> > safely ignore the missing file as %changelog is always an optional 
> > part of the spec.
>  
>  I ran in to some issues when using %include for the changelog and it may be
>  related to this.  I made my changelog the Source1 file for the package and
>  did:
>  
>       %changelog
>       %include %{SOURCE1}
>  
>  It works for the packages I'm using it in.
>   
This will essentially break "dnf builddep" for that spec, because the 
%{_sourcedir} for root is different from the user. It'd be less of an 
issue if it always defaulted to current directory, but that's a can of 
worms of its own.

It'll also cause general breakage and pain for those who expect the spec 
to be standalone parseable. For example the "all of rawhide" spec 
snapshot tarball becomes rather useless if specs use %include.

	- Panu -

[Vitaly Zaitsev]

On 10.01.2020 17:36, Pierre-Yves Chibon wrote:
>  Do we want to drop release and changelog from our spec file? 
YES. Changelogs can be automatically generated from Fedora Git SCM commits.

-- 
Sincerely,
  Vitaly Zaitsev (vitaly(a)easycoding.org)

[Petr Pisar]

On Fri, Jan 10, 2020 at 05:36:46PM +0100, Pierre-Yves Chibon wrote:
>  
>  The release field would need to be set by koji ignoring whatever is in the
>  spec file. How do we want to do this?
>    - Based on dates?
>    - Using an always increasing integer?
>    - Using the number of successful builds since the last time the version
>      field changed?
>    - Another idea?
>   
Obsoletes RPM tag is usually specific up to release value. Therefore the new
release scheme must hold two properties:

(1) The value must be predictable. I.e. a packager must know what release
the pacakge will be assigned before commiting a change with the Obsoletes tag.

(2) The new values must be larger than all historical values (across all
historical Fedora releases). That assures than a new build won't become
obsoleted because of a decreased release.

-- Petr

[Nils Philippsen]

Hey Petr!

On Mon, 2020-01-13 at 10:34 +0100, Petr Pisar wrote:
>  (2) The new values must be larger than all historical values (across
>  all historical Fedora releases). That assures than a new build won't
>  become obsoleted because of a decreased release. 
can you clarify what you mean with "historical values/releases" here?
Would it include all currently active releases at some point in time
(i.e. everything up to F32/rawhide ATM)?

The way I see it, we should have a differently phrased requirement,
ensuring that within an upstream version of a package, the release of a
build in a higher Fedora release should be "newer" than of any previous
build (for the same version) in an older release.

I mean that we e.g. can't cater for the case where a package is built
in an older Fedora release, but no correspondent build is done in a
newer one, just as without automation: if a package has say releases
-1.fc30, -1.fc31 and -1.fc32 and then the packager only bumps to
-2.fc30 and builds in F-30, there's not much we can do about it (other
than chiding them for it ;)).

Nils
-- 
Nils Philippsen    "Those who would give up Essential Liberty to
Software Engineer   purchase a little Temporary Safety, deserve neither
Red Hat             Liberty nor Safety."  --  Benjamin Franklin, 1759
PGP fingerprint:  D0C1 1576 CDA6 5B6E BBAE  95B2 7D53 7FCA E9F6 395D
            old:  C4A8 9474 5C4C ADE3 2B8F  656D 47D8 9B65 6951 3011

[Petr Pisar]

On Mon, Jan 20, 2020 at 05:20:08PM +0100, Nils Philippsen wrote:
>  On Mon, 2020-01-13 at 10:34 +0100, Petr Pisar wrote:
>  > (2) The new values must be larger than all historical values (across
>  > all historical Fedora releases). That assures than a new build won't
>  > become obsoleted because of a decreased release.
>  
>  can you clarify what you mean with "historical values/releases" here?
>  Would it include all currently active releases at some point in time
>  (i.e. everything up to F32/rawhide ATM)?
>   
Not ontly currently active. Users also upgrade from just end-of-lifed
distribution to a next one. It must also cover at least one distribution
before the latest active one.

>  The way I see it, we should have a differently phrased requirement,
>  ensuring that within an upstream version of a package, the release of a
>  build in a higher Fedora release should be "newer" than of any previous
>  build (for the same version) in an older release.
>   
Yes. You are right. My requirement was too strong. Probably stemming from the
fact that many packages do not recieve any version bump and the only changing
part is the Release string.

-- Petr

[Ken Dreyer]

On Fri, Jan 10, 2020 at 9:37 AM Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:
>  Thus our call for input to accept or reject the idea and if the
> former
>  scope/define the system. 
Whatever you decide, please try it out on a small set of packages that
you personally maintain for a long time. This "field testing" will
ensure that you're covering all the corner cases that you know about
(and many of the ones that you do not).

With the packages I maintain with rdopkg, we generally do it the other
way around: We write a human-readable %changelog entry, and then
"rdopkg amend" will amend my dist-git commit log to match the text
that I wrote in the .spec file.

- Ken

[Pavel Raiskup]

On Friday, January 10, 2020 5:36:46 PM CET Pierre-Yves Chibon wrote:
>  Do we want to drop release and changelog from our spec file? 
No.  People continuously tend to forget that '%changelog' is for
end-users.  Especially if some distributions already claim they can live
fine without %changelog...

Unless product managers say that 'rpm -q --changelog' is not a thing
nowadays, we should at least _allow_ being "nice" to end users.  So
whatever approach we use by default -- the maintainers still have to have
a chance to maintain %changelog manually.

That said, to not loose the freedom, yes - we can implement some
automation - but only if that can be turned off.  By those who care about
%changelog.

Regarding automation, I'm sceptic.  See how GNU maintains ChangeLog files
... and how difficult is to edit the ChangeLog entries retrospectively
when some automation breaks it.  If people claim maintaining %changelog is
too expensive so they want it generated -- having it generated is even
more expensive.  I mean if maintainer cares to have '-q --changelog' nice.

>  With the changelog it becomes a little bit more tricky.
>  We currently have 3 changelogs in Fedora with 3 different target audience (this
>  is how I understand them):
>    - One for the files in the git repository, meant to be "consumed" by our
>      fellow packagers, not meant to leave the Fedora infrastructure
>    - One in the spec file describing the changes applied to it. This one is meant
>      to be accessible to sysadmins who want to know/check what changed in a package
>    - One in bodhi, meant for end-user consumption and which should give some
>      explanation as to why the package was updated or where information about the
>      update can be found 
In ideal world, shouldn't the bodhi change description be equivalent to
%changelog, or at least a super-set of %changelog?  If these were equivalent,
maintainers woudl have to think more about %changelog.

>  So we need to, somehow, merge two changelogs into one while realizing
> that some
>  information in one may not be desirable in the other (for example the world
>  famous commit message: "oops I've forgot to upload the sources" does not
> need to
>  appear in the RPM's changelog).
>  Would it be easier to merge the git changelog with the spec changelog or the
>  spec changelog with the bodhi notes? 
spec changelog with bodhi notes

>  For the former one easy way to achieve this is to consider all the
> commits since
>  the last successful build and have a magic keyword to either include or exclude
>  a commit message in the changelog.
>  For the latter, we discussed the idea of using annotated git tags this fall.
>
>  Both have their pros and cons, do people have some experience with either and
>  could share their feedback? 
See the GNU (e.g. gnulib) `make ChangeLog`.  The annotated tags are IMO
used in rpkg-util, and for regular git user they are "magic".  People will
start
asking where the changelog is defined, how to change it, etc.

Pavel

[Vít Ondruch]

Dne 21. 01. 20 v 9:36 Pavel Raiskup napsal(a):
>  On Friday, January 10, 2020 5:36:46 PM CET Pierre-Yves Chibon wrote:
> > Do we want to drop release and changelog from our spec file?
>  No.  People continuously tend to forget that '%changelog' is for
>  end-users.  Especially if some distributions already claim they can live
>  fine without %changelog... 

I assume that there will be some tags which allow to exclude some
specific messages from changelog. E.g. yesterday, I did two rebuilds of
libguestfs:


https://src.fedoraproject.org/rpms/libguestfs/c/d33d482cf7b0cf4e1fa48d229...

https://koji.fedoraproject.org/koji/taskinfo?taskID=40786204


and


https://src.fedoraproject.org/rpms/libguestfs/c/cbdfa80fe9eb4a26a08075841...

https://koji.fedoraproject.org/koji/taskinfo?taskID=40786518


The first failed due to some Koji issues. Therefore I had to bump the
release and do another one. If this situation happened when the
changelog is generated from git log, I would expect that adding some
keyword such as "[skip changelog]" (there are quite commonly used
similar hints for CI nowadays [1]) would instruct the generator to leave
the second commit out of the changelog, because it does not provide any
additional value to user.


>  Unless product managers say that 'rpm -q --changelog' is not a thing
>  nowadays, we should at least _allow_ being "nice" to end users.  So
>  whatever approach we use by default -- the maintainers still have to have
>  a chance to maintain %changelog manually.
>
>  That said, to not loose the freedom, yes - we can implement some
>  automation - but only if that can be turned off.  By those who care about
>  %changelog.
>
>  Regarding automation, I'm sceptic.  See how GNU maintains ChangeLog files
>  ... and how difficult is to edit the ChangeLog entries retrospectively
>  when some automation breaks it.  If people claim maintaining %changelog is
>  too expensive so they want it generated -- having it generated is even
>  more expensive.  I mean if maintainer cares to have '-q --changelog' nice.
>
> > With the changelog it becomes a little bit more tricky.
> > We currently have 3 changelogs in Fedora with 3 different target audience (this
> > is how I understand them):
> >   - One for the files in the git repository, meant to be "consumed" by our
> >     fellow packagers, not meant to leave the Fedora infrastructure
> >   - One in the spec file describing the changes applied to it. This one is meant
> >     to be accessible to sysadmins who want to know/check what changed in a package
> >   - One in bodhi, meant for end-user consumption and which should give some
> >     explanation as to why the package was updated or where information about the
> >     update can be found
>  In ideal world, shouldn't the bodhi change description be equivalent to
>  %changelog, or at least a super-set of %changelog?  If these were equivalent,
>  maintainers woudl have to think more about %changelog. 

Don't forget that single Bodhi update might ship several builds.


Vít


[1] https://docs.travis-ci.com/user/customizing-the-build/#skipping-a-build


> > So we need to, somehow, merge two changelogs into one while realizing that some
> > information in one may not be desirable in the other (for example the world
> > famous commit message: "oops I've forgot to upload the sources" does
> not need to
> > appear in the RPM's changelog).
> > Would it be easier to merge the git changelog with the spec changelog or the
> > spec changelog with the bodhi notes?
>  spec changelog with bodhi notes
>
> > For the former one easy way to achieve this is to consider all the commits since
> > the last successful build and have a magic keyword to either include or exclude
> > a commit message in the changelog.
> > For the latter, we discussed the idea of using annotated git tags this fall.
> >
> > Both have their pros and cons, do people have some experience with either and
> > could share their feedback?
>  See the GNU (e.g. gnulib) `make ChangeLog`.  The annotated tags are IMO
>  used in rpkg-util, and for regular git user they are "magic".  People will
> start
>  asking where the changelog is defined, how to change it, etc.
>
>  Pavel
>

[Pavel Raiskup]

On Tuesday, January 21, 2020 10:20:10 AM CET Vít Ondruch wrote:
>  I would expect that adding some keyword such as "[skip
> changelog]" (there are
>  quite commonly used similar hints for CI nowadays [1]) would instruct the
>  generator to leave the second commit out of the changelog, because it does not
>  provide any additional value to user. 
Yes, but people can forget to put there the keyword, and push - and later
still affect the %changelog - and so we would have to have a way to adjust
changelog by "patch" or something.

Again, as example, take a look at gnulib, to build-aux/gitlog-to-changelog
and it treats *.amend files, --ignore-matching, --ignore-line, etc.

>  > In ideal world, shouldn't the bodhi change description be
> equivalent to
>  > %changelog, or at least a super-set of %changelog?  If these were equivalent,
>  > maintainers woudl have to think more about %changelog.
>  
>  
>  Don't forget that single Bodhi update might ship several builds. 
Correct, no need to not provide all the %changelogs.

Pavel

[Neal Gompa]

On Tue, Jan 21, 2020 at 4:39 AM Pavel Raiskup <praiskup(a)redhat.com&gt; wrote:
>  On Tuesday, January 21, 2020 10:20:10 AM CET Vít Ondruch wrote:
>  > I would expect that adding some keyword such as "[skip changelog]" (there
> are
>  > quite commonly used similar hints for CI nowadays [1]) would instruct the
>  > generator to leave the second commit out of the changelog, because it does not
>  > provide any additional value to user.
>
>  Yes, but people can forget to put there the keyword, and push - and later
>  still affect the %changelog - and so we would have to have a way to adjust
>  changelog by "patch" or something.
>
>  Again, as example, take a look at gnulib, to build-aux/gitlog-to-changelog
>  and it treats *.amend files, --ignore-matching, --ignore-line, etc.
>
>  > > In ideal world, shouldn't the bodhi change description be equivalent to
>  > > %changelog, or at least a super-set of %changelog?  If these were equivalent,
>  > > maintainers woudl have to think more about %changelog.
>  >
>  >
>  > Don't forget that single Bodhi update might ship several builds.
>
>  Correct, no need to not provide all the %changelogs. 
I don't think people realize this, but while from DNF the updateinfo
and changelog data are separate, when Bodhi generates update notices
that are sent as emails and used in feeds, both pieces of information
are presented together.

So ideally, there should be different information in updateinfo and
changelog. The former is user-centric and the latter is packager
centric.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Pierre-Yves Chibon]

On Tue, Jan 21, 2020 at 09:36:27AM +0100, Pavel Raiskup wrote:
>  On Friday, January 10, 2020 5:36:46 PM CET Pierre-Yves Chibon wrote:
>  > Do we want to drop release and changelog from our spec file?
>  
>  No.  People continuously tend to forget that '%changelog' is for
>  end-users.  Especially if some distributions already claim they can live
>  fine without %changelog...
>  
>  Unless product managers say that 'rpm -q --changelog' is not a thing
>  nowadays, we should at least _allow_ being "nice" to end users.  So
>  whatever approach we use by default -- the maintainers still have to have
>  a chance to maintain %changelog manually. 
rpm -q --changelog is a thing, which is why we're discussing the idea to remove
the changelog from the *spec file* not from the (s)RPM.

And yes I agree that whichever approach is designed, it should be made opt-in,
if only to allow to gather feedback and improve the process.


Pierre

[Pavel Raiskup]

On Tuesday, January 21, 2020 10:24:53 AM CET Pierre-Yves Chibon wrote:
>  On Tue, Jan 21, 2020 at 09:36:27AM +0100, Pavel Raiskup wrote:
>  > On Friday, January 10, 2020 5:36:46 PM CET Pierre-Yves Chibon wrote:
>  > > Do we want to drop release and changelog from our spec file?
>  > 
>  > No.  People continuously tend to forget that '%changelog' is for
>  > end-users.  Especially if some distributions already claim they can live
>  > fine without %changelog...
>  > 
>  > Unless product managers say that 'rpm -q --changelog' is not a thing
>  > nowadays, we should at least _allow_ being "nice" to end users.  So
>  > whatever approach we use by default -- the maintainers still have to have
>  > a chance to maintain %changelog manually.
>  
>  rpm -q --changelog is a thing, which is why we're discussing the idea to remove
>  the changelog from the *spec file* not from the (s)RPM.
>  
>  And yes I agree that whichever approach is designed, it should be made opt-in,
>  if only to allow to gather feedback and improve the process. 
Thanks.

While we are on it..  did you consider to implement some nice
Release/%changelog workflow in src.fedoraproject.org?  I filled now
https://pagure.io/pagure/issue/4714

Pavel