Review Request and Questions

[Brian (bex) Exelbierd]

Hi,

I have submitted the first of my dependency review requests here:

https://bugzilla.redhat.com/show_bug.cgi?id=1712280

Feedback is greatly appreciated so I can clean anything up on the next
set of dependencies.

I am working on the main application packaging for gocryptfs
(https://github.com/rfjakob/gocryptfs/) and have hit a few questions:

1. The upstream provides specific source tarballs for each version
here, https://github.com/rfjakob/gocryptfs/releases - I am planning to
use these as opposed to the gomacros to determine a git tag to pull.
Is there a reason not to do this?

2. The upstream build script sets up some values to be included in the
executables version string.  I presume this helps with debugging.  I
am told that the Go SIG would prefer to see the macros used for
building rather than the upstream build script.  This is fine, except
I am not sure how to handle these values.  Specifically, they are:

GITVERSION=$(cat VERSION)
GITVERSIONFUSE=$(rpm -q golang-github-hanwen-fuse-devel --queryformat
'%{version}')
BUILDDATE=$(date +%Y-%m-%d)

These can be set, presumably, somehow in the spec file (i haven't
tested my ideas here yet).

They are passed to `go build` as part of the ldflags, specifically as:

"-X main.GitVersion=$GITVERSION -X main.GitVersionFuse=$GITVERSIONFUSE
-X main.BuildDate=$BUILDDATE" prepended to any existing ldflags

How do you pass this information with the macros?

thanks,

bex

[Robert-André Mauchin]

On Tuesday, 21 May 2019 10:43:04 CEST Brian (bex) Exelbierd wrote:
> Hi,
>
> I have submitted the first of my dependency review requests here:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1712280
>
> Feedback is greatly appreciated so I can clean anything up on the next
> set of dependencies.
>
> I am working on the main application packaging for gocryptfs
> (https://github.com/rfjakob/gocryptfs/) and have hit a few questions:
> The upstream provides specific source tarballs for each version
>
> here, https://github.com/rfjakob/gocryptfs/releases - I am planning to
> use these as opposed to the gomacros to determine a git tag to pull.
> Is there a reason not to do this?

Use gocryptfs_vX.Y_src.tar.gz, otherwise the usual source does not contain the 
VERSION file.
> The upstream build script sets up some values to be included in the
>
> executables version string.  I presume this helps with debugging.  I
> am told that the Go SIG would prefer to see the macros used for
> building rather than the upstream build script.  This is fine, except
> I am not sure how to handle these values.  Specifically, they are:
>
> GITVERSION=$(cat VERSION)
> GITVERSIONFUSE=$(rpm -q golang-github-hanwen-fuse-devel --queryformat
> '%{version}')
> BUILDDATE=$(date +%Y-%m-%d)
>
> These can be set, presumably, somehow in the spec file (i haven't
> tested my ideas here yet).
>
> They are passed to `go build` as part of the ldflags, specifically as:
>
> "-X main.GitVersion=$GITVERSION -X main.GitVersionFuse=$GITVERSIONFUSE
> -X main.BuildDate=$BUILDDATE" prepended to any existing ldflags
>
> How do you pass this information with the macros?

Just export your LDFLAGS.

For example in micro I have:

export LDFLAGS="-X main.Version=%{version} \
                -X main.CommitHash=%{shortcommit} \
                -X 'main.CompileDate=%{compiledate}'"

This will be picked up by %gobuild

[Brian (bex) Exelbierd]

Thank you for the reply - I'll work on that.

regards,

bex

On Tue, May 21, 2019 at 3:25 PM Robert-André Mauchin  wrote:
> On Tuesday, 21 May 2019 10:43:04 CEST Brian (bex) Exelbierd wrote:
> > Hi,
> >
> > I have submitted the first of my dependency review requests here:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1712280
> >
> > Feedback is greatly appreciated so I can clean anything up on the next
> > set of dependencies.
> >
> > I am working on the main application packaging for gocryptfs
> > (https://github.com/rfjakob/gocryptfs/) and have hit a few questions:
> > The upstream provides specific source tarballs for each version
> >
> > here, https://github.com/rfjakob/gocryptfs/releases - I am planning to
> > use these as opposed to the gomacros to determine a git tag to pull.
> > Is there a reason not to do this?
>
> Use gocryptfs_vX.Y_src.tar.gz, otherwise the usual source does not contain the
> VERSION file.
> > The upstream build script sets up some values to be included in the
> >
> > executables version string.  I presume this helps with debugging.  I
> > am told that the Go SIG would prefer to see the macros used for
> > building rather than the upstream build script.  This is fine, except
> > I am not sure how to handle these values.  Specifically, they are:
> >
> > GITVERSION=$(cat VERSION)
> > GITVERSIONFUSE=$(rpm -q golang-github-hanwen-fuse-devel --queryformat
> > '%{version}')
> > BUILDDATE=$(date +%Y-%m-%d)
> >
> > These can be set, presumably, somehow in the spec file (i haven't
> > tested my ideas here yet).
> >
> > They are passed to `go build` as part of the ldflags, specifically as:
> >
> > "-X main.GitVersion=$GITVERSION -X main.GitVersionFuse=$GITVERSIONFUSE
> > -X main.BuildDate=$BUILDDATE" prepended to any existing ldflags
> >
> > How do you pass this information with the macros?
>
> Just export your LDFLAGS.
>
> For example in micro I have:
>
> export LDFLAGS="-X main.Version=%{version} \
>                 -X main.CommitHash=%{shortcommit} \
>                 -X 'main.CompileDate=%{compiledate}'"
>
> This will be picked up by %gobuild
>
> golang mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]

[Nicolas Mailhot]

Le mardi 21 mai 2019 à 10:43 +0200, Brian (bex) Exelbierd a écrit :
> Hi,
>
> I have submitted the first of my dependency review requests here:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1712280
>
> Feedback is greatly appreciated so I can clean anything up on the
> next
> set of dependencies.
>
> I am working on the main application packaging for gocryptfs
> (https://github.com/rfjakob/gocryptfs/) and have hit a few questions:
> The upstream provides specific source tarballs for each version
>
> here, https://github.com/rfjakob/gocryptfs/releases - I am planning
> to
> use these as opposed to the gomacros to determine a git tag to pull.
> Is there a reason not to do this

The downside is that you will lose the ability to switch easily to a
tag or commit shall you need in the future. So, that depends if your
upstream is reactive or not. If your upstream always releases manually
when you need it to, you don't need to use the source our macro
computes
> The upstream build script sets up some values to be included in
>
> the
> executables version string.  I presume this helps with debugging.  I
> am told that the Go SIG would prefer to see the macros used for
> building rather than the upstream build script.  This is fine, except
> I am not sure how to handle these values.  Specifically, they are:
>
> GITVERSION=$(cat VERSION)

I don't remember if the macro version in fedora already does this, but
the one that will land in devel soonish will compute that kind of
ldflag by default for all go packages
https://pagure.io/go-rpm-macros/blob/master/f/rpm/lua/rpm/go.lua#_61
> GITVERSIONFUSE=$(rpm -q golang-github-hanwen-fuse-devel --queryformat
> '%{version}')

That really begs for
https://github.com/rpm-software-management/rpm/issues/607

It's sad that eveyone is reinventing its own hack to implement
something taht should be done by default by rpm. If you have the time,
please explain rpm upstream why tracking this kind of info is useful.
> BUILDDATE=$(date +%Y-%m-%d)

Please don't do that, that breaks build reproducibility.

Regards,

[Brian (bex) Exelbierd]

On Tue, May 21, 2019 at 4:53 PM Nicolas Mailhot
 wrote:
> Le mardi 21 mai 2019 à 10:43 +0200, Brian (bex) Exelbierd a écrit :
> > Hi,
> >
> > I have submitted the first of my dependency review requests here:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1712280
> >
> > Feedback is greatly appreciated so I can clean anything up on the
> > next
> > set of dependencies.
> >
> > I am working on the main application packaging for gocryptfs
> > (https://github.com/rfjakob/gocryptfs/) and have hit a few questions:
> > The upstream provides specific source tarballs for each version
> >
> > here, https://github.com/rfjakob/gocryptfs/releases - I am planning
> > to
> > use these as opposed to the gomacros to determine a git tag to pull.
> > Is there a reason not to do this
>
> The downside is that you will lose the ability to switch easily to a
> tag or commit shall you need in the future. So, that depends if your
> upstream is reactive or not. If your upstream always releases manually
> when you need it to, you don't need to use the source our macro
> computes

If the upstream changes release mechanisms I am happy to revisit the packaging.
> > The upstream build script sets up some values to be included in
> >
> > the
> > executables version string.
> > GITVERSION=$(cat VERSION)
>
> I don't remember if the macro version in fedora already does this, but
> the one that will land in devel soonish will compute that kind of
> ldflag by default for all go packages
> https://pagure.io/go-rpm-macros/blob/master/f/rpm/lua/rpm/go.lua#_61

As this has not landed, I need to move forward with this for now.
Additionally, as we are pulling the value from a file and not
calculating, I would prefer to see the upstream value retained just in
case they choose to put additional details we don't calculate by
default.
> > GITVERSIONFUSE=$(rpm -q golang-github-hanwen-fuse-devel --queryformat
> > '%{version}')
>
> That really begs for
> https://github.com/rpm-software-management/rpm/issues/607

While this issue may have value, it seems to tackle a different problem.
> > BUILDDATE=$(date +%Y-%m-%d)
>
> Please don't do that, that breaks build reproducibility.

My understanding is that reproducible build systems typically provide
a build date value.  Looking through
https://docs.fedoraproject.org/en-US/packaging-guidelines/ I don't see
one for Fedora.  Do you know what it is?

thanks,

bex

[Neal Gompa]

On Tue, May 21, 2019 at 11:32 AM Brian (bex) Exelbierd
 wrote:
> On Tue, May 21, 2019 at 4:53 PM Nicolas Mailhot
>  wrote:
> > Le mardi 21 mai 2019 à 10:43 +0200, Brian (bex) Exelbierd a écrit :
> > > BUILDDATE=$(date +%Y-%m-%d)
> >
> > Please don't do that, that breaks build reproducibility.
>
> My understanding is that reproducible build systems typically provide
> a build date value.  Looking through
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ I don't see
> one for Fedora.  Do you know what it is?

Set at the top of your spec:

%global source_date_epoch_from_changelog Y

Then use $SOURCE_DATE_EPOCH variable.

--
真実はいつも一つ！/ Always, there's only one truth!

[Brian (bex) Exelbierd]

On Tue, May 21, 2019 at 5:40 PM Neal Gompa  wrote:
> On Tue, May 21, 2019 at 11:32 AM Brian (bex) Exelbierd
>  wrote:
> > On Tue, May 21, 2019 at 4:53 PM Nicolas Mailhot
> >  wrote:
> > > Le mardi 21 mai 2019 à 10:43 +0200, Brian (bex) Exelbierd a écrit :
> > > > BUILDDATE=$(date +%Y-%m-%d)
> > >
> > > Please don't do that, that breaks build reproducibility.
> >
> > My understanding is that reproducible build systems typically provide
> > a build date value.  Looking through
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/ I don't see
> > one for Fedora.  Do you know what it is?
>
> Set at the top of your spec:
>
> %global source_date_epoch_from_changelog Y
>
> Then use $SOURCE_DATE_EPOCH variable.

Thank you Neal.  This feels like a good compromise for reproducible
builds and what the upstream may expect from a version string in
Fedora.

regards,

bex

[Nicolas Mailhot]

Le mardi 21 mai 2019 à 17:31 +0200, Brian (bex) Exelbierd a écrit :
> > > BUILDDATE=$(date +%Y-%m-%d)
> >
> > Please don't do that, that breaks build reproducibility.
>
> My understanding is that reproducible build systems typically provide
> a build date value.

A reproducible build means the same inputs produce the same (bit-
perfect) outputs. Putting a date in the produced builds breaks that.
The people Fedora and Debian-side that work on making builds
reproducible and auditable have been hunting down this kind of breakage
for years, yet people keep reintroducing them.

Just don't put build dates into produced files. It's an anti-pattern

https://reproducible-builds.org/

“First, the build system needs to be made entirely deterministic:
transforming a given source must always create the same result. For
example, the current date and time must not be recorded and output
always has to be written in the same order.”

Regards,