Build system clocks?

List overview All Threads
Download

newer

older

Fedora rawhide compose report:...

mingw sysroot paths (and...

Chris Adams

7 May 2023 7 May '23

7:59 a.m.

I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

I submitted a bug against redhat-rpm-config; the forge macro code calls "date" with -u (for UTC), so that shouldn't happen. But then I thought: unless somehow there's a clock wrong? Like maybe something thinks the hardware clock is UTC but it accidentally is local time? That seems rather unlikely, but wondered if somebody might could check.

For now, I just explicitly defined the date string to use, and that worked, but it's still weird.

-- Chris Adams linux@cmadams.net

Show replies by date

Kevin Fenzi

7 May 7 May

12:15 p.m.

On Sun, May 07, 2023 at 07:59:26AM -0500, Chris Adams wrote:

...

I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

I submitted a bug against redhat-rpm-config; the forge macro code calls "date" with -u (for UTC), so that shouldn't happen. But then I thought: unless somehow there's a clock wrong? Like maybe something thinks the hardware clock is UTC but it accidentally is local time? That seems rather unlikely, but wondered if somebody might could check.

All the builders are using chrony and seem to be pretty close, so I don't think thats it. :)

Chris Adams

12:20 p.m.

Once upon a time, Kevin Fenzi kevin@scrye.com said:

...

All the builders are using chrony and seem to be pretty close, so I don't think thats it. :)

It was pretty much grasphing at straws, but I do appreciate the quick check.

It's weird; the forge macro Lua support is definitely using UTC when it gets the date from the file... I'm wondering if something else is actually setting the "date" macro first (the Lua support code only sets it if is isn't already set), and using some different logic that may be missing UTC for example.

-- Chris Adams linux@cmadams.net

Kevin Kofler

2:48 p.m.

Chris Adams wrote:

...

I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

Could it be that you built the package over midnight UTC, so the SRPM was still built with 20230506, but when the build happened, it was already 20230507?

Kevin Kofler

Blaise Pabon

3:37 p.m.

There ought to be a doc on "lifecycle of a timestamp".... I'm willing to help if we don't already have one.

Tldr there are various levels of abstraction when it comes to representing time. Developers overlook the details until something like this happens.

System clocks don't know about locales, dates and timezones. So the discrepancy may be due to the config of whatever container was doing the logging.

On Sun, May 7, 2023, 12:49 PM Kevin Kofler via devel < devel@lists.fedoraproject.org> wrote:

...

Chris Adams wrote:

...
I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

Could it be that you built the package over midnight UTC, so the SRPM was still built with 20230506, but when the build happened, it was already 20230507?
    Kevin Kofler
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

Chris Adams

3:58 p.m.

Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

...

Chris Adams wrote:

...
I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

Could it be that you built the package over midnight UTC, so the SRPM was still built with 20230506, but when the build happened, it was already 20230507?

It wasn't that, I tried it several times. Also, the forge macro is using the timestamp of the source file, not the wall clock (so the system clock being wrong wouldn't have caused it either).

-- Chris Adams linux@cmadams.net

Elliott Sales de Andrade

4:11 p.m.

On Sun, May 7, 2023 at 4:59 PM Chris Adams linux@cmadams.net wrote:

...

Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

...
Chris Adams wrote:

...
I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When

I

...
...
tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

Could it be that you built the package over midnight UTC, so the SRPM

was

...
still built with 20230506, but when the build happened, it was already 20230507?

It wasn't that, I tried it several times. Also, the forge macro is using the timestamp of the source file, not the wall clock (so the system clock being wrong wouldn't have caused it either).

It uses the timestamp of the source file, but are those timestamps the same? When you download from the forge, the timestamp on the file will be the time that you downloaded it. When you upload to the lookaside cache, the timestamp will be for when it was put there, but your local file has not changed. When fedpkg or koji download from the lookaside cache, it will copy the timestamp from it for reproducibility.

However, if the file already exists, fedpkg won't touch it. I suspect if you delete the tarball and have fedpkg download it again, everything will appear consistent again.

-- Elliott

Fabio Valentini

8 May 8 May

5:42 a.m.

On Sun, May 7, 2023 at 11:11 PM Elliott Sales de Andrade quantum.analyst@gmail.com wrote:

...

On Sun, May 7, 2023 at 4:59 PM Chris Adams linux@cmadams.net wrote:

...
Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

...
Chris Adams wrote:

...
I updated the source of a package of mine last night. The upstream is on Github, and I use the %forgemeta macro for an easy spec file. When I tried to run "fedpkg build" though, it failed - the build system rejected the build because it was expecting an SRPM with a release string including 20230507, but instead got one with 20230506.

Could it be that you built the package over midnight UTC, so the SRPM was still built with 20230506, but when the build happened, it was already 20230507?

It wasn't that, I tried it several times. Also, the forge macro is using the timestamp of the source file, not the wall clock (so the system clock being wrong wouldn't have caused it either).

It uses the timestamp of the source file, but are those timestamps the same? When you download from the forge, the timestamp on the file will be the time that you downloaded it. When you upload to the lookaside cache, the timestamp will be for when it was put there, but your local file has not changed. When fedpkg or koji download from the lookaside cache, it will copy the timestamp from it for reproducibility.

However, if the file already exists, fedpkg won't touch it. I suspect if you delete the tarball and have fedpkg download it again, everything will appear consistent again.

Yeah, the way the forge macros determine "snapshot date" is a bit broken / produces inconsistent results. It uses the source file *modification time* ("mtime"), which might or might not be consistent in all environments ... it's also an attempt to get the "date the snapshot was taken" (not the "date the specified commit was committed") which always seemed a bit misguided to me, but what do I know :)

Anyway, those are the reasons why - back in the days when I did more Go packaging - I added a way to override the date with a fixed value to make the "Release" tag reproducible when using the %forge macros.

(Side note: The %forge macros don't even follow the new Versioning guidelines for snapshots - i.e. using caret and tilde - which puts the snapshot info into the "Version" tag instead of the "Release" tag ... but I'm not sure if anybody is still around who could fix that.)

Fabio

Florian Weimer

6:26 a.m.

* Fabio Valentini:

...

Yeah, the way the forge macros determine "snapshot date" is a bit broken / produces inconsistent results. It uses the source file *modification time* ("mtime"), which might or might not be consistent in all environments ... it's also an attempt to get the "date the snapshot was taken" (not the "date the specified commit was committed") which always seemed a bit misguided to me, but what do I know :)

fedpkg-minimal uses curl -R (--remote-time) to get the timestamp from the lookaside cache. But maybe the lookaside cache servers are not guaranteed to provide Last-Modified: headers? The public view appears to be okay, though.

It would be interesting to see the timestamps the server returned (curl -v output).

But maybe it's something else entirely and the build manages the modify the tarball somehow.

Thanks, Florian

916

Age (days ago)

917

Last active (days ago)

devel@lists.fedoraproject.org

8 comments

7 participants

tags (0)

participants (7)

Blaise Pabon
Chris Adams
Elliott Sales de Andrade
Fabio Valentini
Florian Weimer
Kevin Fenzi
Kevin Kofler