Re: Reminder: GitHub etc. auto-generated archives are not stable in time

[Jan Pokorný]

On 24/05/17 09:58 -0500, Jorge Gallegos wrote:
>  On Wed, May 24, 2017 at 04:19:05PM +0200, Jan Pokorný wrote:
> > today, I've accidentally attested there are no stability guarantees
> > with the on-demand archives from common git hosting sites when preparing
> > a new pacemaker update, redownloading "spectool -s 0 pacemaker.spec"
> > of the original (-0.1.rc1, from 2 weeks ago) spec and comparing the
>  
>  Are you pointing to a _tag_ (or as github likes to call them: release) ?
>  As far as I know tags can be re-created, isn't that what is happening
>  here? 
Nope, the point is that nothing has changed in the codebase or, for
that matter, tags.  It must have been GitHub that changed how its
equivalent of "git archive" behaves.

> > hashes, which (surprisingly to me) didn't match (they were at
> any similar
> > test in the past).  Then I looked at the adiff output:
> > 
> >> diff -ru Unpack-2241/pacemaker-Pacemaker-1.1.17-rc1/configure.ac
> Unpack-6255/pacemaker-Pacemaker-1.1.17-rc1/configure.ac
> >> --- Unpack-2241/pacemaker-Pacemaker-1.1.17-rc1/configure.ac2017-05-09
> 00:55:15.000000000 +0200
> >> +++ Unpack-6255/pacemaker-Pacemaker-1.1.17-rc1/configure.ac2017-05-09
> 00:55:15.000000000 +0200
> >> @@ -1159,7 +1159,7 @@
> >>  AC_PATH_PROGS(GIT, git false)
> >>  AC_MSG_CHECKING(build version)
> >>  
> >> -BUILD_VERSION=0459f40
> >> +BUILD_VERSION=0459f40958
> >>  if test  != ":%h$"; then
> >>     AC_MSG_RESULT(archive hash: )
> >  
> > for configure.ac that indeed has export-subst git attribute set
> > and the change itself arises from "$Format:%h$" substitution.
> > This likely means GitHub was internally updated to use equivalent
> > of git 2.11 feature of abbreviation length autoscaling within
> > last 14 days.
>  
>  This is the other bit that makes me think it was actually the
>  maintainers hand that moved this, I don't believe github does anything
>  special to the code once it's stored there. There is no way for github
>  to alter code afaik? 
Once more, see "git help archive", ATTRIBUTES section, export-subst
in particular.  That exactly stands for the varying part, which is
implementation-specific, and GH implementation has apparently changed,
leading to changed contents of numerous archives to be downloaded from
that very point.

Is/will the pagure.io be immune to this sort of retroactive changes
once/if such tarball extraction is implemented
(see https://pagure.io/pagure/issue/861)?

>  This would suggest to me the maintainer:
>  
>  a) Modified the code (either via script or otherwise)
>  b) Deleted the tag (and thus, the github "release")
>  c) Submitted a new tag (and created a new gh release)
>  
>  Of course if the .spec is pointing to an archive url pointing to a git
>  SHA my theory is busted.
>  
> > 
> > Hope this will be useful for some (e.g. fedora-review tool
> > has a check to redownload and diff sources against SRPM content,
> > IIRC). 
-- 
Poki