[PATCH] distgit: Upload files to both the new and old path

[Pierre-Yves Chibon]

On Thu, May 28, 2015 at 10:36:54AM -0500, Dennis Gilmore wrote:
> On Thursday, May 28, 2015 02:05:44 PM Mathieu Bridon wrote:
> > From: Mathieu Bridon <bochecha at daitauha.fr>
> > 
> > Currently, the CGI script is set to upload files:
> > - to the old path if the upload uses md5
> > - to the new path if the upload uses sha512
> > 
> > The old path is as follows:
> >     /%(srpmname)s/%(filename)s/%(hash)s/%(filename)s
> > 
> > The new path is:
> >     /%(srpmname)s/%(filename)s/%(hashtype)s/%(hash)s/%(filename)s
> > 
> > This was meant to ensure compatibility with current fedpkg which
> > always downloads from the old path, but will eventually download from
> > the new path when we move to sha512.
> > 
> > However, working more on this, I now think it would make for a smoother
> > transition if we instead always stored the files at the new path, but
> > just hardlinked to the old path if the upload is using md5.
> > 
> > This is what this patch achieves.
> > 
> > With this deployed in production, fedpkg could be patched to try
> > downloading from the new path, and fallback to the old one if necessary,
> > which decouples the migration to the new path from the migration to the
> > new hash.
> > ---
> >  roles/distgit/files/dist-git-upload.cgi | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> > 
> > diff --git a/roles/distgit/files/dist-git-upload.cgi
> > b/roles/distgit/files/dist-git-upload.cgi index b4fda74..38c40db 100644
> > --- a/roles/distgit/files/dist-git-upload.cgi
> > +++ b/roles/distgit/files/dist-git-upload.cgi
> > @@ -112,11 +112,6 @@ def main():
> >      hash_dir = os.path.join(module_dir, filename, hash_type, checksum)
> >      msgpath = os.path.join(name, module_dir, filename, hash_type, checksum,
> > filename)
> > 
> > -    if hash_type == "md5":
> > -        # Preserve compatibility with the current folder hierarchy for md5
> > -        hash_dir = os.path.join(module_dir, filename, checksum)
> > -        msgpath = os.path.join(name, module_dir, filename, checksum,
> > filename) -
> >      unwanted_prefix = '/srv/cache/lookaside/pkgs/'
> >      if msgpath.startswith(unwanted_prefix):
> >          msgpath = msgpath[len(unwanted_prefix):]
> > @@ -180,6 +175,11 @@ def main():
> >      print >> sys.stderr, '[username=%s] Stored %s (%d bytes)' % (username,
> > dest_file, filesize) print 'File %s size %d %s %s stored OK' % (filename,
> > filesize, hash_type.upper(), checksum)
> > 
> > +    # Add the file to the old path, where fedpkg is currently looking for
> > it +    if hash_type == "md5":
> > +        old_path = os.path.join(module_dir, filename, checksum, filename)
> > +        os.link(dest_file, old_path)
> > +
> >      # Emit a fedmsg message.  Load the config to talk to the fedmsg-relay.
> >      try:
> >          config = fedmsg.config.load_config([], None)
> 
> The idea is fine, but it is a bit hard to tell from this patch what exactly is 
> going on as there is no context.

You mean in the code? I find the commit message to be pretty explanatory. What
are you looking for?


Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150528/83d42e06/attachment-0001.sig>