[PATCH] distgit: Upload files to both the new and old path

[Mathieu Bridon]

On Fri, 2015-05-29 at 11:32 +0200, Mathieu Bridon wrote:
> Given that I received two +1, and that you agreed to the general idea,
> I've pushed it, and Pierre-Yves is helping me run the playbook to test
> it in staging.

And it's good we did, because there was a problem (see my followup patch
pushed to ansible), it's now working just fine in testing:

On the server:

    # find /srv/cache/lookaside/pkgs/libcangjie/

On the client:

    $ fedpkgstg new-sources libcangjie-1.3.tar.xz 
    Uploading: libcangjie-1.3.tar.xz
    ######################################################################## 100.0%
    Source upload succeeded. Don't forget to commit the sources file

Back on the server:

    # find /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5/e50ed193b0e82b07d2d32ee6e62720b9
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5/e50ed193b0e82b07d2d32ee6e62720b9/libcangjie-1.3.tar.xz
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9/libcangjie-1.3.tar.xz

So now, in staging, when uploading a new source file, it is uploaded in both location.

If the file only exists in the old location:

On the server:

    # find /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9/libcangjie-1.3.tar.xz

On the client:

    $ fedpkgstg new-sources libcangjie-1.3.tar.xz 
    Uploading: libcangjie-1.3.tar.xz
    ######################################################################## 100.0%
    Source upload succeeded. Don't forget to commit the sources file


Back on the server:

    # find /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/e50ed193b0e82b07d2d32ee6e62720b9/libcangjie-1.3.tar.xz
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5/e50ed193b0e82b07d2d32ee6e62720b9
    /srv/cache/lookaside/pkgs/libcangjie/libcangjie-1.3.tar.xz/md5/e50ed193b0e82b07d2d32ee6e62720b9/libcangjie-1.3.tar.xz

The file just gets reuploaded, and hardlinked over, which is just fine.
(they have the same hash after all, so they are expected to be identical
anyway)

If everybody agrees this is the desired behaviour, can we deploy it in
production?

This would help a lot with continuing the migration, as it essentially
means we don't need a "flag day" during which we cut uploads, run a big
script to hardlink all the existing sources to the new path, then allow
uploads again.

We'd still need to run such a script, but only for old tarballs, uploads
of new sources could continue uninterrupted just fine.


-- 
Mathieu