Change Request: Race condition syncing static websites to proxies

[Dennis Gilmore]

On Tuesday 11 May 2010 01:19:09 pm Mike McGrath wrote:
> On Tue, 11 May 2010, Toshio Kuratomi wrote:
> > mmcgrath noted that we have a race condition in our syncStatic.sh script
> > that is leading to proxies not having copies of the website.  Here's a
> > patch to the syncStatic script to fix that:
> > 
> > diff --git a/modules/fedora-web/files/syncStatic.sh
> > b/modules/fedora-web/files/syncStatic.sh index 23f5181..9c1b7ac 100644
> > --- a/modules/fedora-web/files/syncStatic.sh
> > +++ b/modules/fedora-web/files/syncStatic.sh
> > @@ -26,7 +26,7 @@ cd fedora-web
> > 
> >  pushd fedoraproject.org > /dev/null
> >  make > /dev/null 2>&1
> > 
> > -rsync -qa --delete out/* /srv/web/fedoraproject.org/
> > +rsync -qa --delete out/ /srv/web/fedoraproject.org/
> > 
> >  popd > /dev/null
> >  
> >  # Make sure everything else builds from master.
> > 
> > @@ -34,26 +34,22 @@ popd > /dev/null
> > 
> >  /usr/bin/git checkout -q master || exit 1
> >  
> >  pushd spins.fedoraproject.org > /dev/null
> > 
> > -/bin/rm -rf /srv/web/spins.fedoraproject.org/*
> > 
> >  make > /dev/null 2>&1
> > 
> > -rsync -qa --delete out/* /srv/web/spins.fedoraproject.org/
> > +rsync -qa --delete out/ /srv/web/spins.fedoraproject.org/
> > 
> >  popd > /dev/null
> >  
> >  pushd talk.fedoraproject.org > /dev/null
> > 
> > -/bin/rm -rf /srv/web/talk.fedoraproject.org/*
> > 
> >  make > /dev/null 2>&1
> > 
> > -rsync -qa --delete out/* /srv/web/talk.fedoraproject.org/
> > +rsync -qa --delete out/ /srv/web/talk.fedoraproject.org/
> > 
> >  popd > /dev/null
> >  
> >  pushd boot.fedoraproject.org > /dev/null
> > 
> > -/bin/rm -rf /srv/web/boot.fedoraproject.org/*
> > 
> >  make > /dev/null 2>&1
> > 
> > -rsync -qa --delete out/* /srv/web/boot.fedoraproject.org/
> > +rsync -qa --delete out/ /srv/web/boot.fedoraproject.org/
> > 
> >  popd > /dev/null
> >  
> >  pushd start.fedoraproject.org > /dev/null
> > 
> > -/bin/rm -rf /srv/web/start.fedoraproject.org/*
> > 
> >  make > /dev/null 2>&1
> > 
> > -rsync -qa --delete out/* /srv/web/start.fedoraproject.org/
> > +rsync -qa --delete out/ /srv/web/start.fedoraproject.org/
> > 
> >  popd > /dev/null
> > 
> > The patch does two things:
> > 
> > 1) Removes the rm -rf step as that is causing the race (when proxies sync
> > data after the rm -rf and before hte rsync, they get an empty directory)
> > 
> > 2) Removes the wildcard from the rsync.  The wildcard was preventing the
> > --delete from functioning as expected.  I believe (but ricky can confirm)
> > that the --delete not removing files that are no longer needed from the
> > final output directory is why we had the rm -rf in the first place.
> > 
> > Could I get two +1's for this change?
> > 
> > -Toshio
> 
> +1 from me, this was causing problems.
+1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
Url : http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20100511/c1590fba/attachment.bin