RFE: prevent broken rawhide network install due to repo change

[Andrew Farris]

RFE:  prevent broken rawhide network install due to repo change

Problem: network installs will currently fail if the repository changes during 
the installation, which can take 2-4 hours or more, and changed files are 
removed from the repo, making anaconda fail to download the expected version of 
that changed file

If a user has started a network install then any change in files not yet 
downloaded will completely break that install.  The user will see anaconda fail 
to get the file 10 times, then suggest rebooting and starting the install again. 
  This wastes time and network traffic, since the repository could have only 
changed the last file that installer needed... and they will subsequently 
download them all again (this time hopefully missing the period the repo changes).

To solve the problem all that is necessary is for the repo to hold all changed 
files for a period (probably 3-4 hours for the typical install time) so that any 
installs that have begun before the changes to the repo data can finish.  The 
depsolving in anaconda does not recognize a change in the repodata and attempt 
to resume the install with the new data, so either 1) that needs implemented, or 
2) the mirrors need to hold the files for awhile even though the repodata no 
longer has them included.

Would it be possible for the repository data to be created while excluding any 
files that were updated during that build, leaving the old files out of 
listings, but actually leave the file present and accessible by direct request?

This would improve the network install experience because it is difficult to 
anticipate when the repo changes will break the install process for \insert 
random mirror\.

The caveat to the solution is that the repository needs to be cleaned some time 
later by another process which would know which files are now old (not included 
in the repo data) and delete them.  In the meantime (again suggesting 3-4 hour 
overlap) there will be a slightly larger disk space needed because of the 
duplication.

Any mirrors that sync to rawhide while there are duplicated files should have 
those files already (from the day before) and would only fetch the new files. 
Those mirrors could either 1) keep the duplicate files all day until the next 
sync, or 2) have their own cleaning process such as a cron job.  In either case, 
anyone requesting an installation while the duplicates are there would not be 
effected.

I've run into this problem several times when doing network rawhide installs, 
especially if started late night in California (meaning the repo build and 
mirror syncs are likely to overlap the install).

I am unaware obviously of where the infrastructure would need to be changed to 
handle this, though I assume a combination of koji and elsewhere (both in 
building the repo data and afterward in deleting the temporarily remaining files).

If there is anything already in place to solve/work on this issue point me in 
the right direction, because its not working. ;)

Also I think this is only an issue with rawhide since the release mirrors keep 
the original released files and then updates go elsewhere.  Development on the 
other hand has issues with the changes because files disappear.

I'd be happy to move this suggestion to where it belongs (bugzilla against ?)

-- 
Andrew Farris <lordmorgul at gmail.com> <ajfarris at gmail.com>
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----