Software Management call for RFEs

Florian Weimer fweimer at redhat.com
Mon May 27 08:53:40 UTC 2013


On 05/27/2013 10:24 AM, Jan Zelený wrote:
>> Debian repository policy varies quite a bit.  Some repositories keep old
>> versions, some don't.  Mostly the latter, actually, because not all
>> repository managers (there a couple of implementations) can deal with
>> multiple versions for a single package/architecture combination.
>
> I'm sorry but the Debian repositories say otherwise, see Iceweasel for
> instance:
>
> http://ftp.cz.debian.org/debian/pool/main/i/iceweasel/
>
> Multiple old versions are kept in there. That's why they don't need to update
> their metadata every single time - the old ones are simply still valid.

Ah!  That's the physical directory structure on the mirrors.  Package 
pools where introduced to allow physical sharing of 
.deb/.diff.gz/.orig.tar.gz files across architectures and between 
different versions of the distribution.  I've checked a few of the 
examples in the directory, and all the multiple versions I checked are a 
result of that.  For example, 17.0.6esr-1 has not yet been built on 
armhf and mips, so unstable still contains the previous 17.0.5esr-1 
versions, and both are listed in the directory.  Some older versions are 
only present in experimental, which is not subject to autobuilding on 
all architectures, and that's why they haven't been superseded by newer 
versions yet.

They key point is that none of this triggers listing of an outdated 
package version in the distribution-specific package manifest, so it's 
completely invisible to clients which just look at the manifest.  For 
example, 
<http://ftp.de.debian.org/debian/dists/unstable/main/binary-amd64/Packages.bz2> 
contains just one record for the iceweasel package:

Package: iceweasel
Version: 17.0.6esr-1

Directory listing is not consistent across mirrors because it depends on 
web server configuration, so this really isn't something apt-get could 
use anyway.

Debian might eventually start shipping outdated versions if they were 
used as a build dependency for another package, for strict copyleft 
compliance, but I don't think this has yet been implemented.

> This part is about disk read-write but that was not what I was writing about.
> From my experience users mostly complain about the metadata download which is
> explained above.

Metadata download is mainly accelerated by two things: First of all, 
it's not implicit you have to manually requested it with "apt-get 
update".  And there package diffs, which are ed-style diffs of the 
Packages file I mentioned above.  This approach would work quite well 
for primary.xml because it doesn't contain cross-references between 
packages using non-natural keys.  It doesn't work for the SQLite 
database, either in binary or SQL dump format, because of the reliance 
on artificial primary keys (such as package IDs).

However, for many users that follow unstable or testing, package diffs 
are currently slower than downloading the full Packages file because the 
diffs are incremental (i.e., they contain the changes from file version 
N to N+1, and you have to apply all of them to get to the current 
version) and apt-get can easily write 100 MB or more because the 
Packages file is rewritten locally multiple times.

-- 
Florian Weimer / Red Hat Product Security Team


More information about the devel mailing list