A patch to change the location of the yum cache directory.
This enables existing yum cache to be used, which in our case is especially useful because that cache is populated already. Most importantly, livecd-creator doesn't bind mount the yum cache directory.
Kind regards,
Jeroen van Meeuwen -kanarip
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
On Sun, 2007-08-05 at 20:33 +0200, Jeroen van Meeuwen wrote:
A patch to change the location of the yum cache directory.
This enables existing yum cache to be used, which in our case is especially useful because that cache is populated already. Most importantly, livecd-creator doesn't bind mount the yum cache directory.
Why not do the bind mount? By bind mounting the cache as /var/cache/yum in the chroot, you don't have to do any mucking with the yum config which should make it so that you just have to get the option and if it doesn't exist, fall back to the default. It also makes the cachedir available to the chroot in %post which people could take advantage of if they wanted
Jeremy
On Tue, 2007-08-07 at 17:26 -0400, Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
If using the system yum cache works, that makes a lot of sense to me.
My worry was the sqlite etc. files in the system directory and that if the system yum runs at the same time as the chroot one they'd stomp on each other's data.
On Tue, 2007-08-07 at 17:49 -0400, Colin Walters wrote:
On Tue, 2007-08-07 at 17:26 -0400, Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
If using the system yum cache works, that makes a lot of sense to me.
The nice thing with allowing it to be specified is that you can either end up with a) unspecified does a private cache, just like now (this is an important case to keep working, especially when doing multiple builds at a time from a fast local source) b) you can specify the system cache if you're pointing at the same repos as your system. that can save you download time, etc if you do keepcache=0 in your system yum.conf or do your system update after creating a live image c) you can specify something like /var/tmp/livecd-cache if you want to have a livecd-specific cache.
Jeremy
On Wed, 2007-08-08 at 12:37 -0400, Jeremy Katz wrote:
On Tue, 2007-08-07 at 17:49 -0400, Colin Walters wrote:
On Tue, 2007-08-07 at 17:26 -0400, Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
If using the system yum cache works, that makes a lot of sense to me.
The nice thing with allowing it to be specified is that you can either end up with a) unspecified does a private cache, just like now (this is an important case to keep working, especially when doing multiple builds at a time from a fast local source)
I don't quite understand this one - the cache is in the tempdir as far as I can see; therefore it's not shared between builds?
b) you can specify the system cache if you're pointing at the same repos as your system. that can save you download time, etc if you do keepcache=0 in your system yum.conf or do your system update after creating a live image
(keepcache=1 I assume you mean) But is there the potential for things to break if multiple yum processes write to it?
c) you can specify something like /var/tmp/livecd-cache if you want to have a livecd-specific cache.
I don't really want it to be livecd-specific. The goal is just to have tools that by default do not repeatedly download huge files over the internet. I have plenty of gigs of hard drive space. Mock drove me absolutely nuts for this reason (yes, it has some magic --cache option that's not the default, requires manual intervention to update, and was broken anyways last I tried).
Hmm. Would anything break if we just pointed all the tools at /var/cache/yum? Does yum delete all files it sees there if it has keepcache=0? And what about the concurrency issue?
On Wed, 2007-08-08 at 13:31 -0400, Colin Walters wrote:
On Wed, 2007-08-08 at 12:37 -0400, Jeremy Katz wrote:
On Tue, 2007-08-07 at 17:49 -0400, Colin Walters wrote:
On Tue, 2007-08-07 at 17:26 -0400, Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
If using the system yum cache works, that makes a lot of sense to me.
The nice thing with allowing it to be specified is that you can either end up with a) unspecified does a private cache, just like now (this is an important case to keep working, especially when doing multiple builds at a time from a fast local source)
I don't quite understand this one - the cache is in the tempdir as far as I can see; therefore it's not shared between builds?
Right -- because if I'm creating multiple images at a time, you otherwise run some risk (at least for now) of stomping over things. So having the default be the safe, though slower, route isn't a bad thing.
b) you can specify the system cache if you're pointing at the same repos as your system. that can save you download time, etc if you do keepcache=0 in your system yum.conf or do your system update after creating a live image
(keepcache=1 I assume you mean) But is there the potential for things to break if multiple yum processes write to it?
Yeah -- right now, the locking is at the "one yum process at a time" level. livecd-creator isn't taking that lock as, by default, we're not using those caches. It's probably worth looking at per-repo locks in yum for the future to avoid this sort of thing. But that starts to get a little bit more painful.
c) you can specify something like /var/tmp/livecd-cache if you want to have a livecd-specific cache.
I don't really want it to be livecd-specific. The goal is just to have tools that by default do not repeatedly download huge files over the internet. I have plenty of gigs of hard drive space. Mock drove me absolutely nuts for this reason (yes, it has some magic --cache option that's not the default, requires manual intervention to update, and was broken anyways last I tried).
Hmm. Would anything break if we just pointed all the tools at /var/cache/yum? Does yum delete all files it sees there if it has keepcache=0? And what about the concurrency issue?
As of right now, pointing everything at the same place is likely a recipe for disaster. Making it _possible_ for things to share when the user is aware and wants to isn't unreasonable. And then, when we eventually get finer grained locking in place, we can look at moving the default to be shared between everything
Jeremy
Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
On Sun, 2007-08-05 at 20:33 +0200, Jeroen van Meeuwen wrote:
A patch to change the location of the yum cache directory.
This enables existing yum cache to be used, which in our case is especially useful because that cache is populated already. Most importantly, livecd-creator doesn't bind mount the yum cache directory.
Why not do the bind mount? By bind mounting the cache as /var/cache/yum in the chroot, you don't have to do any mucking with the yum config which should make it so that you just have to get the option and if it doesn't exist, fall back to the default. It also makes the cachedir available to the chroot in %post which people could take advantage of if they wanted
I'm not sure I understand what you mean by 'any mucking with the yum config'. Is it the 'temp' yum config livecd-tools you're talking about, LiveCDYum._writeConf()? That config isn't used in our case so it doesn't need any mucking, but I understand your point -if the --yumcache option is in livecd-tools and it doesn't get bind mounted, that'd need mucking.
Bind mounting or not bind mounting is a different matter; it doesn't need discussion as much, because there's no specific reason we currently don't bind mount. It's no show-stopper -that I know of.
If the patch did the bind mount and let's one choose the yum cache directory to use (e.g., to bind mount), all is fine, right?
Kind regards,
Jeroen van Meeuwen -kanarip
On Wed, 2007-08-08 at 15:51 +0200, Jeroen van Meeuwen wrote:
Jeremy Katz wrote:
So, I commented a little on Colin's related patch... I'm leaning towards this approach, though.
On Sun, 2007-08-05 at 20:33 +0200, Jeroen van Meeuwen wrote:
A patch to change the location of the yum cache directory.
This enables existing yum cache to be used, which in our case is especially useful because that cache is populated already. Most importantly, livecd-creator doesn't bind mount the yum cache directory.
Why not do the bind mount? By bind mounting the cache as /var/cache/yum in the chroot, you don't have to do any mucking with the yum config which should make it so that you just have to get the option and if it doesn't exist, fall back to the default. It also makes the cachedir available to the chroot in %post which people could take advantage of if they wanted
I'm not sure I understand what you mean by 'any mucking with the yum config'. Is it the 'temp' yum config livecd-tools you're talking about, LiveCDYum._writeConf()?
Yep.
That config isn't used in our case so it doesn't need any mucking, but I understand your point -if the --yumcache option is in livecd-tools and it doesn't get bind mounted, that'd need mucking.
Bind mounting or not bind mounting is a different matter; it doesn't need discussion as much, because there's no specific reason we currently don't bind mount. It's no show-stopper -that I know of.
Right, I think that bind mounting makes the patch simpler and still gets the same result. So if there's not a reason the bind mount won't work for you, let's go that route
If the patch did the bind mount and let's one choose the yum cache directory to use (e.g., to bind mount), all is fine, right?
Yep, should be good to go
Jeremy
livecd@lists.fedoraproject.org
-
Colin Walters -
Jeremy Katz -
Jeroen van Meeuwen