Re: Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[David Zeuthen]

Hi,

On Wed, 2006-09-20 at 10:48 -0700, Toshio Kuratomi wrote:
>  Does pilgrim make an attempt to integrate any of stateless's
> work?  In
>  my mind integrating stateless with livecd creation just makes sense.
>  But I don't think there's been much work done on that front since
>  Jeremy's proof of concept fork of kadischi which I don't think he's been
>  updating. 
Nope, I think it's much more elegant to just use dm-snapshot to provide
a real rw rootfs. Not sure what Bill Nottingham (Cc'ed) or people
working on stateless team thinks of this, they might have a number of
good reasons that I haven't thought out. I still think stateless makes
sense for non-livecd work however.

Btw, If someone could talk davej into including unionfs into the Fedora
kernel, we'd use that instead of dm-snapshot and we'd have persistence
more easily solved [1].

    David

[1] : we can already do this for our livecd but it will be tied to the
specific build you're using, e.g. in practice it's tied to the physical
media you created it with. 

With unionfs things might look much better and we'd easily be able to do
a harddisk install of "livecd + your changes made" instead of harddisk
install becomes contents of "stock livecd", ie. without your changes.
That said, I'm not sure that this really matters in real life.

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[Bill Nottingham]

David Zeuthen (davidz(a)redhat.com) said: 
>  Nope, I think it's much more elegant to just use dm-snapshot to
> provide
>  a real rw rootfs. Not sure what Bill Nottingham (Cc'ed) or people
>  working on stateless team thinks of this, they might have a number of
>  good reasons that I haven't thought out. I still think stateless makes
>  sense for non-livecd work however. 
The reason we didn't use dm-snapshot is that it removes the security
benefits of readonly-root (after all, you don't need 99% of the system
to actually be read-write); moreover, you can't selectively apply it
(it has to be done at the whole block device level.)

>  Btw, If someone could talk davej into including unionfs into the
> Fedora
>  kernel, we'd use that instead of dm-snapshot and we'd have persistence
>  more easily solved [1]. 
Flaming death. Deadlocks, oopses, etc. (it might be better now)

Bill

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[Dave Jones]

On Wed, Sep 20, 2006 at 04:07:12PM -0400, David Zeuthen wrote:

>  Btw, If someone could talk davej into including unionfs into the
> Fedora
>  kernel, we'd use that instead of dm-snapshot and we'd have persistence
>  more easily solved [1]. 
I think the comments Al Viro had on it the last time it was reviewed
were for the most part unprintable. I wouldn't hold your breath for
this to appear any time soon.

	Dave

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[David Zeuthen]

Hey Dave,

On Wed, 2006-09-20 at 20:20 -0400, Dave Jones wrote:
>  On Wed, Sep 20, 2006 at 04:07:12PM -0400, David Zeuthen wrote:
>  
>   > Btw, If someone could talk davej into including unionfs into the Fedora
>   > kernel, we'd use that instead of dm-snapshot and we'd have persistence
>   > more easily solved [1].
>  
>  I think the comments Al Viro had on it the last time it was reviewed
>  were for the most part unprintable. I wouldn't hold your breath for
>  this to appear any time soon. 
Right. My understanding is that the controversial part of unionfs is the
ability to join multiple writable file systems into a single tree. Is
this correct?

If so, note that this is not a feature livecd nor stateless needs, the
one part is always read-only, the other parts is just a single overlay
where we take damage.

How hard would it be to do a unionfs-ro (read only) with the following
semantics

 1. Support exactly two underlying directories, the first assumed to be
    read only

 2. If some file exists in both trees, pick file with latest ctime

I dunno much about the kernel VFS layer to say whether this is easy or
hard but I do hope this is a lot simpler than what current unionfs is
doing. So.. is this hard?

Btw, justification for 2. ("pick file with latest ctime", not just "if
file exist in rw branch, always pick rw branch"): suppose I use a livecd
using this unionfs-ro fs and updates my bash package. The changes are
now written to a USB stick such that I have a persistent session. I now
download a newer version of the livecd where the bash package is newer.
When using this together with my USB stick, we'll pick the
newest /bin/bash file.

    David

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[Dave Jones]

<subscriber only list removed from Cc>

On Thu, Sep 21, 2006 at 08:31:48PM -0400, David Zeuthen wrote:
 
>  On Wed, 2006-09-20 at 20:20 -0400, Dave Jones wrote:
>  > On Wed, Sep 20, 2006 at 04:07:12PM -0400, David Zeuthen wrote:
>  > 
>  >  > Btw, If someone could talk davej into including unionfs into the Fedora
>  >  > kernel, we'd use that instead of dm-snapshot and we'd have
> persistence
>  >  > more easily solved [1].
>  > 
>  > I think the comments Al Viro had on it the last time it was reviewed
>  > were for the most part unprintable. I wouldn't hold your breath for
>  > this to appear any time soon.
>  
>  Right. My understanding is that the controversial part of unionfs is the
>  ability to join multiple writable file systems into a single tree. Is
>  this correct? 
tbh, since Al reviewed it unfavorably, I haven't even bothered looking at it,
so I'm the wrong person to be asking.  But ISTR there were a number
of issues rather than one sticking point.

Al would be a better person to ask about the gory details.

	Dave

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[Bill Nottingham]

David Zeuthen (davidz(a)redhat.com) said: 
>  Right. My understanding is that the controversial part of unionfs is
> the
>  ability to join multiple writable file systems into a single tree. Is
>  this correct? 
Well, in my experience, all I tried was one readable + one writable, and
*that* blew up. So, my objection was that the basic case failed.

Bill

Pilgrim, kadischi, and stateless (was Re: [Fedora-livecd-list] pilgrim livecd work)

[David Zeuthen]

On Thu, 2006-09-21 at 21:26 -0400, Bill Nottingham wrote:
>  David Zeuthen (davidz(a)redhat.com) said: 
>  > Right. My understanding is that the controversial part of unionfs is the
>  > ability to join multiple writable file systems into a single tree. Is
>  > this correct? 
Just found the page here,

 http://www.am-utils.org/project-unionfs.html

And yes, it appears it's designed to do a lot of things instead of just
doing one thing really well.

>  Well, in my experience, all I tried was one readable + one writable,
> and
>  *that* blew up. So, my objection was that the basic case failed. 
I'm more curious how hard it can be given the assumptions I listed in my
other mail. Then again, I'm not a kernel hacker and got too much on my
plate already :-)

     David