[PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs

[Douglas McClendon]

attached is a patch which gaging from the history of this list, will be welcomed.

I know there may already be some similar functionality already in revisor, this 
patch adds it to livecd-tools

The two new options are

--addidir=<directory>
and
--addsdir=<directory>

Both may be invoked any number of times.  The contents of the directories 
specified, will be copied to the target livecd.  addsdir adds the contents to 
the rootfs of the target, while addidir adds the contents to the iso9660fs of 
the target.

An example might be to use this to add an index.html and autorun.bat to the iso 
so that when the iso is popped in a winblowz machine, a webpage from the cdrom 
will automatically launch.

The addidir copy happens just before mkisofs, so this can be used to easily 
replace /isolinux/splash.jpg

The addsdir copy happens after installation, but before the selinux relabel and 
optional prelink.

These semantics are up for debate.  Test and enjoy...

As always, comments, criticisms, suggestions are welcome.

-dmc

[Jeremy Katz]

A few thoughts...

On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote:
>  I know there may already be some similar functionality already in
> revisor, this 
>  patch adds it to livecd-tools
>
>  The two new options are
>  
>  --addidir=<directory>
>  and
>  --addsdir=<directory> 
First off, at a glance, the options just look like gobbledy-gook to
me :-)  In cases like this, it's almost always better to spell out what
is meant rather than an abbreviation that isn't immediately obvious.

The second thing is that having this as command-line options leads to an
increased chance of the live image build not being reproducible.  As
much as possible, the movement has been to specifying things within the
kickstart config rather than having command line options for specifying
things.  

In fact, I'd like to overall entirely deprecate the use of most of the
command-line arguments in favor of having them specified in the
kickstart config.  And at the same time, I also want to look at making
sure that we include the kickstart config on the image that's created so
that people can more easily see how the image was created, modify it,
and then create their own.  What do other people think of this?  The
things which really "matter" which aren't currently expressed in the
config are prelink and the uncompressed size of the rootfs.  The latter
is easy (we just add interpretation of part / --size ...).  Adding the
former is probably not terrible to do either.

>  Both may be invoked any number of times.  The contents of the
> directories 
>  specified, will be copied to the target livecd.  addsdir adds the contents to 
>  the rootfs of the target, while addidir adds the contents to the iso9660fs of 
>  the target. 
Then when and what of these occurring seems fine.  Although I wonder if
we'd be better off just making it so that you could do %post --nochroot
and also moving %post to be run later

Jeremy

[Douglas McClendon]

Jeremy Katz wrote:
>  A few thoughts...
>  
>  On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote:
> > I know there may already be some similar functionality already in revisor, this 
> > patch adds it to livecd-tools
> >
> > The two new options are
> >
> > --addidir=<directory>
> > and
> > --addsdir=<directory>
>  
>  First off, at a glance, the options just look like gobbledy-gook to
>  me :-)  In cases like this, it's almost always better to spell out what
>  is meant rather than an abbreviation that isn't immediately obvious. 
aesthetics.  rename however you like.

>  The second thing is that having this as command-line options leads to
> an
>  increased chance of the live image build not being reproducible.  As
>  much as possible, the movement has been to specifying things within the
>  kickstart config rather than having command line options for specifying
>  things.   
I think this is where ideals meet practicality.  But to work with you, would you 
be happier with a patch, where say, instead of --addsdir, there was 
--add-user-system-data, which behaved exactly the same, but the implementation 
mechanism included rolling the directory into a dirt simple noarch rpm with 
almost no data in the specfile?  I.e. do the rpmbuild -bs, rpmbuild --rebuild, 
and createrepo for the user?

I suppose it really isn't that hard (though it will be months before such a 
thing makes it to the top of my priorities).

>  
>  In fact, I'd like to overall entirely deprecate the use of most of the
>  command-line arguments in favor of having them specified in the
>  kickstart config.  And at the same time, I also want to look at making
>  sure that we include the kickstart config on the image that's created so
>  that people can more easily see how the image was created, modify it,
>  and then create their own.  What do other people think of this?  The
>  things which really "matter" which aren't currently expressed in the
>  config are prelink and the uncompressed size of the rootfs.  The latter
>  is easy (we just add interpretation of part / --size ...).  Adding the
>  former is probably not terrible to do either.
>  
> > Both may be invoked any number of times.  The contents of the directories 
> > specified, will be copied to the target livecd.  addsdir adds the contents to 
> > the rootfs of the target, while addidir adds the contents to the iso9660fs of 
> > the target.
>  
>  Then when and what of these occurring seems fine.  Although I wonder if
>  we'd be better off just making it so that you could do %post --nochroot
>  and also moving %post to be run later 
I just saw a mention on kickstart-list, about using just such a scenario, and 
wondered if it was really possible.  Of course, as far as reproducability goes, 
it is a bit wonky, as %post --nochroot seems to me to be just as much a supreme 
breaking of reproducability as the --addsdir thing.  I.e. the environment 
outside the chroot, is going to be entirely different for every build 
(especially traditional anaconda kickstart usage, versus livecd-creator usage).

Also, in my opinion, adding things to the isodir really has /nothing/ to do with 
the system being defined in the kickstart, and really should be separate (maybe 
even separate tool, 'livecd-repacker'?).  I mean, suppose you want to roll your 
own mp3/ogg library on your CD.  Not the sort of thing that seems appropriate 
for a kickstart.

-dmc

[Jeremy Katz]

On Thu, 2007-07-26 at 10:22 -0500, Douglas McClendon wrote:
>  Jeremy Katz wrote:
>  > A few thoughts...
>  > 
>  > On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote:
>  >> I know there may already be some similar functionality already in revisor, this
>
>  >> patch adds it to livecd-tools
>  >>
>  >> The two new options are
>  >>
>  >> --addidir=<directory>
>  >> and
>  >> --addsdir=<directory>
>  > 
>  > First off, at a glance, the options just look like gobbledy-gook to
>  > me :-)  In cases like this, it's almost always better to spell out what
>  > is meant rather than an abbreviation that isn't immediately obvious.
>  
>  aesthetics.  rename however you like. 
Yeah, definitely a minor quibble that's easy to resolve.

>  > The second thing is that having this as command-line options
> leads to an
>  > increased chance of the live image build not being reproducible.  As
>  > much as possible, the movement has been to specifying things within the
>  > kickstart config rather than having command line options for specifying
>  > things.  
>  
>  I think this is where ideals meet practicality.  But to work with you, would you 
>  be happier with a patch, where say, instead of --addsdir, there was 
>  --add-user-system-data, which behaved exactly the same, but the implementation 
>  mechanism included rolling the directory into a dirt simple noarch rpm with 
>  almost no data in the specfile?  I.e. do the rpmbuild -bs, rpmbuild --rebuild, 
>  and createrepo for the user? 
That doesn't really change anything, though.  It still fundamentally
means that the image can't be reproducibly created from its config.

>  > In fact, I'd like to overall entirely deprecate the use of
> most of the
>  > command-line arguments in favor of having them specified in the
>  > kickstart config.  And at the same time, I also want to look at making
>  > sure that we include the kickstart config on the image that's created so
>  > that people can more easily see how the image was created, modify it,
>  > and then create their own.  What do other people think of this?  The
>  > things which really "matter" which aren't currently expressed in the
>  > config are prelink and the uncompressed size of the rootfs.  The latter
>  > is easy (we just add interpretation of part / --size ...).  Adding the
>  > former is probably not terrible to do either.
>  > 
>  >> Both may be invoked any number of times.  The contents of the directories 
>  >> specified, will be copied to the target livecd.  addsdir adds the contents to 
>  >> the rootfs of the target, while addidir adds the contents to the iso9660fs of 
>  >> the target.
>  > 
>  > Then when and what of these occurring seems fine.  Although I wonder if
>  > we'd be better off just making it so that you could do %post --nochroot
>  > and also moving %post to be run later
>  
>  I just saw a mention on kickstart-list, about using just such a scenario, and 
>  wondered if it was really possible.  Of course, as far as reproducability goes, 
>  it is a bit wonky, as %post --nochroot seems to me to be just as much a supreme 
>  breaking of reproducability as the --addsdir thing.  I.e. the environment 
>  outside the chroot, is going to be entirely different for every build 
>  (especially traditional anaconda kickstart usage, versus livecd-creator usage). 
The difference is that you can at least keep it in mind and be doing,
eg, a wget to grab files.  Sure, it doesn't prevent you from doing
something like using a local file, but there's nothing now that stops
you from using a file:// repo.  

>  Also, in my opinion, adding things to the isodir really has /nothing/
> to do with 
>  the system being defined in the kickstart, and really should be separate (maybe 
>  even separate tool, 'livecd-repacker'?).  I mean, suppose you want to roll your 
>  own mp3/ogg library on your CD.  Not the sort of thing that seems appropriate 
>  for a kickstart. 
But if I want to include a library of demos or videos about Fedora to
help educate a new user, I do want them there.  So yes, there are cases
why reproducibility might not matter.  But as a general case, it really
does matter quite a bit

Jeremy

[Douglas McClendon]

Jeremy Katz wrote:
>  On Thu, 2007-07-26 at 10:22 -0500, Douglas McClendon wrote:
> > Jeremy Katz wrote:
> >> A few thoughts...
> >>
> >> On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote:
> >>> I know there may already be some similar functionality already in revisor,
> this 
> >>> patch adds it to livecd-tools
> >>>
> >>> The two new options are
> >>>
> >>> --addidir=<directory>
> >>> and
> >>> --addsdir=<directory> 
> >> The second thing is that having this as command-line options
> leads to an
> >> increased chance of the live image build not being reproducible.  As
> >> much as possible, the movement has been to specifying things within the
> >> kickstart config rather than having command line options for specifying
> >> things.  
> > I think this is where ideals meet practicality.  But to work with you, would you 
> > be happier with a patch, where say, instead of --addsdir, there was 
> > --add-user-system-data, which behaved exactly the same, but the implementation 
> > mechanism included rolling the directory into a dirt simple noarch rpm with 
> > almost no data in the specfile?  I.e. do the rpmbuild -bs, rpmbuild --rebuild, 
> > and createrepo for the user?
>  
>  That doesn't really change anything, though.  It still fundamentally
>  means that the image can't be reproducibly created from its config. 
If reproducibility from a single text file config is that important to you, then 
you are correct.  Short of uuencoding tarballs, there is no way to get there 
from here.

-dmc

[Jeremy Katz]

On Thu, 2007-07-26 at 15:19 -0500, Douglas McClendon wrote:
>  Jeremy Katz wrote:
>  > That doesn't really change anything, though.  It still fundamentally
>  > means that the image can't be reproducibly created from its config.
>  
>  If reproducibility from a single text file config is that important to you, then 
>  you are correct.  Short of uuencoding tarballs, there is no way to get there 
>  from here. 
We're already dependent on being able to get to the repo -- being
dependent on grabbing things from a web server or similar also for
"added" pieces isn't crazy I don't think.  Although doing it in a way
that's flexible enough to _allow_ local files is fine, I don't want to
be showing that as the "one true way"

Jeremy

[Douglas McClendon]

Jeremy Katz wrote:
>  On Thu, 2007-07-26 at 15:19 -0500, Douglas McClendon wrote:
> > Jeremy Katz wrote:
> >> That doesn't really change anything, though.  It still fundamentally
> >> means that the image can't be reproducibly created from its config.
> > If reproducibility from a single text file config is that important to you, then 
> > you are correct.  Short of uuencoding tarballs, there is no way to get there 
> > from here.
>  
>  We're already dependent on being able to get to the repo -- being
>  dependent on grabbing things from a web server or similar also for
>  "added" pieces isn't crazy I don't think.  Although doing it in a way
>  that's flexible enough to _allow_ local files is fine, I don't want to
>  be showing that as the "one true way" 
I'd honestly by perfectly happy if they were completely undocumented options, 
except at the bottom of the existing /usr/share/doc/livecd-tools-version/HACKING 
file.

-dmc