I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
#1 - mayflower.conf - add PROGRAMS and FILES
I'd like it if mayflower, in addition to supporting MODULES+= lines, would also support PROGRAMS+= and FILES+=
PROGRAMS+= would be used for example, in my prior persistence patch, to add things like ntfsmount.
FILES+= would differ from PROGRAMS, in that the auto shared library dependency check would not apply to them. Maybe that doesn't matter, and you could just have FILES.
#2 - support user specified mayflower.conf location (i.e. not just /etc/mayflower.conf)
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
You would of course supply your own custom program via #1 above.
Now, you are no doubt asking, what would this be used for...
Answers:
1) support SELinux enabled livecd creation from an SELinux disabled build system.
2) support building of anaconda rpm as a non root user
3) support livecd-creator as a non root user
Now you are no doubt asking, how do #1 #2 & #3 get you 1) 2) and 3)?
The answer is by implementing the qmkfs program I alluded to a long time ago on fedora-devel, and it's more general incarnation, qrr (qemu root run)
The idea is this- qrr is a script. qrr reads it's input, a config file, describing a program and some data. qrr then invokes mayflower (as a user, not root, with #2) to generate an initramfs, that is basically the same as the livecd initramfs, but with extra programs, specified by the config file, and added with #1. Then, qemu is invoked with #3, somewhat like this-
mkdir ./my_input_output_dir cp -av <user supplied input data> ./my_input_output_dir qemu-img create scratch.img 10G qemu -kernel /boot/vmlinuz-$(uname -r) \ -initrd ./my_mayflower_generated_initrd.img \ -append "program=/my_custom_qrr_program" \ -hda ./scratch.img \ -hdb fat:rw:./my_input_output_dir
Thus, the mayflower initramfs boots, runs the user supplied my_custom_qrr_program, which mounts the my_input_output_dir, and proceeds to process it with root privs (i.e. it can do loopback mounts, selinux relabeling, etc...), and then leaves its output (e.g. a filesystem image) in my_input_output_dir.
I think that the changes involved with #1 #2 and #3 are fairly minor, elegant, and safe. Actually implementing 1) and 2) are a bit tricky. 3) is probably a lot tricky, but arguably worth the effort.
Mainly I want this feature (qrr) for my own currently private project, though I would definitely go ahead and implement 1) as part of the patchset to be submitted for merging, which I think is justification enough for #1 #2 and #3.
In general, the ability to use mayflower to construct arbitrarily useful custom purpose initramfs-s, seems quite useful. I imagine that when people really start to think about this, and what it allows, they will come up with many uses that I would probably never imagine.
Personally I am hoping for a koji/pungi/livecd-creator/revisor that does not require root privileges at all. It seems to me like there should be no reason why a non-root user cannot take the fedora cvs/git trees, and output (customized) F9 media sets.
I realize this may sound like a lot of stuff, but please focus on how small and safe the patches to support #1 #2 and #3 really are.
Also, I have basically done this somewhat manually. (tweaked mayflower, generated initramfs as user, booted qemu to get arbitrary program to run)
questions/comments/criticisms?
-dmc
Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
...
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
Also, #3 would be invoked by an optional argument to mayflower, e.g. --program
(as always, feel free to suggest better names for things)
Likewise, in the scenario of my persistence patch, presumably an enable/disable flag (or configfile option somehow) would be passed to livecd-creator. Then, if appropriate, the persistence stuff (findoverlay script, vfat module, ntfsmount and fuse module maybe) would be added to the resultant livecd initramfs via the mayflower.conf.
-dmc
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file? Then (I believe) someone could have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages: * A sample kickstart and a sample config with lots of comments and all the options plus a basic tutorial would be all the documentation many people would need * It would make it much simpler to return to a project months later and tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
Tim Wood
On Aug 12, 2007, at 10:13 PM, Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
#1 - mayflower.conf - add PROGRAMS and FILES
I'd like it if mayflower, in addition to supporting MODULES+= lines, would also support PROGRAMS+= and FILES+=
PROGRAMS+= would be used for example, in my prior persistence patch, to add things like ntfsmount.
FILES+= would differ from PROGRAMS, in that the auto shared library dependency check would not apply to them. Maybe that doesn't matter, and you could just have FILES.
#2 - support user specified mayflower.conf location (i.e. not just /etc/mayflower.conf)
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
You would of course supply your own custom program via #1 above.
Now, you are no doubt asking, what would this be used for...
Answers:
- support SELinux enabled livecd creation from an SELinux disabled
build system.
support building of anaconda rpm as a non root user
support livecd-creator as a non root user
Now you are no doubt asking, how do #1 #2 & #3 get you 1) 2) and 3)?
The answer is by implementing the qmkfs program I alluded to a long time ago on fedora-devel, and it's more general incarnation, qrr (qemu root run)
The idea is this- qrr is a script. qrr reads it's input, a config file, describing a program and some data. qrr then invokes mayflower (as a user, not root, with #2) to generate an initramfs, that is basically the same as the livecd initramfs, but with extra programs, specified by the config file, and added with #1. Then, qemu is invoked with #3, somewhat like this-
mkdir ./my_input_output_dir cp -av <user supplied input data> ./my_input_output_dir qemu-img create scratch.img 10G qemu -kernel /boot/vmlinuz-$(uname -r) \ -initrd ./my_mayflower_generated_initrd.img \ -append "program=/my_custom_qrr_program" \ -hda ./scratch.img \ -hdb fat:rw:./my_input_output_dir
Thus, the mayflower initramfs boots, runs the user supplied my_custom_qrr_program, which mounts the my_input_output_dir, and proceeds to process it with root privs (i.e. it can do loopback mounts, selinux relabeling, etc...), and then leaves its output (e.g. a filesystem image) in my_input_output_dir.
I think that the changes involved with #1 #2 and #3 are fairly minor, elegant, and safe. Actually implementing 1) and 2) are a bit tricky. 3) is probably a lot tricky, but arguably worth the effort.
Mainly I want this feature (qrr) for my own currently private project, though I would definitely go ahead and implement 1) as part of the patchset to be submitted for merging, which I think is justification enough for #1 #2 and #3.
In general, the ability to use mayflower to construct arbitrarily useful custom purpose initramfs-s, seems quite useful. I imagine that when people really start to think about this, and what it allows, they will come up with many uses that I would probably never imagine.
Personally I am hoping for a koji/pungi/livecd-creator/revisor that does not require root privileges at all. It seems to me like there should be no reason why a non-root user cannot take the fedora cvs/ git trees, and output (customized) F9 media sets.
I realize this may sound like a lot of stuff, but please focus on how small and safe the patches to support #1 #2 and #3 really are.
Also, I have basically done this somewhat manually. (tweaked mayflower, generated initramfs as user, booted qemu to get arbitrary program to run)
questions/comments/criticisms?
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Tim Wood wrote:
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file?
Most of my RFC was basically about extending the functionality of the mayflower.conf config file. But currently, as far as livecd-creator (and presumably revisor) is concerned, a mayflower.conf is hard-coded.
The question is whether or not there is any reason to expose the livecd-creator or revisor user to these sorts of options.
For the selinux enabled on an selinux disabled system, I think this should all happen inside livecd creator, and cause things to 'just work' (versus now, where it detects the situation, and says 'too bad, you can't do this').
For the anaconda rpm build as non-root user, this really has nothing to do with livecd-creator or revisor.
And for the vastly trickier idea of livecd-creator as non-root, again, there is no aspect that the livecd-creator or revisor user should care about, other than having it just work.
Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and all
the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later and
tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
1) files added to the iso filesystem. E.g. like ubuntu's inclusion of the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
2) the persistence feature.
OTOH, here is how I guess I could imagine cramming everything into the kickstart-
1) have some truly magic livecd kickstart command to add files to the isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
2) and this is rather key- put the mayflower invocation in the %post of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case. Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
2 as described (with or without remerge with anaconda) also gives you the ability to customize the initramfs (e.g. add persistence and similar mayflower optional features) in the %post of the kickstart.
But I want to emphasize to anyone reading this far, that none of this really has anything to do with the simple modifications and simple functionality enhancements that I was aiming for in the parent RFC. This has been a tangent going down rearchitecting the kickstart/config file and invocation of livecd-creator.
And actually, given what I described, I kind of like the uglier and uglier, but single kickstart solution. (the other aspects which I've complained about in the past, I think I can see workarounds for as well)
-dmc
Douglas McClendon wrote:
Tim Wood wrote:
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file?
Most of my RFC was basically about extending the functionality of the mayflower.conf config file. But currently, as far as livecd-creator (and presumably revisor) is concerned, a mayflower.conf is hard-coded.
The question is whether or not there is any reason to expose the livecd-creator or revisor user to these sorts of options.
For the selinux enabled on an selinux disabled system, I think this should all happen inside livecd creator, and cause things to 'just work' (versus now, where it detects the situation, and says 'too bad, you can't do this').
For the anaconda rpm build as non-root user, this really has nothing to do with livecd-creator or revisor.
And for the vastly trickier idea of livecd-creator as non-root, again, there is no aspect that the livecd-creator or revisor user should care about, other than having it just work.
Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and
all the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later
and tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
I agree, Kickstart should not be the_ only_ file for configuration.
- files added to the iso filesystem. E.g. like ubuntu's inclusion of
the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
Why only LiveCD, think beyond the liveCD. Eg. Small systems can be fit on to a USBstick, Rescue systems, small network servers, small streamers etc...
- the persistence feature.
It is a must have feature. But again this should be configurable. Currently Which I developed is doing a persistence at the time of instalation(again a bit hard coded).
OTOH, here is how I guess I could imagine cramming everything into the kickstart-
- have some truly magic livecd kickstart command to add files to the
isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
- and this is rather key- put the mayflower invocation in the %post
of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case. Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
2 as described (with or without remerge with anaconda) also gives you the ability to customize the initramfs (e.g. add persistence and similar mayflower optional features) in the %post of the kickstart.
But I want to emphasize to anyone reading this far, that none of this really has anything to do with the simple modifications and simple functionality enhancements that I was aiming for in the parent RFC. This has been a tangent going down rearchitecting the kickstart/config file and invocation of livecd-creator.
And actually, given what I described, I kind of like the uglier and uglier, but single kickstart solution. (the other aspects which I've complained about in the past, I think I can see workarounds for as well)
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
ashok shankar das wrote:
Douglas McClendon wrote:
Tim Wood wrote:
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file?
Most of my RFC was basically about extending the functionality of the mayflower.conf config file. But currently, as far as livecd-creator (and presumably revisor) is concerned, a mayflower.conf is hard-coded.
The question is whether or not there is any reason to expose the livecd-creator or revisor user to these sorts of options.
For the selinux enabled on an selinux disabled system, I think this should all happen inside livecd creator, and cause things to 'just work' (versus now, where it detects the situation, and says 'too bad, you can't do this').
For the anaconda rpm build as non-root user, this really has nothing to do with livecd-creator or revisor.
And for the vastly trickier idea of livecd-creator as non-root, again, there is no aspect that the livecd-creator or revisor user should care about, other than having it just work.
Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and
all the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later
and tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
I agree, Kickstart should not be the_ only_ file for configuration.
Actually I sort of argued both sides of that coin. And I ended up changing my mind. What sorts of option and configuration do you think should not be in kickstart, but somewhere else?
- files added to the iso filesystem. E.g. like ubuntu's inclusion of
the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
Why only LiveCD, think beyond the liveCD. Eg. Small systems can be fit on to a USBstick, Rescue systems, small network servers, small streamers etc...
usbstick (especially with livecd-iso-to-disk) is really just an alternate form of livecd.
But yes, the changes I proposed to mayflower can also be used to generate initramfss that are useful as rescue systems, and perhaps even as entire-system-in-initramfs for small servers/routers. I never suggested it was useful for livecd only.
- the persistence feature.
It is a must have feature. But again this should be configurable. Currently Which I developed is doing a persistence at the time of instalation(again a bit hard coded).
I'm still curious where this implementation you speak of that you claim was posted to this list is. Please repost or give a link to an archive post.
I'm curious whether your implementation has design aspects that are better than the implementation I posted a couple weeks ago, and discussed at length on this list in the couple weeks before that.
(as I've told Tim offlist, I'll be posting an updated version of that implementation within the next couple of days. I'm just honing my git skills at the moment)
-dmc
OTOH, here is how I guess I could imagine cramming everything into the kickstart-
- have some truly magic livecd kickstart command to add files to the
isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
- and this is rather key- put the mayflower invocation in the %post
of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case. Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
2 as described (with or without remerge with anaconda) also gives you the ability to customize the initramfs (e.g. add persistence and similar mayflower optional features) in the %post of the kickstart.
But I want to emphasize to anyone reading this far, that none of this really has anything to do with the simple modifications and simple functionality enhancements that I was aiming for in the parent RFC. This has been a tangent going down rearchitecting the kickstart/config file and invocation of livecd-creator.
And actually, given what I described, I kind of like the uglier and uglier, but single kickstart solution. (the other aspects which I've complained about in the past, I think I can see workarounds for as well)
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Douglas McClendon wrote:
ashok shankar das wrote:
Douglas McClendon wrote:
Tim Wood wrote:
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file?
Most of my RFC was basically about extending the functionality of the mayflower.conf config file. But currently, as far as livecd-creator (and presumably revisor) is concerned, a mayflower.conf is hard-coded.
The question is whether or not there is any reason to expose the livecd-creator or revisor user to these sorts of options.
For the selinux enabled on an selinux disabled system, I think this should all happen inside livecd creator, and cause things to 'just work' (versus now, where it detects the situation, and says 'too bad, you can't do this').
For the anaconda rpm build as non-root user, this really has nothing to do with livecd-creator or revisor.
And for the vastly trickier idea of livecd-creator as non-root, again, there is no aspect that the livecd-creator or revisor user should care about, other than having it just work.
Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and
all the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later
and tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
I agree, Kickstart should not be the_ only_ file for configuration.
Actually I sort of argued both sides of that coin. And I ended up changing my mind. What sorts of option and configuration do you think should not be in kickstart, but somewhere else?
- files added to the iso filesystem. E.g. like ubuntu's inclusion
of the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
Why only LiveCD, think beyond the liveCD. Eg. Small systems can be fit on to a USBstick, Rescue systems, small network servers, small streamers etc...
usbstick (especially with livecd-iso-to-disk) is really just an alternate form of livecd.
But yes, the changes I proposed to mayflower can also be used to generate initramfss that are useful as rescue systems, and perhaps even as entire-system-in-initramfs for small servers/routers. I never suggested it was useful for livecd only.
- the persistence feature.
It is a must have feature. But again this should be configurable. Currently Which I developed is doing a persistence at the time of instalation(again a bit hard coded).
I'm still curious where this implementation you speak of that you claim was posted to this list is. Please repost or give a link to an archive post.
I have a very vague vison fo what to be separated. I think the target specific things should be on a separate file. from which the KS file can read and work acordingly. As an example the drivers which goes into the initrd are hard coded. Suppose i got a weired hardware which needs some special drivers then i am helpless in current scenario.
why should be there a ISO-to-USB script? This sort of configuration should go into separate config file too.
Try using KS file only for the package instalation and post configuration of the installed system. Try to Implement a Config based image creation and target setup.
I can say this much at this time.
I'm curious whether your implementation has design aspects that are better than the implementation I posted a couple weeks ago, and discussed at length on this list in the couple weeks before that.
(as I've told Tim offlist, I'll be posting an updated version of that implementation within the next couple of days. I'm just honing my git skills at the moment)
-dmc
OTOH, here is how I guess I could imagine cramming everything into the kickstart-
- have some truly magic livecd kickstart command to add files to
the isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
- and this is rather key- put the mayflower invocation in the %post
of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case. Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
2 as described (with or without remerge with anaconda) also gives you the ability to customize the initramfs (e.g. add persistence and similar mayflower optional features) in the %post of the kickstart.
But I want to emphasize to anyone reading this far, that none of this really has anything to do with the simple modifications and simple functionality enhancements that I was aiming for in the parent RFC. This has been a tangent going down rearchitecting the kickstart/config file and invocation of livecd-creator.
And actually, given what I described, I kind of like the uglier and uglier, but single kickstart solution. (the other aspects which I've complained about in the past, I think I can see workarounds for as well)
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
ashok shankar das wrote:
Douglas McClendon wrote:
ashok shankar das wrote:
Douglas McClendon wrote:
Tim Wood wrote:
The one comment I'd have is would it be possible to feed some (or all) of these options in as a config file?
Most of my RFC was basically about extending the functionality of the mayflower.conf config file. But currently, as far as livecd-creator (and presumably revisor) is concerned, a mayflower.conf is hard-coded.
The question is whether or not there is any reason to expose the livecd-creator or revisor user to these sorts of options.
For the selinux enabled on an selinux disabled system, I think this should all happen inside livecd creator, and cause things to 'just work' (versus now, where it detects the situation, and says 'too bad, you can't do this').
For the anaconda rpm build as non-root user, this really has nothing to do with livecd-creator or revisor.
And for the vastly trickier idea of livecd-creator as non-root, again, there is no aspect that the livecd-creator or revisor user should care about, other than having it just work.
Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and
all the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later
and tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
I agree, Kickstart should not be the_ only_ file for configuration.
Actually I sort of argued both sides of that coin. And I ended up changing my mind. What sorts of option and configuration do you think should not be in kickstart, but somewhere else?
- files added to the iso filesystem. E.g. like ubuntu's inclusion
of the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
Why only LiveCD, think beyond the liveCD. Eg. Small systems can be fit on to a USBstick, Rescue systems, small network servers, small streamers etc...
usbstick (especially with livecd-iso-to-disk) is really just an alternate form of livecd.
But yes, the changes I proposed to mayflower can also be used to generate initramfss that are useful as rescue systems, and perhaps even as entire-system-in-initramfs for small servers/routers. I never suggested it was useful for livecd only.
- the persistence feature.
It is a must have feature. But again this should be configurable. Currently Which I developed is doing a persistence at the time of instalation(again a bit hard coded).
I'm still curious where this implementation you speak of that you claim was posted to this list is. Please repost or give a link to an archive post.
I have a very vague vison fo what to be separated. I think the target specific things should be on a separate file. from which the KS file can read and work acordingly. As an example the drivers which goes into the initrd are hard coded.
%post mayflower invocation
Suppose i got a weired hardware which needs some special drivers then i am helpless in current scenario.
%post mayflower invocation
why should be there a ISO-to-USB script?
because it's so blaringly obviously convenient and useful. Especially for people that downloaded the f7 livecd, want to boot from usb, and have no interest in running livecd-creator or something similar on their system.
This sort of configuration
should go into separate config file too.
Try using KS file only for the package instalation and post configuration of the installed system. Try to Implement a Config based image creation and target setup.
I'm still not seeing any example where its necessary.
I can say this much at this time.
So what you're saying, is that you are waving your hands about this alleged persistence implementation of yours, which offlist you claim has already been posted to the list, but when asked to actually provide a link to the post or the implementation, you've got nuthin.
-dmc
I'm curious whether your implementation has design aspects that are better than the implementation I posted a couple weeks ago, and discussed at length on this list in the couple weeks before that.
(as I've told Tim offlist, I'll be posting an updated version of that implementation within the next couple of days. I'm just honing my git skills at the moment)
-dmc
OTOH, here is how I guess I could imagine cramming everything into the kickstart-
- have some truly magic livecd kickstart command to add files to
the isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
- and this is rather key- put the mayflower invocation in the %post
of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case. Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
2 as described (with or without remerge with anaconda) also gives you the ability to customize the initramfs (e.g. add persistence and similar mayflower optional features) in the %post of the kickstart.
But I want to emphasize to anyone reading this far, that none of this really has anything to do with the simple modifications and simple functionality enhancements that I was aiming for in the parent RFC. This has been a tangent going down rearchitecting the kickstart/config file and invocation of livecd-creator.
And actually, given what I described, I kind of like the uglier and uglier, but single kickstart solution. (the other aspects which I've complained about in the past, I think I can see workarounds for as well)
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
On Mon, 2007-08-13 at 01:07 -0500, Douglas McClendon wrote:
Tim Wood wrote: Then (I believe) someone could
have a kickstart and a config file to handle everything. Then Revisor would need one additional field to select the path to this config file. A couple of advantages:
- A sample kickstart and a sample config with lots of comments and all
the options plus a basic tutorial would be all the documentation many people would need
- It would make it much simpler to return to a project months later and
tweak/tune/use it (find 2 files you left in /etc/revisor/... vs. files there, notes somewhere else and maybe some custom bash code somewhere else)
What you are getting at here, is really the crux of the debate I had with jeremy over the addsdir/addidir patch, and the ideal of reproducability from a _single_ config file.
The basic problem, which I think you solved with your outline above, is that kickstart is simply not appropriate to be the one _single_ config file for a livecd project. The two examples that immediately spring to mind are
People are caught up, though, on "you can't add anything to kickstart". Which is a patently false statement. Also, the idea is that we're also moving pungi also over to use the kickstart syntax (+parser). That way, you have _one_ config file that can be used for any of the following: 1) Create a live image 2) Create an installable tree 3) Install from an everything type tree
Which starts to be pretty flexible. For some of the things like initial config, etc -- well, we're starting to want to let that be specified when creating an installable tree so that you can create an installable tree with a certain set of default config stuff that anaconda listens to.
- files added to the iso filesystem. E.g. like ubuntu's inclusion of
the windows firefox installer. Or a generic web page to be viewed under windows when the livecd is inserted in a windows system.
1A) have some truly magic livecd kickstart command to add files to the isodir. This command would be silently ignored in the non-livecd case (or perhaps files copied to /iso or some arbitrary directory in a non-livecd kickstart invocation).
It's definitely going to be okay if there are things that livecd-creator + pungi use when they're doing things that the normal install case ignores. Much like today, livecd-creator ignores a few things that aren't relevant to it.
- the persistence feature.
2A) and this is rather key- put the mayflower invocation in the %post of the kickstart. Perhaps even enclose it in a conditional, based on some variable that only gets defined in the livecd case.
I'm not sure how persistence really requires anything in the config. There's not any reason why you wouldn't want your live image to be _capable_ of persistence. You may configure it to not enable it by default, but that's a different story (and from our earlier discussion, a boot arg is probably sufficient for this and with the handling of the bootloader argument that I was pointing kanarip to last week, this just falls out, voila! :-)
Then perhaps even merge the livecd-creator code back into anaconda, ala the old kadischi anaconda rootpath livecd creation facility. (yes Jeremy, I'm trying to give you nightmares ;)
FWIW, I keep coming back here -- the amount of code that we're basically copying and pasting from anaconda is a little depressing. And especially with markmc's patches to sort of abstract out the "what's being installed to", we get even closer. And by cleaning up anaconda and moving over to using it, we get a UI for "free". I don't know what the long-term answer here is :-/
Jeremy
follow up since I've no urge to look like a newb in the bat-it-over the fence group on this list... what is Mayflower? Link to let me RTFM is fine o'course...
Tim
On Aug 12, 2007, at 10:13 PM, Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
#1 - mayflower.conf - add PROGRAMS and FILES
I'd like it if mayflower, in addition to supporting MODULES+= lines, would also support PROGRAMS+= and FILES+=
PROGRAMS+= would be used for example, in my prior persistence patch, to add things like ntfsmount.
FILES+= would differ from PROGRAMS, in that the auto shared library dependency check would not apply to them. Maybe that doesn't matter, and you could just have FILES.
#2 - support user specified mayflower.conf location (i.e. not just /etc/mayflower.conf)
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
You would of course supply your own custom program via #1 above.
Now, you are no doubt asking, what would this be used for...
Answers:
- support SELinux enabled livecd creation from an SELinux disabled
build system.
support building of anaconda rpm as a non root user
support livecd-creator as a non root user
Now you are no doubt asking, how do #1 #2 & #3 get you 1) 2) and 3)?
The answer is by implementing the qmkfs program I alluded to a long time ago on fedora-devel, and it's more general incarnation, qrr (qemu root run)
The idea is this- qrr is a script. qrr reads it's input, a config file, describing a program and some data. qrr then invokes mayflower (as a user, not root, with #2) to generate an initramfs, that is basically the same as the livecd initramfs, but with extra programs, specified by the config file, and added with #1. Then, qemu is invoked with #3, somewhat like this-
mkdir ./my_input_output_dir cp -av <user supplied input data> ./my_input_output_dir qemu-img create scratch.img 10G qemu -kernel /boot/vmlinuz-$(uname -r) \ -initrd ./my_mayflower_generated_initrd.img \ -append "program=/my_custom_qrr_program" \ -hda ./scratch.img \ -hdb fat:rw:./my_input_output_dir
Thus, the mayflower initramfs boots, runs the user supplied my_custom_qrr_program, which mounts the my_input_output_dir, and proceeds to process it with root privs (i.e. it can do loopback mounts, selinux relabeling, etc...), and then leaves its output (e.g. a filesystem image) in my_input_output_dir.
I think that the changes involved with #1 #2 and #3 are fairly minor, elegant, and safe. Actually implementing 1) and 2) are a bit tricky. 3) is probably a lot tricky, but arguably worth the effort.
Mainly I want this feature (qrr) for my own currently private project, though I would definitely go ahead and implement 1) as part of the patchset to be submitted for merging, which I think is justification enough for #1 #2 and #3.
In general, the ability to use mayflower to construct arbitrarily useful custom purpose initramfs-s, seems quite useful. I imagine that when people really start to think about this, and what it allows, they will come up with many uses that I would probably never imagine.
Personally I am hoping for a koji/pungi/livecd-creator/revisor that does not require root privileges at all. It seems to me like there should be no reason why a non-root user cannot take the fedora cvs/ git trees, and output (customized) F9 media sets.
I realize this may sound like a lot of stuff, but please focus on how small and safe the patches to support #1 #2 and #3 really are.
Also, I have basically done this somewhat manually. (tweaked mayflower, generated initramfs as user, booted qemu to get arbitrary program to run)
questions/comments/criticisms?
-dmc
-- Fedora-livecd-list mailing list Fedora-livecd-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list
Tim Wood wrote:
follow up since I've no urge to look like a newb in the bat-it-over the fence group on this list... what is Mayflower? Link to let me RTFM is fine o'course...
Tim
mayflower is functionally akin to mkinitrd, except that it makes initramfss suitable for booting livecds, as opposed to booting normal systems. It is located in the livecd-tools rpm, and lives in /usr/lib/livecd-tools/mayflower. During livecd-creator, it gets copied to the system being built, run in a chroot to create the initramfs, which is then put in the livecds isolinux directory and used to boot it.
Query- Does anybody else think its time to create some sort of mailinglist distinction between
a) livecd users b) livecd-tools/revisor users c) livecd-tools/revisor developers
-dmc
On Sun, 2007-08-12 at 23:13 -0500, Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
So from a big high-level point of view, I'd really rather _not_ add lots more that people depend on from mayflower. Having two different codebases to handle the initramfs (mayflower + mkinitrd) is incredibly painful for having to adapt both for changes in the OS, etc.
And given that overall mkinitrd "does more", I think it's going to be better to get the live image specific stuff integrated into mkinitrd rather than spending a lot of time making mayflower more flexible for the sake of flexibility.
But to comment on the concepts...
#1 - mayflower.conf - add PROGRAMS and FILES
[snip]
FILES+= would differ from PROGRAMS, in that the auto shared library dependency check would not apply to them. Maybe that doesn't matter, and you could just have FILES.
Having two different lists is just asking for people to pick the wrong one. Since I can't think of a real reason that you'd want a binary without its deps, just lump them together.
#2 - support user specified mayflower.conf location (i.e. not just /etc/mayflower.conf)
Not terribly against or for this. Having the ability to specify config locations is often useful enough for writing test cases that it's worthwhile
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
Why not just use init= ? Other than the fact that doing so is currently broken with mayflower (... see above comments about two implementations of the same thing :-)
Jeremy
Jeremy Katz wrote:
On Sun, 2007-08-12 at 23:13 -0500, Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
So from a big high-level point of view, I'd really rather _not_ add lots more that people depend on from mayflower. Having two different codebases to handle the initramfs (mayflower + mkinitrd) is incredibly painful for having to adapt both for changes in the OS, etc.
I agree that it would be cool if mayflower got absorbed into mkinitrd (or vice versa). But that seems a litte far off right now, wheras the changes I presented were all very small, and only served to structure mayflower in a better way, which I imagine would make it's absorbtion into mkinitrd easier, not harder. I.e.
#1 - this can be used to replace all that ugly cp /bin/stuff bin/ with an elegant FILES= syntax in the existing mayflower.conf
#2 - as you agreed, just makes sense to not hard-code /etc/mayflower.conf
#3 - consider how my reply (--program) would be implemented, i.e. by creating scriptlets in variables that get conditionally expanded in the inline init creation. This sort of method of modularizing what gets put in the init (think the OLPC specific parts, and everything else), makes it much more elegant.
And given that overall mkinitrd "does more", I think it's going to be better to get the live image specific stuff integrated into mkinitrd rather than spending a lot of time making mayflower more flexible for the sake of flexibility.
But to comment on the concepts...
#1 - mayflower.conf - add PROGRAMS and FILES
[snip]
FILES+= would differ from PROGRAMS, in that the auto shared library dependency check would not apply to them. Maybe that doesn't matter, and you could just have FILES.
Having two different lists is just asking for people to pick the wrong one. Since I can't think of a real reason that you'd want a binary without its deps, just lump them together.
Agreed.
#2 - support user specified mayflower.conf location (i.e. not just /etc/mayflower.conf)
Not terribly against or for this. Having the ability to specify config locations is often useful enough for writing test cases that it's worthwhile
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
Why not just use init= ? Other than the fact that doing so is currently broken with mayflower (... see above comments about two implementations of the same thing :-)
The reason is that I very much like all the mayflower generated init stuff that happens up to the point of the existing shell escape. I.e. setting up an environment where bash can run and I can see my devices. By specifying my own init, I'd basically be duplicating the mayflower init generation code, except with my program= escape point, at which point... hey, I'm back where this proposal started.
-dmc
On Mon, 2007-08-13 at 12:03 -0500, Douglas McClendon wrote:
Jeremy Katz wrote:
On Sun, 2007-08-12 at 23:13 -0500, Douglas McClendon wrote:
I have a few changes I'd like to request to mayflower. I'd like to get some feedback before actually laying down the effort of producing patches.
So from a big high-level point of view, I'd really rather _not_ add lots more that people depend on from mayflower. Having two different codebases to handle the initramfs (mayflower + mkinitrd) is incredibly painful for having to adapt both for changes in the OS, etc.
I agree that it would be cool if mayflower got absorbed into mkinitrd (or vice versa). But that seems a litte far off right now, wheras the changes I presented were all very small, and only served to structure mayflower in a better way, which I imagine would make it's absorbtion into mkinitrd easier, not harder. I.e.
It's actually not that much work. The first step (switching to using bash for the initramfs so that you don't have to implement conditionals in nash) is actually already done and working -- see the bash-branch of mkinitrd. I just haven't had time to then get the livecd bits using that. I had hoped to for F8, but time is rapidly running out and that's going to be lower on my "caring" list.
And note, I'm not really against the changes, I just breathe a heavy sigh every time something else is done in mayflower rather than spending the time on fixing the whole "two initrd" infrastructures thing :/
#3 - optional program, sort of like existing shell cmdline arg
Have a cmdline argument of program= and eprogram= which would cause the specified program to be executed. program= would happen right after the current shell, and eprogram would happen right after the current eshell.
Why not just use init= ? Other than the fact that doing so is currently broken with mayflower (... see above comments about two implementations of the same thing :-)
The reason is that I very much like all the mayflower generated init stuff that happens up to the point of the existing shell escape. I.e. setting up an environment where bash can run and I can see my devices. By specifying my own init, I'd basically be duplicating the mayflower init generation code, except with my program= escape point, at which point... hey, I'm back where this proposal started.
init= doesn't replace the initramfs's init -- it's what is supposed to be run after you've got the real system. eg, init=/bin/bash to just get dropped to a shell rather than running /sbin/init. But yeah, after a quick try, something about it is busted with mayflower. Try it on a "normal" system, though, to get the effect that's intended
Jeremy
Jeremy Katz wrote:
The reason is that I very much like all the mayflower generated init stuff that happens up to the point of the existing shell escape. I.e. setting up an environment where bash can run and I can see my devices. By specifying my own init, I'd basically be duplicating the mayflower init generation code, except with my program= escape point, at which point... hey, I'm back where this proposal started.
init= doesn't replace the initramfs's init -- it's what is supposed to be run after you've got the real system. eg, init=/bin/bash to just get dropped to a shell rather than running /sbin/init. But yeah, after a quick try, something about it is busted with mayflower. Try it on a "normal" system, though, to get the effect that's intended
Assuming mayflower gets fixed, would there be any issue with the fact that in my usage scenarios, there is no real root filesystem, and that the init I'm trying to specify is a program in the initramfs?
-dmc
livecd@lists.fedoraproject.org
-
ashok shankar das -
Douglas McClendon -
Jeremy Katz -
Tim Wood