Hi,
Today I was running through runs of livemedia-creator in mock to figure out what exactly we need to get koji to run in order to produce the liveiso's I got things running and was able to produce a live iso. aside from things like the iso being ~1G bigger than what we get from livecd-creator I had one big surprise. The surprise was the output that we got. In Release engineering we want the tooling to make the artifact as we are going to ship it. This means that someone running the tool at home can make the same thing as what we make. I was expecting for output something along the lines of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd- creator. Instead of just an iso what I got was a tree
EFI EFI/BOOT EFI/BOOT/BOOTX64.EFI EFI/BOOT/MokManager.efi EFI/BOOT/grubx64.efi EFI/BOOT/grub.cfg EFI/BOOT/fonts EFI/BOOT/fonts/unicode.pf2 images images/pxeboot images/pxeboot/vmlinuz images/pxeboot/initrd.img images/install.img images/efiboot.img images/macboot.img images/product.img images/boot.iso isolinux isolinux/vmlinuz isolinux/initrd.img isolinux/isolinux.bin isolinux/vesamenu.c32 isolinux/ldlinux.c32 isolinux/libcom32.c32 isolinux/libutil.c32 isolinux/isolinux.cfg isolinux/grub.conf isolinux/boot.msg isolinux/splash.png isolinux/memtest LiveOS LiveOS/squashfs.img
where the only thing that we will ship is images/boot.iso but we will have to do manipulation of the tree to keep what we want and name it correctly and discard all the extra bits. this means that every use that makes livecds will need to write tooling to extract the boot.iso, rename it and discard the extra bits.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
I am happy to write the code but want to make sure that it will be welcome before I go off and do so. This is a feature we are trying to get into the compose tools and process for Fedora 24 so that we can get rid of livecd- creator and appliance-creator once and for all
Regards
Dennis
Forgot to cc acarter on the original email
On Tuesday, December 15, 2015 06:13:44 PM Dennis Gilmore wrote:
Hi,
Today I was running through runs of livemedia-creator in mock to figure out what exactly we need to get koji to run in order to produce the liveiso's I got things running and was able to produce a live iso. aside from things like the iso being ~1G bigger than what we get from livecd-creator I had one big surprise. The surprise was the output that we got. In Release engineering we want the tooling to make the artifact as we are going to ship it. This means that someone running the tool at home can make the same thing as what we make. I was expecting for output something along the lines of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd- creator. Instead of just an iso what I got was a tree
EFI EFI/BOOT EFI/BOOT/BOOTX64.EFI EFI/BOOT/MokManager.efi EFI/BOOT/grubx64.efi EFI/BOOT/grub.cfg EFI/BOOT/fonts EFI/BOOT/fonts/unicode.pf2 images images/pxeboot images/pxeboot/vmlinuz images/pxeboot/initrd.img images/install.img images/efiboot.img images/macboot.img images/product.img images/boot.iso isolinux isolinux/vmlinuz isolinux/initrd.img isolinux/isolinux.bin isolinux/vesamenu.c32 isolinux/ldlinux.c32 isolinux/libcom32.c32 isolinux/libutil.c32 isolinux/isolinux.cfg isolinux/grub.conf isolinux/boot.msg isolinux/splash.png isolinux/memtest LiveOS LiveOS/squashfs.img
where the only thing that we will ship is images/boot.iso but we will have to do manipulation of the tree to keep what we want and name it correctly and discard all the extra bits. this means that every use that makes livecds will need to write tooling to extract the boot.iso, rename it and discard the extra bits.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
I am happy to write the code but want to make sure that it will be welcome before I go off and do so. This is a feature we are trying to get into the compose tools and process for Fedora 24 so that we can get rid of livecd- creator and appliance-creator once and for all
Regards
Dennis
На 16.12.2015 в 04:11, Dennis Gilmore написа:
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
Hi Dennis, FYI if you add this, please make it an option so that existing livemedia-creator behavior isn't affected (where possible). Just recently QE has developed some tests which use not only the boot.iso but other bits from the generated tree.
-- Alex
On Wed, 2015-12-16 at 10:34 +0200, Alexander Todorov wrote:
На 16.12.2015 в 04:11, Dennis Gilmore написа:
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
Hi Dennis, FYI if you add this, please make it an option so that existing livemedia-creator behavior isn't affected (where possible). Just recently QE has developed some tests which use not only the boot.iso but other bits from the generated tree.
Hmm, well that answers one of my questions. ;) What tests are these?
На 16.12.2015 в 19:54, Adam Williamson написа:
On Wed, 2015-12-16 at 10:34 +0200, Alexander Todorov wrote:
На 16.12.2015 в 04:11, Dennis Gilmore написа:
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
Hi Dennis, FYI if you add this, please make it an option so that existing livemedia-creator behavior isn't affected (where possible). Just recently QE has developed some tests which use not only the boot.iso but other bits from the generated tree.
Hmm, well that answers one of my questions. ;) What tests are these?
Using lmc create a Live ISO or a disk image. Then use virt-install to boot the ISO/image. Once the VM is running ssh into it and make sure it's operational.
I can provide you with more details in private if you need them.
-- Alex
On Thu, 2015-12-17 at 16:42 +0200, Alexander Todorov wrote:
На 16.12.2015 в 19:54, Adam Williamson написа:
On Wed, 2015-12-16 at 10:34 +0200, Alexander Todorov wrote:
На 16.12.2015 в 04:11, Dennis Gilmore написа:
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
Hi Dennis, FYI if you add this, please make it an option so that existing livemedia-creator behavior isn't affected (where possible). Just recently QE has developed some tests which use not only the boot.iso but other bits from the generated tree.
Hmm, well that answers one of my questions. ;) What tests are these?
Using lmc create a Live ISO or a disk image. Then use virt-install to boot the ISO/image. Once the VM is running ssh into it and make sure it's operational.
But what other bits from the generated tree would that use?
На 17.12.2015 в 19:16, Adam Williamson написа:
On Thu, 2015-12-17 at 16:42 +0200, Alexander Todorov wrote:
На 16.12.2015 в 19:54, Adam Williamson написа:
On Wed, 2015-12-16 at 10:34 +0200, Alexander Todorov wrote:
На 16.12.2015 в 04:11, Dennis Gilmore написа:
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
Hi Dennis, FYI if you add this, please make it an option so that existing livemedia-creator behavior isn't affected (where possible). Just recently QE has developed some tests which use not only the boot.iso but other bits from the generated tree.
Hmm, well that answers one of my questions. ;) What tests are these?
Using lmc create a Live ISO or a disk image. Then use virt-install to boot the ISO/image. Once the VM is running ssh into it and make sure it's operational.
But what other bits from the generated tree would that use?
install.img
On Tue, Dec 15, 2015 at 06:13:44PM -0600, Dennis Gilmore wrote:
Hi,
Today I was running through runs of livemedia-creator in mock to figure out what exactly we need to get koji to run in order to produce the liveiso's I got things running and was able to produce a live iso. aside from things like the iso being ~1G bigger than what we get from livecd-creator I had one big
You can probably thank DNF for that. It's idea of the right packages is different from yums.
surprise. The surprise was the output that we got. In Release engineering we want the tooling to make the artifact as we are going to ship it. This means that someone running the tool at home can make the same thing as what we make. I was expecting for output something along the lines of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd- creator. Instead of just an iso what I got was a tree
Remember that this is not livecd-creator, and it was not designed to give you the same output.
where the only thing that we will ship is images/boot.iso but we will have to do manipulation of the tree to keep what we want and name it correctly and discard all the extra bits. this means that every use that makes livecds will need to write tooling to extract the boot.iso, rename it and discard the extra bits.
Don't you already have to do that with the new lmc koji code to grab the output from the run inside mock? I really don't see the need to add yet this to lmc when your code can easily move the boot.iso to the final location/name and remove the intermediate directory.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
lmc uses lorax to create the iso output, which it why it looks very much like a boot.iso tree. I want to minimize differences from lorax as much as possible.
I am happy to write the code but want to make sure that it will be welcome before I go off and do so. This is a feature we are trying to get into the compose tools and process for Fedora 24 so that we can get rid of livecd- creator and appliance-creator once and for all
Thanks for doing this, in this case I think it belongs in the new koji code, not lmc.
On Wed, 2015-12-16 at 09:23 -0800, Brian C. Lane wrote:
surprise. The surprise was the output that we got. In Release engineering we want the tooling to make the artifact as we are going to ship it. This means that someone running the tool at home can make the same thing as what we make. I was expecting for output something along the lines of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd- creator. Instead of just an iso what I got was a tree
Remember that this is not livecd-creator, and it was not designed to give you the same output.
Well, yes, but also, it's supposed to be livecd-creator's replacement, and it's supposed to perform the same role. Surely one of its #1 goals should be "to produce what release engineering wants it to produce", since that's probably its most critical role. If releng is telling you its output is not what they want, that seems like something to consider.
where the only thing that we will ship is images/boot.iso but we will have to do manipulation of the tree to keep what we want and name it correctly and discard all the extra bits. this means that every use that makes livecds will need to write tooling to extract the boot.iso, rename it and discard the extra bits.
Don't you already have to do that with the new lmc koji code to grab the output from the run inside mock? I really don't see the need to add yet this to lmc when your code can easily move the boot.iso to the final location/name and remove the intermediate directory.
You didn't provide any response to dgilmore's objection to this design: that anyone else using lmc would have to recreate that work. Obviously not everyone who wants to build a live image for testing has a Koji instance they can run it through.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
lmc uses lorax to create the iso output, which it why it looks very much like a boot.iso tree. I want to minimize differences from lorax as much as possible.
But presumably you also want a tool that produces what its users actually want. As someone who uses livecd-creator regularly, I want what dgilmore wants: a live ISO file. I don't want all that other stuff. Naming is a bit trickier, but it certainly seems reasonable to at least provide the ability to choose the output file name with a parameter (as livecd-creator did).
Perhaps a useful question would be: is there anyone who *would* want all that stuff, when building a live image? And if not, what reason is there to provide it?
On Wed, Dec 16, 2015 at 09:53:25AM -0800, Adam Williamson wrote:
Perhaps a useful question would be: is there anyone who *would* want all that stuff, when building a live image? And if not, what reason is there to provide it?
There are, and they do. Also remember that lmc has been in use by others for quite a while now and that you and Fedora releng are not the only users.
On Wednesday, December 16, 2015 10:32:32 AM Brian C. Lane wrote:
On Wed, Dec 16, 2015 at 09:53:25AM -0800, Adam Williamson wrote:
Perhaps a useful question would be: is there anyone who *would* want all that stuff, when building a live image? And if not, what reason is there to provide it?
There are, and they do. Also remember that lmc has been in use by others for quite a while now and that you and Fedora releng are not the only users.
What are the use cases for the extra things? and are they something we want to support?
Dennis
On Wednesday, December 16, 2015 09:53:25 AM Adam Williamson wrote:
On Wed, 2015-12-16 at 09:23 -0800, Brian C. Lane wrote:
surprise. The surprise was the output that we got. In Release engineering we want the tooling to make the artifact as we are going to ship it. This means that someone running the tool at home can make the same thing as what we make. I was expecting for output something along the lines of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd- creator. Instead of just an iso what I got was a tree
Remember that this is not livecd-creator, and it was not designed to give you the same output.
Well, yes, but also, it's supposed to be livecd-creator's replacement, and it's supposed to perform the same role. Surely one of its #1 goals should be "to produce what release engineering wants it to produce", since that's probably its most critical role. If releng is telling you its output is not what they want, that seems like something to consider.
anyone at home making livecds's or people doing remixes will be expecting the same output as livecd-creator
where the only thing that we will ship is images/boot.iso but we will have to do manipulation of the tree to keep what we want and name it correctly and discard all the extra bits. this means that every use that makes livecds will need to write tooling to extract the boot.iso, rename it and discard the extra bits.
Don't you already have to do that with the new lmc koji code to grab the output from the run inside mock? I really don't see the need to add yet this to lmc when your code can easily move the boot.iso to the final location/name and remove the intermediate directory.
There are many cases where people make livecds without koji. each one of them needs to write something?
You didn't provide any response to dgilmore's objection to this design: that anyone else using lmc would have to recreate that work. Obviously not everyone who wants to build a live image for testing has a Koji instance they can run it through.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as an example of how we ship live media it is not a great thing to have to do. I strongly feel that a small piece of additional code in livemedia-creator to correctly name the iso given that there is a --image-name flag is reasonable and will save every person that makes livemedia from having to rewrite basically the same function, or us just writing a wrapper around livemedia- creator to do so.
lmc uses lorax to create the iso output, which it why it looks very much like a boot.iso tree. I want to minimize differences from lorax as much as possible.
But presumably you also want a tool that produces what its users actually want. As someone who uses livecd-creator regularly, I want what dgilmore wants: a live ISO file. I don't want all that other stuff. Naming is a bit trickier, but it certainly seems reasonable to at least provide the ability to choose the output file name with a parameter (as livecd-creator did).
anyone that respins fedora lives's either adding updates, or making a remix adding things we can not ship is going to want what I am asking for and will not have koji and will have to re-implement something to get the output needed. There are many use cases outside of releng that wants to make a livecd and just a livecd.
There is nothing wrong with code reuse. I do not see how we can ship what is produced even if we decide that some of the extra stuff is useful. we can not ship 20 isos called boot.iso. people regularly grab isos from different parts of the tree and sync them to a common directory.
Dennis
On Wed, Dec 16, 2015, at 02:13 PM, Dennis Gilmore wrote:
anyone at home making livecds's or people doing remixes will be expecting the same output as livecd-creator
I'd say anyone using all of this tooling is going to have to be ready to adapt to some degree.
There are many cases where people make livecds without koji. each one of them needs to write something?
FWIW rpm-ostree-toolbox presently also wraps livemedia-creator and does do output postprocessing - I didn't find it really arduous to maintain.
But it'd also seem sane to me to have a --output-exactly-like-livecd-creator option in lmc.
Here's the PR to add a --iso-only option to livemedia-creator
https://github.com/rhinstaller/lorax/pull/80
I ended up not touching the templates at all, it would have made things much messier. The change is pretty self-contained in the make-iso section of the code and only depends on the output from the templates having images/boot.iso present.
You can also pass --resultdir to control where it gets placed and --image-name to rename the boot.iso
On Thursday, January 07, 2016 10:25:37 AM Brian C. Lane wrote:
Here's the PR to add a --iso-only option to livemedia-creator
https://github.com/rhinstaller/lorax/pull/80
I ended up not touching the templates at all, it would have made things much messier. The change is pretty self-contained in the make-iso section of the code and only depends on the output from the templates having images/boot.iso present.
You can also pass --resultdir to control where it gets placed and --image-name to rename the boot.iso
Thanks Brian,
I have tested the patches and the work and give us what we need.
Dennis
anaconda-devel@lists.fedoraproject.org
-
Adam Williamson -
Alexander Todorov -
Brian C. Lane -
Colin Walters -
Dennis Gilmore