1:53 p.m.
This is the first in a bunch of OLPC needs I will be writing about on
the list. It is super simple for me to hack up solutions that work for
us but I would like to find a common solution that benefits all Fedora
based projects.
One of the things we need to for OLPC from the LiveCD creator is the
ability to output configuration files from a template. We do this
currently in Pilgrim and can do this in the LiveCD tools via the %post
as shell scripts. This is pretty unwieldy and tends to be error prone
with typos and missing character escapes causing sometimes hard to debug
problems.
A simple idea is to have standard template mechanism such as using
pythons string substitutions. For example take the file /etc/issue. In
shell we do this:
cat <<EOF > $INSTALL_ROOT/etc/issue
OLPC build $BUILD (stream $STREAM; variant $VARIANT)
Kernel \r on an \m
EOF
It would be nice to have a template file which read something like:
OLPC build %(build) (stream %(stream); variant %(variant))
Kernel \r on an \m
and simply passed in a hash with substitutions from the livecd tools.
Python templates or whatever templates is not really the issue here.
What is the issue is defining a standard template format to use,
defining how the tool finds the templates and defining how substitutions
are specified.
Any suggestions? I am willing to do the code myself but in the lack of
any interest I'm going to end up creating a one off solution, so mucho
input is desired.
--
John (J5) Palmieri <johnp(a)redhat.com>
11:58 a.m.
Hi,
On Tue, 2007-04-24 at 14:53 -0400, John (J5) Palmieri wrote:
Any suggestions? I am willing to do the code myself but in the lack
of any interest I'm going to end up creating a one off solution, so
mucho input is desired.
I think munging configuration files might be outside the scope of the
livecd-tools project; sure, granted it's something people want to do but
often it's used as a hack; for example on my system
$ rpm -qf /etc/issue
fedora-release-6.93-1
so I think what you *could* do instead is to have olpc-release or
similar on OLPC provide that file (would be based on fedora-release). Of
course, since /etc/issue is marked as %config(noreplace) in the
fedora-release it's still sane for you to edit it in %post. I guess it
all depends on the configuration file in question.
So I think my point of view right now is that it's good enough to do
this in %post for most cases; I mean, if downstreams (like OLPC) need to
change a ton of configuration files/defaults perhaps there is something
broken with the apps you try to include?
Perhaps you can come up with concrete examples of what configuration
files / defaults you need to edit /change? I think that would be useful.
Thanks.
David
3:51 p.m.
On Wed, 2007-04-25 at 12:58 -0400, David Zeuthen wrote:
Hi,
On Tue, 2007-04-24 at 14:53 -0400, John (J5) Palmieri wrote:
> Any suggestions? I am willing to do the code myself but in the lack
> of any interest I'm going to end up creating a one off solution, so
> mucho input is desired.
I think munging configuration files might be outside the scope of the
livecd-tools project; sure, granted it's something people want to do but
often it's used as a hack; for example on my system
$ rpm -qf /etc/issue
fedora-release-6.93-1
And I'm going to package that up for every daily release? And what
about countries? Too heavy and engineered for an image based
distribution.
so I think what you *could* do instead is to have olpc-release or
similar on OLPC provide that file (would be based on fedora-release). Of
course, since /etc/issue is marked as %config(noreplace) in the
fedora-release it's still sane for you to edit it in %post. I guess it
all depends on the configuration file in question.
So I think my point of view right now is that it's good enough to do
this in %post for most cases; I mean, if downstreams (like OLPC) need to
change a ton of configuration files/defaults perhaps there is something
broken with the apps you try to include?
No just that apps can be configured for different purposes. For
instance gedit can be made into a development environment but requires
me to write out a bunch of gconf files. We have Sabyon for a reason.
It basically does what I want to do in the case of gconf and some other
settings. It is a dump of configuration files.
Perhaps you can come up with concrete examples of what configuration
files / defaults you need to edit /change? I think that would be useful.
Thanks.
/etc/sysconfig/modules/olpc-1.modules for the module lists to load
/etc/rwtab for stateless tmpfs mounted stuff
/etc/asound.state to put the correct default sound levels
/etc/modprobe.d/olpc.conf.dist so we get the correct module parameters
/boot/olpc_build for build info that the bootloader uses
/etc/issue for build info the user can read
/etc/securetty for adding the serial console
/etc/inittab for removing unneeded memory waisting VT's
/usr/share/fonts/dejavu-lgc/fonts.dir for mapping legacy fonts
/usr/share/fonts/dejavu-lgc/fonts.scale same as above
/usr/share/fonts/dejavu-lgc/fonts.alias same
/etc/X11/xorg.conf and its variants for booting on any device and
getting the correct xorg config file with the correct keyboard laysouts
/home/olpc/.xinitrc for launching sugar
/boot/grub/grub.conf for the occasional qemu boot
/etc/init.d/olpc-configure init script for boot time configuration
/usr/share/X11/xkb/symbols/* for keyboard configuration that is not yet
upstream (note that these must be added to an already existing file)
/boot/olpc.fth - for booting from openfirmware
Note that we can't move a lot of these to an RPM because RPM conflict
rules would prevent them from being installed. Some files need input
such as BUILD number and some files change so rapidly during the testing
phase that taking the time to build an rpm and place it in the right
spot and rebuild the repo would be a unneeded bottleneck.
--
John (J5) Palmieri <johnp(a)redhat.com>
8:44 p.m.
On Thu, 2007-04-26 at 16:51 -0400, John (J5) Palmieri wrote:
> I think munging configuration files might be outside the scope
of the
> livecd-tools project; sure, granted it's something people want to do but
> often it's used as a hack; for example on my system
>
> $ rpm -qf /etc/issue
> fedora-release-6.93-1
And I'm going to package that up for every daily release? And what
about countries? Too heavy and engineered for an image based
distribution.
[...]
> so I think what you *could* do instead is to have olpc-release
or
> similar on OLPC provide that file (would be based on fedora-release). Of
> course, since /etc/issue is marked as %config(noreplace) in the
> fedora-release it's still sane for you to edit it in %post. I guess it
> all depends on the configuration file in question.
I think I said that both approaches works. Just make sure all files
being changed are tagged with %config(noreplace) if you care about RPM
upgrade paths (which you probably don't need to for OLPC but it's nice
nonetheless if someone wants to pick this up for hardware that can cope
with yum/rpm etc.).
> So I think my point of view right now is that it's good
enough to do
> this in %post for most cases; I mean, if downstreams (like OLPC) need to
> change a ton of configuration files/defaults perhaps there is something
> broken with the apps you try to include?
No just that apps can be configured for different purposes. For
instance gedit can be made into a development environment but requires
me to write out a bunch of gconf files. We have Sabyon for a reason.
It basically does what I want to do in the case of gconf and some other
settings. It is a dump of configuration files.
Sure, or you can use gconftool-2 or Puppet as Mark mentioned or
basically anything that gets the job done. My point really was that I
don't think configuration file munging belongs in livecd-tools proper.
Do you think that's wrong?
> Perhaps you can come up with concrete examples of what
configuration
> files / defaults you need to edit /change? I think that would be useful.
> Thanks.
/etc/sysconfig/modules/olpc-1.modules for the module lists to load
/etc/rwtab for stateless tmpfs mounted stuff
/etc/asound.state to put the correct default sound levels
/etc/modprobe.d/olpc.conf.dist so we get the correct module parameters
/boot/olpc_build for build info that the bootloader uses
/etc/issue for build info the user can read
/etc/securetty for adding the serial console
/etc/inittab for removing unneeded memory waisting VT's
/usr/share/fonts/dejavu-lgc/fonts.dir for mapping legacy fonts
/usr/share/fonts/dejavu-lgc/fonts.scale same as above
/usr/share/fonts/dejavu-lgc/fonts.alias same
/etc/X11/xorg.conf and its variants for booting on any device and
getting the correct xorg config file with the correct keyboard laysouts
/home/olpc/.xinitrc for launching sugar
/boot/grub/grub.conf for the occasional qemu boot
/etc/init.d/olpc-configure init script for boot time configuration
/usr/share/X11/xkb/symbols/* for keyboard configuration that is not yet
upstream (note that these must be added to an already existing file)
/boot/olpc.fth - for booting from openfirmware
Note that we can't move a lot of these to an RPM because RPM conflict
rules would prevent them from being installed. Some files need input
such as BUILD number and some files change so rapidly during the testing
phase that taking the time to build an rpm and place it in the right
spot and rebuild the repo would be a unneeded bottleneck.
So perhaps one maintainable way for you is to have a single script
(packaged up in a OLPC specific RPM) that does all the munging and your
%post simple invokes this? That script would know about per-country
things and what kind of "image" (e.g. jffs2, livecd, disk_ext3 etc.)
you're building by having the %post pass it as options? Wouldn't that
work?
Of course, the question I raised in an earlier mail of how we pass items
to the %post script in the Kickstart file still stands. Jeremy? Perhaps
just setting environment variables works out of the box. I'd be
surprised if this is not the case.
Also, external variables used by higher level constructs that does batch
builds in a structured way (e.g. pilgrim-autobuild) needs to pass
(OLPC-specific) variables too such as BUILD_NUMBER, TARGET_COUNTRY etc.
to %post. I think the way I'd like this to work is that such variables
are transparently passed by livecd-tools and then the %post takes care
of it. Perhaps it would look like this
%post
olpc-configure-system --image-type $LIVECD_TOOLS_IMAGE_TYPE \
--olpc-build-number $OLPC_IMAGE_NUMBER \
--olpc-target $OLPC_TARGET_COUNTRY
Would that work for you?
David
10:59 a.m.
New subject: SSH defualt on livecd bootup
I noticed I can't ssh telnet into a livecd - I'm guessing this was
disabled for security reasons.
can someone tell me why exactly?
I would have loved to boot an old box with f7-test4 i386 and telneted
over, configured vncserver to allow me to control it from remote. then
install it from the cd to the harddrive and reboot it without hooking a
monitor up to it.
12:27 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 11:59 -0400, Steve Berry wrote:
I noticed I can't ssh telnet into a livecd - I'm guessing
this was
disabled for security reasons.
can someone tell me why exactly?
I would have loved to boot an old box with f7-test4 i386 and telneted
over, configured vncserver to allow me to control it from remote. then
install it from the cd to the harddrive and reboot it without hooking a
monitor up to it.
Most likely the root password is not set which automatically disables
ssh for security reasons.
--
John (J5) Palmieri <johnp(a)redhat.com>
1:37 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 11:59 -0400, Steve Berry wrote:
I noticed I can't ssh telnet into a livecd - I'm guessing
this was
disabled for security reasons.
can someone tell me why exactly?
I would have loved to boot an old box with f7-test4 i386 and telneted
over, configured vncserver to allow me to control it from remote. then
install it from the cd to the harddrive and reboot it without hooking a
monitor up to it.
Mostly that ssh server key generation takes ages thus slowing down the
already slow boot process. Also, the passwords are all blank.
David
2 p.m.
New subject: SSH defualt on livecd bootup
David Zeuthen wrote:
Mostly that ssh server key generation takes ages thus slowing down the
already slow boot process. Also, the passwords are all blank.
David
I think the slow boot process is due to the Ext3 filesystem you have
decided to use.
Also, the SSH key generation doesn't take more than 6 seconds on a dual
PII@300MHz, less than 2
seconds on a Intel Celeron at 2.6GHz. That isn't exactly "ages". I would
contribute it to
the filesystem you have decided to use.
J. Hartline
2:16 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 14:00 -0500, Jasper Hartline wrote:
David Zeuthen wrote:
>
> Mostly that ssh server key generation takes ages thus slowing down the
> already slow boot process. Also, the passwords are all blank.
>
> David
>
I think the slow boot process is due to the Ext3 filesystem you have
decided to use.
I hate to repeat myself, but here we go. Using ext3 is a very conscious
decision:
1) it support very quick live cd installs; just dd the data instead of
doing an RPM transaction that requires you to carry at least twice
the amount of data.
2) ext3 actually supports xattrs so SELinux just works out of the box.
This is pretty important from a "let's market Fedora" point of view.
3) it's the standard file system used by Fedora
Also, the SSH key generation doesn't take more than 6 seconds on
a dual
PII@300MHz, less than 2
seconds on a Intel Celeron at 2.6GHz. That isn't exactly "ages".
I seem to remember it's a lot more but whatever. Even if this was fast
it wouldn't make sense for generic live CD's to include this feature as
the passwords would still be empty. Custom live cd's is another matter
of course.
Btw, the key generation process is CPU-bound so it's a bit hard for me
to understand why you blame the file system since that's mostly IO
performance. I also don't think it's useful for you to make
unsubstantiated claims about ext3's performance or the lack of. Do you
numbers to back your statements up? Because if you have I'd love to see
them so I can pass them on to our ext3 developers. Thanks!
David
2:32 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 15:16 -0400, David Zeuthen wrote:
I seem to remember it's a lot more but whatever. Even if this was
fast
it wouldn't make sense for generic live CD's to include this feature as
the passwords would still be empty. Custom live cd's is another matter
of course.
And as it currently stands, it's easy to set the root pass with
livecd-creator and _not_ disable sshd if that's what you want to do.
Jeremy
2:41 p.m.
New subject: SSH defualt on livecd bootup
Jeremy Katz wrote:
On Fri, 2007-04-27 at 15:16 -0400, David Zeuthen wrote:
> I seem to remember it's a lot more but whatever. Even if this was fast
> it wouldn't make sense for generic live CD's to include this feature as
> the passwords would still be empty. Custom live cd's is another matter
> of course.
>
And as it currently stands, it's easy to set the root pass with
livecd-creator and _not_ disable sshd if that's what you want to do.
Absolutely.
This is the value of having the tools which are used to master a LiveCD
available to the public as Open Source Software.
2:40 p.m.
New subject: SSH defualt on livecd bootup
David Zeuthen wrote:
On Fri, 2007-04-27 at 14:00 -0500, Jasper Hartline wrote:
> David Zeuthen wrote:
>
>> Mostly that ssh server key generation takes ages thus slowing down the
>> already slow boot process. Also, the passwords are all blank.
>>
>> David
>>
>>
> I think the slow boot process is due to the Ext3 filesystem you have
> decided to use.
>
I hate to repeat myself, but here we go. Using ext3 is a very conscious
decision:
I'm not arguing why you chose Ext3, I'm saying the slow boot process is
probably due
to Ext3.
> Btw, the key generation process is CPU-bound so it's a bit
hard for me
> to understand why you blame the file system since that's mostly IO
> performance. I also don't think it's useful for you to make
> unsubstantiated claims about ext3's performance or the lack of. Do you
> numbers to back your statements up? Because if you have I'd love to see
> them so I can pass them on to our ext3 developers. Thanks!
>
>
I never said the SSH key generation was due to Ext3, I said the boot
process you describe is probably due to Ext3.
Learn how to read, when you do come back with a valid retort.
J. Hartline
2:57 p.m.
New subject: SSH defualt on livecd bootup
On 4/27/07, Jasper Hartline <jasperhartline(a)adelphia.net> wrote:
Learn how to read, when you do come back with a valid retort.
Jasper, that was not necessary. :(
Regards,
--
Thomas Chung
http://fedoraproject.org/wiki/ThomasChung
3:08 p.m.
New subject: SSH defualt on livecd bootup
Thomas Chung wrote:
On 4/27/07, Jasper Hartline <jasperhartline(a)adelphia.net>
wrote:
> Learn how to read, when you do come back with a valid retort.
Jasper, that was not necessary. :(
Regards,
Niether was his assumption I was talking about the SSH key generation
being slow due to Ext3.
Nowhere in my statement did I say "The SSH key generation is slow due to
Ext3".
He also said my claims were unsubstantiated, again going on an
assumption of what I said which I never had.
It's plain to see he is not a native English speaker.
J. Hartline
3:33 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 15:08 -0500, Jasper Hartline wrote:
He also said my claims were unsubstantiated, again going on an
assumption of what I said which I never had.
Not at all. I asked you to clarify "I think the slow boot process is due
to the Ext3 filesystem you have decided to use." because I don't think
you really know what you're talking about. I'm still waiting for that
clarification.
It's plain to see he is not a native English speaker.
Would you rather I replied in Danish? Would that make you feel better?
Anyway, Jasper, I know enough English to know that it's rude to talk
about people present in the third person. I'm here. I've got a name.
Refer to me as David, not as "he". Thanks.
Your mail and whole line or arguing just makes me sad. It's so obvious,
even to non-native English speakers such as myself, that you're just
trolling obviously because I'm the person that ate your lunch by making
Kadishci mostly irrelevant with pilgrim/livecd-tools. Well sorry about
that Jasper. But that doesn't really justify you starting a witch-hunt
against non-native English speakers.
David
3:46 p.m.
New subject: SSH defualt on livecd bootup
David Zeuthen wrote:
On Fri, 2007-04-27 at 15:08 -0500, Jasper Hartline wrote:
> He also said my claims were unsubstantiated, again going on an
> assumption of what I said which I never had.
>
Not at all. I asked you to clarify "I think the slow boot process is due
to the Ext3 filesystem you have decided to use." because I don't think
you really know what you're talking about. I'm still waiting for that
clarification.
SquashFS is much faster than the uncompressed Ext3 which Pilgrim and
livecd-tools uses.
> It's plain to see he is not a native English speaker.
>
Would you rather I replied in Danish? Would that make you feel better?
Anyway, Jasper, I know enough English to know that it's rude to talk
about people present in the third person. I'm here. I've got a name.
Refer to me as David, not as "he". Thanks.
I was replying to someone who was commenting on my comment to you, I
wasn't talking to you.
Again, you are far from knowing enough English to know this obviously.
Your mail and whole line or arguing just makes me sad. It's so
obvious,
even to non-native English speakers such as myself, that you're just
trolling obviously because I'm the person that ate your lunch by making
Kadishci mostly irrelevant with pilgrim/livecd-tools. Well sorry about
that Jasper. But that doesn't really justify you starting a witch-hunt
against non-native English speakers.
You were the one who made the assumption I was talking about SSH keys
generation being slow due to Ext3.
Let me clarify:
You state starting SSHD would not be good for the already slow boot process.
I stated, the boot process is probably slow due to Ext3.
I also added that SSH key generation on a dual PII@300MHz takes a mere 6
seconds or less
and that on an Intel Celeron at 2.6GHz took maybe 2 seconds.
Now you're going to try to sit here and cover up the fact you are far
from an English major, and that you did
not make a mistake assuming what you did, and try to sidetrack your
flaws in personality claiming I'm on a witch hunt
because of some software?
Explain.
J. Hartline
4:44 p.m.
New subject: SSH defualt on livecd bootup
David Zeuthen wrote:
On Fri, 2007-04-27 at 15:46 -0500, Jasper Hartline wrote:
> Explain.
>
Jasper, for det første skylder jeg ikke dig nogen forklaring. For det
andet har jeg bedre ting at bruge min tid på. Held og lykke, du har
_virkelig_ brug for det.
You sure felt like explaining why you chose Ext3, when I was not
debating WHY you chose
Ext3, but merely suggested it is probably what makes the boot process slow.
So are you just confused, or are you seriously going to try to win such
an argument
stating opinions as facts you cannot prove?
J. Hartline
4:54 p.m.
New subject: SSH defualt on livecd bootup
Jasper Hartline wrote:
David Zeuthen wrote:
> On Fri, 2007-04-27 at 15:46 -0500, Jasper Hartline wrote:
>
>> Explain.
>>
>
> Jasper, for det første skylder jeg ikke dig nogen forklaring. For det
> andet har jeg bedre ting at bruge min tid på. Held og lykke, du har
> _virkelig_ brug for det.
>
You sure felt like explaining why you chose Ext3, when I was not
debating WHY you chose
Ext3, but merely suggested it is probably what makes the boot process slow.
So are you just confused, or are you seriously going to try to win such
an argument
stating opinions as facts you cannot prove?
Jasper, your original message was confusing to me too. Your comparison
on SSH key generation times vs cpu speed atleast implied that you
thought the process was cpu bound. This is not something related to
whether someone speaks English as their native language or not.
At any rate this conversation is going nowhere useful. Just drop it and
move on folks.
Rahul
3:36 p.m.
New subject: SSH defualt on livecd bootup
On Fri, 2007-04-27 at 15:08 -0500, Jasper Hartline wrote:
Thomas Chung wrote:
> On 4/27/07, Jasper Hartline <jasperhartline(a)adelphia.net> wrote:
>> Learn how to read, when you do come back with a valid retort.
>
> Jasper, that was not necessary. :(
> Regards,
Niether was his assumption I was talking about the SSH key generation
being slow due to Ext3.
Nowhere in my statement did I say "The SSH key generation is slow due to
Ext3".
He also said my claims were unsubstantiated, again going on an
assumption of what I said which I never had.
It's plain to see he is not a native English speaker.
Woh, calm down here. I think the issue is the context of the tread was
ssh. Moving it to issues with ext3 is a whole other topic and your post
was easy to misinterpret as saying ext3 was the reason ssh is slow.
Language being language, missing a comma by the reader or author here or
there can least to misinterpretations. Lets not get a simple
misunderstanding to become an invitation to personal attacks.
--
John (J5) Palmieri <johnp(a)redhat.com>
12:45 p.m.
On Thu, 2007-04-26 at 21:44 -0400, David Zeuthen wrote:
On Thu, 2007-04-26 at 16:51 -0400, John (J5) Palmieri wrote:
> > I think munging configuration files might be outside the scope of the
> > livecd-tools project; sure, granted it's something people want to do but
> > often it's used as a hack; for example on my system
> >
> > $ rpm -qf /etc/issue
> > fedora-release-6.93-1
>
> And I'm going to package that up for every daily release? And what
> about countries? Too heavy and engineered for an image based
> distribution.
[...]
> > so I think what you *could* do instead is to have olpc-release or
> > similar on OLPC provide that file (would be based on fedora-release). Of
> > course, since /etc/issue is marked as %config(noreplace) in the
> > fedora-release it's still sane for you to edit it in %post. I guess it
> > all depends on the configuration file in question.
I think I said that both approaches works. Just make sure all files
being changed are tagged with %config(noreplace) if you care about RPM
upgrade paths (which you probably don't need to for OLPC but it's nice
nonetheless if someone wants to pick this up for hardware that can cope
with yum/rpm etc.).
> > So I think my point of view right now is that it's good enough to do
> > this in %post for most cases; I mean, if downstreams (like OLPC) need to
> > change a ton of configuration files/defaults perhaps there is something
> > broken with the apps you try to include?
>
> No just that apps can be configured for different purposes. For
> instance gedit can be made into a development environment but requires
> me to write out a bunch of gconf files. We have Sabyon for a reason.
> It basically does what I want to do in the case of gconf and some other
> settings. It is a dump of configuration files.
Sure, or you can use gconftool-2 or Puppet as Mark mentioned or
basically anything that gets the job done. My point really was that I
don't think configuration file munging belongs in livecd-tools proper.
Do you think that's wrong?
> > Perhaps you can come up with concrete examples of what configuration
> > files / defaults you need to edit /change? I think that would be useful.
> > Thanks.
>
> /etc/sysconfig/modules/olpc-1.modules for the module lists to load
> /etc/rwtab for stateless tmpfs mounted stuff
> /etc/asound.state to put the correct default sound levels
> /etc/modprobe.d/olpc.conf.dist so we get the correct module parameters
> /boot/olpc_build for build info that the bootloader uses
> /etc/issue for build info the user can read
> /etc/securetty for adding the serial console
> /etc/inittab for removing unneeded memory waisting VT's
> /usr/share/fonts/dejavu-lgc/fonts.dir for mapping legacy fonts
> /usr/share/fonts/dejavu-lgc/fonts.scale same as above
> /usr/share/fonts/dejavu-lgc/fonts.alias same
> /etc/X11/xorg.conf and its variants for booting on any device and
> getting the correct xorg config file with the correct keyboard laysouts
> /home/olpc/.xinitrc for launching sugar
> /boot/grub/grub.conf for the occasional qemu boot
> /etc/init.d/olpc-configure init script for boot time configuration
> /usr/share/X11/xkb/symbols/* for keyboard configuration that is not yet
> upstream (note that these must be added to an already existing file)
> /boot/olpc.fth - for booting from openfirmware
>
> Note that we can't move a lot of these to an RPM because RPM conflict
> rules would prevent them from being installed. Some files need input
> such as BUILD number and some files change so rapidly during the testing
> phase that taking the time to build an rpm and place it in the right
> spot and rebuild the repo would be a unneeded bottleneck.
So perhaps one maintainable way for you is to have a single script
(packaged up in a OLPC specific RPM) that does all the munging and your
%post simple invokes this? That script would know about per-country
things and what kind of "image" (e.g. jffs2, livecd, disk_ext3 etc.)
you're building by having the %post pass it as options? Wouldn't that
work?
Of course, the question I raised in an earlier mail of how we pass items
to the %post script in the Kickstart file still stands. Jeremy? Perhaps
just setting environment variables works out of the box. I'd be
surprised if this is not the case.
Also, external variables used by higher level constructs that does batch
builds in a structured way (e.g. pilgrim-autobuild) needs to pass
(OLPC-specific) variables too such as BUILD_NUMBER, TARGET_COUNTRY etc.
to %post. I think the way I'd like this to work is that such variables
are transparently passed by livecd-tools and then the %post takes care
of it. Perhaps it would look like this
%post
olpc-configure-system --image-type $LIVECD_TOOLS_IMAGE_TYPE \
--olpc-build-number $OLPC_IMAGE_NUMBER \
--olpc-target $OLPC_TARGET_COUNTRY
Would that work for you?
In some ways it does but then again it isn't a standard solution but
just a workaround. It still means no quick turnaround for testing but
that is all a failing of our build system.
%post to me is a workaround all together. I would really have liked
this to be more declarative, then tools could be written which had
context to what is going on instead of this RPM black box. A couple of
things are already declarative in ks such as turning on and off default
init scripts. I would have liked to have had more intelligent systems.
For instance with the RPM method inheritance is at the whim of how the
post in the RPM is written. For the most part people will get it right
but there is also the issue that I can't optimize either. If it was
declarative a tool could say this file is going to be overwritten by a
later declaration so there is no need to write it out.
I guess I am in a camp which believes we need to get away from the
flexibility of scripting around deficiencies in our higher level tools
and instead make the tools more robust and able to handle the corner
cases.
1:43 p.m.
On Fri, 2007-04-27 at 13:45 -0400, John (J5) Palmieri wrote:
> %post
> olpc-configure-system --image-type $LIVECD_TOOLS_IMAGE_TYPE \
> --olpc-build-number $OLPC_IMAGE_NUMBER \
> --olpc-target $OLPC_TARGET_COUNTRY
>
> Would that work for you?
In some ways it does but then again it isn't a standard solution but
just a workaround. It still means no quick turnaround for testing but
that is all a failing of our build system.
%post to me is a workaround all together. I would really have liked
this to be more declarative, then tools could be written which had
context to what is going on instead of this RPM black box. A couple of
things are already declarative in ks such as turning on and off default
init scripts. I would have liked to have had more intelligent systems.
For instance with the RPM method inheritance is at the whim of how the
post in the RPM is written. For the most part people will get it right
but there is also the issue that I can't optimize either. If it was
declarative a tool could say this file is going to be overwritten by a
later declaration so there is no need to write it out.
I guess I am in a camp which believes we need to get away from the
flexibility of scripting around deficiencies in our higher level tools
and instead make the tools more robust and able to handle the corner
cases.
Yeah, it's not that I disagree with that; it's just the way some things
work in the distro are not really optimized for custom distros just yet.
Note that this is not something specific to livecd-tools; you want these
features to apply to normal spins and other installation methods as
well. I believe both Bill and Mark did a lot of thinking on this topic
and Bill even went as far as to propose it for Fedora 7
http://fedoraproject.org/wiki/Releases/FeatureCustomDistro
but that feature didn't make it in. Definitely worth thinking more
about...
David
1:44 p.m.
On Fri, 2007-04-27 at 13:45 -0400, John (J5) Palmieri wrote:
On Thu, 2007-04-26 at 21:44 -0400, David Zeuthen wrote:
> On Thu, 2007-04-26 at 16:51 -0400, John (J5) Palmieri wrote:
> > > I think munging configuration files might be outside the scope of the
> > > livecd-tools project; sure, granted it's something people want to do
but
> > > often it's used as a hack; for example on my system
> > >
> > > $ rpm -qf /etc/issue
> > > fedora-release-6.93-1
> >
> > And I'm going to package that up for every daily release? And what
> > about countries? Too heavy and engineered for an image based
> > distribution.
>
> [...]
>
> > > so I think what you *could* do instead is to have olpc-release or
> > > similar on OLPC provide that file (would be based on fedora-release). Of
> > > course, since /etc/issue is marked as %config(noreplace) in the
> > > fedora-release it's still sane for you to edit it in %post. I guess
it
> > > all depends on the configuration file in question.
>
> I think I said that both approaches works. Just make sure all files
> being changed are tagged with %config(noreplace) if you care about RPM
> upgrade paths (which you probably don't need to for OLPC but it's nice
> nonetheless if someone wants to pick this up for hardware that can cope
> with yum/rpm etc.).
>
> > > So I think my point of view right now is that it's good enough to do
> > > this in %post for most cases; I mean, if downstreams (like OLPC) need to
> > > change a ton of configuration files/defaults perhaps there is something
> > > broken with the apps you try to include?
> >
> > No just that apps can be configured for different purposes. For
> > instance gedit can be made into a development environment but requires
> > me to write out a bunch of gconf files. We have Sabyon for a reason.
> > It basically does what I want to do in the case of gconf and some other
> > settings. It is a dump of configuration files.
>
> Sure, or you can use gconftool-2 or Puppet as Mark mentioned or
> basically anything that gets the job done. My point really was that I
> don't think configuration file munging belongs in livecd-tools proper.
> Do you think that's wrong?
>
> > > Perhaps you can come up with concrete examples of what configuration
> > > files / defaults you need to edit /change? I think that would be useful.
> > > Thanks.
> >
> > /etc/sysconfig/modules/olpc-1.modules for the module lists to load
> > /etc/rwtab for stateless tmpfs mounted stuff
> > /etc/asound.state to put the correct default sound levels
> > /etc/modprobe.d/olpc.conf.dist so we get the correct module parameters
> > /boot/olpc_build for build info that the bootloader uses
> > /etc/issue for build info the user can read
> > /etc/securetty for adding the serial console
> > /etc/inittab for removing unneeded memory waisting VT's
> > /usr/share/fonts/dejavu-lgc/fonts.dir for mapping legacy fonts
> > /usr/share/fonts/dejavu-lgc/fonts.scale same as above
> > /usr/share/fonts/dejavu-lgc/fonts.alias same
> > /etc/X11/xorg.conf and its variants for booting on any device and
> > getting the correct xorg config file with the correct keyboard laysouts
> > /home/olpc/.xinitrc for launching sugar
> > /boot/grub/grub.conf for the occasional qemu boot
> > /etc/init.d/olpc-configure init script for boot time configuration
> > /usr/share/X11/xkb/symbols/* for keyboard configuration that is not yet
> > upstream (note that these must be added to an already existing file)
> > /boot/olpc.fth - for booting from openfirmware
> >
> > Note that we can't move a lot of these to an RPM because RPM conflict
> > rules would prevent them from being installed. Some files need input
> > such as BUILD number and some files change so rapidly during the testing
> > phase that taking the time to build an rpm and place it in the right
> > spot and rebuild the repo would be a unneeded bottleneck.
>
> So perhaps one maintainable way for you is to have a single script
> (packaged up in a OLPC specific RPM) that does all the munging and your
> %post simple invokes this? That script would know about per-country
> things and what kind of "image" (e.g. jffs2, livecd, disk_ext3 etc.)
> you're building by having the %post pass it as options? Wouldn't that
> work?
>
> Of course, the question I raised in an earlier mail of how we pass items
> to the %post script in the Kickstart file still stands. Jeremy? Perhaps
> just setting environment variables works out of the box. I'd be
> surprised if this is not the case.
>
> Also, external variables used by higher level constructs that does batch
> builds in a structured way (e.g. pilgrim-autobuild) needs to pass
> (OLPC-specific) variables too such as BUILD_NUMBER, TARGET_COUNTRY etc.
> to %post. I think the way I'd like this to work is that such variables
> are transparently passed by livecd-tools and then the %post takes care
> of it. Perhaps it would look like this
>
> %post
> olpc-configure-system --image-type $LIVECD_TOOLS_IMAGE_TYPE \
> --olpc-build-number $OLPC_IMAGE_NUMBER \
> --olpc-target $OLPC_TARGET_COUNTRY
>
> Would that work for you?
In some ways it does but then again it isn't a standard solution but
just a workaround. It still means no quick turnaround for testing but
that is all a failing of our build system.
%post to me is a workaround all together. I would really have liked
this to be more declarative, then tools could be written which had
context to what is going on instead of this RPM black box. A couple of
things are already declarative in ks such as turning on and off default
init scripts. I would have liked to have had more intelligent systems.
For instance with the RPM method inheritance is at the whim of how the
post in the RPM is written. For the most part people will get it right
but there is also the issue that I can't optimize either. If it was
declarative a tool could say this file is going to be overwritten by a
later declaration so there is no need to write it out.
I guess I am in a camp which believes we need to get away from the
flexibility of scripting around deficiencies in our higher level tools
and instead make the tools more robust and able to handle the corner
cases.
Actually the RPM idea still has some flaws. It leaves junk config
templates laying around. Even if you delete them on systems where we
want the ability to rpm upgrade they will get the new templates. Not
ideal.
--
John (J5) Palmieri <johnp(a)redhat.com>
2:52 p.m.
On Fri, 2007-04-27 at 13:45 -0400, John (J5) Palmieri wrote:
%post to me is a workaround all together. I would really have liked
this to be more declarative, then tools could be written which had
context to what is going on instead of this RPM black box. A couple of
things are already declarative in ks such as turning on and off default
init scripts.
The problem is that arbitrary substitutions within files really isn't
declarative. Adding things that are (like services, etc) doesn't tend
to be hard or that controversial. But when you get to the degree of
what you're wanting, you really want a config management system like
puppet, cfengine, etc. And I think that the _eventual_ answer is that
we'll standardize on something like that as The Config Management
Solution at which point a stronger integration into the config will make
plenty of sense. But you don't really want to have integration with
cfengine and puppet and ... -- instead, we need to get to where we can
make an educated decision and stick with it.
Jeremy
4:29 a.m.
On Tue, 2007-04-24 at 14:53 -0400, John (J5) Palmieri wrote:
One of the things we need to for OLPC from the LiveCD creator is the
ability to output configuration files from a template.
One thing I've been meaning to do for a while, based on David
Lutterkort's suggestion, is to try using puppet[1] from the
livecd-creator ... i.e. run puppet from %post with a manifest that would
change whatever configuration files you need. One reason puppet might be
nice is that it does inheritance, templating etc.
Dunno, I'm not convinced yet, but it's worth trying.
Cheers,
Mark.
[1] - http://reductivelabs.com/projects/puppet/
11:56 a.m.
On Thu, 2007-04-26 at 10:29 +0100, Mark McLoughlin wrote:
On Tue, 2007-04-24 at 14:53 -0400, John (J5) Palmieri wrote:
> One of the things we need to for OLPC from the LiveCD creator is the
> ability to output configuration files from a template.
One thing I've been meaning to do for a while, based on David
Lutterkort's suggestion, is to try using puppet[1] from the
livecd-creator ... i.e. run puppet from %post with a manifest that would
change whatever configuration files you need. One reason puppet might be
nice is that it does inheritance, templating etc.
I gave this a shot. What I did was:
- Packaged, in an RPM, a manifest for changing the iptables config
- Demonstrated a form of inheritance - i.e. there's a base iptables
configuration, and the manifest adds to that
- Installed puppet and the manifest RPM via the kickstart package list
- In %post, passed the manifest to puppet and then removed puppet
again
See the attached kickstart file and puppet files.
So, some comments:
- I don't want to have to require setting up a puppet server in order
to build images
- Having to package the manifest in an RPM sucks, we should probably
have support for copying the manifest into the chroot e.g.
$> livecd-creator -t raw -c ./test-image.ks -p iptables.pp --p myiptables.pp
- One reason why I include the iptables config inline in the puppet
manifest is that I don't like having stuff scattered over many
different files
- The main reason, though, is that I wanted the derived iptables
config to just add some rules rather than completely override the
entire config. Can we do something similar with templating?
- This is all fairly clunky, is there a better way?
Cheers,
Mark.
12:28 p.m.
On Wed, 2007-05-16 at 17:56 +0100, Mark McLoughlin wrote:
- The main reason, though, is that I wanted the derived iptables
config to just add some rules rather than completely override the
entire config. Can we do something similar with templating?
Here's a quick prototype of using templating. Hmm.
Cheers,
Mark.
12:37 p.m.
On Wed, 2007-05-16 at 17:56 +0100, Mark McLoughlin wrote:
- I don't want to have to require setting up a puppet server in
order
to build images
Absolutely; for livecd, you want to use the standalone command line
clinet for puppet (as you did)
- Having to package the manifest in an RPM sucks, we should
probably
have support for copying the manifest into the chroot e.g.
$> livecd-creator -t raw -c ./test-image.ks -p iptables.pp --p myiptables.pp
To integrate more tightly, you want to do two things for puppet; (1)
copy a number of puppet modules [1] into the chroot and (2) have a
toplevel manifest that uses those modules and changes the config in the
chroot. It's pretty much what you have, mostly some small changes to
make the setup a little more flexible.
- One reason why I include the iptables config inline in the puppet
manifest is that I don't like having stuff scattered over many
different files
- The main reason, though, is that I wanted the derived iptables
config to just add some rules rather than completely override the
entire config. Can we do something similar with templating?
- This is all fairly clunky, is there a better way?
One thing that makes that clunky is that AFAIK no good way to ask
whether the current iptables has a certain rule installed, and if not
install it. If something like that existed, it would be fairly easy to
write a puppet type that made this a little less clunky.
The underlying wrangling with iptables and /etc/sysconfig/iptables
doesn't really belong into a config mgmt tool like puppet, but into
tools that make iptables more manageable. This is similar to how
useradd/usermod/userdel make dealing with /etc/passwd pretty
straightforward, and puppet exposes that with the user type.
[1] http://reductivelabs.com/trac/puppet/wiki/ModuleOrganisation
2:19 p.m.
Hi,
On Wednesday 16 May 2007 19:37, David Lutterkort wrote:
One thing that makes that clunky is that AFAIK no good way to ask
whether the current iptables has a certain rule installed, and if not
install it. If something like that existed, it would be fairly easy to
write a puppet type that made this a little less clunky.
I'm doing the same what is done with puppet here, with FAI
(http://www.informatik.uni-koeln.de/fai/ and http://faiwiki.debian.net). I'll
post a short howto tomorrow.
And with FAI I would use a cfengine script for such "edits", and I think you
can use cfengine with puppet, too.
FAI used to be a debian only installer, but since some time (years..) it's
also possible to update and manage running systems with it, also rpm based
systems. (It doesn't support installations of rpm based systems yet, mostly
because there seems to be no aquivalent to debootstrap.)
regards,
Holger
6187
days inactive
6209
days old
livecd@lists.fedoraproject.org
28 comments
10 participants
participants (10)
-
David Lutterkort -
David Zeuthen -
Holger Levsen -
J. Hartline -
Jeremy Katz -
John (J5) Palmieri -
Mark McLoughlin -
Rahul Sundaram -
Steve Berry -
Thomas Chung