Debian cobbler import

[Chris Sarginson]

Hi guys,

I've just tried doing an import of an ubuntu CD after downloading the latest cobbler from git and got the following:

[Errno 36] File name too long: '/var/lib/cobbler/config/distros.d/ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-install-i386'

Can anyone point me in the right direction with regards to fixing this?

This is on a Centos 5 box if that matters.

Chris

[Javier Palacios]

On Thu, Nov 13, 2008 at 10:22 AM, Chris Sarginson
 wrote:
> Hi guys,
>
> I've just tried doing an import of an ubuntu CD after downloading the latest cobbler from git and got the following:
>
> [Errno 36] File name too long: '/var/lib/cobbler/config/distros.d/ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-install-i386'
>
> Can anyone point me in the right direction with regards to fixing this?

This was the very first thing that happened when starting to import
debian distros, and was supposed to be fixed time ago.

Which CD did you tried? Test with the alternate ones for the latest
ubuntu is on my todo today list.

Javier Palacios

[Michael DeHaan]

Chris Sarginson wrote:
> Hi guys,
>
> I've just tried doing an import of an ubuntu CD after downloading the latest cobbler from git and got the following:
>
> [Errno 36] File name too long: '/var/lib/cobbler/config/distros.d/ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-install-i386'
>
> Can anyone point me in the right direction with regards to fixing this?
>
> This is on a Centos 5 box if that matters.
>
> Chris
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

Where you using the devel branch from git (master will almost certaintly 
not work -- and is close to 1.2.8 code)?    Also, did you supply 
--breed=debian to the import command?

I think Javier resolved this one earlier, but he'd know for sure.

--Michael

[Chris Sarginson]

OK, using the correct branch now - sorry about that, I am however, getting a different error:

cobbler import --breed=debian --name=ubuntu --mirror=/mnt/

results in

- found content (breed=ubuntu) at /var/www/cobbler/ks_mirror/ubuntu/install

This is with an Ubuntu 8.04 ISO - has ubuntu been tested or is it Debian specific so far?

Many Thanks for your help guys!

Chris

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Michael DeHaan
Sent: 13 November 2008 12:38
To: cobbler mailing list
Subject: Re: Debian cobbler import

Chris Sarginson wrote:
> Hi guys,
>
> I've just tried doing an import of an ubuntu CD after downloading the latest cobbler from git and got the following:
>
> [Errno 36] File name too long: '/var/lib/cobbler/config/distros.d/ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-install-i386'
>
> Can anyone point me in the right direction with regards to fixing this?
>
> This is on a Centos 5 box if that matters.
>
> Chris
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

Where you using the devel branch from git (master will almost certaintly
not work -- and is close to 1.2.8 code)?    Also, did you supply
--breed=debian to the import command?

I think Javier resolved this one earlier, but he'd know for sure.

--Michael

_______________________________________________
cobbler mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/cobbler

[Javier Palacios]

> found content (breed=ubuntu) at /var/www/cobbler/ks_mirror/ubuntu/install
>
> This is with an Ubuntu 8.04 ISO - has ubuntu been tested or is it Debian specific so far?

As far as I remember I did only made import test with ubuntu, no deployment.

Have a look at
https://fedorahosted.org/cobbler/wiki/DebianDeployment
and you will see how it is with debian, and try to follow. In case the
init images are able
to network install, it could be easier. If not you need find a network
boot image for ubuntu.

Javier Palacios

[Chris Sarginson]

I spotted where I was going wrong - --breed now supports ubuntu specifically rather than --breed=debian - I've done a test import with the patch you submitted earlier and this worked fine.

Thanks
chris

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Javier Palacios
Sent: 14 November 2008 09:06
To: cobbler mailing list
Subject: Re: Debian cobbler import
> found content (breed=ubuntu) at /var/www/cobbler/ks_mirror/ubuntu/install
>
> This is with an Ubuntu 8.04 ISO - has ubuntu been tested or is it Debian specific so far?

As far as I remember I did only made import test with ubuntu, no deployment.

Have a look at
https://fedorahosted.org/cobbler/wiki/DebianDeployment
and you will see how it is with debian, and try to follow. In case the
init images are able
to network install, it could be easier. If not you need find a network
boot image for ubuntu.

Javier Palacios
_______________________________________________
cobbler mailing list
[e-mail]
https://fedorahosted.org/mailman/listinfo/cobbler

[Javier Palacios]

On Fri, Nov 14, 2008 at 12:54 PM, Chris Sarginson
 wrote:
> I spotted where I was going wrong - --breed now supports ubuntu specifically rather than --breed=debian - I've done a test import with the patch you submitted earlier and this worked fine.

Actually, you shouldn't need to supply the breed. It should detect the
breed automatically. If not, that might be considered a bug.

JaviP

[Michael DeHaan]

Chris Sarginson wrote:
> I spotted where I was going wrong - --breed now supports ubuntu specifically rather than --breed=debian - I've done a test import with the patch you submitted earlier and this worked fine.
>
> Thanks
> chris

All of those patches should be applied to the development branch of 
Cobbler now, if not, let me know.

Also if Ubuntu doesn't work that's probably a reason for making 
--breed=ubuntu raise an error for now. Thoughts?

I know it works if you "cobbler distro add" it manually.
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Javier Palacios
> Sent: 14 November 2008 09:06
> To: cobbler mailing list
> Subject: Re: Debian cobbler import
> > found content (breed=ubuntu) at /var/www/cobbler/ks_mirror/ubuntu/install
> >
> > This is with an Ubuntu 8.04 ISO - has ubuntu been tested or is it Debian specific so far?
>
> As far as I remember I did only made import test with ubuntu, no deployment.
>
> Have a look at
> https://fedorahosted.org/cobbler/wiki/DebianDeployment
> and you will see how it is with debian, and try to follow. In case the
> init images are able
> to network install, it could be easier. If not you need find a network
> boot image for ubuntu.
>
> Javier Palacios
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Javier Palacios]

On Mon, Nov 17, 2008 at 8:51 PM, Michael DeHaan  wrote:
> Chris Sarginson wrote:
> > I spotted where I was going wrong - --breed now supports ubuntu specifically rather than --breed=debian - I've done a test import with the patch you submitted earlier and this worked fine.
>
> All of those patches should be applied to the development branch of
> Cobbler now, if not, let me know.

Unles Michael founds I problem I didn't, the debian patches usually go
quite fast to devel branch, but in case someone is extremely
impatient, the github where I commit my patches is at
http://github.com/javiplx/cobbler/tree/debian.
It might happen that non-debian things don't work, but debian related
imports, repos and so should work. That means debian and ubuntu. Any
other debian derived distro should require a new Importer class
implementing at least the same methods than the UbuntuImporter does.

JaviP

[Michael DeHaan]

Michael DeHaan wrote:
> Chris Sarginson wrote:
> > I spotted where I was going wrong - --breed now supports ubuntu specifically rather than --breed=debian - I've done a test import with the patch you submitted earlier and this worked fine.
> >
> > Thanks
> > chris
>
> All of those patches should be applied to the development branch of 
> Cobbler now, if not, let me know.
>
> Also if Ubuntu doesn't work that's probably a reason for making 
> --breed=ubuntu raise an error for now. Thoughts?
>
> I know it works if you "cobbler distro add" it manually.
> > -----Original Message-----
> > From: [e-mail] [mailto:[e-mail]] On Behalf Of Javier Palacios
> > Sent: 14 November 2008 09:06
> > To: cobbler mailing list
> > Subject: Re: Debian cobbler import
> > > found content (breed=ubuntu) at /var/www/cobbler/ks_mirror/ubuntu/install
> > >
> > > This is with an Ubuntu 8.04 ISO - has ubuntu been tested or is it Debian specific so far?
> >
> > As far as I remember I did only made import test with ubuntu, no deployment.
> >
> > Have a look at
> > https://fedorahosted.org/cobbler/wiki/DebianDeployment
> > and you will see how it is with debian, and try to follow. In case the
> > init images are able
> > to network install, it could be easier. If not you need find a network
> > boot image for ubuntu.
> >
> > Javier Palacios
> > _______________________________________________
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
> > _______________________________________________
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

Aside -- one thing I notice now about the Debian instructions is that 
the source DVD had a /very/ large download to just get the source RPMs 
(4.4GB) which I imagine we don't actually need many of.

Can't this be made to work with just binary content, or better, ship 
it's own excludes file and use an rsync mirror?

I know you mentioned you had some info here: 
https://fedorahosted.org/cobbler/wiki/AlternateDebianDeployment but this 
seems exceedingly sparse.

How much would it take to get this going?

Otherwise do we want to hold off claiming Debian import is a feature 
until 1.5?

--Michael

[Javier Palacios]

> Aside -- one thing I notice now about the Debian instructions is that
> the source DVD had a /very/ large download to just get the source RPMs
> (4.4GB) which I imagine we don't actually need many of.

I'm afraid that there is no work around. That is the way debian is, 3 DVD
download for etch install, although you require something like a half one for
a working system. The more sensible way a found is to use the multiarch
media, with three archs plus sources, which is a lot of waste but at least is
a reduced and manageable version.

That's large, but after your first attempt to sync a repository (single arch, no
sources), you will realize that the proper is actually "huge".
> How much would it take to get this goin'mg?

I will try to enlarge that explanation a bit, but in brief you should import
http://ftp.es.debian.org/debian/dists/etch/main/installer-i386/20070308etch3...
which I'm not sure if will create distros and profiles properly (in
any case, will do in short).
Probably the ks_meta will need manual setup, but only setting up
repositories (and syncing)
should be enough.
> Otherwise do we want to hold off claiming Debian import is a feature
> until 1.5?

Maybe just mention that it is not the most comfortable import (but that is not
a cobbler fault). It might require some time to consolidate, but it's probably
better to push a not yet perfect feature in order to gain feedback compared
to keeping that just on devel branch. Although it's obviously your opinion the
one that rules this.

Besides debmirror and apt-mirror, there is a third tool called
apt-proxy that I have used.
It is basically a local cache, and reduces the size of downloads. But
although it is written
in python and could be more or less easily integrated, it breaks
completelly the way cobbler
manages the repos.

Javier Palacios

[Javier Palacios]

> I know you mentioned you had some info here:
> https://fedorahosted.org/cobbler/wiki/AlternateDebianDeployment but this
> seems exceedingly sparse.
>
> How much would it take to get this going?

I have updated that page. But not without patching:

http://github.com/javiplx/cobbler/commit/cf88bd0a7df1a066b09097f5d6274a038aa...

To improve this, a --os-version command parameter is needed. I can add
in the same
way that I did with the --breed, but have a question before. With
this, we have tree options
(breed, arch and version) that import should be able to detect in
normal media. I'm
thinking that it might be a good idea to fork the import command into
an intelligent one and
the "stupid" one that requires many parameters. For the upcoming
1.3/1.4 releases
the --os-version could be fine, but in the longer term, having two
commands might help,
for example, to easily introduce other distros without requiring a
completely automated
discovery.

Javier Palacios

[Michael DeHaan]

Javier Palacios wrote:
> > I know you mentioned you had some info here:
> > https://fedorahosted.org/cobbler/wiki/AlternateDebianDeployment but this
> > seems exceedingly sparse.
> >
> > How much would it take to get this going?
>
> I have updated that page. But not without patching:
>
> http://github.com/javiplx/cobbler/commit/cf88bd0a7df1a066b09097f5d6274a038aa...

Looks small enough, I'll check this out.
> To improve this, a --os-version command parameter is needed. I can add
> in the same
> way that I did with the --breed, but have a question before.

So far this sounds fine. In that case, if breed is supplied and is 
debian, it should raise an error if --os-version is not set, to avoid 
suprises and keep the user from knowing they have to supply this.
> With
> this, we have tree options
> (breed, arch and version) that import should be able to detect in
> normal media. I'm
> thinking that it might be a good idea to fork the import command into
> an intelligent one and
> the "stupid" one that requires many parameters. For the upcoming
> 1.3/1.4 releases
> the --os-version could be fine, but in the longer term, having two
> commands might help,
> for example, to easily introduce other distros without requiring a
> completely automated
> discovery.

What would be the two different syntaxes in the above? What would be the 
advantage of having two commands?
> Javier Palacios
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Javier Palacios]

> > With
> > this, we have tree options
> > (breed, arch and version) that import should be able to detect in
> > normal media. I'm
> > thinking that it might be a good idea to fork the import command into
> > an intelligent one and
> > the "stupid" one that requires many parameters. For the upcoming
> > 1.3/1.4 releases
> > the --os-version could be fine, but in the longer term, having two
> > commands might help,
> > for example, to easily introduce other distros without requiring a
> > completely automated
> > discovery.
>
> What would be the two different syntaxes in the above? What would be the
> advantage of having two commands?

Let say the intelligent accepts only name, path, and available-as, and
discovers everything. And the other one requires breed, arch, version.

I cannot point any real advantage of this fork, except that it is faster to
implement a new breed or arch if you have a place where you don't
need to care so much about undesired interferences. And that a
a complete import of a complex tree of distros is quite impressive.

But I also believe that the faster to implement means the slower to
integrate, so the fork could be a step back. Let's keep around the
idea of specialized imports without thinking on that for the moment.

JaviP

[Javier Palacios]

> > [Errno 36] File name too long: '/var/lib/cobbler/config/distros.d/ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-ubuntu-install-i386'
>
> Where you using the devel branch from git (master will almost certaintly
> not work -- and is close to 1.2.8 code)?    Also, did you supply
> --breed=debian to the import command?
>
> I think Javier resolved this one earlier, but he'd know for sure.

The bug should not happen even in master, as the patch for scientific linux
should prevent symlink traversal.

But in any case, it does not happen on current devel, neither with 8.04.1 nor
with 8.10 alternate CD. In the case of latest ubuntu, an exception is raised.
The patch is ready, but I will not be able to upload to github in a while, so
I will mail when available.

Javier Palacios