Future of Satellite

[Konrad Scherer]

Hello all,

I have seen this mentioned in several other places (puppet users) but not here. 
As cobbler is integrated with Satellite I think this is of interest to the 
people subscribed to this list. At the 2011 RedHat summit there was a 
presentation at about the future of Satellite[1].

There is a pdf of the slides of the presentation[2]. Of interest to cobbler 
users is the new provisioning engine: The Foreman (pg 39). Interestingly v5 of 
the pdf described The Foreman as "cobbler on steroids" and that item has been 
removed from v7.

Some obvious questions:
1) Will the next major version of Satellite support both cobbler and foreman?
2) Will there be any tools to help users of cobbler migrate to foreman?

As a happy and grateful user of cobbler I have noticed that the cobbler project 
is not as active as it used to be. Is it time for me to move on?

Konrad

[1] 
http://www.redhat.com/webinars/?zPage=2011-07-07-Future-of-Red-Hat-Networ...
[2] 
http://www.redhat.com/summit/2011/presentations/summit/whats_next/thursda...

[Jörgen Maas]

On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
<konrad.scherer(a)windriver.com&gt; wrote:
>  As a happy and grateful user of cobbler I have noticed that the cobbler project
>  is not as active as it used to be. Is it time for me to move on?
>  
Unfortunately I have come to the same conclusions.
The cobbler project looks like it's dying a slow death.

So yes, I guess it's time to move on ;).

-- 
Grtz,
Jörgen Maas

[Jonathan Underwood]

On 19 September 2011 20:29, Jörgen Maas <jorgen.maas(a)gmail.com&gt; wrote:
>  On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
>  <konrad.scherer(a)windriver.com&gt; wrote:
> >
> > As a happy and grateful user of cobbler I have noticed that the cobbler project
> > is not as active as it used to be. Is it time for me to move on?
> >
>
>  Unfortunately I have come to the same conclusions.
>  The cobbler project looks like it's dying a slow death.
>
>  So yes, I guess it's time to move on ;).
>  
I think if you were to look at the activity in the source code
repository, you might draw a different conclusion. Also, you might
want to look at this:

http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/

That said, the perception that cobbler is dying as a project probably
has something to do with the lack of release management of late...

J.

[James Cammarata]

On Mon, Sep 19, 2011 at 4:44 PM, Jonathan Underwood
<jonathan.underwood(a)gmail.com&gt; wrote:
>  On 19 September 2011 20:29, Jörgen Maas <jorgen.maas(a)gmail.com&gt;
> wrote:
> > On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
> > <konrad.scherer(a)windriver.com&gt; wrote:
> >>
> >> As a happy and grateful user of cobbler I have noticed that the cobbler project
> >> is not as active as it used to be. Is it time for me to move on?
> >>
> >
> > Unfortunately I have come to the same conclusions.
> > The cobbler project looks like it's dying a slow death.
> >
> > So yes, I guess it's time to move on ;).
> >
>
>  I think if you were to look at the activity in the source code
>  repository, you might draw a different conclusion. Also, you might
>  want to look at this:
>
>  http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/
>
>  That said, the perception that cobbler is dying as a project probably
>  has something to do with the lack of release management of late... 
That post was one of the reasons I wrote the mongodb backend :)

But yes, cobbler always does seem to go through lulls in activity,
something I try to change periodically by contributing fixes and an
occasional major feature (see import modules). I use cobbler now more
than I ever had in the past, and I see a lot of people out there using
it still. I'm not a fan of foreman, and I haven't found anything else
I like as much as my cobbler/puppet combo.

If you think cobbler is fading, change that by contributing. There's
always a need for snippets, especially for the newer supported distros
like debian and ubuntu.

[Bill Peck]

On 09/19/2011 05:44 PM, Jonathan Underwood wrote:
>  On 19 September 2011 20:29, Jörgen Maas<jorgen.maas(a)gmail.com> 
> wrote:
> > On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
> > <konrad.scherer(a)windriver.com&gt;  wrote:
> >> As a happy and grateful user of cobbler I have noticed that the cobbler project
> >> is not as active as it used to be. Is it time for me to move on?
> >>
> > Unfortunately I have come to the same conclusions.
> > The cobbler project looks like it's dying a slow death.
> >
> > So yes, I guess it's time to move on ;).
> >
>  I think if you were to look at the activity in the source code
>  repository, you might draw a different conclusion. Also, you might
>  want to look at this:
>
>  http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/ 
Unless I'm mistaken the solution Michael gives for the scaling issue is 
to use the backend couchDB.  The problem is that doesn't solve anything 
since it still loads *everything* into ram.  And that gets back to the 
real problem, cobbler wasn't designed to scale, simple as that.

>  That said, the perception that cobbler is dying as a project
> probably
>  has something to do with the lack of release management of late...
>
>  J.
>  _______________________________________________
>  cobbler-devel mailing list
>  cobbler-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler-devel

[brett lentz]

On Tue, Sep 20, 2011 at 8:43 AM, Bill Peck <bpeck(a)redhat.com&gt; wrote:
>  On 09/19/2011 05:44 PM, Jonathan Underwood wrote:
> > On 19 September 2011 20:29, Jörgen Maas&lt;jorgen.maas(a)gmail.com&gt;  wrote:
> >> On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
> >> <konrad.scherer(a)windriver.com&gt;  wrote:
> >>> As a happy and grateful user of cobbler I have noticed that the cobbler
> project
> >>> is not as active as it used to be. Is it time for me to move on?
> >>>
> >> Unfortunately I have come to the same conclusions.
> >> The cobbler project looks like it's dying a slow death.
> >>
> >> So yes, I guess it's time to move on ;).
> >>
> > I think if you were to look at the activity in the source code
> > repository, you might draw a different conclusion. Also, you might
> > want to look at this:
> >
> > http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/
>
>  Unless I'm mistaken the solution Michael gives for the scaling issue is
>  to use the backend couchDB.  The problem is that doesn't solve anything
>  since it still loads *everything* into ram.  And that gets back to the
>  real problem, cobbler wasn't designed to scale, simple as that.
>  
The whole "not designed to scale" comment, while a nice sound bite, is
meaningless and a bit of a red herring.

As the saying goes, "If you can't measure it, it's not science." The
same is true here. Performance is defined by a rate (i.e. some value
that changes with respect to some other value). Who cares if couchDB
loads everything into RAM? How many bytes does "everything" take up?
Is there enough RAM to hold it? Without any numbers, the statement is
pure FUD.

None of the detractors has yet really mentioned any specific numbers
around what their requirements really are (or whether Cobbler meets
those requirements). Mike's post mentions a few cases of a few
installs running upwards of 20k systems. I'd be interested to know how
cobbler performs in those settings; how quickly it can serve a
rendered kickstart; how quickly it can lookup a particular system
record; how much time it takes an admin to work with the whole system.

I have my own gripes with Cobbler's UI and data model, especially with
respect to how it integrates with puppet. Personally, I think there is
enough room for improvement in those areas that vague hand-waving
about performance is unnecessary.

If there's a likely reason for why the Satellite folks are moving away
from Cobbler, it's probably less about performance and got more to do
with the fact that Foreman was built with integrating with Puppet/Chef
in mind. (That's just a guess.) In Cobbler, that integration is quite
obviously an after-thought (and a poorly implemented one at that).
Life of a system doesn't stop when the installer exits, yet Cobbler
doesn't seem to consider any of those areas. There's little to no
consideration given to issues of inventory, life cycle, or config
management.

If all you care about is installing an OS, Cobbler is a great tool. It
does that job very very well. The problem is, Satellite, Foreman, and
other similar tools are thinking about more than just OS installation.

---Brett.

[Bill Peck]

On 09/20/2011 09:56 AM, brett lentz wrote:
>  On Tue, Sep 20, 2011 at 8:43 AM, Bill Peck<bpeck(a)redhat.com> 
> wrote:
> > On 09/19/2011 05:44 PM, Jonathan Underwood wrote:
> >> On 19 September 2011 20:29, Jörgen Maas<jorgen.maas(a)gmail.com>    wrote:
> >>> On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
> >>> <konrad.scherer(a)windriver.com&gt;    wrote:
> >>>> As a happy and grateful user of cobbler I have noticed that the cobbler
> project
> >>>> is not as active as it used to be. Is it time for me to move on?
> >>>>
> >>> Unfortunately I have come to the same conclusions.
> >>> The cobbler project looks like it's dying a slow death.
> >>>
> >>> So yes, I guess it's time to move on ;).
> >>>
> >> I think if you were to look at the activity in the source code
> >> repository, you might draw a different conclusion. Also, you might
> >> want to look at this:
> >>
> >> http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/
> > Unless I'm mistaken the solution Michael gives for the scaling issue is
> > to use the backend couchDB.  The problem is that doesn't solve anything
> > since it still loads *everything* into ram.  And that gets back to the
> > real problem, cobbler wasn't designed to scale, simple as that.
> >
>  The whole "not designed to scale" comment, while a nice sound bite, is
>  meaningless and a bit of a red herring.
>
>  As the saying goes, "If you can't measure it, it's not science." The
>  same is true here. Performance is defined by a rate (i.e. some value
>  that changes with respect to some other value). Who cares if couchDB
>  loads everything into RAM? How many bytes does "everything" take up?
>  Is there enough RAM to hold it? Without any numbers, the statement is
>  pure FUD. 
Its not pure fud.  Slapping a DB on the backend just to use as a file 
store does nothing for you.  It doesn't enforce data integrity and it 
doesn't speed anything up either.  We have bz's filed about the 
integrity issues where the in memory version of the dataset gets 
confused and has profiles referencing other profiles that don't exist 
any more.  These errors cause cobbler to fall down quite badly, no other 
profile operations can continue because cobbler wants to re-write all 
the pxe records when you update one.

We have hundreds of systems in cobbler which isn't much, but we do have 
thousands of distro/profile records and this is an area where cobbler 
doesn't scale.  cobbler sync takes almost 10 minutes to complete which 
may not seem like much but during that time you can't be sure systems 
will pxe boot correctly since a file or record may be missing.  This is 
better now that a .link_cache directory is used but then we have to 
worry about that directory having old stale data which needs to be cleaned.

Don't get me wrong, cobbler has helped us tremendously but its reached 
its limit for us and the above issues have been resolved in foreman.  
I'm sure we will experience other pain points in foreman but at this 
point it looks like a more solid foundation to stand on.

If cobbler works for you then great.

>  None of the detractors has yet really mentioned any specific numbers
>  around what their requirements really are (or whether Cobbler meets
>  those requirements). Mike's post mentions a few cases of a few
>  installs running upwards of 20k systems. I'd be interested to know how
>  cobbler performs in those settings; how quickly it can serve a
>  rendered kickstart; how quickly it can lookup a particular system
>  record; how much time it takes an admin to work with the whole system.
>
>  I have my own gripes with Cobbler's UI and data model, especially with
>  respect to how it integrates with puppet. Personally, I think there is
>  enough room for improvement in those areas that vague hand-waving
>  about performance is unnecessary.
>
>  If there's a likely reason for why the Satellite folks are moving away
>  from Cobbler, it's probably less about performance and got more to do
>  with the fact that Foreman was built with integrating with Puppet/Chef
>  in mind. (That's just a guess.) In Cobbler, that integration is quite
>  obviously an after-thought (and a poorly implemented one at that).
>  Life of a system doesn't stop when the installer exits, yet Cobbler
>  doesn't seem to consider any of those areas. There's little to no
>  consideration given to issues of inventory, life cycle, or config
>  management.
>
>  If all you care about is installing an OS, Cobbler is a great tool. It
>  does that job very very well. The problem is, Satellite, Foreman, and
>  other similar tools are thinking about more than just OS installation.
>
>  ---Brett.
>  _______________________________________________
>  cobbler-devel mailing list
>  cobbler-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler-devel

[Douglas Kilpatrick]

Bill Peck wrote:
>  We have hundreds of systems in cobbler which isn't much, but we
> do have 
>  thousands of distro/profile records and this is an area where cobbler 
>  doesn't scale.  cobbler sync takes almost 10 minutes to complete which 
>  may not seem like much but during that time you can't be sure systems  
Have you considered using the python tftp server instead?  It grabs the 
profile from the datastore at request time, rather than needing to 
pre-render it, which should solve your specific problem.

Doug
-- 
Doug Kilpatrick
kilpatds(a)oppositelock.org

[Jörgen Maas]

On Tue, Sep 20, 2011 at 4:43 PM, Bill Peck <bpeck(a)redhat.com&gt; wrote:
<snip>
>  We have bz's filed about the
>  integrity issues where the in memory version of the dataset gets
>  confused and has profiles referencing other profiles that don't exist
>  any more. 
Then how were these issues resolved for your (i assume) paying customers?

-- 
Grtz,
Jörgen Maas

[Bill Peck]

On 09/21/2011 01:44 PM, Jörgen Maas wrote:
>  On Tue, Sep 20, 2011 at 4:43 PM, Bill Peck<bpeck(a)redhat.com> 
> wrote:
>  <snip>
> > We have bz's filed about the
> > integrity issues where the in memory version of the dataset gets
> > confused and has profiles referencing other profiles that don't exist
> > any more.
>  Then how were these issues resolved for your (i assume) paying customers?
>  
My paying customer is ourselves. :-)  Its part of our test system which 
does automatic provisioning of systems.  Right now I have a cobbler_fsck 
script which goes through the records and tries to fix things.  I'm not 
sure how the corruption is happening in the first place.

[Bill Peck]

On 09/21/2011 02:32 PM, Bill Peck wrote:
>  On 09/21/2011 01:44 PM, Jörgen Maas wrote:
> > On Tue, Sep 20, 2011 at 4:43 PM, Bill Peck<bpeck(a)redhat.com>   wrote:
> > <snip>
> >> We have bz's filed about the
> >> integrity issues where the in memory version of the dataset gets
> >> confused and has profiles referencing other profiles that don't exist
> >> any more.
> > Then how were these issues resolved for your (i assume) paying customers?
> >
>  My paying customer is ourselves. :-)  Its part of our test system which
>  does automatic provisioning of systems.  Right now I have a cobbler_fsck
>  script which goes through the records and tries to fix things.  I'm not
>  sure how the corruption is happening in the first place.
>  _______________________________________________
>  cobbler-devel mailing list
>  cobbler-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler-devel 
I should clarify that I'm not with the group that does Satellite.

[brett lentz]

On Wed, Sep 21, 2011 at 2:32 PM, Bill Peck <bpeck(a)redhat.com&gt; wrote:
>  On 09/21/2011 01:44 PM, Jörgen Maas wrote:
> > On Tue, Sep 20, 2011 at 4:43 PM, Bill Peck&lt;bpeck(a)redhat.com&gt;  wrote:
> > <snip>
> >> We have bz's filed about the
> >> integrity issues where the in memory version of the dataset gets
> >> confused and has profiles referencing other profiles that don't exist
> >> any more.
> > Then how were these issues resolved for your (i assume) paying customers?
> >
>
>  My paying customer is ourselves. :-)  Its part of our test system which
>  does automatic provisioning of systems.  Right now I have a cobbler_fsck
>  script which goes through the records and tries to fix things.  I'm not
>  sure how the corruption is happening in the first place. 
At my previous employer, we had a cobbler install that had been around
since the 1.x era, but had been upgraded all the way to 2.0.11. There
was a ton of cruft and broken json files that were causing us all
sorts of grief due to various bugs that have (afaik) since been fixed
in the latest version of Cobbler.

What I ended up doing was manually fixing the offending json files.
Once they had corrected (and current) syntax, all of the weirdness
disappeared.

Perhaps you're having a similar issue?

---Brett.

[Corey Osman]

I meant to comment on this a while ago.  I am in the midst of switching completely away
from cobbler to theforeman.  I have been using both (cobbler for provisioning, foreman for
puppet stuff) and now I just want a single cohesive application.  I don't think
cobbler is dead but it just serves a particular use case where a shop needs an easy to
setup, well documented, provisioning management system.  Foreman on the other hand is just
one piece ( large piece) of the overall puzzle, cobbler could never have solved on its
own.  I think we all should embrace foreman with open arms as it solves many of the
problems cobbler could not.  Furthermore, foreman needs our ideas and experience since
foreman is trying to solve similar problems that cobbler solved years ago.  We just need
to translate that cobbler code into foreman code (python vs ruby/rails).  Cobbler is
awesome, but foreman does so much more.  I didn't really use the foreman provisioning
piece till a few weeks ago and am happy that it can adjust to my network of twenty really
slow WAN links.  Ever used cobbler across a 1.5 Mbps WAN?  Foreman has a completely
different approach to this problem and does it well.

Additionally, I like foreman so much that I built a mobile app around it that can
essentially control an entire datacenter.  Check it out:  http://www.remoteadmin.co


-- cobbler user since version 0.3


Corey Osman

On Sep 20, 2011, at 7:43 AM, Bill Peck wrote:

>  On 09/20/2011 09:56 AM, brett lentz wrote:
> > On Tue, Sep 20, 2011 at 8:43 AM, Bill Peck<bpeck(a)redhat.com>  wrote:
> >> On 09/19/2011 05:44 PM, Jonathan Underwood wrote:
> >>> On 19 September 2011 20:29, Jörgen Maas<jorgen.maas(a)gmail.com>   
> wrote:
> >>>> On Mon, Sep 19, 2011 at 9:19 PM, Konrad Scherer
> >>>> <konrad.scherer(a)windriver.com&gt;    wrote:
> >>>>> As a happy and grateful user of cobbler I have noticed that the
> cobbler project
> >>>>> is not as active as it used to be. Is it time for me to move on?
> >>>>> 
> >>>> Unfortunately I have come to the same conclusions.
> >>>> The cobbler project looks like it's dying a slow death.
> >>>> 
> >>>> So yes, I guess it's time to move on ;).
> >>>> 
> >>> I think if you were to look at the activity in the source code
> >>> repository, you might draw a different conclusion. Also, you might
> >>> want to look at this:
> >>> 
> >>> http://michaeldehaan.net/2011/05/14/about-that-scaling-thing/
> >> Unless I'm mistaken the solution Michael gives for the scaling issue is
> >> to use the backend couchDB.  The problem is that doesn't solve anything
> >> since it still loads *everything* into ram.  And that gets back to the
> >> real problem, cobbler wasn't designed to scale, simple as that.
> >> 
> > The whole "not designed to scale" comment, while a nice sound bite, is
> > meaningless and a bit of a red herring.
> > 
> > As the saying goes, "If you can't measure it, it's not science."
> The
> > same is true here. Performance is defined by a rate (i.e. some value
> > that changes with respect to some other value). Who cares if couchDB
> > loads everything into RAM? How many bytes does "everything" take up?
> > Is there enough RAM to hold it? Without any numbers, the statement is
> > pure FUD.
>  
>  Its not pure fud.  Slapping a DB on the backend just to use as a file 
>  store does nothing for you.  It doesn't enforce data integrity and it 
>  doesn't speed anything up either.  We have bz's filed about the 
>  integrity issues where the in memory version of the dataset gets 
>  confused and has profiles referencing other profiles that don't exist 
>  any more.  These errors cause cobbler to fall down quite badly, no other 
>  profile operations can continue because cobbler wants to re-write all 
>  the pxe records when you update one.
>  
>  We have hundreds of systems in cobbler which isn't much, but we do have 
>  thousands of distro/profile records and this is an area where cobbler 
>  doesn't scale.  cobbler sync takes almost 10 minutes to complete which 
>  may not seem like much but during that time you can't be sure systems 
>  will pxe boot correctly since a file or record may be missing.  This is 
>  better now that a .link_cache directory is used but then we have to 
>  worry about that directory having old stale data which needs to be cleaned.
>  
>  Don't get me wrong, cobbler has helped us tremendously but its reached 
>  its limit for us and the above issues have been resolved in foreman.  
>  I'm sure we will experience other pain points in foreman but at this 
>  point it looks like a more solid foundation to stand on.
>  
>  If cobbler works for you then great.
>  
> > None of the detractors has yet really mentioned any specific numbers
> > around what their requirements really are (or whether Cobbler meets
> > those requirements). Mike's post mentions a few cases of a few
> > installs running upwards of 20k systems. I'd be interested to know how
> > cobbler performs in those settings; how quickly it can serve a
> > rendered kickstart; how quickly it can lookup a particular system
> > record; how much time it takes an admin to work with the whole system.
> > 
> > I have my own gripes with Cobbler's UI and data model, especially with
> > respect to how it integrates with puppet. Personally, I think there is
> > enough room for improvement in those areas that vague hand-waving
> > about performance is unnecessary.
> > 
> > If there's a likely reason for why the Satellite folks are moving away
> > from Cobbler, it's probably less about performance and got more to do
> > with the fact that Foreman was built with integrating with Puppet/Chef
> > in mind. (That's just a guess.) In Cobbler, that integration is quite
> > obviously an after-thought (and a poorly implemented one at that).
> > Life of a system doesn't stop when the installer exits, yet Cobbler
> > doesn't seem to consider any of those areas. There's little to no
> > consideration given to issues of inventory, life cycle, or config
> > management.
> > 
> > If all you care about is installing an OS, Cobbler is a great tool. It
> > does that job very very well. The problem is, Satellite, Foreman, and
> > other similar tools are thinking about more than just OS installation.
> > 
> > ---Brett.
> > _______________________________________________
> > cobbler-devel mailing list
> > cobbler-devel(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/cobbler-devel
>  
>  _______________________________________________
>  cobbler-devel mailing list
>  cobbler-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler-devel