A How-to Question: Capturing a server configuration in command line form

[Dan White]

I am looking for a way to preserve a server configuration for disaster recovery purposes.

Is there a way to "dump" the contents/configuration of a cobbler server such that it can be used as command line input to rebuild/duplicate the server ?

The report output has all the necessary information, but it is not in the same form as it would be when being entered on the command line.

Thanks.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

[David Lee]

Dan White wrote:
> I am looking for a way to preserve a server configuration for disaster recovery purposes.
>
> Is there a way to "dump" the contents/configuration of a cobbler server such that it can be used as command line input to rebuild/duplicate the server ?
>
> The report output has all the necessary information, but it is not in the same form as it would be when being entered on the command line.
>
> Thanks.
> [...]

We're new to cobbler, but are developing something resembling this 
(using 2.0.11 at present).

But first a little detour into how we might view Disaster Recovery (DR).

<detour>

People often think of DR as having a live service in one location and 
something cold (possibly even powered off) and gathering dust until the 
fateful day, in a different physical location.

Certainly you need both locations!

But I would suggest a possible variant view of "live service" for 
consideration.  Try the question: "When the fateful days arrives, how 
confident am I that I can construct the service, in all its inevitable 
complexity, on that cold system?"  Or, even worse, include the thought 
"...and I was caught in the disaster, therefore I am unavailable, and 
someone else will have to restore...".

Why not have your service running live, day-to-day, and distributed 
across both locations.  Then you KNOW that your DR location basically 
works.  There may still be other subtle trouble when the main site goes 
away; this can only ever be verified by regular testing.  But at least 
you'll have some confidence that the copy of the server itself in the DR 
location has been delivering live, good service.

</detour>

So don't think in terms of setting up a cobbler server.  Rather consider 
setting up a cobbler service comprising two (or more) actual servers, 
and day-by-day keeping those server components in step with each other, 
and configuring clients to use them both.  (Or if you have to have a 
single IP, then regularly switch that day-by-day hosting-IP of your 
service between the servers.)

The basic technology to keep multiple cobbler servers in step is 
"cobbler replicate".

(And we're supplementing that with a "configuration management" tool 
which itself configures and maintains those multiple cobbler servers. 
But that detail is outside scope of the cobbler-specific question.)

Hope that helps.

(Despite my appearing to give advice, I would also welcome advice as our 
own group ourselves are relative newbies as we embarking on this.)

[James Cammarata]

On Wed, Oct 19, 2011 at 3:54 AM, David Lee  wrote:
> Dan White wrote:
> > I am looking for a way to preserve a server configuration for disaster recovery purposes.
> >
> > Is there a way to "dump" the contents/configuration of a cobbler server such that it can be used as command line input to rebuild/duplicate the server ?
> >
> > The report output has all the necessary information, but it is not in the same form as it would be when being entered on the command line.
> >
> > Thanks.
> > [...]
>
> We're new to cobbler, but are developing something resembling this
> (using 2.0.11 at present).
>
> But first a little detour into how we might view Disaster Recovery (DR).
>
> <detour>
> </detour>
>
> So don't think in terms of setting up a cobbler server.  Rather consider
> setting up a cobbler service comprising two (or more) actual servers,
> and day-by-day keeping those server components in step with each other,
> and configuring clients to use them both.  (Or if you have to have a
> single IP, then regularly switch that day-by-day hosting-IP of your
> service between the servers.)
>
> The basic technology to keep multiple cobbler servers in step is
> "cobbler replicate".
>
> (And we're supplementing that with a "configuration management" tool
> which itself configures and maintains those multiple cobbler servers.
> But that detail is outside scope of the cobbler-specific question.)

Cobbler replicate is definitely the way to go, or if you use some kind
of SAN replication like SRDF you can store all of your data on a
replicated LUN.

[Dan White]

I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.

Such info could be used to make a separate server for a different group for a different project by tweaking parameters.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- James Cammarata  wrote:
> Cobbler replicate is definitely the way to go, or if you use some kind
> of SAN replication like SRDF you can store all of your data on a
> replicated LUN.

[Harry Hoffman]

While maybe not "clonable" when building a brand new cobbler server I 
use root's .bash_history to guide me through what I've done.

For every config file I edit I always issue the command "cp $file 
$file-`date -I`" so I have the original to diff against to know exactly 
what changes were made.

Once done building the new cobbler server I take the history file, 
remove all of the things that are uneccesary (ls; pwd; etc) and put full 
paths on file and commands I've used the write all of this out to our wiki.

It works pretty well even when you don't get to document for days or 
weeks after you've finished the build.

Cheers,
Harry

On 10/19/2011 09:00 AM, Dan White wrote:
> I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.
>
> Such info could be used to make a separate server for a different group for a different project by tweaking parameters.
>
> “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin&  Hobbes)
>
> ----- James Cammarata  wrote:
> > Cobbler replicate is definitely the way to go, or if you use some kind
> > of SAN replication like SRDF you can store all of your data on a
> > replicated LUN.
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[James Cammarata]

On Wed, Oct 19, 2011 at 8:21 AM, Harry Hoffman
 wrote:
> While maybe not "clonable" when building a brand new cobbler server I
> use root's .bash_history to guide me through what I've done.
>
> For every config file I edit I always issue the command "cp $file
> $file-`date -I`" so I have the original to diff against to know exactly
> what changes were made.
>
> Once done building the new cobbler server I take the history file,
> remove all of the things that are uneccesary (ls; pwd; etc) and put full
> paths on file and commands I've used the write all of this out to our wiki.
>
> It works pretty well even when you don't get to document for days or
> weeks after you've finished the build.

I'd recommend looking into a configuration management system like
puppet. You can lay down a consistent configuration very quickly using
that.

[Harry Hoffman]

Sure, I use cfengine... but it's a chicken and egg game and you have to 
start somewhere.

The setup I describe is when there are no other machines and I'm walking 
in to build something from the ground up.

Cheers,
Harry

On 10/19/2011 09:28 AM, James Cammarata wrote:
> On Wed, Oct 19, 2011 at 8:21 AM, Harry Hoffman
>   wrote:
> > While maybe not "clonable" when building a brand new cobbler server I
> > use root's .bash_history to guide me through what I've done.
> >
> > For every config file I edit I always issue the command "cp $file
> > $file-`date -I`" so I have the original to diff against to know exactly
> > what changes were made.
> >
> > Once done building the new cobbler server I take the history file,
> > remove all of the things that are uneccesary (ls; pwd; etc) and put full
> > paths on file and commands I've used the write all of this out to our wiki.
> >
> > It works pretty well even when you don't get to document for days or
> > weeks after you've finished the build.
>
> I'd recommend looking into a configuration management system like
> puppet. You can lay down a consistent configuration very quickly using
> that.
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Dan White]

I am using puppet, but I use cobbler for PXE-kickstarting

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- James Cammarata  wrote:
> I'd recommend looking into a configuration management system like
> puppet. You can lay down a consistent configuration very quickly using
> that.
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Dan White]

All my config files are in Subversion, but I want to capture the commands to set up the distros, repos, profiles, and systems

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- Harry Hoffman  wrote:
> While maybe not "clonable" when building a brand new cobbler server I 
> use root's .bash_history to guide me through what I've done.
>
> For every config file I edit I always issue the command "cp $file 
> $file-`date -I`" so I have the original to diff against to know exactly 
> what changes were made.
>
> Once done building the new cobbler server I take the history file, 
> remove all of the things that are uneccesary (ls; pwd; etc) and put full 
> paths on file and commands I've used the write all of this out to our wiki.
>
> It works pretty well even when you don't get to document for days or 
> weeks after you've finished the build.
>
> Cheers,
> Harry
>
> On 10/19/2011 09:00 AM, Dan White wrote:
> > I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.
> >
> > Such info could be used to make a separate server for a different group for a different project by tweaking parameters.
> >
> > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > Bill Waterson (Calvin&  Hobbes)
> >
> > ----- James Cammarata  wrote:
> > > Cobbler replicate is definitely the way to go, or if you use some kind
> > > of SAN replication like SRDF you can store all of your data on a
> > > replicated LUN.
> >
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Harry Hoffman]

Sure, you can get all of that from the "history" command.

On 10/19/2011 10:34 AM, Dan White wrote:
> All my config files are in Subversion, but I want to capture the commands to set up the distros, repos, profiles, and systems
>
> “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin&  Hobbes)
>
> ----- Harry Hoffman  wrote:
> > While maybe not "clonable" when building a brand new cobbler server I
> > use root's .bash_history to guide me through what I've done.
> >
> > For every config file I edit I always issue the command "cp $file
> > $file-`date -I`" so I have the original to diff against to know exactly
> > what changes were made.
> >
> > Once done building the new cobbler server I take the history file,
> > remove all of the things that are uneccesary (ls; pwd; etc) and put full
> > paths on file and commands I've used the write all of this out to our wiki.
> >
> > It works pretty well even when you don't get to document for days or
> > weeks after you've finished the build.
> >
> > Cheers,
> > Harry
> >
> > On 10/19/2011 09:00 AM, Dan White wrote:
> > > I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.
> > >
> > > Such info could be used to make a separate server for a different group for a different project by tweaking parameters.
> > >
> > > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > > Bill Waterson (Calvin&   Hobbes)
> > >
> > > ----- James Cammarata   wrote:
> > > > Cobbler replicate is definitely the way to go, or if you use some kind
> > > > of SAN replication like SRDF you can store all of your data on a
> > > > replicated LUN.
> > >
> > > cobbler mailing list
> > > [e-mail]
> > > https://fedorahosted.org/mailman/listinfo/cobbler
> >
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Dan White]

Yes, provided that I did it from the command line.
Silly me, I used the web interface for some of it and THAT's the part I am trying to recapture.

I may have to just try on a scratch machine.

I appreciate all the responses I am getting on this.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- Harry Hoffman  wrote:
> Sure, you can get all of that from the "history" command.
>
> On 10/19/2011 10:34 AM, Dan White wrote:
> > All my config files are in Subversion, but I want to capture the commands to set up the distros, repos, profiles, and systems
> >
> > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > Bill Waterson (Calvin&  Hobbes)
> >
> > ----- Harry Hoffman  wrote:
> > > While maybe not "clonable" when building a brand new cobbler server I
> > > use root's .bash_history to guide me through what I've done.
> > >
> > > For every config file I edit I always issue the command "cp $file
> > > $file-`date -I`" so I have the original to diff against to know exactly
> > > what changes were made.
> > >
> > > Once done building the new cobbler server I take the history file,
> > > remove all of the things that are uneccesary (ls; pwd; etc) and put full
> > > paths on file and commands I've used the write all of this out to our wiki.
> > >
> > > It works pretty well even when you don't get to document for days or
> > > weeks after you've finished the build.
> > >
> > > Cheers,
> > > Harry
> > >
> > > On 10/19/2011 09:00 AM, Dan White wrote:
> > > > I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.
> > > >
> > > > Such info could be used to make a separate server for a different group for a different project by tweaking parameters.
> > > >
> > > > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > > > Bill Waterson (Calvin&   Hobbes)
> > > >
> > > > ----- James Cammarata   wrote:
> > > > > Cobbler replicate is definitely the way to go, or if you use some kind
> > > > > of SAN replication like SRDF you can store all of your data on a
> > > > > replicated LUN.
> > > >
> > > > cobbler mailing list
> > > > [e-mail]
> > > > https://fedorahosted.org/mailman/listinfo/cobbler
> > >
> > > cobbler mailing list
> > > [e-mail]
> > > https://fedorahosted.org/mailman/listinfo/cobbler
> >
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Harry Hoffman]

Oh, either I missed that part or you failed to mention it, LOL.

On 10/19/2011 10:59 AM, Dan White wrote:
> Yes, provided that I did it from the command line.
> Silly me, I used the web interface for some of it and THAT's the part I am trying to recapture.
>
> I may have to just try on a scratch machine.
>
> I appreciate all the responses I am getting on this.
>
> “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin&  Hobbes)
>
> ----- Harry Hoffman  wrote:
> > Sure, you can get all of that from the "history" command.
> >
> > On 10/19/2011 10:34 AM, Dan White wrote:
> > > All my config files are in Subversion, but I want to capture the commands to set up the distros, repos, profiles, and systems
> > >
> > > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > > Bill Waterson (Calvin&   Hobbes)
> > >
> > > ----- Harry Hoffman   wrote:
> > > > While maybe not "clonable" when building a brand new cobbler server I
> > > > use root's .bash_history to guide me through what I've done.
> > > >
> > > > For every config file I edit I always issue the command "cp $file
> > > > $file-`date -I`" so I have the original to diff against to know exactly
> > > > what changes were made.
> > > >
> > > > Once done building the new cobbler server I take the history file,
> > > > remove all of the things that are uneccesary (ls; pwd; etc) and put full
> > > > paths on file and commands I've used the write all of this out to our wiki.
> > > >
> > > > It works pretty well even when you don't get to document for days or
> > > > weeks after you've finished the build.
> > > >
> > > > Cheers,
> > > > Harry
> > > >
> > > > On 10/19/2011 09:00 AM, Dan White wrote:
> > > > > I agree, replicate is a great way to "clone" the existing server, but I am tasked to record the steps it took to create the server such that someone else could do it.
> > > > >
> > > > > Such info could be used to make a separate server for a different group for a different project by tweaking parameters.
> > > > >
> > > > > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
> > > > > Bill Waterson (Calvin&    Hobbes)
> > > > >
> > > > > ----- James Cammarata    wrote:
> > > > > > Cobbler replicate is definitely the way to go, or if you use some kind
> > > > > > of SAN replication like SRDF you can store all of your data on a
> > > > > > replicated LUN.
> > > > >
> > > > > cobbler mailing list
> > > > > [e-mail]
> > > > > https://fedorahosted.org/mailman/listinfo/cobbler
> > > >
> > > > cobbler mailing list
> > > > [e-mail]
> > > > https://fedorahosted.org/mailman/listinfo/cobbler
> > >
> > > cobbler mailing list
> > > [e-mail]
> > > https://fedorahosted.org/mailman/listinfo/cobbler
> >
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Jennings, Jared L CTR USAF AFMC 46 SK/CCI]

Dan, I've never used the web UI, but when I make changes using the cobbler command, I see changes to JSON files stored under /var/lib/cobbler. For profiles and systems, I've pretty much been able to just track those changes with SVN and be fine. (Diffs would mean more if the json were not all on one line...) Messing with distros and repos, though, also touches /var/www/cobbler. I don't have any better thing going to duplicate what I did with those than .bash_history.

From this experience I can tell you that cobbler gets frustrated fast when some of the profiles refer to distros that don't exist.

There is or used to be some Cobbler feature that would automatically check in changes to a version control system when you change stuff in Cobbler, but it appeared more suited to Git than Subversion, and I don't think it's prepared to tell Subversion about my smartcard.
> -----Original Message-----
> From: [e-mail] [mailto:cobbler-
> [e-mail]] On Behalf Of Dan White
> Sent: Wednesday, October 19, 2011 9:59 AM
> To: cobbler mailing list
> Subject: Re: A How-to Question: Capturing a server configuration in
> commandline form
>
> Yes, provided that I did it from the command line.
> Silly me, I used the web interface for some of it and THAT's the part I
> am trying to recapture.
>
> I may have to just try on a scratch machine.
>
> I appreciate all the responses I am getting on this.
>
> “Sometimes I think the surest sign that intelligent life exists
> elsewhere in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin & Hobbes)
>
> ----- Harry Hoffman  wrote:
> > Sure, you can get all of that from the "history" command.
> >
> > On 10/19/2011 10:34 AM, Dan White wrote:
> > > All my config files are in Subversion, but I want to capture the
>
> commands to set up the distros, repos, profiles, and systems
> > > “Sometimes I think the surest sign that intelligent life exists
>
> elsewhere in the universe is that none of it has tried to contact us.”
> > > Bill Waterson (Calvin&  Hobbes)
> > >
> > > ----- Harry Hoffman  wrote:
> > > > While maybe not "clonable" when building a brand new cobbler
>
> server I
> > > > use root's .bash_history to guide me through what I've done.
> > > >
> > > > For every config file I edit I always issue the command "cp $file
> > > > $file-`date -I`" so I have the original to diff against to know
>
> exactly
> > > > what changes were made.
> > > >
> > > > Once done building the new cobbler server I take the history file,
> > > > remove all of the things that are uneccesary (ls; pwd; etc) and
>
> put full
> > > > paths on file and commands I've used the write all of this out to
>
> our wiki.
> > > > It works pretty well even when you don't get to document for days
>
> or
> > > > weeks after you've finished the build.
> > > >
> > > > Cheers,
> > > > Harry
> > > >
> > > > On 10/19/2011 09:00 AM, Dan White wrote:
> > > > > I agree, replicate is a great way to "clone" the existing server,
>
> but I am tasked to record the steps it took to create the server such
> that someone else could do it.
> > > > > Such info could be used to make a separate server for a different
>
> group for a different project by tweaking parameters.
> > > > > “Sometimes I think the surest sign that intelligent life exists
>
> elsewhere in the universe is that none of it has tried to contact us.”
> > > > > Bill Waterson (Calvin&   Hobbes)
> > > > >
> > > > > ----- James Cammarata   wrote:
> > > > > > Cobbler replicate is definitely the way to go, or if you use
>
> some kind
> > > > > > of SAN replication like SRDF you can store all of your data on a
> > > > > > replicated LUN.
> > > > >
> > > > > cobbler mailing list
> > > > > [e-mail]
> > > > > https://fedorahosted.org/mailman/listinfo/cobbler
> > > >
> > > > cobbler mailing list
> > > > [e-mail]
> > > > https://fedorahosted.org/mailman/listinfo/cobbler
> > >
> > > cobbler mailing list
> > > [e-mail]
> > > https://fedorahosted.org/mailman/listinfo/cobbler
> >
> > cobbler mailing list
> > [e-mail]
> > https://fedorahosted.org/mailman/listinfo/cobbler
>
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler

[Dan White]

Thanks for the reply.
You make a very valid point.

However, part of my tasking is to document what I did to set up the server and provide sufficient information that the process can be repeated -- possibly by another group for another collection of machines.

I was hoping to reduce the server setup to a series of command line inputs and then preserve the whole thing in a Subversion repository.

As I said, all the info I believe I neeed comes out in a report, but I would perfer to avoid the painful trial-and-error process of mapping the report to the command line, especially if someone else has already done this.  I am an enthusiastic advocate of Avoiding the Re-Invention of the Wheel.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- David Lee  wrote:
> Dan White wrote:
> > I am looking for a way to preserve a server configuration for disaster recovery purposes.
> >
> > Is there a way to "dump" the contents/configuration of a cobbler server such that it can be used as command line input to rebuild/duplicate the server ?
> >
> > The report output has all the necessary information, but it is not in the same form as it would be when being entered on the command line.
> >
> > Thanks.
> > [...]
>
> We're new to cobbler, but are developing something resembling this 
> (using 2.0.11 at present).
>
> But first a little detour into how we might view Disaster Recovery (DR).
>
> <detour>
>
> People often think of DR as having a live service in one location and 
> something cold (possibly even powered off) and gathering dust until the 
> fateful day, in a different physical location.
>
> Certainly you need both locations!
>
> But I would suggest a possible variant view of "live service" for 
> consideration.  Try the question: "When the fateful days arrives, how 
> confident am I that I can construct the service, in all its inevitable 
> complexity, on that cold system?"  Or, even worse, include the thought 
> "...and I was caught in the disaster, therefore I am unavailable, and 
> someone else will have to restore...".
>
> Why not have your service running live, day-to-day, and distributed 
> across both locations.  Then you KNOW that your DR location basically 
> works.  There may still be other subtle trouble when the main site goes 
> away; this can only ever be verified by regular testing.  But at least 
> you'll have some confidence that the copy of the server itself in the DR 
> location has been delivering live, good service.
>
> </detour>
>
> So don't think in terms of setting up a cobbler server.  Rather consider 
> setting up a cobbler service comprising two (or more) actual servers, 
> and day-by-day keeping those server components in step with each other, 
> and configuring clients to use them both.  (Or if you have to have a 
> single IP, then regularly switch that day-by-day hosting-IP of your 
> service between the servers.)
>
> The basic technology to keep multiple cobbler servers in step is 
> "cobbler replicate".
>
> (And we're supplementing that with a "configuration management" tool 
> which itself configures and maintains those multiple cobbler servers. 
> But that detail is outside scope of the cobbler-specific question.)
>
> Hope that helps.
>
> (Despite my appearing to give advice, I would also welcome advice as our 
> own group ourselves are relative newbies as we embarking on this.)
>
> -- 
> : David Lee
> : ECMWF (Data Handling System)
> : Shinfield Park
> : Reading  RG2 9AX
> : Berkshire
> :
> : tel:    [phone]
> : email:  [e-mail]
> _______________________________________________
> cobbler mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/cobbler