proposal: webserver/site configs in puppet

[Seth Vidal]

Hi folks,
 A problem I've been having recently is how we configure/maintain our
webserver configs in puppet. Right now we use a common class that has
definitions for a all the common functions/setups we use for our apache
setups. It's good from a programmatic code-reuse standpoint to make sure
we're not having to make N edits all over the place. It's bad b/c it
makes it next to impossible to know that when you edit

httpd::proxy in modules/httpd/manifests/init.pp that you're going to
impact the following systems:
 proxy*
 puppet*
 collab*
 secondary*

Since we run so many different kinds of websites and types of website
services I'm going suggest we stop thinking of 'httpd' as the base layer
and start thinking of the name of website itself as the base layer.

so instead of 'httpd' the module you'd care about would be:

'infrastructure.fedoraproject.org'

or

'proxy.fedoraproject.org'

etc, etc

The advantage is - if I want to modify infrastructure - I don't have to
worry if my modification will change things on other systems I'm not
aware of. It lets people make changes quickly, safe and confident that
they are only modifying the site they think they are modifying.

The disadvantage is we may have to make certain kinds of changes in a
number of places when we want to make a change.

I personally, think I'm better at running git-grep to know where else
has the same config than I am at parsing puppet configs in my head to
know what is or is not actually using a specific import.

I'm not sold on how this dir structure should be setup, yet and I'm
curious for some feed back.

thoughts?

-sv

[Tristan Santore]

On 08/06/11 05:49, seth vidal wrote:
> Hi folks,
>  A problem I've been having recently is how we configure/maintain our
> webserver configs in puppet. Right now we use a common class that has
> definitions for a all the common functions/setups we use for our apache
> setups. It's good from a programmatic code-reuse standpoint to make sure
> we're not having to make N edits all over the place. It's bad b/c it
> makes it next to impossible to know that when you edit
>
> httpd::proxy in modules/httpd/manifests/init.pp that you're going to
> impact the following systems:
>  proxy*
>  puppet*
>  collab*
>  secondary*
>
> Since we run so many different kinds of websites and types of website
> services I'm going suggest we stop thinking of 'httpd' as the base layer
> and start thinking of the name of website itself as the base layer.
>
> so instead of 'httpd' the module you'd care about would be:
>
> 'infrastructure.fedoraproject.org'
>
> or
>
> 'proxy.fedoraproject.org'
>
> etc, etc
>
> The advantage is - if I want to modify infrastructure - I don't have to
> worry if my modification will change things on other systems I'm not
> aware of. It lets people make changes quickly, safe and confident that
> they are only modifying the site they think they are modifying.
>
> The disadvantage is we may have to make certain kinds of changes in a
> number of places when we want to make a change.
>
> I personally, think I'm better at running git-grep to know where else
> has the same config than I am at parsing puppet configs in my head to
> know what is or is not actually using a specific import.
>
> I'm not sold on how this dir structure should be setup, yet and I'm
> curious for some feed back.
>
> thoughts?
>
> -sv
>
> infrastructure mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Seth,

that is what subclasses should be used for imho. Id keep httpd as http
baseclass, but only have the usual stuff in there like service,
packages, etc.., maybe some base config, which then gets amended through
augeas or even through overriding the actual config files to use. Maybe
even use site specific definitions, which could also be used as exported
resources, and then a change in a site domain.com would propagate to
proxy01 automagically.

Then I would realize resources as required.
I haven't kept track through as to exactly gets amended when changes are
pushed, in terms of the httpd side.

Regards,

Tristan

P.s.: But obviously this always depends on the situation and setup.
Probably a million other ways to do this.

[Adam M. Dutko]

It seems like the Apache "Include" directive is being overlooked.
Can't we have a base http/https file with essential "global
directives" like ServerTokens, some (not all) modules and etc. then
maintain a seperate set of include files for each site that has any
needed changes?

We could then use puppet to change which files are included on a per
host basis, no?

[Seth Vidal]

On Wed, 2011-06-08 at 08:29 -0400, Adam M. Dutko wrote:
> It seems like the Apache "Include" directive is being overlooked.
> Can't we have a base http/https file with essential "global
> directives" like ServerTokens, some (not all) modules and etc. then
> maintain a seperate set of include files for each site that has any
> needed changes?
>
> We could then use puppet to change which files are included on a per
> host basis, no?
> On Wed, 2011-06-08 at 19:24 +0530, ranjib dey wrote:
> hi seth,
> > I prefer to maintain the whole apache configs in debian style. Where
>
> I have base apache module for minimal setup + sites-available +
> mods-available, and individual use cases (passenger /mod_wsgi etc or
> virtualhosts/proxypasses) are modelled as dependant modules of apache
> where they selectively enable/disable sites or mods. Individual mods
> are loaded by separate .load file and configured by separate .conf
> files. Also we have separate puppet sub modules for individual mods.
> This way i can readily assemble my node specific apache configs
> without tampering the base puppet modules even if they need fin
> grained customizations.
> >

Wanted to reply to both of these. I think this is what I'm aiming for.

I suspect our base apache configuration on all of our hosts of a single
distro ver are shockingly similar. However, I do not believe the base
apache config file that ships in rhel is limited enough.

So we'll probably need a minimal apache config and then a bunch of .conf
files included in. Which is, ostensibly, what we do now, but we've
abstracted that 2 or 3 layers more which is where the complexity is
coming from.

so I think I'm in favor of not using the classes (or subclasses as that
just means we end up going deeper into puppetisms when, realistically,
we should be doing more apache-isms since that is what we're actually
running)

If we have:

httpd.conf
  and a simple site-include directive in puppet to pull in a dir/.conf
file.

this makes it similar to how we lay things out on the systems now - but
makes our puppet configs look more like what things do look like on the
system. :)

thinking on this some more - seems like working on this for the rhel6
change over for our various servers is a good idea. Instead of trying to
force this back onto rhel5.

-sv

[Seth Vidal]

On Wed, 2011-06-08 at 12:44 -0400, seth vidal wrote:
> On Wed, 2011-06-08 at 08:29 -0400, Adam M. Dutko wrote:
> > It seems like the Apache "Include" directive is being overlooked.
> > Can't we have a base http/https file with essential "global
> > directives" like ServerTokens, some (not all) modules and etc. then
> > maintain a seperate set of include files for each site that has any
> > needed changes?
> >
> > We could then use puppet to change which files are included on a per
> > host basis, no?
> > On Wed, 2011-06-08 at 19:24 +0530, ranjib dey wrote:
> > hi seth,
> > > I prefer to maintain the whole apache configs in debian style. Where
> >
> > I have base apache module for minimal setup + sites-available +
> > mods-available, and individual use cases (passenger /mod_wsgi etc or
> > virtualhosts/proxypasses) are modelled as dependant modules of apache
> > where they selectively enable/disable sites or mods. Individual mods
> > are loaded by separate .load file and configured by separate .conf
> > files. Also we have separate puppet sub modules for individual mods.
> > This way i can readily assemble my node specific apache configs
> > without tampering the base puppet modules even if they need fin
> > grained customizations.
> > >
>
> Wanted to reply to both of these. I think this is what I'm aiming for.
>
> I suspect our base apache configuration on all of our hosts of a single
> distro ver are shockingly similar. However, I do not believe the base
> apache config file that ships in rhel is limited enough.
>
> So we'll probably need a minimal apache config and then a bunch of .conf
> files included in. Which is, ostensibly, what we do now, but we've
> abstracted that 2 or 3 layers more which is where the complexity is
> coming from.
>
> so I think I'm in favor of not using the classes (or subclasses as that
> just means we end up going deeper into puppetisms when, realistically,
> we should be doing more apache-isms since that is what we're actually
> running)
>
> If we have:
>
> httpd.conf
>   and a simple site-include directive in puppet to pull in a dir/.conf
> file.
>
> this makes it similar to how we lay things out on the systems now - but
> makes our puppet configs look more like what things do look like on the
> system. :)
>
> thinking on this some more - seems like working on this for the rhel6
> change over for our various servers is a good idea. Instead of trying to
> force this back onto rhel5.

Talking on irc and looking at infrastructure.fedoraproject.org as a
prototype helped congeal some ideas, these are a  melange from myself
and other people - I'm not taking credit for them, just posting them.

1. we modify the default httpd.conf to modify this line:

Include conf.d/*.conf
  to
   Include fi-conf.d/*.conf

which is where we will store all of our site configs - this gets us
around the random welcome.conf issue and anyone else stepping on our
configs later.

It also means we need to make sure when we gather up things we may want
from pkgs we install that include a conf file into conf.d to actually be
included - but that's not much of a deal breaker I don't think.

2. we keep our current habit of:
    $website.fedoraproject.org.conf file and $website.fedoraproject.org
dir full of files.

3. we stop generating the items in #2 by templates.

4. under puppet/configs/httpd/websites/ we start housing the .conf files
and the dirs for items in #2.

5. those dirs and files are entirely (and recursively) deployed using a
simple module in puppet which just does that.

This means if we need to make an adjustment to a site we can make it to
JUST that site and not have to guess what else it might impact.

It also means if we want to make the same adjustment to several sites
that we may have to make it several times. But I believe in the power of
git-grep. :)

6. For ssl sites we will need to have a similar structure in private for
the keys/certs/etc - that will need some working out of details but it
is not terrible.

I'm going to work on a version of this for
infrastructure.fedoraproject.org and we'll go from there.

Thanks to StylusEater and mmcgrath for helping with their thoughts here.

-sv

[Ranjib Dey]

hi seth,
I prefer to maintain the whole apache configs in debian style. Where I have
base apache module for minimal setup + sites-available + mods-available, and
individual use cases (passenger /mod_wsgi etc or virtualhosts/proxypasses)
are modelled as dependant modules of apache where they selectively
enable/disable sites or mods. Individual mods are loaded by separate .load
file and configured by separate .conf files. Also we have separate puppet
sub modules for individual mods. This way i can readily assemble my node
specific apache configs without tampering the base puppet modules even if
they need fin grained customizations.

regards
ranjib

On Wed, Jun 8, 2011 at 10:19 AM, seth vidal wrote:
> Hi folks,
>  A problem I've been having recently is how we configure/maintain our
> webserver configs in puppet. Right now we use a common class that has
> definitions for a all the common functions/setups we use for our apache
> setups. It's good from a programmatic code-reuse standpoint to make sure
> we're not having to make N edits all over the place. It's bad b/c it
> makes it next to impossible to know that when you edit
>
> httpd::proxy in modules/httpd/manifests/init.pp that you're going to
> impact the following systems:
>  proxy*
>  puppet*
>  collab*
>  secondary*
>
> Since we run so many different kinds of websites and types of website
> services I'm going suggest we stop thinking of 'httpd' as the base layer
> and start thinking of the name of website itself as the base layer.
>
> so instead of 'httpd' the module you'd care about would be:
>
> 'infrastructure.fedoraproject.org'
>
> or
>
> 'proxy.fedoraproject.org'
>
> etc, etc
>
> The advantage is - if I want to modify infrastructure - I don't have to
> worry if my modification will change things on other systems I'm not
> aware of. It lets people make changes quickly, safe and confident that
> they are only modifying the site they think they are modifying.
>
> The disadvantage is we may have to make certain kinds of changes in a
> number of places when we want to make a change.
>
> I personally, think I'm better at running git-grep to know where else
> has the same config than I am at parsing puppet configs in my head to
> know what is or is not actually using a specific import.
>
> I'm not sold on how this dir structure should be setup, yet and I'm
> curious for some feed back.
>
> thoughts?
>
> -sv
>
> infrastructure mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure