QA Client Network Isolation and FAS Groups

[Tim Flink]

On Fri, 10 Jan 2014 11:29:40 -0700
Kevin Fenzi <kevin at scrye.com> wrote:

> On Fri, 10 Jan 2014 11:09:16 -0700
> Tim Flink <tflink at redhat.com> wrote:
> 
> > One of the things that QA is focusing on ATM is getting better
> > automation support for Fedora. We're currently doing this in two
> > ways (that will eventually combine):
> > 
> >  - taskotron (replacing autoqa)
> >  - beaker (different type of system - used by Red Hat QE, who is
> >    interested in running some of their tests upstream in Fedora)
> > 
> > Before we start granting access to the systems to Fedora
> > contributors, we want to a) make the process relatively easy and b)
> > isolate the client machines so that if something goes wrong or a
> > bad actor gains access to them, any potential problems can be
> > limited.
> 
> Absolutely. :) 
> 
> > To do this, I'd like to propose one change and ask for input on
> > another.
> > 
> > The first change is with FAS groups and shell access. To the best of
> > my knowledge, in order to ssh into the bastion hosts, a user needs
> > to be a member of the sysadmin group. While this works well for
> > sysadmin people, I'm not sure I see a benefit of sending all
> > sysadmin email to folks who are just looking to debug beaker or
> > taskotron clients. I'd like to propose the creation of a new FAS
> > shell group for qa automation and separate groups for beaker users,
> > beaker admins, taskotron users and taskotron admins (qa-automation,
> > beaker, beaker-admin, taskotron, taskotron-admin).
> 
> So, it's a bit more complex than that. ;) 
> 
> All 'sysadmin-*' groups require you to be a member of 'sysadmin'
> first. This means if you are being added to 'sysadmin-qa' or
> something, you have to first get added to sysadmin. sysadmin is a
> tracking group, it provides no shell access to anything, but it does
> provide other things: 
> 
> * nagios alerts
> * all commits from puppet/ansible/etc repos via email
> * Can push commits to puppet/ansible/etc repos (but need to be in some
>   other group that provides them shell access to bastion and then
>   lockbox01 where the repos are). 
> 
> So, the idea is that anyone is a sysadmin subgroup gets all the info
> and commits and can see whats going on, etc. 

Ha, I'm pretty sure I've been corrected on this misunderstanding
before and forgot. Thanks for the reminder :)


> > The qa-automation group wouldn't need access to bastion01, just
> > whichever bastion host has access to the automation clients
> > (bastion-comm01 for now). The other groups would be kind of like the
> > relationship between the various sysadmin FAS groups and the base
> > sysadmin group (ie, sysadmin access for all members, sysadmin-dns,
> > sysadmin-qa etc. for farther access).
> > 
> > Does this seem reasonable and do-able? If so, I'd like to get the
> > FAS group stuff done in the relatively near future - not "drop
> > everything and do it now" but ideally in the next couple of weeks
> > if possible.
> 
> I have no objection. When we setup the sysadmin-qa group we made it
> like the rest of the sysadmin groups, but mostly that was due to me
> not being sure there wasn't something else going on with sysadmin
> group. :) 
> 
> I'm not sure you need admin and users groups seperate? Could you just
> have one group and make some people admins in the group and key off
> that? 

For the moment, it doesn't matter much beyond shell access because we
don't have FAS integration for either taskotron or beaker. There are
admin tasks for beaker that and there will be admin tasks for
taskotron. I'd prefer not to have more interfaces for group/permission
management than we need to have but as long as we can manage shell
access and base group membership to the two systems, we'll make it work.
 
> I think yeah, bastion-comm01 should mostly work fine, except if people
> in those groups need to commit to ansible/puppet/etc repos. Would
> they? Or would that be something we keep with sysadmin-qa?

The taskotron user and admin groups would be separate from any sysadmin
groups - I'd like to keep sysadmin-qa separate. I don't want to be
giving shell access to all our virthosts, write access to the infra
playbooks etc. to every user of beaker and taskotron who needs shell
access to the clients.

> > For network isolation, I don't pretend to be an expert on networking
> > so I'll describe the functionality that we're looking for and what I
> > think might work for a solution, but I'll defer to the expertise
> > here on whether it's a good idea or not :)
> > 
> > The beaker and taskotron clients will need network access to several
> > Fedora systems in order to work.
> > 
> > Taskotron Clients:
> >  - Taskotron buildmaster
> >  - bodhi, koji, repos, dist-git, task-git (part of taskotron, not
> > yet created), resultsdb (also part of taskotron)
> > 
> > Beaker Clients:
> >  - Beaker server and lab controller (same system for now)
> >  - repos, maybe grabbing packages from koji/bodhi
> > 
> > I put together a quick diagram with the various network connections:
> > http://tflink.fedorapeople.org/taskotron/client-network-connections.png
> > 
> > From a few previous conversations, I think that a private network
> > for the clients could provide the isolation that we're looking for.
> > As far as getting network access to the systems needed to function,
> > I figured that the beaker server and taskotron master would have
> > network interfaces on this private network and a gateway could be
> > used to restrict outgoing traffic to only the resources required.
> > 
> > All of the clients would be hosted on the qa virthosts, which are
> > currently in the same rack. I was thinking that it would be possible
> > to use one of the network interfaces in these virthosts to create
> > this private network (assuming that the network switch capacity is
> > available) but I'm definitely open to other ideas.
> > 
> > Does this idea for network access and isolation seem reasonable and
> > do-able? I figure that the network isolation/access part will
> > require more discussion and time for implementation after a
> > decision is reached. Our systems will work fine with the current
> > network configuration but I wanted to get this part of the
> > conversation started so that the implementation could happen before
> > we get too far with automation development.
> 
> Let me ponder on this part... part of the problem is that we do not
> control switches or routers. We have to ask RHIT about those... so we
> can't just do things and it's always better when we do something that
> requires them to not have to do anything. ;) 

No worries, I didn't think that any networking changes would be
immediate. I don't have my heart set on that exact solution, either. It
just made the most sense to me. If there are easier alternatives that
accomplish the same goals of isolation, I'm game.

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20140110/d6f08f95/attachment.sig>