Security testing: need for a security policy, and a security-critical package process

[Gene Czarcinski]

Although I have read all of the messages on this thread as of the date/time of 
this message, I am replying to this first message with all of my comments.

My background: I am currently retired but a few years ago I was still being 
paid the big bucks for working on computer security and security assessment of 
systems in US classified environments.

On the whole, I believe that Adam has outlined a good approach to the problem 
of doing QA on security for Fedora!

General comment:  I have read messages which claim that the approach is wrong 
and that we need to look at things that a user can do rather than what a user 
cannot do.  I disagree.  While the right approach for design/development is to 
assume that a user can do nothing except what they are specifically authorized 
to do, for security QA this needs to be turned around and the testing needs to 
demonstrate what a user cannot do.

On Monday 23 November 2009 17:08:31 Adam Williamson wrote:
> We can't do any meaningful security testing without knowing exactly what
> we should be testing for, in which packages. I believe Seth Vidal's
> upcoming proposal for covering 'major changes' may touch on this, but I
> doubt they'll cover exactly the same ground.
> 
> So, if we are to have meaningful security testing in future releases -
> which QA believes would be a good thing - we need the project to define
> a security policy. We believe there's a genuine need for this anyway, as
> the introduction and widespread adoption of PolicyKit will likely lead
> to much more complex and significant potential changes in security
> posture than any previous change.
> 
> It's not QA's role to define exactly what the security policy should
> look like or what it should cover, but from the point of view of
> testing, what we really need are concrete requirements. The policy does
> not have to be immediately comprehensive - try and cover every possible
> security-related issue - to be valuable. Something as simple as spot's
> proposed list of things an unprivileged user must not be able to do -
> http://spot.livejournal.com/312216.html - would serve a valuable purpose
> here.
+1

A written description of the security policy is a must!  Without it being 
written down in simple english (with translations as appropriate), there will 
be far too much subjective interpretation of what the policy is.  I believe 
spot's list is a good starting point for F13.  

However, the policy should consider how Fedora should work with respect to 
security and not how it does work as currently implemented.  For example, you 
cannot currently login as root from the gui (gdm) interface but you can login 
as root from a virtual terminal ... is this the way the system should work?

Keep it simple (KISS) for the initial attempt.  It will grow more complicated 
all by itself as time passes.

BTW, the security policy should assume that a grub password is in use so that 
a user cannot do something like disabling selinux by editing the kernel 
command line.  This should be tested by the security QA.

> 
> The second thing QA would require, aside from a policy with concrete and
> testable requirements, is a list of security-sensitive components to
> test. Obviously we couldn't test every package in the entire
> distribution for compliance with even such a simple list as spot's, and
> it would be a waste of time to try. 
+1

You definitely need to define a "reference implementation" that will be tested.  
Security assurance testing is done on "as-built" systems ... not "as 
designed"!  It may be possible but is not practical to test everything. [or 
will take so long that the release will no longer be supported]

Furthermore, I believe you should initially focus on a small subset of what is 
in Fedora (perhaps gnome only) and with a selected set of services (servers).

At this point in time, considering all of the various windows implementations 
(gnome, kde, openbox, xfce, etc.) will result in a lot of motion but little of 
it in a forward direction.  KISS!!!

...........
Given a written security policy for Fedora and a written description of the 
"reference implementation" that will be tested, these need to be vetted and 
"tuned" from comments.

After a reasonable amount of time, these documents/specifications should be 
approved by the Fedora Executive Committee (or whatever).  Any variation or 
change, should require additional approval.  There should be some independent 
oversight of the security QA process to minimize subjective 
(re)interpretation.

This will NOT make everyone happy.  Sorry, but there is only so much resources 
and you really do not want this to be a black hole which consumes everything 
else.

Start small, grow, and learn.  Two years from now, the security policy, the 
reference installation/configurations, and the QA process for securtiy will 
likely be very different.

Gene