Although I have read all of the messages on this thread as of the date/time of
this message, I am replying to this first message with all of my comments.

My background: I am currently retired but a few years ago I was still being
paid the big bucks for working on computer security and security assessment of
systems in US classified environments.

On the whole, I believe that Adam has outlined a good approach to the problem
of doing QA on security for Fedora!

General comment:  I have read messages which claim that the approach is wrong
and that we need to look at things that a user can do rather than what a user
cannot do.  I disagree.  While the right approach for design/development is to
assume that a user can do nothing except what they are specifically authorized
to do, for security QA this needs to be turned around and the testing needs to
demonstrate what a user cannot do.

On Monday 23 November 2009 17:08:31 Adam Williamson wrote:
> We can't do any meaningful security testing without knowing exactly what
> we should be testing for, in which packages. I believe Seth Vidal's
> upcoming proposal for covering 'major changes' may touch on this, but I
> doubt they'll cover exactly the same ground.
>
> So, if we are to have meaningful security testing in future releases -
> which QA believes would be a good thing - we need the project to define
> a security policy. We believe there's a genuine need for this anyway, as
> the introduction and widespread adoption of PolicyKit will likely lead
> to much more complex and significant potential changes in security
> posture than any previous change.
>
> It's not QA's role to define exactly what the security policy should
> look like or what it should cover, but from the point of view of
> testing, what we really need are concrete requirements. The policy does
> not have to be immediately comprehensive - try and cover every possible
> security-related issue - to be valuable. Something as simple as spot's
> proposed list of things an unprivileged user must not be able to do -
> http://spot.livejournal.com/312216.html - would serve a valuable purpose
> here.

+1

A written description of the security policy is a must!  Without it being
written down in simple english (with translations as appropriate), there will
be far too much subjective interpretation of what the policy is.  I believe
spot's list is a good starting point for F13.

However, the policy should consider how Fedora should work with respect to
security and not how it does work as currently implemented.  For example, you
cannot currently login as root from the gui (gdm) interface but you can login
as root from a virtual terminal ... is this the way the system should work?

Keep it simple (KISS) for the initial attempt.  It will grow more complicated
all by itself as time passes.

BTW, the security policy should assume that a grub password is in use so that
a user cannot do something like disabling selinux by editing the kernel
command line.  This should be tested by the security QA.

>
> The second thing QA would require, aside from a policy with concrete and
> testable requirements, is a list of security-sensitive components to
> test. Obviously we couldn't test every package in the entire
> distribution for compliance with even such a simple list as spot's, and
> it would be a waste of time to try.

+1

You definitely need to define a "reference implementation" that will be tested.
Security assurance testing is done on "as-built" systems ... not "as
designed"!  It may be possible but is not practical to test everything. [or
will take so long that the release will no longer be supported]

Furthermore, I believe you should initially focus on a small subset of what is
in Fedora (perhaps gnome only) and with a selected set of services (servers).

At this point in time, considering all of the various windows implementations
(gnome, kde, openbox, xfce, etc.) will result in a lot of motion but little of
it in a forward direction.  KISS!!!

...........
Given a written security policy for Fedora and a written description of the
"reference implementation" that will be tested, these need to be vetted and
"tuned" from comments.

After a reasonable amount of time, these documents/specifications should be
approved by the Fedora Executive Committee (or whatever).  Any variation or
change, should require additional approval.  There should be some independent
oversight of the security QA process to minimize subjective
(re)interpretation.

This will NOT make everyone happy.  Sorry, but there is only so much resources
and you really do not want this to be a black hole which consumes everything
else.

Start small, grow, and learn.  Two years from now, the security policy, the
reference installation/configurations, and the QA process for securtiy will
likely be very different.

Gene