SELinux testing

[James Morris]

Many packages are not being tested with SELinux enabled before release,
which is increasing the number of software defects being shipped, and
causing users to disable SELinux.  It is well documented that the cost of
fixing software defects rises dramatically the further from the source
they are experienced, and the worst possible case is after the software is
shipped and deployed.  For Fedora, these costs can be thought of in terms
of cost to the project's goodwill and cost of everyone's time dealing with
usually trivially addressable SELinux policy issues.

Given that SELinux is a core security feature of the OS, enabled by 
default, I'd like to propose that the Fedora project establishes a simple 
guideline for package maintainers in relation to SELinux testing.

This guideline would request that developers test their package with 
SELinux enabled (and by this I mean in enforcing mode) and follow a simple 
procedure:

1. Ensure they have the latest SELiunx policy installed.
2. Boot with selinux=1 and in enforcing mode.
3. Perform the normal testing of their application.
4. Check syslog (or /var/log/audit/audit.log if audit is enabled) for AVC 
   messages related to their package.

If there are any bugs or AVC messages:

5. Obtain support from the SELinux team.  The best way to do this I
believe is to file a bugzilla against the selinux-policy package.  They
should note that they are a Fedora packager (and expect a high priority
response).  If SELinux is running all or most of the time, issues will be
caught and fixed eariler in their dev cycle.

6. Don't release the package until the SELinux issue is resolved.

7. If for some reason, #2 is not possible, and the release of the package 
is important enough to warrant disabling a core security feature of the 
OS:

7a. Make a note of the bugzilla # from (1) in the rpm info, cvs commit and
release notes, with an explanation.  Also include a standardized
disclaimer in the rpm info which advises the user of the security risks
arising from disabling SELinux.  This should only happen in truly
exceptional cases.  I'm not sure how we can reliably notify users that
SELinux can be re-enabled again, and whether they'll tolerate the entire
fs being relabeled on reboot.  Really, this just should not happen.

Perhaps we can establish some kind of approval scheme for releasing a
package which requires SELinux to be disabled (or for that matter, any
security feature).



Thoughts?




-- 
James Morris
<jmorris(a)redhat.com&gt;

[James Morris]

On Fri, 8 Sep 2006, James Morris wrote:

>  6. Don't release the package until the SELinux issue is
> resolved.
>  
>  7. If for some reason, #2 is not possible, and the release of the package  
                         ^^^ that should be #6 

-- 
James Morris
<jmorris(a)redhat.com&gt;

[Jason L Tibbitts III]

> >>>> "JM" == James Morris
> <jmorris(a)redhat.com&gt; writes: 
JM> This guideline would request that developers test their package
JM> with SELinux enabled (and by this I mean in enforcing mode) and
JM> follow a simple procedure:

Just like the IPv6 thing, I don't think this is an appropriate topic
for the packaging committee to consider.

If it were in our purview, we could require that packages operate with
SELinux targeted enforcing, but forcing reviewers and package
maintainers to do this is a good way to make sure we have no package
maintainers or reviewers (except for the ones who are paid by Red Hat,
of course).  I mean, FC5 as shipped won't even boot in my environment
with SELinux turned on.  (Yes, I reported the problems and they were
quickly fixed, but that still doesn't get me a system I can boot to
the point of getting updates.)  So I think it's way too early to be
forcing people to test with SELinux on.

For Extras, an SELinux SIG would be great; they could go through and
test applications, probably the server ones first.  Core could of
course make their own policy.  It's not for the packaging committee to
dictate either of those policies.

Now, the packaging committee could publish guidelines for how to
include SELinux rules in a package; that would be great.

 - J<

[Greg DeKoenigsberg]

On Fri, 8 Sep 2006, Jason L Tibbitts III wrote:

>  >>>>> "JM" == James Morris
> <jmorris(a)redhat.com&gt; writes:
>  
>  JM> This guideline would request that developers test their package
>  JM> with SELinux enabled (and by this I mean in enforcing mode) and
>  JM> follow a simple procedure:
>  
>  Just like the IPv6 thing, I don't think this is an appropriate topic
>  for the packaging committee to consider.
>  
>  If it were in our purview, we could require that packages operate with
>  SELinux targeted enforcing, but forcing reviewers and package
>  maintainers to do this is a good way to make sure we have no package
>  maintainers or reviewers (except for the ones who are paid by Red Hat,
>  of course).   
A big +1 here.

We must *always* remember when working with community packagers: they do
this work to accomplish *their* needs.  The fact that they accomplish
*our* needs as well is almost always a fortunate side effect. 

>  I mean, FC5 as shipped won't even boot in my environment with
> SELinux
>  turned on.  (Yes, I reported the problems and they were quickly fixed,
>  but that still doesn't get me a system I can boot to the point of
>  getting updates.)  So I think it's way too early to be forcing people to
>  test with SELinux on.
>  
>  For Extras, an SELinux SIG would be great; they could go through and
>  test applications, probably the server ones first.  Core could of
>  course make their own policy.  It's not for the packaging committee to
>  dictate either of those policies. 
Another big +1.  The unfortunate side effect here is that it's possible -- 
even likely -- that most community packagers won't give two craps about 
the SELinux SIG.

>  Now, the packaging committee could publish guidelines for how to
>  include SELinux rules in a package; that would be great. 
+1 again.

--g

-------------------------------------------------------------
Greg DeKoenigsberg || Fedora Project || fedoraproject.org
Be an Ambassador || http://fedoraproject.org/wiki/Ambassadors
-------------------------------------------------------------

[Jason L Tibbitts III]

> >>>> "GD" == Greg DeKoenigsberg
> <gdk(a)redhat.com&gt; writes: 
GD> Another big +1.  The unfortunate side effect here is that it's
GD> possible -- even likely -- that most community packagers won't
GD> give two craps about the SELinux SIG.

Well, don't underestimate your reviewers.  I know enough to point out
some packages which will likely have selinux issues even though I am
not generally able to test them under selinux.

What has been a problem (for instance, in the Bugzilla review) is that
even given an SELinux policy to test, nobody involved could do the
testing.  I tried but I just ended up wasting a couple of hours on
relabels and ending up with a machine that needed to be reinstalled.
Fortunately I have plenty of machines around that I can hack on; many
reviewers and packagers aren't so blessed.

I note that the submitter/maintainer of the Extras Bugzilla package is
@redhat.com.

The review ticket is
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=188359

 - J<

[Nicolas Mailhot]

Le vendredi 08 septembre 2006 à 16:09 -0500, Jason L Tibbitts III a
écrit :
>   So I think it's way too early to be
>  forcing people to test with SELinux on. 
It's a vicious circle - if you don't test next release won't work with
selinux, which mean you won't test, and so on.

I'd probably relax the rules as "all packages must be tested with
SeLinux by FC7T1", with the selinux team making sure selinux does not
change too much afterwards so :
1. tested packages keep working at least till FC9T1
2. FC7 is solid selinux-wise and from FC7 on we *can* require new
packages to be selinux-proof

Note that FC7T1 is an *optimistic* deadline, it requires the selinux
team to stabilise code tools and documentation well before, *and* block
the human resources that will be necessary to help other packagers at
the test date, and fix packages before FC7 proper. If FC7 is half-fixed
like the previous ones, it *will* quickly degenerate. (network effects)

-- 
Nicolas Mailhot

[Bill Nottingham]

Jason L Tibbitts III (tibbs(a)math.uh.edu) said: 
>  Now, the packaging committee could publish guidelines for how to
>  include SELinux rules in a package; that would be great. 
Well, there's a draft standard, but you know that. :)

http://fedoraproject.org/wiki/PackagingDrafts/SELinux

Bill

[Hans Ulrich Niedermann]

James Morris <jmorris(a)redhat.com&gt; writes:

>  This guideline would request that developers test their package with
>
>  SELinux enabled (and by this I mean in enforcing mode) and follow a simple 
>  procedure:
>
>  1. Ensure they have the latest SELiunx policy installed.
>  2. Boot with selinux=1 and in enforcing mode.
>  3. Perform the normal testing of their application. 
Using which policy? targeted? strict? mls?

Testing with "targeted" should be a "MUST" requirement IMHO, but
requiring "strict" or "mls" will cause problems.

>  4. Check syslog (or /var/log/audit/audit.log if audit is enabled) for
> AVC 
>     messages related to their package. 
Gruß,

Uli

[James Morris]

On Sat, 9 Sep 2006, Hans Ulrich Niedermann wrote:

>  James Morris <jmorris(a)redhat.com&gt; writes:
>  
>  > This guideline would request that developers test their package with 
>  > SELinux enabled (and by this I mean in enforcing mode) and follow a simple 
>  > procedure:
>  >
>  > 1. Ensure they have the latest SELiunx policy installed.
>  > 2. Boot with selinux=1 and in enforcing mode.
>  > 3. Perform the normal testing of their application.
>  
>  Using which policy? targeted? strict? mls?
>  
>  Testing with "targeted" should be a "MUST" requirement IMHO, but
>  requiring "strict" or "mls" will cause problems. 
Yes, we only worry about targeted for general users.



-- 
James Morris
<jmorris(a)redhat.com&gt;

[Paul Howarth]

On Fri, 2006-09-08 at 18:09 -0400, Bill Nottingham wrote:
>  Jason L Tibbitts III (tibbs(a)math.uh.edu) said: 
>  > Now, the packaging committee could publish guidelines for how to
>  > include SELinux rules in a package; that would be great.
>  
>  Well, there's a draft standard, but you know that. :)
>  
>  http://fedoraproject.org/wiki/PackagingDrafts/SELinux 
That particular page needs a bit of work now, as some of the content has
moved here:

http://fedoraproject.org/wiki/PackagingDrafts/SELinux/PolicyModules

Paul.

[Steven Pritchard]

On Fri, Sep 08, 2006 at 04:50:44PM -0400, James Morris wrote:
>  7. If for some reason, #2 is not possible, and the release of the
> package 
>  is important enough to warrant disabling a core security feature of the 
>  OS:
>  
>  7a. Make a note of the bugzilla # from (1) in the rpm info, cvs commit and
>  release notes, with an explanation.  Also include a standardized
>  disclaimer in the rpm info which advises the user of the security risks
>  arising from disabling SELinux.  This should only happen in truly
>  exceptional cases.  I'm not sure how we can reliably notify users that
>  SELinux can be re-enabled again, and whether they'll tolerate the entire
>  fs being relabeled on reboot.  Really, this just should not happen. 
Can the policy for one application be turned off?  (I honestly don't
know...  I haven't been able to justify spending the time to really
wrap my brain around SELinux yet.)

If not, that seems like a major flaw.  It seems to me that if a user
could just toggle off checks for a particular application (and reboot,
I would assume) and have everything work well enough, there would be
an incentive to fix the one application to work with SELinux instead
of just turning off SELinux entirely.

BTW, my limited experience with SELinux issues with one of my packages
is here:

  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=187305

The time it took to resolve that bug really should be a hint that
we're not ready to require SELinux compatibility in Extras yet.

Steve
-- 
Steven Pritchard - K&S Pritchard Enterprises, Inc.
Email: steve(a)kspei.com             http://www.kspei.com/
Phone: [phone]               Mobile: [phone]

[Paul Howarth]

On Sat, 2006-09-09 at 11:15 -0500, Steven Pritchard wrote:
>  On Fri, Sep 08, 2006 at 04:50:44PM -0400, James Morris wrote:
>  > 7. If for some reason, #2 is not possible, and the release of the package 
>  > is important enough to warrant disabling a core security feature of the 
>  > OS:
>  > 
>  > 7a. Make a note of the bugzilla # from (1) in the rpm info, cvs commit and
>  > release notes, with an explanation.  Also include a standardized
>  > disclaimer in the rpm info which advises the user of the security risks
>  > arising from disabling SELinux.  This should only happen in truly
>  > exceptional cases.  I'm not sure how we can reliably notify users that
>  > SELinux can be re-enabled again, and whether they'll tolerate the entire
>  > fs being relabeled on reboot.  Really, this just should not happen.
>  
>  Can the policy for one application be turned off?  (I honestly don't
>  know...  I haven't been able to justify spending the time to really
>  wrap my brain around SELinux yet.) 
This is usually possible, by setting the xxx_disable_trans SELinux
boolean, service xxx doesn't transition from the unconfined domain and
effectively runs with SELinux protection turned off.

>  If not, that seems like a major flaw.  It seems to me that if a user
>  could just toggle off checks for a particular application (and reboot,
>  I would assume) and have everything work well enough, there would be
>  an incentive to fix the one application to work with SELinux instead
>  of just turning off SELinux entirely. 
Reboot isn't necessary; restarting the service should suffice.

Paul.

[Bill Nottingham]

James Morris (jmorris(a)redhat.com) said: 
>  This guideline would request that developers test their package with
>
>  SELinux enabled (and by this I mean in enforcing mode) and follow a simple 
>  procedure:
>  
>  1. Ensure they have the latest SELiunx policy installed.
>  2. Boot with selinux=1 and in enforcing mode.
>  3. Perform the normal testing of their application.
>  4. Check syslog (or /var/log/audit/audit.log if audit is enabled) for AVC 
>     messages related to their package.
>  
>  If there are any bugs or AVC messages:
>  
>  5. Obtain support from the SELinux team.  The best way to do this I
>  believe is to file a bugzilla against the selinux-policy package.  They
>  should note that they are a Fedora packager (and expect a high priority
>  response).  If SELinux is running all or most of the time, issues will be
>  caught and fixed eariler in their dev cycle.
>  
>  6. Don't release the package until the SELinux issue is resolved. 
I'd suggest all of the following except #6 - make sure the issues are
known, give a reasonable amount of time for fixes, but not necessarily
hold until release. For example, fixes may not be backported to earlier
releases, or the SELinux changes might require kernel fixes that are
non-trivial to implement.

Bill