Script to check security?

[Bob Gustafson]

With all of the possible variations in security settings - strict,
permissive, local, lots of users, only daemons, etc.

Is there a script around somewhere - something like 'configure' which is
used at the beginning of a component build - which will query various
pieces of a system, do a 'setenforce 1' and then try various programs and
grep the output to give some binary answer, then do 'setenforce 0' and try
the same program, etc.

This script would help to give struggling sysadmins some degree of
confidence that what is being done to their 'policy.local' or whatever, is
benign.

Of course the script could be corrupted or buggy - one more thing to add to
when adding or changing the SELinux system, but there would be advantages:

Just as the 'no child left behind' program uses testing to measure the
effectiveness of public expenditures on schools ( :-) ), a security testing
script could help to test the effectiveness of the SELinux system as it
evolves.

A testing script would also help to rein in the tendency to add wrinkles
and grow the complexity of the system - each wrinkle would have a test
module to check it.

BobG

On Thu, 27 May 2004 11:59:24 CDT, Bob Gustafson <bobgus(a)rcn.com&gt;  said:

>  Is there a script around somewhere - something like
> 'configure' which is
>  used at the beginning of a component build - which will query various
>  pieces of a system, do a 'setenforce 1' and then try various programs and
>  grep the output to give some binary answer, then do 'setenforce 0' and try
>  the same program, etc. 
"Testing can reveal the presence of flaws, but not their absence" -- Dykstra

Writing such a test harness for a program is a daunting challenge - the biggest
hurdle is that although you can cover 75% of the issues simply by doing a
'setenforce 1' and seeing if the program will even start up, devising harness
cases for the other 25% is very difficult - it's often stuff like "initial
one-time
file creation" or "error handling (I've had the joy of trying to debug an
application
that got a permission error while trying to open an error message catalog to get
the human-readable form of "permission error" - instant recursive error ;)

My posting about mysql the other day was related to another project of mine that
involves a multi-gigabyte mysql database.  The as-shipped mysql.fc labels files
with the assumption that /var/lib/mysql/<dbname> is where the database lives.
Now, either I get to live with a 40-gigabyte /var, or I also stick a mysqld_db_t
on the /datastore/<dbname> tree where the database actually resides.

Now for those of you listening at home - devise a test that will catch the
difference between these two lines:

/datastore/mydata(/.*)?               system_u:object_r:mysqld_db_t
/datastore(/.*)?               system_u:object_r:mysqld_db_t

(Hint - what happens if there's a /datastore/otherstuff directory?)

>  This script would help to give struggling sysadmins some degree of
>  confidence that what is being done to their 'policy.local' or whatever, is
>  benign. 
It's feasible to set up a script that verifies that a given program is given
"enough" access - see 'audit2allow'.  It's another challenge
entirely to verify
that it is in fact the minimal set of required access - mostly because it has
no way to identify what "proper" means.

(Hmm... I'm trying to figure out if the generic case of computing "minimal
set"
is the equivalent of the Halting Problem.  It's actually probably fairly doable
with static code analysis, except that programmers have this very annoying
tendency to do stuff like call sprintf(foo,"%s", user_file); and then
open(foo)... And sometimes they actually *want* a "../.." pattern in foo. ;)

[Bob Gustafson]

On Thu, 27 May 2004 14:00:53 -0400, Valdis.Kletnieks wrote:
> On Thu, 27 May 2004 11:59:24 CDT, Bob Gustafson <bobgus(a)rcn.com&gt;
>  said:
>
> > Is there a script around somewhere - something like 'configure' which is
> > used at the beginning of a component build - which will query various
> > pieces of a system, do a 'setenforce 1' and then try various programs and
> > grep the output to give some binary answer, then do 'setenforce 0' and try
> > the same program, etc.
>
> "Testing can reveal the presence of flaws, but not their absence" -- Dykstra
>
> Writing such a test harness for a program is a daunting challenge - the
> biggest
> hurdle is that although you can cover 75% of the issues simply by doing a
> 'setenforce 1' and seeing if the program will even start up, devising harness
> cases for the other 25% is very difficult - it's often stuff like "initial
> one-time
> file creation" or "error handling (I've had the joy of trying to debug an
> application
> that got a permission error while trying to open an error message catalog
> to get
> the human-readable form of "permission error" - instant recursive error ;)
>
> My posting about mysql the other day was related to another project of
> mine that
> involves a multi-gigabyte mysql database.  The as-shipped mysql.fc labels
> files
> with the assumption that /var/lib/mysql/<dbname> is where the database lives.
> Now, either I get to live with a 40-gigabyte /var, or I also stick a
> mysqld_db_t
> on the /datastore/<dbname> tree where the database actually resides.
>
> Now for those of you listening at home - devise a test that will catch the
> difference between these two lines:
>
> /datastore/mydata(/.*)?               system_u:object_r:mysqld_db_t
> /datastore(/.*)?               system_u:object_r:mysqld_db_t
>
> (Hint - what happens if there's a /datastore/otherstuff directory?) 
(I do need to read more of the FAQs and docs, but here is a stab..)

Assuming that /datastore/mydata(/.*) is more restrictive than
/datastore(/.*), the testing probe could be a small program that 'looks
like' mysqld (assumes same roles with same selinux tags as mysqld) which
tries to access files in the 'crack' between /datastore/mydata and
/datastore. As part of the testing procedure, files could be dropped in the
'crack' for this test program to access.

(File access could be generalized to cover shared memory access, sockets, etc.)

Another wrinkle would be to add an error return code (or codes) to all of
the OS file access routines which would tell a calling program that it was
trying to open a file without sufficient SELinux permission (maybe this has
been done already). This feature would help to make probes a bit more
intelligent

If probe programs were written in great numbers, the /var/log/messages file
would be a large problem. Something would need to be done that would turn
off logging during probe testing, or some such. This would have to be done
carefully as it is would be a big door for hackers.

With a good testing scheme, patterns would begin to emerge which could be
used to simplify and regularize OS access needs. 'Simple' means that proof
testing would have a better chance.

> > This script would help to give struggling sysadmins some degree of
> > confidence that what is being done to their 'policy.local' or whatever, is
> > benign.
>
> It's feasible to set up a script that verifies that a given program is given
> "enough" access - see 'audit2allow'.  It's another challenge
> entirely to
> verify
> that it is in fact the minimal set of required access - mostly because it has
> no way to identify what "proper" means.
>
> (Hmm... I'm trying to figure out if the generic case of computing "minimal
> set"
> is the equivalent of the Halting Problem.  It's actually probably fairly
> doable
> with static code analysis, except that programmers have this very annoying
> tendency to do stuff like call sprintf(foo,"%s", user_file); and then
> open(foo)... And sometimes they actually *want* a "../.." pattern in foo. ;)
>

On Fri, 28 May 2004 11:51:38 CDT, Bob Gustafson <bobgus(a)rcn.com&gt;  said:
>  >/datastore/mydata(/.*)?              
> system_u:object_r:mysqld_db_t
>  >/datastore(/.*)?               system_u:object_r:mysqld_db_t
>  >
>  > (Hint - what happens if there's a /datastore/otherstuff directory?) 
>  Assuming that /datastore/mydata(/.*) is more restrictive than
>  /datastore(/.*), the testing probe could be a small program that 'looks
>  like' mysqld (assumes same roles with same selinux tags as mysqld) which
>  tries to access files in the 'crack' between /datastore/mydata and
>  /datastore. As part of the testing procedure, files could be dropped in the
>  'crack' for this test program to access. 
Yes.  However, you just forgot to verify that SAS still works when accessing
its datasets in /datastore/otherstuff because it's labeled mysql_db_t instead
of whatever it should have been for SAS...

Or maybe it wasn't SAS, but Mathematica.  Or was it that other app???

(Yes, it was a trick question to make a point....)

[Bob Gustafson]

On Fri, 28 May 2004 13:11:14 -0400, Valdis.Kletnieks(a)vt.edu wrote:
> On Fri, 28 May 2004 11:51:38 CDT, Bob Gustafson <bobgus(a)rcn.com&gt;
>  said:
> > >/datastore/mydata(/.*)?               system_u:object_r:mysqld_db_t
> > >/datastore(/.*)?               system_u:object_r:mysqld_db_t
> > >
> > > (Hint - what happens if there's a /datastore/otherstuff directory?)
>
> > Assuming that /datastore/mydata(/.*) is more restrictive than
> > /datastore(/.*), the testing probe could be a small program that 'looks
> > like' mysqld (assumes same roles with same selinux tags as mysqld) which
> > tries to access files in the 'crack' between /datastore/mydata and
> > /datastore. As part of the testing procedure, files could be dropped in the
> > 'crack' for this test program to access.
>
> Yes.  However, you just forgot to verify that SAS still works when accessing
> its datasets in /datastore/otherstuff because it's labeled mysql_db_t instead
> of whatever it should have been for SAS...
>
> Or maybe it wasn't SAS, but Mathematica.  Or was it that other app??? 
The 'test suite', just like the build tool 'configure', would be a
collection of probes and test wrappers. SAS and Mathematica would also have
their own modular piece of the test script (and have their files properly
labeled).

Perhaps these test script modules will converge to some common form which
can be customized a bit for the 'next' application. Much of the GNU build
process has evolved to mostly the same code for different applications -
just a few names are changed at a higher level.

The process of doing it, finding the problems, and then finding the
commonality in the checks, and using the commonality to simplify the system
is very important. SELinux is a whack at developing a useful language for
this commonality. Using the policy to nudge the kernel development is the
next step. (or maybe the step after next..)

> (Yes, it was a trick question to make a point....) 
Keep 'em coming.

[Russell Coker]

On Sat, 29 May 2004 11:14, Bob Gustafson <bobgus(a)rcn.com&gt; wrote:
>  The 'test suite', just like the build tool
> 'configure', would be a
>  collection of probes and test wrappers. SAS and Mathematica would also have
>  their own modular piece of the test script (and have their files properly
>  labeled). 
This MAY detect that the program operates correctly (which is more difficult 
than you may imagine in the case of programs that launch other programs in 
different domains - think of checking password access by pam_unix.so).  
However it does not help for the more difficult problems, such as determining 
that the program is not given excessive permissions.

Many programs routinely try to access things that they don't need, one of the 
most difficult parts of writing SE Linux policy is deciding when to use a 
dontaudit rule for such things.

>  Perhaps these test script modules will converge to some common form
> which
>  can be customized a bit for the 'next' application. Much of the GNU build
>  process has evolved to mostly the same code for different applications -
>  just a few names are changed at a higher level. 
I've been trying to encourage post-grad students to do a Ph.D thesis on a 
related topic.  So far no-one has been interested.  It's generally regarded 
that this problem is not solvable in the general case, but I think that a 
good Ph.D project could get a usable partial solution.

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page