Proactive SELinux fixes from automatic collection of logs

[Rahul Sundaram]

Hi

There are many instances where SELinux policy causes AVC denials while 
running programs. Some of these are policy issues, some actual bugs in 
the program or security issues and others where the denial is rather 
harmless and can be ignored for all practical purposes.

It is sometimes tedious to go and file a bug report methodologically on 
all these denials in hope that we uncover and fix real policy issues. 
What would be better is for users to run in some opt-in program that 
automatically sends either the audit or messages log or both to central 
server and the SELinux developers proactively fix policy issues without 
the overhead of users filing bug reports.

I would gladly run a program and I would guess that many users would 
find this a much better and easier way to report issues. We could even 
tie this to a GUI and first boot in the installer. Kind of a smolt 
(http://smolt.fedoraproject.org/stats) for SELinux if you will.  Comments?

Rahul

[John Dennis]

On Fri, 2007-06-29 at 06:37 +0530, Rahul Sundaram wrote:
>  Hi
>  
>  There are many instances where SELinux policy causes AVC denials while 
>  running programs. Some of these are policy issues, some actual bugs in 
>  the program or security issues and others where the denial is rather 
>  harmless and can be ignored for all practical purposes.
>  
>  It is sometimes tedious to go and file a bug report methodologically on 
>  all these denials in hope that we uncover and fix real policy issues. 
>  What would be better is for users to run in some opt-in program that 
>  automatically sends either the audit or messages log or both to central 
>  server and the SELinux developers proactively fix policy issues without 
>  the overhead of users filing bug reports.
>  
>  I would gladly run a program and I would guess that many users would 
>  find this a much better and easier way to report issues. We could even 
>  tie this to a GUI and first boot in the installer. Kind of a smolt 
>  (http://smolt.fedoraproject.org/stats) for SELinux if you will.  Comments? 
We already have something much like you're suggesting. A while ago it
was recognized that diagnosing and addressing SELlinux AVC denials was a
significant problem. We designed and built a tool to help with that,
it's called setroubleshoot. It has a daemon that runs with root
priveleges and a user space desktop GUI component that will alert you to
any AVC denial and analyze it. You get a notification on your desktop
and GUI tool which allows you to browse your AVC denials in a user
friendly interpretation.

See: https://hosted.fedoraproject.org/projects/setroubleshoot

During the design phase of the tool we considered automatic bug
reporting and the design of the tool would support automatic bug
reporting. However, we elected to not implement automatic bug reporting
for the following reasons:

1) Not all AVC denials are bugs. In fact many are due to correctly
operations the sys admin must explicitly enable via a policy boolean.
One of the primary jobs of setroubleshoot is to automatically detect
these cases and tell the user how to configure the policy.

2) The information contained in an AVC denial is security sensitive. It
would be a huge security hole to automatically transmit any of this
information in the form of a bug report or other notification channel.

3) Automatic collection of user generated reports was an extra
development effort which also requires a central service. Implementing
the feature and resources to then manage this central service was deemed
out of scope, especially taking into consideration points 1 and 2 above.

The conclusion is that when user's see an AVC and have it automatically
interpreted (very easy with setroubleshoot) it is then up to them to
decide if it's really a bug (could just be policy configuration) and if
it is a bug if they want to export information which might be security
sensitive. If they elect to report then they can simply copy the report
out of the setroublehoot GUI and paste it into a bugzilla. 

Should there be a button which says "submit as bug report" in the GUI.
Sure that might be a very handy feature but at the moment bugzilla
requires an authenticated account to generate a new bug report. We also
don't want to flood bugzilla with duplicates via automated submission.
When setroubleshoot was designed it took the duplicate report issue into
account and it can recognize the same issue occuring on multiple systems
so that only one report would be generated. But that requires submitting
a setroubleshoot report to an intermediary central server which then
interacts with bugzilla.
-- 
John Dennis <jdennis(a)redhat.com&gt;

[Rahul Sundaram]

John Dennis wrote:
>  On Fri, 2007-06-29 at 06:37 +0530, Rahul Sundaram wrote:
> > Hi
> >
> > There are many instances where SELinux policy causes AVC denials while 
> > running programs. Some of these are policy issues, some actual bugs in 
> > the program or security issues and others where the denial is rather 
> > harmless and can be ignored for all practical purposes.
> >
> > It is sometimes tedious to go and file a bug report methodologically on 
> > all these denials in hope that we uncover and fix real policy issues. 
> > What would be better is for users to run in some opt-in program that 
> > automatically sends either the audit or messages log or both to central 
> > server and the SELinux developers proactively fix policy issues without 
> > the overhead of users filing bug reports.
> >
> > I would gladly run a program and I would guess that many users would 
> > find this a much better and easier way to report issues. We could even 
> > tie this to a GUI and first boot in the installer. Kind of a smolt 
> > (http://smolt.fedoraproject.org/stats) for SELinux if you will.  Comments?
>  
>  We already have something much like you're suggesting. A while ago it
>  was recognized that diagnosing and addressing SELlinux AVC denials was a
>  significant problem. We designed and built a tool to help with that,
>  it's called setroubleshoot.  
This requires a GUI right? My idea would work on any Fedora system.

>  1) Not all AVC denials are bugs. In fact many are due to correctly
>  operations the sys admin must explicitly enable via a policy boolean. 
This is in fact one of my reasons for favoring a more automated 
collection of AVC denials. Some of the systems don't have any GUI and I 
don't report bugs unless it prevents programs from working correctly. 
Maybe there are policy improvements to be made but it is not worth the 
trouble in many occasions to go and file bug reports for every AVC denial.

>  2) The information contained in an AVC denial is security sensitive.
> It
>  would be a huge security hole to automatically transmit any of this
>  information in the form of a bug report or other notification channel. 
Encrypt it before transmission and scrub the data before revealing 
anything. Also this concern is already somewhat offset from the effort 
described below.

>  3) Automatic collection of user generated reports was an extra
>  development effort which also requires a central service. Implementing
>  the feature and resources to then manage this central service was deemed
>  out of scope, especially taking into consideration points 1 and 2 above. 
Since there is a effort now to create infrastructure that allows people 
to upload logs and get analysis it wouldn't be too much additional 
effort. Smolt also already has similar infrastructure in place which 
would be a good example to learn from.

Rahul

[John Dennis]

On Mon, 2007-07-02 at 22:30 +0530, Rahul Sundaram wrote:
>  John Dennis wrote:
>  > We already have something much like you're suggesting. A while ago it
>  > was recognized that diagnosing and addressing SELlinux AVC denials was a
>  > significant problem. We designed and built a tool to help with that,
>  > it's called setroubleshoot. 
>  
>  This requires a GUI right? My idea would work on any Fedora system. 
No a GUI is not required, setroubleshoot-server can be installed on a
headless machine. In this configuration the alerts setroubleshoot
generates can be sent via email or one can use sealert in either the GUI
or the command line mode to connect to the remote node and view the
analysis or one can ssh into the machine and use the command line mode
of sealert. That means at the moment there are 3 different ways you can
get setroubleshoot analysis from a machine without an installed GUI.

There are probably more favorable ways of gathering the data from
setroubleshoot when managing a collection of nodes. We do have a
requirement to better support general auditing from a collection of
nodes. Work is proceeding on that front and the plans are to have
setroubleshoot be a component in 'aggregate auditing.

>  > 1) Not all AVC denials are bugs. In fact many are due to
> correctly
>  > operations the sys admin must explicitly enable via a policy boolean.
>  
>  This is in fact one of my reasons for favoring a more automated 
>  collection of AVC denials. Some of the systems don't have any GUI and I 
>  don't report bugs unless it prevents programs from working correctly. 
>  Maybe there are policy improvements to be made but it is not worth the 
>  trouble in many occasions to go and file bug reports for every AVC denial. 
Just one problem here, who is going to be responsible for triaging every
AVC denial on every installed Fedora system to figure out if it's user
error, noise, or a genuine issue? The sheer volume would be
overwhelming. One need only consider how long many genuine bugs languish
in bugzilla due to inattention and one has to question just what
forwarding all AVC denials is going to accomplish in practical terms.
Putting an intelligent human in between the denial event and a bug
report gains enormous efficiency right?

>  
>  > 2) The information contained in an AVC denial is security sensitive. It
>  > would be a huge security hole to automatically transmit any of this
>  > information in the form of a bug report or other notification channel.
>  
>  Encrypt it before transmission and scrub the data before revealing 
>  anything. Also this concern is already somewhat offset from the effort 
>  described below. 
Automatically sending security information to a remote third party is
not going to be accepted by most users and certainly could not be
enabled by default. If automatic transmission is not enabled by default
then what is gained over an administrator of the system being
automatically notified of a denial by setroubleshoot and letting them
evaluate if this particular AVC denial needs to be elevated to a bug
report?

>  > 3) Automatic collection of user generated reports was an extra
>  > development effort which also requires a central service. Implementing
>  > the feature and resources to then manage this central service was deemed
>  > out of scope, especially taking into consideration points 1 and 2 above.
>  
>  Since there is a effort now to create infrastructure that allows people 
>  to upload logs and get analysis it wouldn't be too much additional 
>  effort. Smolt also already has similar infrastructure in place which 
>  would be a good example to learn from. 
I will take a look a smolt to see what it offers and what it's model is.
Perhaps there are things in smolt we could benefit from.

-- 
John Dennis <jdennis(a)redhat.com&gt;

[Stephen John Smoogen]

On 7/2/07, John Dennis <jdennis(a)redhat.com&gt; wrote:
>  On Mon, 2007-07-02 at 22:30 +0530, Rahul Sundaram wrote: 
>  > > 2) The information contained in an AVC denial is security
> sensitive. It
>  > > would be a huge security hole to automatically transmit any of this
>  > > information in the form of a bug report or other notification channel.
>  >
>  > Encrypt it before transmission and scrub the data before revealing
>  > anything. Also this concern is already somewhat offset from the effort
>  > described below.
>
>  Automatically sending security information to a remote third party is
>  not going to be accepted by most users and certainly could not be
>  enabled by default. If automatic transmission is not enabled by default
>  then what is gained over an administrator of the system being
>  automatically notified of a denial by setroubleshoot and letting them
>  evaluate if this particular AVC denial needs to be elevated to a bug
>  report?
>  
Also scrubbing the data can be very hard since the information that
could be sensitive is more than user name/ip address. While there
might be some statistical information that could be picked up (hmmm
4000 users have problems with /xen installations... maybe we should
see if there is a problem with the policy and what people think they
are doing.

Another issue I could see is that if someone opted into the program,
and Fedora 'witnesses' a breakin (or some other criminal act) via a
Selinux report... what are the reporting requirements (depending on
the nation that the servers are in and where the client is.)



-- 
Stephen J Smoogen. -- CSIRT/Linux System Administrator
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"

[Bruno Wolff III]

On Fri, Jun 29, 2007 at 06:37:21 +0530,
  Rahul Sundaram <sundaram(a)fedoraproject.org&gt; wrote:
>  
>  I would gladly run a program and I would guess that many users would 
>  find this a much better and easier way to report issues. We could even 
>  tie this to a GUI and first boot in the installer. Kind of a smolt 
>  (http://smolt.fedoraproject.org/stats) for SELinux if you will.  Comments? 
Make sure you have a way to detect people doing odd things so you don't
waste your time tracking down nonproblems. For example when playing with
MCS labels recently I managed to install some package updates with additional
labels that prevented me from using them later. So you need to think about
what kind of customization information you need to include along with
the acv messages.