Audit overhead and default rules

[Andrew Lutomirski]

On Mon, Feb 10, 2014 at 1:02 PM, Steve Grubb <sgrubb at redhat.com> wrote:
> On Monday, February 10, 2014 12:41:08 PM Andrew Lutomirski wrote:
>> >> There are, indeed, many ways for me to fix this on my machine.  I'm
>> >> suggesting that Fedora change the default so that no one has
>> >> experiences this overhead by default.
>> >
>> > There are 3 levels of audit performance degradation.
>> > 1) audit is disabled. You get full speed
>> > 2) audit is enabled and no rules. This is the default for Fedora so that
>> > more information can be collected when AVC's occur.
>> > 3) audit is enabled and rules loaded. This does get a performance hit that
>> > can be measured. In this case, the person wanted auditing and is willing
>> > to take any performance hit it may incur.
>> >
>> > The audit system has been set for #2 for the last 8 or 9 years as a
>> > balance
>> > between getting information for avc's, not taking a big performance hit,
>> > and keeping setup easy for when people want to add auditing to their
>> > system.
>>
>> Right.  I'm proposing changing the default from #2 to #1.
>
> I forgot to mention option 0) audit package not installed. I don't think the
> audit package is mandatory and that would be the default. But if you do
> install the audit package its assumed you want auditing in some capacity and
> are willing to take the minimal hit. You also get more audit events such as
> promiscuous socket use, user space events, and a couple other kernel events
> that are security related.
>
>> I think that #2-by-default is a terrible tradeoff.  I suspect I've debugged
>> more selinux denials than the average user, and I have *never* *once*
>> looked at a 'syscall' entry in the log.
>
> The selinux people wanted the syscall event. Once upon a time, you used to
> have to add a rule to get the syscall information. But they decided they want
> more information by default. I would suggest reverting that patch as a test. I
> think the problem was that they needed a file path sometimes and would ask
> people to add an audit rule like "-w /etc/shadow -p w". But then the user
> wouldn't get a recurrence and they couldn't really fix the problem very fast.
> The exact details may be different, but I think this is the gist of it.
>

Here's an example from my logs:

type=AVC msg=audit(1383816002.656:3662): avc:  denied  { execute } for
 pid=32707 comm="sh" name="ldconfig" dev="dm-2" ino=172883
scontext=system_u:system_r:smoltclient_t:s0-s0:c0.c1023
tcontext=system_u:object_r:ldconfig_exec_t:s0 tclass=file
type=SYSCALL msg=audit(1383816002.656:3662): arch=c000003e syscall=59
success=no exit=-13 a0=9c0d30 a1=9c0e00 a2=9bfd70 a3=0 items=0
ppid=32706 pid=32707 auid=999 uid=999 gid=998 euid=999 suid=999
fsuid=999 egid=998 sgid=998 fsgid=998 ses=415 tty=(none) comm="sh"
exe="/usr/bin/bash"
subj=system_u:system_r:smoltclient_t:s0-s0:c0.c1023 key=(null)

The useful things (I think) are "dev" and "ino", both of which are in
the AVC line, not the syscall line.

>
>> I think that subjecting every Fedora user by default to 20-40 ns of extra
>> syscall latency for the sole benefit of getting those 'syscall' messages is a
>> bad tradeoff.
>
> I don't think all Fedora users have audit installed and would not see the hit.

>From the F20 comps:

  <group>
    <id>core</id>
    <_name>Core</_name>
    <_description>Smallest possible installation</_description>
    <default>false</default>
    <uservisible>false</uservisible>
    <packagelist>
      <packagereq type="mandatory">audit</packagereq>

audit is the very first mandatory package :-/

>
>> I'm willing to write kernel code to improve the situation.  The
>> problem is that getting rid of TIF_SYSCALL_AUDIT when there are no
>> audit rules configured is messy.  Better suggestions are welcome.
>
> The problem is getting TIF_SYSCALL_AUDIT back in all processes when auditing
> is enabled. We cannot stop the OS and stab that flag into all processes when
> audit gets re-enabled. Its best not to play with that flag.

Sure we can.  I have patches to do that.  They have other problems,
though, but that's fixable.

>
> The kernel logic is supposed to be something like
>
> if (tif & TIF_SYSCALL_AUDIT)
>   if (current->audit_context)
>     if (audit_ever_enabled)
>       audit_syscall_entry()
>
> So, the overhead when disabled should only be an if statement or two.

On my laptop it's up to 1/3 of *total* syscall time.  Linux fast-path
syscalls are fast, and audit disables the fast path.

--Andy