AVC denials when using RH Cluster Suite's qdiskd and ping heuristic

[Sean E. Millichamp]

I have been experimenting with using a quorum disk with the RH cluster
suite product (qdiskd in the cman RPM).  One of the requirements of
using qdiskd is that you must specify at least one command that can be
run to test the health of the node.  

A typical command to use for this heuristic is ping, although any
command that returns a 0/non-0 exit status is acceptable.

When I configure a simple ping test with qdisk I get:

type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
comm="ping" path="/dev/sda4" dev=tmpfs ino=1051
scontext=root:system_r:ping_t:s0 tcontext=system_u:object_
r:fixed_disk_device_t:s0 tclass=blk_file

type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
comm="ping" path="/dev/sdb4" dev=tmpfs ino=985
scontext=root:system_r:ping_t:s0 tcontext=system_u:object_r
:fixed_disk_device_t:s0 tclass=blk_file

type=SYSCALL msg=audit(1219960233.627:4561): arch=c000003e syscall=59 success=yes exit=0
a0=1f3575a0 a1=1f357610 a2=1f356150 a3=3 items=0 ppid=23163 pid=23174 auid=0 uid=0 gid=0
euid=0 suid=0
 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=189 comm="ping"
exe="/bin/ping" subj=root:system_r:ping_t:s0 key=(null)

The test is: "/bin/ping -c1 -t1 x.x.x.x" where x.x.x.x is a
known-reachable IP.  This works without AVC denials from the command
line.

Interestingly, the ping command seems to complete and return an exit
code properly to qdisk as the log message shows that it is up.

I have used audit2allow which told me I could do:

allow ping_t fixed_disk_device_t:blk_file { read write };

and silence the message, but somehow giving ping access to
fixed_disk_device_t to quiet a log message seems like it defeats the
spirit of limited access :)

I have no idea why ping (I am assuming that I am reading it correctly
and it is ping) would be trying to access /dev/sda4 or /dev/sdb4.  Those
partitions are the extended partition container on my software RAID-1
boot drives.  I've poked briefly at the qdiskd code and it seems to do a
normal fork/exec to invoke the ping.

Has anyone seen anything like this or have any ideas on where I should
look next?  I've spent a while on it so far and I don't have a lot more
time to spend on it, but I'd like to solve it if possible and (if
needed) get a fix out somewhere.

I have been doing my testing with the RHEL versions of these tools as I
don't have a cluster of Fedora machines handy, but I couldn't find a
RHEL SELinux list and this looked like the best bet.

selinux-policy-targeted-2.4.6-137.1.el5
cman-2.0.84-2.el5

I'm still pretty new to trying to sift through SELinux policy/messages
so please be patient with me. :)

Thanks!

Sean

[Sean E. Millichamp]

On Thu, 2008-08-28 at 18:05 -0400, Sean E. Millichamp wrote:

>  When I configure a simple ping test with qdisk I get:
>  
>  type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
> comm="ping" path="/dev/sda4" dev=tmpfs ino=1051
> scontext=root:system_r:ping_t:s0 tcontext=system_u:object_
>  r:fixed_disk_device_t:s0 tclass=blk_file
>  
>  type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
> comm="ping" path="/dev/sdb4" dev=tmpfs ino=985
> scontext=root:system_r:ping_t:s0 tcontext=system_u:object_r
>  :fixed_disk_device_t:s0 tclass=blk_file
>  
>  type=SYSCALL msg=audit(1219960233.627:4561): arch=c000003e syscall=59 success=yes exit=0
> a0=1f3575a0 a1=1f357610 a2=1f356150 a3=3 items=0 ppid=23163 pid=23174 auid=0 uid=0 gid=0
> euid=0 suid=0
>   fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=189 comm="ping"
> exe="/bin/ping" subj=root:system_r:ping_t:s0 key=(null) 
These messages are because qdiskd doesn't properly clean up its file
descriptors before forking and execing ping.  I will clean up my
findings and submit a patch/open a bug report against qdisk.

Thanks,
Sean

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sean E. Millichamp wrote:
>  On Thu, 2008-08-28 at 18:05 -0400, Sean E. Millichamp wrote:
>  
> > When I configure a simple ping test with qdisk I get:
> >
> > type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
> comm="ping" path="/dev/sda4" dev=tmpfs ino=1051
> scontext=root:system_r:ping_t:s0 tcontext=system_u:object_
> > r:fixed_disk_device_t:s0 tclass=blk_file
> >
> > type=AVC msg=audit(1219960233.627:4561): avc:  denied  { read write } for  pid=23174
> comm="ping" path="/dev/sdb4" dev=tmpfs ino=985
> scontext=root:system_r:ping_t:s0 tcontext=system_u:object_r
> > :fixed_disk_device_t:s0 tclass=blk_file
> >
> > type=SYSCALL msg=audit(1219960233.627:4561): arch=c000003e syscall=59 success=yes
> exit=0 a0=1f3575a0 a1=1f357610 a2=1f356150 a3=3 items=0 ppid=23163 pid=23174 auid=0 uid=0
> gid=0 euid=0 suid=0
> >  fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=189 comm="ping"
> exe="/bin/ping" subj=root:system_r:ping_t:s0 key=(null)
>  
>  These messages are because qdiskd doesn't properly clean up its file
>  descriptors before forking and execing ping.  I will clean up my
>  findings and submit a patch/open a bug report against qdisk.
>  
>  Thanks,
>  Sean
>  
>  
>  --
>  fedora-selinux-list mailing list
>  fedora-selinux-list(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-selinux-list 
Great, I was just about
to suggest this.  Whenever you see something
bizarre like ping trying to read write raw disks, I think of leaked file
descriptors. or redirection of stdout.


> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iEYEARECAAYFAki+waYACgkQrlYvE4MpobNRVQCbB8ZKR7N378P18cVG6eRHsY3H
> lqIAn0SQgiRvz400F0lvYABizPbatDAG
> =b8wE
> -----END PGP SIGNATURE-----

[Sean E. Millichamp]

On Wed, 2008-09-03 at 12:56 -0400, Daniel J Walsh wrote:

>  Sean E. Millichamp wrote:
>  > These messages are because qdiskd doesn't properly clean up its file
>  > descriptors before forking and execing ping.  I will clean up my
>  > findings and submit a patch/open a bug report against qdisk. 
>  Great, I was just about to suggest this.  Whenever you see something
>  bizarre like ping trying to read write raw disks, I think of leaked file
>  descriptors. or redirection of stdout. 
It turns out qdiskd was (is) leaking a number of file descriptors to its
forked heuristics - and not always predictably because of the threading
qdiskd uses.  It would have been a very hard bug to spot if not for
SELinux - mark a win for security!

For those interested the qdisk bug report against RHEL 5 (with suggested
patches) is here: https://bugzilla.redhat.com/show_bug.cgi?id=460645

Sean