Re: glusterfs / fusefs / SELinux problems in permissive mode

[Stephen Smalley]

On Fri, 2009-06-05 at 10:22 +0100, Ted Rule wrote:
>  I have a problem running a GlusterFS Client talking to a mounted
>  filesystem, where the SELinux labelling appears to be lost at dismount.
>  
>  The experiments we've tried so far suggest that the problem is in
>  SELinux policy for the underlying FUSE kernel driver. 
See this thread:
http://marc.info/?t=121379726100001&r=1&w=2

What is needed is the ability to automatically detect whether the
underlying fs supports security xattrs, using them if supported or
falling back to the genfscon rule for filesystems that do not support
them.  That is what Eric's patch was trying to do, but it ran into a
deadlock problem on some filesystems.

If you want all of the files in the mount to have a single label on each
mount, you could use a context= mount instead.  But that doesn't support
per-file labeling.

>  All the boxes in question running CentOS5.3, with these relevant
> rpms:
>  
>  $ rpm -q kernel selinux-policy  fuse glusterfs-client
>  kernel-2.6.18-92.el5
>  kernel-2.6.18-128.1.10.el5
>  selinux-policy-2.4.6-203.el5
>  fuse-2.7.4-1.el5.rf
>  glusterfs-client-2.0.0-1
>  
>  where the fuse driver is from Dag repository, and glusterfs-client was
>  locally rebuilt from an SRPM, but everything else from CentOS
>  
>  When the remote filesystem is initially mounted, the tree ends up with
>  fusefs_t labels:
>  
>  [me@web1 ~]$ ls -lZ /mnt/shared
>  ....
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       contexttest
>  drwxr-xr-x  root   root        system_u:object_r:fusefs_t       contexttest2
>  drwxr-xr-x  root   root        system_u:object_r:fusefs_t       contexttest3
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       files
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       icons
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       images
>  drwxrwsr-x  root   weblogs     system_u:object_r:fusefs_t       logfiles
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       navtest
>  
>  ...
>  [me@web1 ~]$
>  
>  
>  However, it's possible to chcon the tree manually and set the locally
>  visible label, as in:
>  
>  [me@web1 ~]$  sudo chcon -t file_t /mnt/shared/contexttest3
>  [me@web1 ~]$  ls -lZ /mnt/shared
>  ...
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       contexttest
>  drwxr-xr-x  root   root        system_u:object_r:fusefs_t       contexttest2
>  drwxr-xr-x  root   root        system_u:object_r:file_t         contexttest3
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       EMMA
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       files
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       icons
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       images
>  drwxrwsr-x  root   weblogs     system_u:object_r:fusefs_t       logfiles
>  drwxrwsr-x  apache sharedfiles system_u:object_r:fusefs_t       navtest
>  ...
>  
>  This change of label is also correctly reflected on the server end of
>  the mount.
>  
>  However, if we now unmount/remount the volume, all the labels revert to
>  fusefs_t.
>  
>  FWIW, this was all with the Client and Server in SELinux permissive.
>  
>  This then suggested to us that the fuse driver somehow fails to read all
>  the extended attributes from the ext3 filesystem on the Server; maybe it
>  only reads fuse-specific attributes,
>  and leaves the remainder blank.
>  
>  After some other experiments, we tried running the Client in SELinux
>  disabled, and lo and behold, the Server's labels were picked up on mount
>  correctly.
>  
>  We then found that the filesystem policy appears to class fuse as a
>  non-xattr filesystem, as in this clause from kernel/filesystem.te:
>  
>  ....
>  type fusefs_t;
>  fs_noxattr_type(fusefs_t)
>  allow fusefs_t self:filesystem associate;
>  allow fusefs_t fs_t:filesystem associate;
>  genfscon fuse / gen_context(system_u:object_r:fusefs_t,s0)
>  genfscon fuseblk / gen_context(system_u:object_r:fusefs_t,s0)
>  genfscon fusectl / gen_context(system_u:object_r:fusefs_t,s0)
>  ....
>  
>  However, we can see that chcon is able to write the label from the
>  client up to the server, so the fuse driver must be xattr capable to
>  some degree.
>  
>  Given that the problem appears to go away when I set SELinux to disabled
>  ( as opposed to permissive ), I presume that it's a policy bug.
>  
>  Is there some relatively simple tweak that could be applied, such as
>  adding a line something like this:
>  
>     fs_use_xattr fuse gen_context(system_u:object_r:fusefs_t,s0);
>  
>  to match what ext3 has:
>  
>     fs_use_xattr ext3 gen_context(system_u:object_r:fs_t,s0);
>  
>  
>  Can you offer any further suggestions?
>  
>  
>  Many thanks,
>  
>  
>  
>   
-- 
Stephen Smalley
National Security Agency