Fedora 15: a NFSv4 to glusterfs migration HOW-TO

Thu Aug 25 12:59:25 UTC 2011

Excellent how to Dr.
Can you add this to the Fedora wiki?

2011/8/25 Dr. Michael J. Chudobiak <mjc at avtechpulse.com>

> As detailed in another thread, we upgraded a few test machines on our
> LAN to Fedora 15 (with gnome-shell and firefox), with user folders
> served from a NFSv4 server (F14 originally, then F15).
>
> It just didn't work. The F15 desktops would freeze frequently. And
> worse, this would freeze ALL desktops on the LAN intermittently, as the
> NFS server struggled with client flakiness.
>
> When it did work, Firefox would lose authenticated logins randomly,
> presumably due to corruption of its cookies.sqlite file. sqlite and NFS
> seems to be a nightmare, for both NFSv3 and NFSv4.
>
> Moving from a NFSv4 server to a glusterfs server solved all of these
> problems, and sped up boot times significantly too. glusterfs looks
> intimidating at first, because of all its fancy replicating features and
> what-not, but it turns out to be trivially easy to set up a simple
> server than will replace 95% of the NFS installations out there.
>
> Luckily, you can easily point both the NFS daemon and the glusterfs
> daemon at the same export folder, so you can migrate clients slowly over
> time.
>
> This HOW-TO is intended to document the process. There are other similar
> HOW-TOs out there, but they are all a little out-of-date or don't show
> how to enable locking correctly, which is critical for Firefox.
>
> In this example, we export the server's /fileserver folder, and mount it
> on /fileserver on the clients. In my server, /fileserver was already
> being served by the NFSv4 server, which is fine.
>
> --------------------------------------------
> 1. On the server:
> - yum install glusterfs-server
>
> --------------------------------------------
> 2. On each client:
> - yum install glusterfs-fuse
> - mkdir /etc/glusterfs/
> - mkdir /fileserver
>
> --------------------------------------------
> 3. On the server, edit the volume configuration file
> (/etc/glusterfs/glusterfsd.vol) so that it looks like this:
>
> volume raw
>   type storage/posix
>   option directory /fileserver
> end-volume
>
> volume brick
>   type features/posix-locks
>   subvolumes raw
> end-volume
>
> volume server
>   type protocol/server
>   option transport-type tcp
>   subvolumes brick
>   option auth.addr.brick.allow *
> end-volume
>
>
> The first stanza selects the basic folder to export.
>
> The second stanza adds file locking to it. This is required to support
> Firefox, and some other applications.
>
> The third stanza authorizes everyone to access this file-locked export
> over the network.
>
> There is also a /etc/glusterfs/glusterfsd.vol file on the system, for
> configuring the management interface. For this simple installation it
> does not need to be modified.
>
> --------------------------------------------
> 4. Restart the server services:
> - service glusterd restart
> - service glusterfsd restart
>
> I believe the first service is a management service, and the second is
> the actual file-export service.
> --------------------------------------------
>
> --------------------------------------------
> 5. On the client, create the /etc/glusterfs/glusterfs.vol configuration
> file, which should look like this:
>
> volume client
>   type protocol/client
>   option transport-type tcp
>   option remote-host 192.168.0.3  # use YOUR server IP here
>   option remote-subvolume brick
> end-volume
>
>
> --------------------------------------------
> 6. On the client, add this line to the end of /etc/rc.d/rc.local:
>
> mount -t glusterfs /etc/glusterfs/glusterfs.vol /fileserver
>
> --------------------------------------------
> 7. On the client, reboot, and check /var/log/messages for errors. On one
> machine, we had an selinux problem that was flagged in the logs. We had
> to manually create the logging file using:
>
> touch /var/log/glusterfs/fileserver.log; reboot
>
> --------------------------------------------
> 8. On the client, see if you can access the files in /fileserver. If
> not, read the /var/log/glusterfs/* files on both the client and the server.
>
> At this point, everything should work!
>
>
>
> Weird things and gotchas:
>
> A. You need the file-locking option to make Firefox work properly.
>
> B. LibreOffice wouldn't start on one system, until we did:
> rm ~/.libreoffice
> rm ~/.openoffice.org
>
> C. selinux prevented the creation of log files on one client, which
> prevented the filesystem from mouting. The manual fix noted above fixed
> that.
>
> D. This HOWTO mounts the glusterfs from /etc/rc.d/rc.local, which is the
> last step in the boot process. In theory, you can mount it from
> /etc/fstab or using autofs. However, we found that autofs mounting just
> didn't work - not sure why. fstab mounting didn't work either - I
> suspect it occurred too early in the boot process. /etc needs to be up
> and running so glusterfs can read the config file, and I don't think the
> current init/systemd files handle this correctly. There are some Debian
> bug reports about this that you can google.
>
>
> I hope this is useful to someone, and that we can finally drive a stake
> through the heart of NFS...
>
>
> - Mike
> --
> users mailing list
> users at lists.fedoraproject.org
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
>

-- 
-- 
Marcos Luis Ortíz Valmaseda
 Software Engineer (UCI)
 Linux User # 418229
 http://marcosluis2186.posterous.com
 http://www.linkedin.com/in/marcosluis2186
 https://fedoraproject.org/wiki/User:Marcosluis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/users/attachments/20110825/407be2c4/attachment.html 