Distributed file system
Bill Rugolsky Jr.
brugolsky at telemetry-investments.com
Thu Feb 24 15:57:53 UTC 2005
On Thu, Feb 24, 2005 at 12:54:57PM +0530, gaurav wrote:
> I have around 50 machines in my lab, since user data is lying
> at central server (all home directory mounted here)....around 50 to 70%
> storage of local machines is un used .. I was thinking if there was a
> file system using which
>
> 1. Is Distributed across all these machines
> 2. Transparent to users (i.e for users can access thru normal path eg
> /dist/tom/data )
> 3. Redundancy factor (Since files are distributed, if one part of
> gets corrupt it should automatically restore)
> 4. ACL
> 5. Scalable
>
> Is this my wild dream ...or stuff like already exists..if yes the pl
> share.I was evaluating CODA but it is not ready for production release
OpenAFS, Lustre, GFS. None of these do precisely what you want, but you
ought to evaluate them.
There are a number of experimental peer-to-peer filesystems based upon
distributed hashes. Several are implemented in Java, e.g., OceanStore,
and have convenient Java APIs, but aren't readily usable as general-purpose
filesystems.
MIT has a (dormant?) project called Ivy that uses SFS and DHash/Chord to
implement a distributed peer-to-peer filesystem:
http://www.pdos.lcs.mit.edu/ivy/
Ivy is log-structured, so reclaiming space of "deleted" files is a problem.
As with NFS, the semantics of most p2p filesystems is not POSIX, so there
are limits on what you can do with them. But for archival storage, they're
great!
A fun/useful project for a bored student would be to port the various
user-space filesystems (CFS, SFS/Ivy, etc.) that use the NFS API (or
Java APIs), to the FUSE (Filesystems in Userspace, fuse.sourceforge.net)
API, which is currently in Andrew Morton's -mm kernel tree, awaiting
more testing before merging.
Regards,
Bill Rugolsky
More information about the users
mailing list