Converting ext4 to xfs

[Suvayu Ali]

Hi everyone,

I was wondering if there was any way I could convert an ext4 partition 
into an xfs partition without copying around files onto a separate 
partition and reformatting. The partition in question is my /home on a 
volume group of its own.

I realised, lately I have been dealing with very large files quite 
often. Ranging from few hundred megs to a gig or two. So I decided to 
switch to xfs. Do you think its worth the effort?

Also since we are talking about my /home here, there will also be small 
conf/settings files along with the large files. Do you think that would 
be disadvantageous somehow? Would it make more sense to shrink my /home 
and have a separate xfs partition in the created space for the large files?

Thanks for any thoughts on this.

[Michael Cronenworth]

Suvayu Ali wrote:
> So I decided to
> switch to xfs. Do you think its worth the effort?

AFAIK there is no conversion for ext4->xfs. However, there is a 
conversion for ext4->btrfs. It is also reversible if you end up not 
liking btrfs. I've switched to btrfs as my file system of choice though. 
(Former xfs user)

[Kwan Lowe]

On Fri, Oct 1, 2010 at 2:43 AM, Suvayu Ali  wrote:
> Hi everyone,
>
> I was wondering if there was any way I could convert an ext4 partition
> into an xfs partition without copying around files onto a separate
> partition and reformatting. The partition in question is my /home on a
> volume group of its own.

I don't have a real answer except perhaps to create a new LVM for XFS
then slowly migrate files from one LVM/fs to the other.  As you fill
space on one, shrink the other filesystem to reclaim those LPs back
into the VG.  It's messy but should work.  AFAIK, there is no in-place
conversion utility as the two filesystems are quite different.
> I realised, lately I have been dealing with very large files quite
> often. Ranging from few hundred megs to a gig or two. So I decided to
> switch to xfs. Do you think its worth the effort?

Curios about this though -- what sort of performance are you seeing
with ext4 vs XFS?  I'm running only a few XFS filesystems but I don't
see a huge performance increase versus ext4.  My workload is typically
small sound files.
> Also since we are talking about my /home here, there will also be small
> conf/settings files along with the large files. Do you think that would
> be disadvantageous somehow? Would it make more sense to shrink my /home
> and have a separate xfs partition in the created space for the large files?
>
> Thanks for any thoughts on this.
>
> --
> Suvayu
> Open source is the future. It sets us free.
>
> users mailing list
> [e-mail]
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

[Michael Cronenworth]

Kwan Lowe wrote:
> I don't have a real answer except perhaps to create a new LVM for XFS
> then slowly migrate files from one LVM/fs to the other.  As you fill
> space on one, shrink the other filesystem to reclaim those LPs back
> into the VG.  It's messy but should work.  AFAIK, there is no in-place
> conversion utility as the two filesystems are quite different.

XFS doesn't support shrinking. :)

(patches existed for it but they were never accepted)

[JD]

On 10/01/2010 11:00 AM, Kwan Lowe wrote:
> On Fri, Oct 1, 2010 at 2:43 AM, Suvayu Ali  wrote:
> > Hi everyone,
> >
> > I was wondering if there was any way I could convert an ext4 partition
> > into an xfs partition without copying around files onto a separate
> > partition and reformatting. The partition in question is my /home on a
> > volume group of its own.
>
> I don't have a real answer except perhaps to create a new LVM for XFS
> then slowly migrate files from one LVM/fs to the other.  As you fill
> space on one, shrink the other filesystem to reclaim those LPs back
> into the VG.  It's messy but should work.  AFAIK, there is no in-place
> conversion utility as the two filesystems are quite different.
> > I realised, lately I have been dealing with very large files quite
> > often. Ranging from few hundred megs to a gig or two. So I decided to
> > switch to xfs. Do you think its worth the effort?
>
> Curios about this though -- what sort of performance are you seeing
> with ext4 vs XFS?  I'm running only a few XFS filesystems but I don't
> see a huge performance increase versus ext4.  My workload is typically
> small sound files.
> > Also since we are talking about my /home here, there will also be small
> > conf/settings files along with the large files. Do you think that would
> > be disadvantageous somehow? Would it make more sense to shrink my /home
> > and have a separate xfs partition in the created space for the large files?
> >
> > Thanks for any thoughts on this.
> >
> > --
> > Suvayu
> > Open source is the future. It sets us free.
> >
> > users mailing list
> > [e-mail]
> > To unsubscribe or change subscription options:
> > https://admin.fedoraproject.org/mailman/listinfo/users
> > Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Hi Kwan,
I had run performance measurement for reading and
writing of multimegabyte files in linux xfs and ext3.
There was absolutely no denying that ext3 outperformed
xfs by a long shot. Unfortunately, I do not recall the exact
numbers.

One thing that is overlooked by a lot of people is that
the linux xfs IS NOT what runs on the big SGI machines.
Many performance features were not ported because the
x86 architecture does not support, for example,  huge pages
(64k page). the SGI 64 bit architecture supports these
large pages, and perhaps even larger. There are several
implementations of the MIPS architecture.
SGI was probably using a much more robust implementation.
Also, high performance xfs requires large amounts of ram
for the btrees that are searched for the exact blocks to access.
Most x86 machines, especially laptops, are poor targets
for such an architecture. So, many of the features were
removed or scaled back to a size that effectively killed
performance on pc's.

Even on Linux x86_64, with 16GB of ram, the xfs is
still a low performance FS as compared with ext3.

Perhaps SGI will embark on a new port for the new intel
64bit architectures (don't hold your breath :)

These tests were done back in 2006.

I am not sure if the linux xfs has undergone
any serious performance improvements since then.

[Suvayu Ali]

Hi Michael,

On 1 October 2010 08:57, Michael Cronenworth  wrote:
> Suvayu Ali wrote:
> > So I decided to
> > switch to xfs. Do you think its worth the effort?
>
> AFAIK there is no conversion for ext4->xfs. However, there is a
> conversion for ext4->btrfs. It is also reversible if you end up not
> liking btrfs. I've switched to btrfs as my file system of choice though.
> (Former xfs user)

Thank you for your input. After your response, I read a little bit
about btrfs. It seems the feature set is quite enticing. I am
considering using it except that there seems to be a lack of gui tools
for it. I am not very familiar with dealing with filesystems,
otherwise I am a quite comfortable command line user.

Apart from a howtoforge article and a couple of pages on the btrfs
wiki I couldn't find beginner level documentation to get familiar with
the command line tools for btrfs. Could you point me to some resources
to get started?

Thanks a lot everyone who responded.

[Nifty Fedora Mitch]

On Fri, Oct 1, 2010 at 11:27 AM, JD  wrote:
> On 10/01/2010 11:00 AM, Kwan Lowe wrote:
> > On Fri, Oct 1, 2010 at 2:43 AM, Suvayu Ali  wrote:
> > > Hi everyone,
> > >
> > > I was wondering if there was any way I could convert an ext4 partition
> > > into an xfs partition without copying around files onto a separate
> > > partition and reformatting.

As other noted shrinking is not an option also it should
be noted that block 1 of any file system cannot move
without special handling....
> > > I realised, lately I have been dealing with very large files quite
> > > often. Ranging from few hundred megs to a gig or two. So I decided to
> > > switch to xfs. Do you think its worth the effort?

It would require some benchmarking.   A lot of people
expect a new file system to run faster than the underlying hardware ;-)
XFS posted its best results on a system with a very large
number of disk controllers i.e. big fast parallel hardware
supporting a parallel RAID/LogicalVolume device.
> > Curios about this though -- what sort of performance are you seeing
> > with ext4 vs XFS?  I'm running only a few XFS filesystems but I don't
> > see a huge performance increase versus ext4.  My workload is typically
> > small sound files.

It is important to benchmark your raw disk and compare the
raw results with the filesystem results.    Small sound files
are closer to the file load that a mail server might have.   There
have been numerous discussions in the context of mail and
net news....
> > > Also since we are talking about my /home here, there will also be small
> > > conf/settings files along with the large files. Do you think that would
> > > be disadvantageous somehow? Would it make more sense to shrink my /home
> > > and have a separate xfs partition in the created space for the large files?

I doubt that it would help more than single digit %age one way or another.
> > > --
> > > Suvayu
> > >
>
> Hi Kwan,
> I had run performance measurement for reading and
> writing of multimegabyte files in linux xfs and ext3.
> There was absolutely no denying that ext3 outperformed
> xfs by a long shot. Unfortunately, I do not recall the exact
> numbers.

Linux and ext3/4 are a good pair.
> One thing that is overlooked by a lot of people is that
> the linux xfs IS NOT what runs on the big SGI machines.
> Many performance features were not ported because the
> x86 architecture does not support, for example,  huge pages
> (64k page). the SGI 64 bit architecture supports these
> large pages, and perhaps even larger. There are several
> implementations of the MIPS architecture.
> SGI was probably using a much more robust implementation.
> Also, high performance xfs requires large amounts of ram
> for the btrees that are searched for the exact blocks to access.
> Most x86 machines, especially laptops, are poor targets
> for such an architecture. So, many of the features were
> removed or scaled back to a size that effectively killed
> performance on pc's.

As far as I know the xfs in Linux is the same XFS that
Irix had.   However Irix was designed to support lots
of parallel hardware, I/O and ccNUMA processing.  It
is the Irix operating system that establishes a foundation
that Linux is just beginning to address.

Large pages are just one of the core operating system
features that made Irix a world class Unix version (still
is).
> Even on Linux x86_64, with 16GB of ram, the xfs is
> still a low performance FS as compared with ext3.
>
> Perhaps SGI will embark on a new port for the new intel
> 64bit architectures (don't hold your breath :)

Do not bet on it... the old SGI is gone.
> These tests were done back in 2006.
>
> I am not sure if the linux xfs has undergone
> any serious performance improvements since then.

You are correct..   yet the design goals of
xfs had a lot to do with durability combined
with the speed that a journaled FS could have.
One of the important design goals of XFS was
how it survives the reset button/ power failure
and how correctly and quickly fsck would finish
on reboot.

It should be noted that XFS supports attributes in
a way that enables SELinux and Trix (security enhanced trusted
Irix) to operate reliably.

At the time these were big and landmark improvements in file system
technology.   Now that these are 'public' open source projects
can build on them....  thus ext4 "should" be an improvement
but your work load and hardware could tip the scales one way
or another.

[stan]

On Fri, 1 Oct 2010 22:02:32 -0700
suvayu ali  wrote:
> Apart from a howtoforge article and a couple of pages on the btrfs
> wiki I couldn't find beginner level documentation to get familiar with
> the command line tools for btrfs. Could you point me to some resources
> to get started?

There are man pages for each of the tools in the btrfs package,
btrfs-progs.

/usr/share/man/man8/btrfs-image.8.gz                                                             
/usr/share/man/man8/btrfs-show.8.gz                                                              
/usr/share/man/man8/btrfsck.8.gz                                                                 
/usr/share/man/man8/btrfsctl.8.gz                                                                
/usr/share/man/man8/mkfs.btrfs.8.gz

Because of your inquiry, I also did some research.  Btrfs sounded like
the bees knees, but I got cold feet when I read all the warnings about
it being suitable only for benchmarking and testing, and not to run any
production or even slightly valuable information on it.  I was going to
run F14 on it when I install it.  Think I'll stick with ext4 now. Or
maybe not. :-)

[stan]

Just a PS for anyone else interested in possibly running btrfs, an
article written last year by a filesystem developer about btrfs.

http://lwn.net/Articles/342892/

[JD]

On 10/02/2010 09:15 AM, stan wrote:
> Just a PS for anyone else interested in possibly running btrfs, an
> article written last year by a filesystem developer about btrfs.
>
> http://lwn.net/Articles/342892/

Thanx for the link Stan.
Very interesting article.