USB 2.0 vs FireWire?

[Chris Ruprecht]

Hello,

I have a Dell PowerEdge 2500 server with FC6 + latest updates installed.

For backup, I want to use 2 external disk drives, 300 GB each (all 
internal stuff is SCSI). I bought a USB 2.0/FireWire combo card and am 
running a few tests and to my great surprise, USB 2 (480 mbps) is way 
faster than FireWire 400 (400 mbps). I get this from hdparm:

Connected as USB 2.0 drive:
hdparm -t -T /dev/sdd

/dev/sdd:
 Timing cached reads:   948 MB in  2.00 seconds = 473.56 MB/sec
 Timing buffered disk reads:   76 MB in  3.03 seconds =  25.11 MB/sec

Connected as FireWire 400 drive:
hdparm -t -T /dev/sdd

/dev/sdd:
 Timing cached reads:   948 MB in  2.00 seconds = 473.05 MB/sec
 Timing buffered disk reads:   32 MB in  3.07 seconds =  10.44 MB/sec

To make sure that hdparm is not messing with me, as the only reference I 
find in its man page is relating to IDE and SCSI drives, I tested some 
simple copy command of a 4 GB file. I did each command twice in a row, 
to avoind any skewing of the results by anything in OS buffers.

USB 2.0 Mode:
time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx3
`/home/db/pm/pm_10.d2' -> `/media/extd/xxx3'

real    3m20.819s
user    0m0.988s
sys     0m54.483s
[root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx4
`/home/db/pm/pm_10.d2' -> `/media/extd/xxx4'

real    3m18.799s
user    0m1.048s
sys     0m54.791s

FireWire Mode:
time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx1
`/home/db/pm/pm_10.d2' -> `/media/extd/xxx1'

real    7m33.109s
user    0m1.196s
sys     0m52.439s
[root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx2
`/home/db/pm/pm_10.d2' -> `/media/extd/xxx2'

real    9m13.851s
user    0m1.108s
sys     0m55.191s

I used the same drive, a Maxtor 320 GB drive that was on sale recently, 
which is mounted inside an Adaptec external USB/FireWire combo case.

Any ideas why FireWire is so much slower than USB? Should I look at the 
card, the drive or the drivers? I would really like to use the FireWire 
side of things, but not at this ridiculous  speed.

Thanks for any pointers.
Chris

[Development discussions related to Fedora]

Chris Ruprecht wrote:
> Hello,
>
> I have a Dell PowerEdge 2500 server with FC6 + latest updates installed.
>
> For backup, I want to use 2 external disk drives, 300 GB each (all 
> internal stuff is SCSI). I bought a USB 2.0/FireWire combo card and am 
> running a few tests and to my great surprise, USB 2 (480 mbps) is way 
> faster than FireWire 400 (400 mbps). I get this from hdparm:
>
> Connected as USB 2.0 drive:
> hdparm -t -T /dev/sdd
>
> /dev/sdd:
> Timing cached reads:   948 MB in  2.00 seconds = 473.56 MB/sec
> Timing buffered disk reads:   76 MB in  3.03 seconds =  25.11 MB/sec
>
> Connected as FireWire 400 drive:
> hdparm -t -T /dev/sdd
>
> /dev/sdd:
> Timing cached reads:   948 MB in  2.00 seconds = 473.05 MB/sec
> Timing buffered disk reads:   32 MB in  3.07 seconds =  10.44 MB/sec

Notice this is only looking at read speeds.
> To make sure that hdparm is not messing with me, as the only reference I 
> find in its man page is relating to IDE and SCSI drives, I tested some 
> simple copy command of a 4 GB file. I did each command twice in a row, 
> to avoind any skewing of the results by anything in OS buffers.
>
> USB 2.0 Mode:
> time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx3
> `/home/db/pm/pm_10.d2' -> `/media/extd/xxx3'
>
> real    3m20.819s
> user    0m0.988s
> sys     0m54.483s
> [root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx4
> `/home/db/pm/pm_10.d2' -> `/media/extd/xxx4'
>
> real    3m18.799s
> user    0m1.048s
> sys     0m54.791s
>
> FireWire Mode:
> time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx1
> `/home/db/pm/pm_10.d2' -> `/media/extd/xxx1'
>
> real    7m33.109s
> user    0m1.196s
> sys     0m52.439s
> [root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx2
> `/home/db/pm/pm_10.d2' -> `/media/extd/xxx2'
>
> real    9m13.851s
> user    0m1.108s
> sys     0m55.191s

You are causing the data to be duplicated on the disk; this might slow 
down with each subsequent write due to disks generally being fastest 
transfer at the begining, and speed trailing of by 20% toward the end of 
the disk.
> I used the same drive, a Maxtor 320 GB drive that was on sale recently, 
> which is mounted inside an Adaptec external USB/FireWire combo case.
>
> Any ideas why FireWire is so much slower than USB? Should I look at the 
> card, the drive or the drivers? I would really like to use the FireWire 
> side of things, but not at this ridiculous  speed.

With {probably} a different drive and dual interface caddy, my brother 
had seen similar results ie usb2 is about 2x the speed of firewire. He 
did notice that it is important to plug the second usb power header 
connector in or either interface could be much slower than expected. I 
think this might be because one interface /the drive might have been 
running close to minimum power limits. {his test was with winxp sp2 on a 
dell notebook}

The second thing to notice is that USB2 is a much newer standard. The 
original "firewire is faster" comes from a comparison with USB1.

Could you try another comparison by writing a single large file eg a 
dvd.iso to the drive ? I'd like to see the difference. Does it change 
much if you blank the drive, and then test via the firewire interface first.

Do you leave both interface cables plugged into the server when you are 
doing the test ? This might cause an extra slowdown.

Also, for reference, what is the size of your previous test {du -s 
whatever} ? And what did the write data-rate work out to be ?

It might be that a pure firewire interface would be better than the dual 
interface box.

In any case, for backups you probably want to use something like:
$ rsync -a /home/db/pm/pm_10.d2 /media/extd/xxx1

DaveT.

[Chris Ruprecht]

David,

thank you for the explanation, specially the "FireWire is much faster" 
bit ;). Yes, indeed it is much faster then USB 1.x.

to clear up some variables:
The machine in question has 8 internal SCSI drives, the 4 GB file 
resides on one of the drives. It doesn't have a DVD drive that I could 
use for the test, and I suppose the DVD drive would be slower than the 
external drive anyway.

The external drive only uses one of the two possible connectors at any 
one time and I completely powered both, the server and the drive, down, 
before conducting the test.

The external drive was blank when I started this test (mke2fs ...).

The pm_10.d2 file is a single large (4 GB) database extent of a database 
that is completely shut down. You might say that it is equivalent to a 4 
GB DVD ISO image, except for the content, which, for this test, should 
not be relevant.

I have since tested this on another machine, an IBM eServer xSeries 342 
running CentOS 4.4, with similar results. I don't think this issue is 
only related to FC, I think it's related to Linux in general. I don't 
have Windoze installed on a server, but I will test this on my other 
desktop box which has built-in as well as another FireWire card. But 
that will only happen tomorrow.

Best regards,
Chris

David Timms wrote:
> Chris Ruprecht wrote:
> > Hello,
> >
> > I have a Dell PowerEdge 2500 server with FC6 + latest updates installed.
> >
> > For backup, I want to use 2 external disk drives, 300 GB each (all 
> > internal stuff is SCSI). I bought a USB 2.0/FireWire combo card and 
> > am running a few tests and to my great surprise, USB 2 (480 mbps) is 
> > way faster than FireWire 400 (400 mbps). I get this from hdparm:
> >
> > Connected as USB 2.0 drive:
> > hdparm -t -T /dev/sdd
> >
> > /dev/sdd:
> > Timing cached reads:   948 MB in  2.00 seconds = 473.56 MB/sec
> > Timing buffered disk reads:   76 MB in  3.03 seconds =  25.11 MB/sec
> >
> > Connected as FireWire 400 drive:
> > hdparm -t -T /dev/sdd
> >
> > /dev/sdd:
> > Timing cached reads:   948 MB in  2.00 seconds = 473.05 MB/sec
> > Timing buffered disk reads:   32 MB in  3.07 seconds =  10.44 MB/sec
>
> Notice this is only looking at read speeds.
> > To make sure that hdparm is not messing with me, as the only 
> > reference I find in its man page is relating to IDE and SCSI drives, 
> > I tested some simple copy command of a 4 GB file. I did each command 
> > twice in a row, to avoind any skewing of the results by anything in 
> > OS buffers.
> >
> > USB 2.0 Mode:
> > time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx3
> > `/home/db/pm/pm_10.d2' -> `/media/extd/xxx3'
> >
> > real    3m20.819s
> > user    0m0.988s
> > sys     0m54.483s
> > [root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx4
> > `/home/db/pm/pm_10.d2' -> `/media/extd/xxx4'
> >
> > real    3m18.799s
> > user    0m1.048s
> > sys     0m54.791s
> >
> > FireWire Mode:
> > time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx1
> > `/home/db/pm/pm_10.d2' -> `/media/extd/xxx1'
> >
> > real    7m33.109s
> > user    0m1.196s
> > sys     0m52.439s
> > [root@mars ~]# time cp -v /home/db/pm/pm_10.d2 /media/extd/xxx2
> > `/home/db/pm/pm_10.d2' -> `/media/extd/xxx2'
> >
> > real    9m13.851s
> > user    0m1.108s
> > sys     0m55.191s
>
> You are causing the data to be duplicated on the disk; this might slow 
> down with each subsequent write due to disks generally being fastest 
> transfer at the begining, and speed trailing of by 20% toward the end 
> of the disk.
> > I used the same drive, a Maxtor 320 GB drive that was on sale 
> > recently, which is mounted inside an Adaptec external USB/FireWire 
> > combo case.
> >
> > Any ideas why FireWire is so much slower than USB? Should I look at 
> > the card, the drive or the drivers? I would really like to use the 
> > FireWire side of things, but not at this ridiculous  speed.
>
> With {probably} a different drive and dual interface caddy, my brother 
> had seen similar results ie usb2 is about 2x the speed of firewire. He 
> did notice that it is important to plug the second usb power header 
> connector in or either interface could be much slower than expected. I 
> think this might be because one interface /the drive might have been 
> running close to minimum power limits. {his test was with winxp sp2 on 
> a dell notebook}
>
> The second thing to notice is that USB2 is a much newer standard. The 
> original "firewire is faster" comes from a comparison with USB1.
>
> Could you try another comparison by writing a single large file eg a 
> dvd.iso to the drive ? I'd like to see the difference. Does it change 
> much if you blank the drive, and then test via the firewire interface 
> first.
>
> Do you leave both interface cables plugged into the server when you 
> are doing the test ? This might cause an extra slowdown.
>
> Also, for reference, what is the size of your previous test {du -s 
> whatever} ? And what did the write data-rate work out to be ?
>
> It might be that a pure firewire interface would be better than the 
> dual interface box.
>
> In any case, for backups you probably want to use something like:
> $ rsync -a /home/db/pm/pm_10.d2 /media/extd/xxx1
>
> DaveT.

[Les Mikesell]

On Sun, 2006-11-12 at 01:06, Chris Ruprecht wrote:
> to clear up some variables:
> The machine in question has 8 internal SCSI drives, the 4 GB file 
> resides on one of the drives. It doesn't have a DVD drive that I could 
> use for the test, and I suppose the DVD drive would be slower than the 
> external drive anyway.
>
> The external drive only uses one of the two possible connectors at any 
> one time and I completely powered both, the server and the drive, down, 
> before conducting the test.
> I have since tested this on another machine, an IBM eServer xSeries 342 
> running CentOS 4.4, with similar results. I don't think this issue is 
> only related to FC, I think it's related to Linux in general.

Maybe it's related to your chipset - I think there may be some
variations.  I gave up on firewire on fedora when disk access
broke completely in mid-FC4 and didn't work till mid-FC5.  I'm
using a WD external case and adaptec controller with the
CentOSplus kernel and get about the same speed you listed for
USB 2.0 - somewhere in the 20+MB/sec range in the hdparm test.
This box doesn't have USB 2.0 for a direct comparison but I
saw about the same speed but more CPU consumption when running
it on USB from a laptop (that was probably under FC3, though).

[Chris Ruprecht]

Les Mikesell wrote:
> On Sun, 2006-11-12 at 01:06, Chris Ruprecht wrote:
>
> Maybe it's related to your chipset - I think there may be some
> variations.  I gave up on firewire on fedora when disk access
> broke completely in mid-FC4 and didn't work till mid-FC5.  I'm
> using a WD external case and adaptec controller with the
> CentOSplus kernel and get about the same speed you listed for
> USB 2.0 - somewhere in the 20+MB/sec range in the hdparm test.
> This box doesn't have USB 2.0 for a direct comparison but I
> saw about the same speed but more CPU consumption when running
> it on USB from a laptop (that was probably under FC3, though).

It might be the chipset of the F/W card, not sure. I have another, older 
card, F/W only, I will try that in a bit.

Thanks,
Chris