On Sat, Jun 27, 2020 at 8:01 AM Konstantin Kharlamov <hi-angel(a)yandex.ru> wrote:
I see no one mentined yet: BTRFS is slow on HDDs. It trivially comes from BTRFS
being COW. So if you changed a bit in a file, BTRFS will copy a block (or maybe
a number of them, not sure this detail matters) to another place, and now your
data got fragmented. SSDs may not care, HDDs on the other hand do.
It's faster on some workloads, slower on others. There are
optimizations to help make up for COW: inline extents for small files,
and random writes that commit together (i.e. in the same 30s window)
will be written as sequential writes. It is true btrfs does not have
nearly as many locality optimizations as ext4 and xfs, but at least
xfs developers have recently proposed removing those HDD optimizations
in favor of optimizations that are more relevant to today's hardware
Another reason worth mentioning: BTRFS per se is slow. If you look at
on Phoronix comparing BTRFS with others, BTRFS is rarely even on par with them.
It wins some. It loses others. Head over to the xfs list and enjoy the
benchmark commentary from people who actually understand benchmarking.
A recurring theme is that a benchmark is only as relevant to the
degree it actually mimics the workload you care about. And most
benchmark tools don't do that very well.
Here's a benchmark that's apples to apples because I'm merely timing
the time to compile the exact same thing each time, twice.
They're all in the same ballpark, except there's a write time hit for
the one with zstd:1 on this particular setup (and the compression hit
isn't consistent across all hardware or setups, it's case by case -
and hence the proposal option for compression indicates applying it
selectively to locations we know there's a benefit across the board).
But also you can tell there's no read time (decompression) hit from
this same data set.
Meanwhile, this is somewhere between embarrassing and comedy:
Hmmm, 21 seconds to launch GNOME Terminal with an NVMe and you aren't
curious about what went wrong? Because obviously something is wrong.
The measurement is wrong or the method is wrong or something in the
setup is fouling things up. How do you get a fast result with SSD but
then such a slow result with NVMe?
It makes no sense, but meh, we'll just publish that shit anyway! LOLZ!
And that is how you light your credibility on fire, because you just
don't give a crap about it.
On my 9 year old laptop with a mere Samsung 840 EVO, barely under 1
second for GNOME Terminal to launch, following a reboot and login so
this is not the result of caching. On my much newer HP Spectre with
NVMe, under 0.5s to launch.
My methodology and metrology? I'm using the "one mississippi" method
from finger click of the actual app icon to the time I see a cursor in
the launched app. Not rocket science.
As a matter of fact, I have two Archlinux laptops on BTRFS with
both only have HDD. I've been using for 3-4 years BTRFS there I think, maybe
more. I made use of BTRFS because I was hoping that using ZSTD would result in
less IO. Well, now my overall experience is that it is not rare that systems
starting to lag terribly, then I execute `grep "" /proc/pressure/*`, and see
someone is hogging IO. Then I pop up `iotop -a` and see among various processes
a `[btrfs-cleaner]` and `[btrfs-transacti]`. It may be because of defrag option,
I'm not sure…
There are many btrfs threads. Those actually make it more performant.
If you look at their total cpu time though, e.g. ps aux, you'll see
it's really small compared to most anything else you might think is
root 366 0.0 0.0 0 0 ? S Jun25 1:22
root 500 0.0 0.0 0 0 ? S Jun25 1:45
dbus 538 0.0 0.0 271548 6968 ? S Jun25 1:13
dbus-broker --log 4 --controller 9 --machine-id
ce3f1eade82d42bd891a8c15714b13cf --max-bytes 536870912 --m
root 1328 0.0 0.1 1273476 10116 ? Sl Jun25 3:00
There is in fact a WTF moment as a result of this partial listing and
it's not btrfs.
BTW this is 2 days of uptime.