BTRFS dropped by RedHat

[Przemek Klosowski]

The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...

    Btrfs has been deprecated

    The Btrfs file system has been in Technology Preview state since the
    initial release of Red Hat Enterprise Linux 6. Red Hat will not be
    moving Btrfs to a fully supported feature and it will be removed in
    a future major release of Red Hat Enterprise Linux.
    The Btrfs file system did receive numerous updates from the upstream
    in Red Hat Enterprise Linux 7.4 and will remain available in the Red
    Hat Enterprise Linux 7 series. However, this is the last planned
    update to this feature.

I think RH roadmap is to use XFS over LVM.

This is a pity---BTRFS features looked attractive:

- integrated RAID that ties low level (block/stripe) issues with 
high-level objects (files); I thought this is important because with 
brfs filesystem integrity features filesystem-level trouble could be 
tied to low level issues like silent failures on one raid element. This 
is important and unique: I had seen failures of large volumes both on 
proprietary RAID hardware and in software RAID, due to silent corruption 
of one element of the array, that propagated to other healthy elements.

- snapshotting/rollbacks that enable recovery system update failures and 
other nice functionality

- scalable support for really large file systems (reasonable fsck times, 
etc)

Are people who care about mass storage issues aware of RedHat's plans 
and are OK with the situation? Are there any other options apart from 
what RedHat is planning?

[Fernando Nasser]

On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
>  The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
>  
Is it only RHEL?

What are other distros doing?

> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>
>      Btrfs has been deprecated
>
>      The Btrfs file system has been in Technology Preview state since
>      the initial release of Red Hat Enterprise Linux 6. Red Hat will
>      not be moving Btrfs to a fully supported feature and it will be
>      removed in a future major release of Red Hat Enterprise Linux.
>      The Btrfs file system did receive numerous updates from the
>      upstream in Red Hat Enterprise Linux 7.4 and will remain available
>      in the Red Hat Enterprise Linux 7 series. However, this is the
>      last planned update to this feature.
>
>  I think RH roadmap is to use XFS over LVM.
>
>  This is a pity---BTRFS features looked attractive:
>
>  - integrated RAID that ties low level (block/stripe) issues with 
>  high-level objects (files); I thought this is important because with 
>  brfs filesystem integrity features filesystem-level trouble could be 
>  tied to low level issues like silent failures on one raid element. 
>  This is important and unique: I had seen failures of large volumes 
>  both on proprietary RAID hardware and in software RAID, due to silent 
>  corruption of one element of the array, that propagated to other 
>  healthy elements.
>
>  - snapshotting/rollbacks that enable recovery system update failures 
>  and other nice functionality
>
>  - scalable support for really large file systems (reasonable fsck 
>  times, etc)
>
>  Are people who care about mass storage issues aware of RedHat's plans 
>  and are OK with the situation? Are there any other options apart from 
>  what RedHat is planning?
>
>
>
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org

[Steven Whitehouse]

On 04/08/17 16:23, Fernando Nasser wrote:
>  On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
> >
> > The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
> >
>
>  Is it only RHEL? 
Yes, it is only RHEL. It does not have any effect on Fedora, that
is 
entirely independent,

Steve.

[Solomon Peachy]

On Fri, Aug 04, 2017 at 11:12:46AM -0400, Przemek Klosowski wrote:
>  This is a pity---BTRFS features looked attractive: 
Regardless of the attractiveness of btr's feature sets, the RH 
annoucnement is an indication that they don't consider it supportable 
for the kinds of users that fork over money for RHEL.

And FWIW, I intend to agree.  For all of btrfs's promise, in my personal 
experience it fails the most basic requirement of a filesystem -- to not 
eat data.  *Every* time I've experimented with btrfs, it's ended with 
massive filesystem corruption.  No unclean shutdowns or other hardware 
failures, and I was also just sticking to basic "give me a simple 
filesystem" feature set.

 - Solomon
-- 
Solomon Peachy			       pizza at shaftnet dot org
Delray Beach, FL                          ^^ (email/xmpp) ^^
Quidquid latine dictum sit, altum videtur.

[Neal Gompa]

On Fri, Aug 4, 2017 at 11:12 AM, Przemek Klosowski
<przemek.klosowski(a)nist.gov&gt; wrote:
>  The release notes for RHEL 7.4 announce that RedHat gave up on
> btrfs:
>
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>
>  Btrfs has been deprecated
>
>  The Btrfs file system has been in Technology Preview state since the initial
>  release of Red Hat Enterprise Linux 6. Red Hat will not be moving Btrfs to a
>  fully supported feature and it will be removed in a future major release of
>  Red Hat Enterprise Linux.
>  The Btrfs file system did receive numerous updates from the upstream in Red
>  Hat Enterprise Linux 7.4 and will remain available in the Red Hat Enterprise
>  Linux 7 series. However, this is the last planned update to this feature.
>
>  I think RH roadmap is to use XFS over LVM.
>
>  This is a pity---BTRFS features looked attractive:
>
>  - integrated RAID that ties low level (block/stripe) issues with high-level
>  objects (files); I thought this is important because with brfs filesystem
>  integrity features filesystem-level trouble could be tied to low level
>  issues like silent failures on one raid element. This is important and
>  unique: I had seen failures of large volumes both on proprietary RAID
>  hardware and in software RAID, due to silent corruption of one element of
>  the array, that propagated to other healthy elements.
>
>  - snapshotting/rollbacks that enable recovery system update failures and
>  other nice functionality
>
>  - scalable support for really large file systems (reasonable fsck times,
>  etc)
>
>  Are people who care about mass storage issues aware of RedHat's plans and
>  are OK with the situation? Are there any other options apart from what
>  RedHat is planning?
>  
I'm not particularly pleased with their decision. I think the path
they're going down is wrong, and they should really reconsider.
However, I'm reading over their Stratis whitepaper before I formulate
a response about it.

I don't think they fully understand how the Stratis path cannot
replace what Btrfs gives you, and the irony about this is that the
Btrfs developers been working on fixing the major remaining issues
with RAID this year and it looks like it'll be much better next year.

All of my CentOS 7 servers (and my one RHEL 7 box) use Btrfs, because
it's so much better than the alternatives.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Josh Boyer]

On Fri, Aug 4, 2017 at 11:12 AM, Przemek Klosowski
<przemek.klosowski(a)nist.gov&gt; wrote:
>  The release notes for RHEL 7.4 announce that RedHat gave up on
> btrfs:
>
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>
>  Btrfs has been deprecated 
This is a statement on RHEL 7.  It does not reflect on Fedora.

>  This is a pity---BTRFS features looked attractive: 
What is a pity is that the attractive looking features and the
stability of the filesystem did not line up as one would hope.  Btrfs
works well for some use cases, but for others it can be error prone
and result in data loss.

>  Are people who care about mass storage issues aware of RedHat's
> plans and
>  are OK with the situation? Are there any other options apart from what
>  RedHat is planning? 
Btrfs remains enabled in Fedora.  Realistically, it will remain
enabled and selectable as an option for installs for the foreseeable
future.  Whether a user wishes to use it or not remains up to them.

josh

[František Zatloukal]

Some insight why was BTRFS dropped by RH:
https://news.ycombinator.com/item?id=14907771

Also check out Stratis:
https://fedoraproject.org/wiki/Changes/StratisStorage

2017-08-04 17:12 GMT+02:00 Przemek Klosowski <przemek.klosowski(a)nist.gov&gt;:

>  The release notes for RHEL 7.4 announce that RedHat gave up on
> btrfs:
>
>  https://access.redhat.com/documentation/en-US/Red_Hat_
>  Enterprise_Linux/7/html/7.4_Release_Notes/chap-Red_Hat_
>  Enterprise_Linux-7.4_Release_Notes-Deprecated_Functionality.html
>
>  Btrfs has been deprecated
>
>  The Btrfs file system has been in Technology Preview state since the
>  initial release of Red Hat Enterprise Linux 6. Red Hat will not be moving
>  Btrfs to a fully supported feature and it will be removed in a future major
>  release of Red Hat Enterprise Linux.
>  The Btrfs file system did receive numerous updates from the upstream in
>  Red Hat Enterprise Linux 7.4 and will remain available in the Red Hat
>  Enterprise Linux 7 series. However, this is the last planned update to this
>  feature.
>
>  I think RH roadmap is to use XFS over LVM.
>
>  This is a pity---BTRFS features looked attractive:
>
>  - integrated RAID that ties low level (block/stripe) issues with
>  high-level objects (files); I thought this is important because with brfs
>  filesystem integrity features filesystem-level trouble could be tied to low
>  level issues like silent failures on one raid element. This is important
>  and unique: I had seen failures of large volumes both on proprietary RAID
>  hardware and in software RAID, due to silent corruption of one element of
>  the array, that propagated to other healthy elements.
>
>  - snapshotting/rollbacks that enable recovery system update failures and
>  other nice functionality
>
>  - scalable support for really large file systems (reasonable fsck times,
>  etc)
>
>  Are people who care about mass storage issues aware of RedHat's plans and
>  are OK with the situation? Are there any other options apart from what
>  RedHat is planning?
>
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>
>

[Neal Gompa]

On Fri, Aug 4, 2017 at 11:23 AM, Fernando Nasser <fnasser(a)redhat.com&gt; wrote:
>  On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
>
>  The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
>
>
>  Is it only RHEL?
>
>  What are other distros doing?
>  
It is only RHEL, but unfortunately that has a *huge* knock on effect
across hundreds of derivative distribution projects and products.

It's an enormous problem that they're doing this...

-- 
真実はいつも一つ！/ Always, there's only one truth!

[David Sommerseth]

On 04/08/17 18:06, Neal Gompa wrote:
>  On Fri, Aug 4, 2017 at 11:23 AM, Fernando Nasser
> <fnasser(a)redhat.com&gt; wrote:
> > On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
> >
> > The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
> >
> >
> > Is it only RHEL?
> >
> > What are other distros doing?
> >
>  
>  It is only RHEL, but unfortunately that has a *huge* knock on effect
>  across hundreds of derivative distribution projects and products.
>  
>  It's an enormous problem that they're doing this... 
Hmmm, that sounds backwards to me.   I would say it is more a problem
that Btrfs haven't reached a stability/trust level over so many years
which makes Enterprise Linux considering to use it.  To me this more
indicates the overall state of Btrfs.

I certainly do hope that Btrfs development doesn't slow down by this,
rather that the pace gets improved and it will come back again in
stronger and more solid during a future RHEL 8 release.

-- 
kind regards,

David Sommerseth

[Richard Hughes]

On 4 August 2017 at 16:46, Solomon Peachy <pizza(a)shaftnet.org&gt; wrote:
>  ...it fails the most basic requirement of a filesystem -- to not
>  eat data. 
Me also; I've tried btfs three times now -- in one case telling me I
had no free space after a few weeks when clearly there was tens GBs
free, one time needing to wipe and reformat as writes trudged to a few
thousand bytes per second and and the other time destroying my
file-system when I ran out of battery power. I think "it's getting
better, we promise" only works for so many years. I've never had a
problem with EXT3/4 or with XFS. YMMV.

Richard.

[Gerald B. Cox]

On Fri, Aug 4, 2017 at 10:18 AM, Richard Hughes <hughsient(a)gmail.com&gt; wrote:

>  On 4 August 2017 at 16:46, Solomon Peachy <pizza(a)shaftnet.org&gt;
> wrote:
>  > ...it fails the most basic requirement of a filesystem -- to not
>  > eat data.
>
>  Me also; I've tried btfs three times now -- in one case telling me I
>  had no free space after a few weeks when clearly there was tens GBs
>  free, one time needing to wipe and reformat as writes trudged to a few
>  thousand bytes per second and and the other time destroying my
>  file-system when I ran out of battery power. I think "it's getting
>  better, we promise" only works for so many years. I've never had a
>  problem with EXT3/4 or with XFS. YMMV.
>  
Yeah... I was very excited about RAID 6 capability and starting using it in
2013.  I took all the necessary precautions, backups, etc. and never lost
data.  I waited and waited for it to be declared stable and followed the
mailing list threads.  Then the announcement came in July 2016 that RAID
5/6 was toxic and a year later, it's still not fixed.  That was it for me
and I switched to XFS and counted my blessings I wasn't burnt as some
people have been.  Then there was this:  https://www.patreon.com/bcachefs
and this:  https://www.youtube.com/watch?v=79fvDDPaIoY&t=18m28s

I believe it's a good decision on Redhat's part.  At some point you have to
fish or cut bait.

[Tomasz Torcz]

On Fri, Aug 04, 2017 at 06:43:51PM +0200, David Sommerseth wrote:
>  On 04/08/17 18:06, Neal Gompa wrote:
>  > On Fri, Aug 4, 2017 at 11:23 AM, Fernando Nasser <fnasser(a)redhat.com&gt; wrote:
>  >> On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
>  >>
>  >> The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
>  >>
>  >>
>  >> Is it only RHEL?
>  >>
>  >> What are other distros doing?
>  >>
>  > 
>  > It is only RHEL, but unfortunately that has a *huge* knock on effect
>  > across hundreds of derivative distribution projects and products.
>  > 
>  > It's an enormous problem that they're doing this...
>  
>  Hmmm, that sounds backwards to me.   I would say it is more a problem
>  that Btrfs haven't reached a stability/trust level over so many years
>  which makes Enterprise Linux considering to use it.  To me this more
>  indicates the overall state of Btrfs. 
  There's more to Enteprise Linux than Red Hat. SUSE is happy
to support subset of btrfs' features for enterprise distribution..
 
>  I certainly do hope that Btrfs development doesn't slow down by
> this,
>  rather that the pace gets improved and it will come back again in
>  stronger and more solid during a future RHEL 8 release. 
  Red Hat has none (I think) developers working on btrfs. So btrfs
development speed is not affected by this. At all.

-- 
Tomasz Torcz                        Once you've read the dictionary,
xmpp: zdzichubg(a)chrome.pl           every other book is just a remix.

[Mauricio Tavares]

On Fri, Aug 4, 2017 at 11:48 AM, Neal Gompa <ngompa13(a)gmail.com&gt; wrote:
>  I'm not particularly pleased with their decision. I think the path
>  they're going down is wrong, and they should really reconsider.
>  However, I'm reading over their Stratis whitepaper before I formulate
>  a response about it.
>
>  I don't think they fully understand how the Stratis path cannot
>  replace what Btrfs gives you, and the irony about this is that the
>  Btrfs developers been working on fixing the major remaining issues
>  with RAID this year and it looks like it'll be much better next year.
>
>  All of my CentOS 7 servers (and my one RHEL 7 box) use Btrfs, because
>  it's so much better than the alternatives.
>  
      Better than ZFS?
>
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org

[Peter Robinson]

On Fri, Aug 4, 2017 at 7:35 PM, Tomasz Torcz <tomek(a)pipebreaker.pl&gt; wrote:
>  On Fri, Aug 04, 2017 at 06:43:51PM +0200, David Sommerseth wrote:
> > On 04/08/17 18:06, Neal Gompa wrote:
> > > On Fri, Aug 4, 2017 at 11:23 AM, Fernando Nasser <fnasser(a)redhat.com&gt;
> wrote:
> > >> On 2017-08-04 11:12 AM, Przemek Klosowski wrote:
> > >>
> > >> The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
> > >>
> > >>
> > >> Is it only RHEL?
> > >>
> > >> What are other distros doing?
> > >>
> > >
> > > It is only RHEL, but unfortunately that has a *huge* knock on effect
> > > across hundreds of derivative distribution projects and products.
> > >
> > > It's an enormous problem that they're doing this...
> >
> > Hmmm, that sounds backwards to me.   I would say it is more a problem
> > that Btrfs haven't reached a stability/trust level over so many years
> > which makes Enterprise Linux considering to use it.  To me this more
> > indicates the overall state of Btrfs.
>
>    There's more to Enteprise Linux than Red Hat. SUSE is happy
>  to support subset of btrfs' features for enterprise distribution..
>
> > I certainly do hope that Btrfs development doesn't slow down by this,
> > rather that the pace gets improved and it will come back again in
> > stronger and more solid during a future RHEL 8 release.
>
>    Red Hat has none (I think) developers working on btrfs. So btrfs
>  development speed is not affected by this. At all. 
No, but the ability to support it and to backport fixes and new
functionality to the enterprise kernel would be.

[Richard W.M. Jones]

On Fri, Aug 04, 2017 at 06:00:58PM +0200, František Zatloukal wrote:
>  Some insight why was BTRFS dropped by RH:
>  https://news.ycombinator.com/item?id=14907771
>  
>  Also check out Stratis:
>  https://fedoraproject.org/wiki/Changes/StratisStorage 
Stratis is a management tool that makes it a bit easier to manage all
the layers together.

Another aspect which seems indicative of Red Hat's direction [I have
no inside information on strategy] is the recent acquisition of
Permabit (http://permabit.com/).  AIUI it's a device mapper module
which does some clever data deduplication and/or compression across LVs:

https://www.theregister.co.uk/2016/06/29/permabit_offering_dedupe_to_linu...

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

[Neal Gompa]

On Sat, Aug 5, 2017 at 8:12 AM, Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:
>  On Fri, Aug 04, 2017 at 06:00:58PM +0200, František Zatloukal wrote:
> > Some insight why was BTRFS dropped by RH:
> > https://news.ycombinator.com/item?id=14907771
> >
> > Also check out Stratis:
> > https://fedoraproject.org/wiki/Changes/StratisStorage
>
>  Stratis is a management tool that makes it a bit easier to manage all
>  the layers together.
>
>  Another aspect which seems indicative of Red Hat's direction [I have
>  no inside information on strategy] is the recent acquisition of
>  Permabit (http://permabit.com/).  AIUI it's a device mapper module
>  which does some clever data deduplication and/or compression across LVs:
>
>
> https://www.theregister.co.uk/2016/06/29/permabit_offering_dedupe_to_linu...
>  
There are plenty of "clever" kernel modules that do bits and pieces of
what Btrfs offers. For example, the place I work for created a kernel
module that makes it possible to do VSS/CoW like snapshotting with
*any* legacy file system (as long as it has a real block device)
called dattobd:

https://github.com/datto/dattobd

It can enable snapshotting for legacy file systems like ext4, xfs,
jfs, etc. whether it's on LVM or not. It can also back up filesystems
on things like LVM or mdraid.

That being said, I've been told that it's unlikely that such
functionality will ever make it into the Linux kernel itself, and it's
not as smooth as if the filesystem itself is able to do these things
(like with Btrfs). The biggest pitfall of this approach? The
filesystem has to be "frozen" for a small period of time while the
snapshot is being taken. It's quite noticeable when you're using it on
an hourly schedule, but it's minimized to the littlest effect possible
without a filesystem like Btrfs.

The effect is similar when using LVM snapshots. Stratis is even less
useful than dattobd because it requires building the storage up in
that way, whereas dattobd works with existing installations and Btrfs
can be seeded from existing filesystems and from there you can
leverage all the benefits of Btrfs.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Richard W.M. Jones]

On Sat, Aug 05, 2017 at 12:58:15PM -0400, Neal Gompa wrote:
>  On Sat, Aug 5, 2017 at 8:12 AM, Richard W.M. Jones
> <rjones(a)redhat.com&gt; wrote:
>  > On Fri, Aug 04, 2017 at 06:00:58PM +0200, František Zatloukal wrote:
>  >> Some insight why was BTRFS dropped by RH:
>  >> https://news.ycombinator.com/item?id=14907771
>  >>
>  >> Also check out Stratis:
>  >> https://fedoraproject.org/wiki/Changes/StratisStorage
>  >
>  > Stratis is a management tool that makes it a bit easier to manage all
>  > the layers together.
>  >
>  > Another aspect which seems indicative of Red Hat's direction [I have
>  > no inside information on strategy] is the recent acquisition of
>  > Permabit (http://permabit.com/).  AIUI it's a device mapper module
>  > which does some clever data deduplication and/or compression across LVs:
>  >
>  >
> https://www.theregister.co.uk/2016/06/29/permabit_offering_dedupe_to_linu...
>  >
>  
>  There are plenty of "clever" kernel modules that do bits and pieces of
>  what Btrfs offers. For example, the place I work for created a kernel
>  module that makes it possible to do VSS/CoW like snapshotting with
>  *any* legacy file system (as long as it has a real block device)
>  called dattobd:
>  
>  https://github.com/datto/dattobd 
That's actually something we could really use and the missing
piece I've been looking for, to do "live" virt-p2v :-/

>  It can enable snapshotting for legacy file systems like ext4, xfs,
>  jfs, etc. whether it's on LVM or not. It can also back up filesystems
>  on things like LVM or mdraid.
>  
>  That being said, I've been told that it's unlikely that such
>  functionality will ever make it into the Linux kernel itself, and it's
>  not as smooth as if the filesystem itself is able to do these things
>  (like with Btrfs). The biggest pitfall of this approach? The
>  filesystem has to be "frozen" for a small period of time while the
>  snapshot is being taken. It's quite noticeable when you're using it on
>  an hourly schedule, but it's minimized to the littlest effect possible
>  without a filesystem like Btrfs. 
I totally accept your point that it's better integrated in the
filesystem.

OTOH back to the P2V case, being able to snapshot arbitrary
filesystems (which as you say Windows can already do with VSS) is
brilliant.  Did they ever try to get this upstream?

>  The effect is similar when using LVM snapshots. Stratis is even less
>  useful than dattobd because it requires building the storage up in
>  that way, whereas dattobd works with existing installations and Btrfs
>  can be seeded from existing filesystems and from there you can
>  leverage all the benefits of Btrfs. 
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

[Gerald B. Cox]

On Sat, Aug 5, 2017 at 9:58 AM, Neal Gompa <ngompa13(a)gmail.com&gt; wrote:

>  There are plenty of "clever" kernel modules that do bits and pieces of
>  what Btrfs offers. 
...there you can leverage all the benefits of Btrfs.
>  
That's all fine and good but features mean nothing if you don't have faith
in the underlying filesystem.
People have been burnt by Btrfs and just don't trust it's design. BcacheFS
now has the best chance of
being the next great thing - but Redhat wants to provide something for
their customers now and is tired
of playing the "it's getting better, we promise" game.  Hence the Stratis
strategy.  It's a wise approach.

[Neal Gompa]

On Sat, Aug 5, 2017 at 1:38 PM, Gerald B. Cox <gbcox(a)bzb.us&gt; wrote:
>  On Sat, Aug 5, 2017 at 9:58 AM, Neal Gompa <ngompa13(a)gmail.com&gt;
> wrote:
> >
> >
> > There are plenty of "clever" kernel modules that do bits and pieces of
> > what Btrfs offers.
> >
> > ...there you can leverage all the benefits of Btrfs.
> >
>
>  That's all fine and good but features mean nothing if you don't have faith
>  in the underlying filesystem.
>  People have been burnt by Btrfs and just don't trust it's design. BcacheFS
>  now has the best chance of
>  being the next great thing - but Redhat wants to provide something for their
>  customers now and is tired
>  of playing the "it's getting better, we promise" game.  Hence the Stratis
>  strategy.  It's a wise approach.
>  
No. Red Hat has *zero* engineers that work on Btrfs. That's the real
reason for this.

If Red Hat *really* wanted it, they would have got someone to work on
it specifically to accelerate the stabilization of the filesystem code
for the next RHEL. That *never* happened since Josef Bacik left Red
Hat for Facebook years ago. Josef still works on Btrfs there, just for
Facebook instead of Red Hat.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Gerald B. Cox]

On Sat, Aug 5, 2017 at 10:56 AM, Neal Gompa <ngompa13(a)gmail.com&gt; wrote:

>  On Sat, Aug 5, 2017 at 1:38 PM, Gerald B. Cox <gbcox(a)bzb.us&gt;
> wrote:
>  >
>  > That's all fine and good but features mean nothing if you don't have
>  faith
>  > in the underlying filesystem.
>  > People have been burnt by Btrfs and just don't trust it's design.
>  BcacheFS
>  > now has the best chance of
>  > being the next great thing - but Redhat wants to provide something for
>  their
>  > customers now and is tired
>  > of playing the "it's getting better, we promise" game.  Hence the
> Stratis
>  > strategy.  It's a wise approach.
>  >
>
>  No. Red Hat has *zero* engineers that work on Btrfs. That's the real
>  reason for this.
>
>  If Red Hat *really* wanted it, they would have got someone to work on
>  it specifically to accelerate the stabilization of the filesystem code
>  for the next RHEL. That *never* happened since Josef Bacik left Red
>  Hat for Facebook years ago. Josef still works on Btrfs there, just for
>  Facebook instead of Red Hat.
>  
Why do you think they have zero engineers that work on it?
If they believed it was worthwhile they would have devoted resources to
it.  All
you have to do is connect the dots.

[Chris Murphy]

On Fri, Aug 4, 2017 at 9:12 AM, Przemek Klosowski
<przemek.klosowski(a)nist.gov&gt; wrote:
>  The release notes for RHEL 7.4 announce that RedHat gave up on
> btrfs:
>
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
I see it as acknowledgment Btrfs is stable enough that it's
nonsensical to keep on calling it a technology preview, and also not
explicitly supporting it for paying customers. Red Hat doesn't have
the developers to support it, so it quite literally has no choice but
to deprecate it.

In contrast, SUSE has had reliable atomic updates and rollbacks for
two years, based on Btrfs, in both their enterprise and community
distros. But they also have a bunch of upstream developers to support
it.

This is a gem. Consider this in-production scale example when deciding
whether Btrfs is stable.
https://www.spinics.net/lists/linux-btrfs/msg67308.html

Facebook (and others) are making substantial and growing use of Btrfs
in container deployments.
https://www.spinics.net/lists/linux-btrfs/msg67885.html

Good read on the Red Hat decision making sense for enterprise
workloads, and where current Btrfs problems are located:
https://www.spinics.net/lists/linux-btrfs/msg67940.html

In the past 18 months, there were 100 Btrfs, 71 ext4, and 63 XFS
contributors. There are thousands of line changes per kernel release
cycle for Btrfs. It's in very active development, so Fedora people
don't need to confuse business decisions related to support contracts,
with what technologies Fedora should use.

I mention here a basis for Fedora using Btrfs where it can earn its keep.
https://pagure.io/atomic-wg/issue/306

On the bug front, I monitor all the fs lists and linux-raid@ and
strictly speaking none are stable in that none are static unchanging
targets. All of them are adding features, which then have some bugs
and cause regressions in edge cases, and there's some fix cycles. I've
been hit with file system bugs on multiple platforms over two decades,
and right now I trust Btrfs for my data more than all except maybe ZFS
and that's just because I had to flip a coin at some point, and now I
know where the bodies are buried with Btrfs. Users have gotten hit
with bugs with ZFS on Linux, and when it goes badly there's no fsck at
all so you're just hosed. So yeah, bugs are annoying, file systems are
hard, backsups are good.

Enospc is largely solved on Btrfs, kernel 4.1 added automatic
deallocation of empty block groups, and 4.8 added ticketed enospc
infrastructure; there is still a super annoying significant minority
(maybe bigger than just an edge case) that get sucked into
micromanaging their file systems to avoid the remaining instances.
That's an active discussion on linux-btrfs@ how to solve this with a
user space policy rather than continuing to wait for a smarter
deallocation/reuse block groups policy. Anyway this is definitely a
lot better, it will continue to get better.

The raid56 parity scrub bug is annoying, but the user was always in a
net better position with Btrfs despite it than the same situation with
md, lvm, or hardware RAID. The precondition is a data strip that's
corrupt (not by Btrfs but just ordinary bad sector or other course in
the stack)  -> from there Btrfs detects this corruption during scrub,
reconstructs good data from parity, passes good data to the
application and overwrites good data to disk to fix the corruption,
and then sometimes due to a race a wrong recomputation of parity
happens which is then written to disk. So bad parity replaces good
parity. But in that same scenario, md, lvm, and hardware raid
propagate corrupt data to the application and if it's a parity
overwrite rather than just a read only check for mismatches, bad
parity is also written to disk. Back to Btrfs, a 2nd scrub fixes the
problem; and during normal reads bad parity is a non-factor; if the
stripe the bad parity strip is a part of is degraded (bad sector,
failed drive) requiring reconstruction, yep you get bad reconstruction
due to bad parity but Btrfs catches this due to data checksums and we
get EIO not propagation of corrupt data to the application.

OK this is probably long enough now.


-- 
Chris Murphy

[Adam Williamson]

On Mon, 2017-08-07 at 14:46 -0600, Chris Murphy wrote:
>  On Fri, Aug 4, 2017 at 9:12 AM, Przemek Klosowski
>  <przemek.klosowski(a)nist.gov&gt; wrote:
>  > The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
>  > 
>  >
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>  
>  I see it as acknowledgment Btrfs is stable enough that it's
>  nonsensical to keep on calling it a technology preview, and also not
>  explicitly supporting it for paying customers. Red Hat doesn't have
>  the developers to support it, so it quite literally has no choice but
>  to deprecate it. 
Well, no, you're getting cause and effect mixed up, there. RH doesn't
have btrfs developers on staff *because* RH, over time, has broadly
come to the conclusion that btrfs isn't the storage tech it wants to
roll with. It's not that RH can't support btrfs for paying customers
because it has no btrfs devs, it's more that RH has decided it doesn't
want to support btrfs for paying customers so it doesn't hire any btrfs
devs.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Gerald B. Cox]

On Mon, Aug 7, 2017 at 1:52 PM, Adam Williamson <adamwill(a)fedoraproject.org&gt;
wrote:

>  Well, no, you're getting cause and effect mixed up, there. RH
> doesn't
>  have btrfs developers on staff *because* RH, over time, has broadly
>  come to the conclusion that btrfs isn't the storage tech it wants to
>  roll with. It's not that RH can't support btrfs for paying customers
>  because it has no btrfs devs, it's more that RH has decided it doesn't
>  want to support btrfs for paying customers so it doesn't hire any btrfs
>  devs.
>  
Notwithstanding even if Redhat wanted it, the Btrfs folks admit that it
isn't ready for Redhat's customer base.

[Chris Murphy]

On Mon, Aug 7, 2017 at 2:52 PM, Adam Williamson
<adamwill(a)fedoraproject.org&gt; wrote:
>  On Mon, 2017-08-07 at 14:46 -0600, Chris Murphy wrote:
> > On Fri, Aug 4, 2017 at 9:12 AM, Przemek Klosowski
> > <przemek.klosowski(a)nist.gov&gt; wrote:
> > > The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
> > >
> > >
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
> >
> > I see it as acknowledgment Btrfs is stable enough that it's
> > nonsensical to keep on calling it a technology preview, and also not
> > explicitly supporting it for paying customers. Red Hat doesn't have
> > the developers to support it, so it quite literally has no choice but
> > to deprecate it.
>
>  Well, no, you're getting cause and effect mixed up, there. RH doesn't
>  have btrfs developers on staff *because* RH, over time, has broadly
>  come to the conclusion that btrfs isn't the storage tech it wants to
>  roll with. It's not that RH can't support btrfs for paying customers
>  because it has no btrfs devs, it's more that RH has decided it doesn't
>  want to support btrfs for paying customers so it doesn't hire any btrfs
>  devs. 
That's a rather stunning comment and now I want to know if that's true
or if it's just your personal speculation.

-- 
Chris Murphy

[Adam Williamson]

On Mon, 2017-08-07 at 15:08 -0600, Chris Murphy wrote:
>  On Mon, Aug 7, 2017 at 2:52 PM, Adam Williamson
>  <adamwill(a)fedoraproject.org&gt; wrote:
>  > On Mon, 2017-08-07 at 14:46 -0600, Chris Murphy wrote:
>  > > On Fri, Aug 4, 2017 at 9:12 AM, Przemek Klosowski
>  > > <przemek.klosowski(a)nist.gov&gt; wrote:
>  > > > The release notes for RHEL 7.4 announce that RedHat gave up on btrfs:
>  > > > 
>  > > >
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>  > > 
>  > > I see it as acknowledgment Btrfs is stable enough that it's
>  > > nonsensical to keep on calling it a technology preview, and also not
>  > > explicitly supporting it for paying customers. Red Hat doesn't have
>  > > the developers to support it, so it quite literally has no choice but
>  > > to deprecate it.
>  > 
>  > Well, no, you're getting cause and effect mixed up, there. RH doesn't
>  > have btrfs developers on staff *because* RH, over time, has broadly
>  > come to the conclusion that btrfs isn't the storage tech it wants to
>  > roll with. It's not that RH can't support btrfs for paying customers
>  > because it has no btrfs devs, it's more that RH has decided it doesn't
>  > want to support btrfs for paying customers so it doesn't hire any btrfs
>  > devs.
>  
>  That's a rather stunning comment and now I want to know if that's true
>  or if it's just your personal speculation. 
It's my personal interpretation of what I've followed about RH's
general position on btrfs. It's not 'official' in any sense (I'm
obviously not in a position to say that, nor am I directly quoting or
paraphrasing anyone who *is*). And if anyone more directly involved in
RH storage tells you I'm wrong, I'm happy to defer to them. But I don't
see why you'd say it's "stunning", I mean, it's pretty much in line
with the observable facts (RH used to talk quite positively in public
about btrfs, and employed at least a couple of people to work on it,
who would continually propose it to become the Fedora default
filesystem in the next release; this stopped happening a while back and
now, at least AFAIK, we don't have anyone paid to work full-time on
btrfs, and RH doesn't talk about a lot in public any more either).
Again, all errors are my own.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net