Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Thanks!
- Chris.
Am 2009-11-17 01:55, schrieb Chris Ball:
Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Thanks!
- Chris.
So this will confuse things a lot if the user doesn't have only rpm stuff on one partition, and everything else on another. This is potentially a major risk. How would that be handled?
On 11/17/2009 02:43 AM, nodata wrote:
Am 2009-11-17 01:55, schrieb Chris Ball:
Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Thanks!
- Chris.
So this will confuse things a lot if the user doesn't have only rpm stuff on one partition, and everything else on another. This is potentially a major risk. How would that be handled?
Mr. nodata,
As the URL notes under "Detailed Description," that is not handled at all. It wraps all file I/O, yum or not, into the snapshot.
A bloody awful solution, especially when you consider that btrfs' maintainer Chris Mason is adding support for real userland transactions (via some additional ioctls).
Jeff
Jeff Garzik wrote:
It wraps all file I/O, yum or not, into the snapshot.
In this case it could only be useful in an "undoable mode" boot option.
You boot in this mode, avoid touching any data (no email downloading) and do something dangerous (distro upgrade).
But how do you handle multiple filesystem?...
On Tue, Nov 17, 2009 at 2:48 AM, Jeff Garzik jgarzik@pobox.com wrote:
On 11/17/2009 02:43 AM, nodata wrote:
Am 2009-11-17 01:55, schrieb Chris Ball:
Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Thanks!
- Chris.
So this will confuse things a lot if the user doesn't have only rpm stuff on one partition, and everything else on another. This is potentially a major risk. How would that be handled?
Mr. nodata,
As the URL notes under "Detailed Description," that is not handled at all. It wraps all file I/O, yum or not, into the snapshot.
A bloody awful solution, especially when you consider that btrfs' maintainer Chris Mason is adding support for real userland transactions (via some additional ioctls).
Yeah but you can't roll back userland transactions. Not to mention you are talking about an interface that may change quite a bit over the next year. We have snapshotting abilities now, and yes it's a big hammer, but just because its a bit of a blunt instrument doesn't mean we shouldn't take advantage of it. Thanks,
Josef
On Tue, 2009-11-17 at 08:10 -0500, Josef Bacik wrote:
On Tue, Nov 17, 2009 at 2:48 AM, Jeff Garzik jgarzik@pobox.com wrote:
As the URL notes under "Detailed Description," that is not handled at all. It wraps all file I/O, yum or not, into the snapshot.
Yeah but you can't roll back userland transactions. Not to mention you are talking about an interface that may change quite a bit over the next year.
That seems like a significant limitation, I'm also not sure that the current transaction API would be usable by rpm. Anyway...
We have snapshotting abilities now, and yes it's a big hammer, but just because its a bit of a blunt instrument doesn't mean we shouldn't take advantage of it.
This implies that "all you have is a hammer" but you can already run "yum history undo".
On Tue, 17 Nov 2009, James Antill wrote:
On Tue, 2009-11-17 at 08:10 -0500, Josef Bacik wrote:
On Tue, Nov 17, 2009 at 2:48 AM, Jeff Garzik jgarzik@pobox.com wrote:
As the URL notes under "Detailed Description," that is not handled at all. It wraps all file I/O, yum or not, into the snapshot.
Yeah but you can't roll back userland transactions. Not to mention you are talking about an interface that may change quite a bit over the next year.
That seems like a significant limitation, I'm also not sure that the current transaction API would be usable by rpm. Anyway...
We have snapshotting abilities now, and yes it's a big hammer, but just because its a bit of a blunt instrument doesn't mean we shouldn't take advantage of it.
This implies that "all you have is a hammer" but you can already run "yum history undo".
which works up to a point. If the older pkgs you had prior to an update are not available anywhere history undo isn't going to be able to 'undo' but so much.
-sv
Hi,
This implies that "all you have is a hammer" but you can already run "yum history undo".
which works up to a point. If the older pkgs you had prior to an update are not available anywhere history undo isn't going to be able to 'undo' but so much.
And of course, it's not going to work if Rawhide's broken something anywhere in the yum/rpm/*kit/etc stack, or if prelink's hosed every binary on your system again, or so on. But this would.
- Chris.
On 11/17/2009 02:48 AM, Jeff Garzik wrote:
On 11/17/2009 02:43 AM, nodata wrote:
Am 2009-11-17 01:55, schrieb Chris Ball:
Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Thanks!
- Chris.
So this will confuse things a lot if the user doesn't have only rpm stuff on one partition, and everything else on another. This is potentially a major risk. How would that be handled?
Mr. nodata,
As the URL notes under "Detailed Description," that is not handled at all. It wraps all file I/O, yum or not, into the snapshot.
A bloody awful solution, especially when you consider that btrfs' maintainer Chris Mason is adding support for real userland transactions (via some additional ioctls).
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
If they do, that'd be really interesting for upgrades.
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
regards, tom lane
On Tue, 2009-11-17 at 11:48 -0500, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
No it's just a new definition of "rollback" than is standard. The idea is that _ideally_ people seem to _want_ to be able to do:
1. update to new foo 2. run random other things that alter data. 3. update to new bar 4. run random other things that alter data. 5. run new foo, which alters it's data. 6. run random other things that alter data. 7. run new bar, which alters it's data. 8. run random other things that alter data. 9a. Decide foo is bad and thus. "rollback" just #1 and #5. 9b. Decide bar is bad and thus. "rollback" just #3 and #7.
...so all solutions tend to be exercises in randomly picking the features you think they really want, from what is desired.
Yum history assumes you are happy with just #1 and/or #3.
Snapshots assume you are happy to take a lot of collateral damage to get #5 and/or #7.
On Tue, Nov 17, 2009 at 1:33 PM, James Antill james@fedoraproject.org wrote:
On Tue, 2009-11-17 at 11:48 -0500, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
No it's just a new definition of "rollback" than is standard. The idea is that _ideally_ people seem to _want_ to be able to do:
- update to new foo
- run random other things that alter data.
- update to new bar
- run random other things that alter data.
- run new foo, which alters it's data.
- run random other things that alter data.
- run new bar, which alters it's data.
- run random other things that alter data.
9a. Decide foo is bad and thus. "rollback" just #1 and #5. 9b. Decide bar is bad and thus. "rollback" just #3 and #7.
...so all solutions tend to be exercises in randomly picking the features you think they really want, from what is desired.
Yum history assumes you are happy with just #1 and/or #3.
Snapshots assume you are happy to take a lot of collateral damage to get #5 and/or #7.
Sure, this isn't a perfect solution, it's just a nice to have feature if you care for it. It's nice to take a complete snapshot of your system right before you update just in case something goes horribly wrong and you lose say configuration files or some such. If you modified other things and have to rollback, you can always just mount the newer snapshot when you boot into the old snapshot and copy the new data that you want back. This isn't for the faint of heart, I envisioned it really for people who want to play the rawhide game with less exposure to its instability. Thanks,
Josef
On Tue, 2009-11-17 at 14:05 -0500, Josef Bacik wrote:
On Tue, Nov 17, 2009 at 1:33 PM, James Antill james@fedoraproject.org wrote:
On Tue, 2009-11-17 at 11:48 -0500, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
No it's just a new definition of "rollback" than is standard. The idea is that _ideally_ people seem to _want_ to be able to do:
- update to new foo
- run random other things that alter data.
- update to new bar
- run random other things that alter data.
- run new foo, which alters it's data.
- run random other things that alter data.
- run new bar, which alters it's data.
- run random other things that alter data.
9a. Decide foo is bad and thus. "rollback" just #1 and #5. 9b. Decide bar is bad and thus. "rollback" just #3 and #7.
...so all solutions tend to be exercises in randomly picking the features you think they really want, from what is desired.
Yum history assumes you are happy with just #1 and/or #3.
Snapshots assume you are happy to take a lot of collateral damage to get #5 and/or #7.
Sure, this isn't a perfect solution, it's just a nice to have feature if you care for it. It's nice to take a complete snapshot of your system right before you update just in case something goes horribly wrong and you lose say configuration files or some such. If you modified other things and have to rollback, you can always just mount the newer snapshot when you boot into the old snapshot and copy the new data that you want back. This isn't for the faint of heart, I envisioned it really for people who want to play the rawhide game with less exposure to its instability. Thanks,
Josef
It also works well for people who have things like homedir backup going nightly, but not full system backup. Restore to the way it was before the last yum update, recover important things from backup. It's an "oh shit" handle that has saved my bacon on other platforms before.
On 11/17/2009 02:07 PM, Jesse Keating wrote:
It also works well for people who have things like homedir backup going nightly, but not full system backup. Restore to the way it was before the last yum update, recover important things from backup. It's an "oh shit" handle that has saved my bacon on other platforms before.
I'm not sure how much of this can/should be automated. From the way btrfs snapshots work the ideal workflow should be:
1) /user/ takes a snapshot. 2) user runs yum 3) rebooting/poking/(possibly) restoring
I think the best value add to this would be a yum plugin that simply emits a warning along the lines of "Your last snapshot is ### hours old. Are you sure you want to continue?"
This also keeps us out of the dangerous territory that comes with using non-ubiquitous FS features (your boot is on ext3, your root is on btrfs, your etc is on xfs and your usr is on jfs. What do you snapshot and how?)
--CJD
Hi,
I'm not sure how much of this can/should be automated.
Sorry, not quite following -- what is the caution around automatically creating a new snapshot before each yum transaction? Why shouldn't it be automated?
This also keeps us out of the dangerous territory that comes with using non-ubiquitous FS features (your boot is on ext3, your root is on btrfs, your etc is on xfs and your usr is on jfs. What do you snapshot and how?)
This feature would snapshot the btrfs / only, but that doesn't matter, because snapshots don't do anything until the user chooses to initiate a rollback. The user who chooses a btrfs / and a jfs /usr knows what happens when they rollback only the btrfs filesystem. (And we can print a warning to make sure of that if we decide it's necessary.)
Thanks,
- Chris.
On 11/17/2009 03:56 PM, Chris Ball wrote:
Hi,
I'm not sure how much of this can/should be automated.
Sorry, not quite following -- what is the caution around automatically creating a new snapshot before each yum transaction? Why shouldn't it be automated?
I somewhat read the initial suggestion as trying to implement transactional behavior via snapshot. Just creating one shouldn't hurt.
This also keeps us out of the dangerous territory that comes with using non-ubiquitous FS features (your boot is on ext3, your root is on btrfs, your etc is on xfs and your usr is on jfs. What do you snapshot and how?)
This feature would snapshot the btrfs / only, but that doesn't matter, because snapshots don't do anything until the user chooses to initiate a rollback. The user who chooses a btrfs / and a jfs /usr knows what happens when they rollback only the btrfs filesystem. (And we can print a warning to make sure of that if we decide it's necessary.)
We've now created a useless and slightly dangerous object though. Regardless of the competence of the user who's problem that is, better to avoid it if we can.
Thanks,
- Chris.
--CJD
On Tue, 2009-11-17 at 16:12 -0500, Casey Dahlin wrote:
On 11/17/2009 03:56 PM, Chris Ball wrote:
Hi,
I'm not sure how much of this can/should be automated.
Sorry, not quite following -- what is the caution around
automatically
creating a new snapshot before each yum transaction? Why shouldn't
it
be automated?
I somewhat read the initial suggestion as trying to implement transactional behavior via snapshot. Just creating one shouldn't hurt.
I fail to understand how a snapshot would differentiate between yum related changes and other changes that occur by an otherwise normally operating daemon (logs for example).
Certainly you don't want to loose your logs when you want to revert a yum update ?
What if you loose is something more important, like shared secrets (a kerberos keytab) that have changed during the transaction ?
Simo.
Hi,
I somewhat read the initial suggestion as trying to implement transactional behavior via snapshot. Just creating one shouldn't hurt.
Ah. Yeah, not at all.
I fail to understand how a snapshot would differentiate between yum related changes and other changes that occur by an otherwise normally operating daemon (logs for example).
If someone wanted to use whole-fs snapshots for yum transactions (which I don't) the way to do it would be to only rollback changes made to files that are present in either of the RPM manifests.
- Chris.
Hi,
We've now created a useless and slightly dangerous object though. Regardless of the competence of the user who's problem that is, better to avoid it if we can.
Okay. Perhaps the automated snapshot creation algorithm should be amended to something like:
"check /proc/mounts for at least one btrfs, and zero other non-btrfs filesystems, except for /boot which can be anything."
Thanks,
- Chris.
On Tue, Nov 17, 2009 at 03:56:16PM -0500, Chris Ball wrote:
Hi,
I'm not sure how much of this can/should be automated.
Sorry, not quite following -- what is the caution around automatically creating a new snapshot before each yum transaction? Why shouldn't it be automated?
AIUI it will waste space. Snapshots are quick and easy to perform, but if you keep them around they can't be garbage collected.
Rich.
On 11/17/2009 12:05 PM, Josef Bacik wrote:
Sure, this isn't a perfect solution, it's just a nice to have feature if you care for it. It's nice to take a complete snapshot of your system right before you update just in case something goes horribly wrong and you lose say configuration files or some such. If you modified other things and have to rollback, you can always just mount the newer snapshot when you boot into the old snapshot and copy the new data that you want back. This isn't for the faint of heart, I envisioned it really for people who want to play the rawhide game with less exposure to its instability. Thanks,
I've long wanted to help with fedora rawhide a bit more but only really have it on the one computer I work on. So this would make it possible for me to do that. I think that's great.
I have a few questions. Suppose I do an update, and it breaks X or some other important piece for me. I reboot into my previous snapshot. From there I continue to work.
Do I have to remove the 'future' snapshot I came back from to continue working in that snapshot? Or is that just best practice. If I move forward, I'd have to move all the work I did in that snapshot to the head/snapshot that just got fixed with an update...
Just thinking out loud here.
On 11/17/2009 11:48 AM, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
Yes, that's why I included the extra words.
On 11/17/2009 11:48 AM, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
So, I guess I should expand some more on what I'm saying.
The problem here is that normal database-like transactions don't help an upgrade much, because you don't really know whether you want to commit the changes until you've rebooted the machine and poked around for a bit.
Or, put another way, what you basically want is:
1 create an fs snapshot 2 do an upgrade 3 reboot machine 4 poke around 5 decide if it's good (6a) or bad (6b) 6 act on #5 a - remove snapshot b - abandon all changes after the snapshot
And if you're talking about implementing that with database-like semantics, then you want something non-traditional such as:
1 start transaction 2 upgrade 3 tentatively commit transaction 4 reboot machine 5 poke around 6 decide if it's good (6a) or bad (6b) 7 act on #6 a - fully commit transaction b - roll back
These obviously aren't the traditional semantics, hence I'm asking if it has semantics like that, because if it doesn't, then I don't really see Jeff's point.
On Tue, Nov 17, 2009 at 2:09 PM, Peter Jones pjones@redhat.com wrote:
On 11/17/2009 11:48 AM, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
So, I guess I should expand some more on what I'm saying.
The problem here is that normal database-like transactions don't help an upgrade much, because you don't really know whether you want to commit the changes until you've rebooted the machine and poked around for a bit.
Or, put another way, what you basically want is:
1 create an fs snapshot 2 do an upgrade 3 reboot machine 4 poke around 5 decide if it's good (6a) or bad (6b) 6 act on #5 a - remove snapshot b - abandon all changes after the snapshot
And if you're talking about implementing that with database-like semantics, then you want something non-traditional such as:
1 start transaction 2 upgrade 3 tentatively commit transaction 4 reboot machine 5 poke around 6 decide if it's good (6a) or bad (6b) 7 act on #6 a - fully commit transaction b - roll back
These obviously aren't the traditional semantics, hence I'm asking if it has semantics like that, because if it doesn't, then I don't really see Jeff's point.
It doesn't. Userspace transactions are _only_ to make sure that a set of operations go to disk at the same time. The original reason this was implemented was for ceph, a distributed filesystem that has a client that runs in userspace and needs to write to an inode and update an xattr on that inode at the same time in order to maintain filesystem consistency. Nowadays there is _no_ guarantee that the write to the inode and the write to the xattr will go into the same transaction, so the userspace transaction just makes sure that they do happen in the same transaction. It's not a snapshot or anything like that, its just a way to guarantee ordering. Thanks,
Josef
On 11/17/2009 02:15 PM, Josef Bacik wrote:
On Tue, Nov 17, 2009 at 2:09 PM, Peter Jones pjones@redhat.com wrote:
On 11/17/2009 11:48 AM, Tom Lane wrote:
Peter Jones pjones@redhat.com writes:
Do they support rollbacks after commit? If they don't, they're not really as useful for this as they could be.
Rollback *after* commit? This must be some other usage of the term "commit" than is standard to database people.
So, I guess I should expand some more on what I'm saying.
The problem here is that normal database-like transactions don't help an upgrade much, because you don't really know whether you want to commit the changes until you've rebooted the machine and poked around for a bit.
Or, put another way, what you basically want is:
1 create an fs snapshot 2 do an upgrade 3 reboot machine 4 poke around 5 decide if it's good (6a) or bad (6b) 6 act on #5 a - remove snapshot b - abandon all changes after the snapshot
And if you're talking about implementing that with database-like semantics, then you want something non-traditional such as:
1 start transaction 2 upgrade 3 tentatively commit transaction 4 reboot machine 5 poke around 6 decide if it's good (6a) or bad (6b) 7 act on #6 a - fully commit transaction b - roll back
These obviously aren't the traditional semantics, hence I'm asking if it has semantics like that, because if it doesn't, then I don't really see Jeff's point.
It doesn't. Userspace transactions are _only_ to make sure that a set of operations go to disk at the same time. The original reason this was implemented was for ceph, a distributed filesystem that has a client that runs in userspace and needs to write to an inode and update an xattr on that inode at the same time in order to maintain filesystem consistency. Nowadays there is _no_ guarantee that the write to the inode and the write to the xattr will go into the same transaction, so the userspace transaction just makes sure that they do happen in the same transaction. It's not a snapshot or anything like that, its just a way to guarantee ordering. Thanks,
Okay, so then I definitely don't see what jgarzik was getting at.
tis 2009-11-17 klockan 02:48 -0500 skrev Jeff Garzik:
A bloody awful solution, especially when you consider that btrfs' maintainer Chris Mason is adding support for real userland transactions (via some additional ioctls).
Yes.
But the draft proposal is presented as "tools for experienced people" rather than "cool feature for all", so I don't see the harm. (Check the uses cases under Benefit to Fedora.)
/abo
Am 2009-11-17 19:33, schrieb Alexander Boström:
tis 2009-11-17 klockan 02:48 -0500 skrev Jeff Garzik:
A bloody awful solution, especially when you consider that btrfs' maintainer Chris Mason is adding support for real userland transactions (via some additional ioctls).
Yes.
But the draft proposal is presented as "tools for experienced people"
It has a gui...
rather than "cool feature for all", so I don't see the harm. (Check the uses cases under Benefit to Fedora.)
On Mon, 2009-11-16 at 19:55 -0500, Chris Ball wrote:
Hi,
I've written up a draft of an F13 filesystem rollback feature using Btrfs snapshots that are automatically created by yum:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
See also: http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs http://blogs.sun.com/erwann/entry/time_slider_screencast http://blogs.sun.com/erwann/entry/new_time_slider_features_in
It was mentioned in the past that we'd want to have this ported to using btrfs snapshots.
Cheers
Hi all,
It'd be great to get feedback on whether this is the right idea, and how exactly the UI interaction should work, before submitting this formally.
Some off-list feedback so far:
* People want independent active snapshots per filesystem (i.e. btrfs /home is the live filesystem, and btrfs / is an older snapshot), which makes sense but makes the UI more complex since it'll have to show a list of (filesystem, snapshot) tuples.
* Ray says not to invent a new system-config-blah, and instead to talk with davidz about Palimpsest. David, what do you think?
* Several people think that the ZFS Time Slider patches to nautilus¹ look good, and want that for btrfs. Sounds plausible², but I'm personally less interested in file versioning via snapshots and more interested in working on ways to let developers feel comfortable upgrading to Rawhide daily, for the next release.
* Someone on the feature's Talk Page suggests that we should encourage people using anaconda to create a btrfs rootfs separate to their /home, so that they can rollback rootfs snapshots without affecting their homedir. Could work; maybe an anaconda dialog that says "hey, I see you're choosing btrfs, we're going to turn on snapshots and we suggest you use a separate /home partition".
Any more thoughts?
Thanks!
- Chris.
¹: http://blogs.sun.com/erwann/entry/zfs_on_the_desktop_zfs http://blogs.sun.com/erwann/entry/time_slider_screencast http://blogs.sun.com/erwann/entry/new_time_slider_features_in
²: It's a bit awkward; you have to do a mount on each snapshot to be able to show the directory listing for it, but it appears that's what ZFS/Time Slider does already, so it won't be any worse..
(I'm not subscribed to fedora-devel-list so if you expect an answer please Cc me)
On Tue, 2009-11-17 at 09:52 -0500, Chris Ball wrote:
- Ray says not to invent a new system-config-blah, and instead to talk with davidz about Palimpsest. David, what do you think?
Yep, we're planning to add support to DeviceKit-disks for exposing the (privileged) operations that btrfs may expose (locked down by polkit, etc etc). There are also plans to expose these operations in the UI in Palimpsest and/or Nautilus. I don't think snapshots is going to have any Palimpsest UI (it belongs in Nautilus I think) but the multi-disk stuff definitely will.
As always, DKD and Palimpsest is supposed to be complementary to the command-line tools. So all this is only relevant for creating nice UIs for managing btrfs.
(Btw there's still some work needed in udev/blkid (but hopefully not in the btrfs on-disk format) to properly export everything we need - e.g. things like number of block devices in the multi-disk filesystem, maybe the "raid" level and so on. I haven't gotten around to looking at this in detail yet. It's on my TODO list though.)
- Several people think that the ZFS Time Slider patches to nautilus¹ look good, and want that for btrfs. Sounds plausible²,
People hacking on Nautilus will have to look into this with both ZFS, btrfs and possibly other filesystems in mind. I haven't seen anyone do work like this and it's not trivial.
(Oh, and if it turns out that creating/destroying btrfs snaphots isn't a privileged operation (I can't remember at this point) it would probably make sense for Nautilus to just use the btrfs tools directly instead of going through a system daemon. There's just no need to overcomplicate things.)
David
On Tue, Nov 17, 2009 at 10:36 AM, David Zeuthen david@fubar.dk wrote:
(I'm not subscribed to fedora-devel-list so if you expect an answer please Cc me)
On Tue, 2009-11-17 at 09:52 -0500, Chris Ball wrote:
- Ray says not to invent a new system-config-blah, and instead to talk
with davidz about Palimpsest. David, what do you think?
Yep, we're planning to add support to DeviceKit-disks for exposing the (privileged) operations that btrfs may expose (locked down by polkit, etc etc). There are also plans to expose these operations in the UI in Palimpsest and/or Nautilus. I don't think snapshots is going to have any Palimpsest UI (it belongs in Nautilus I think) but the multi-disk stuff definitely will.
Talking about sliders and various btrfs bits in Nautilus, I would just like to point out that there are other desktops and file managers to consider. It should be possible to perform the same operations via GUI in KDE or xfce.
Whether this means keeping stuff in a seperate, desktop agnostic, application (like a system-config or Palimpset) or making changes to Dolphin, etc as well I don't know.
Hi David, thanks for the reply.
Yep, we're planning to add support to DeviceKit-disks for exposing the (privileged) operations that btrfs may expose (locked down by polkit, etc etc). There are also plans to expose these operations in the UI in Palimpsest and/or Nautilus. I don't think snapshots is going to have any Palimpsest UI (it belongs in Nautilus I think) but the multi-disk stuff definitely will.
I don't think it makes sense for the filesystem-level snapshot operations I describe in the feature proposal to be in Nautilus:
https://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs
The proposal is about system-administrator decisions on what your root directory on btrfs will look like at next boot; they should only be performed by someone deep into "I am modifying the mount behavior of my fs" mode.
(It does make sense for a Time Slider port to Btrfs, working with file-level operations on snapshots, to live inside Nautilus. That's not part of this proposal, but would be very useful work.)
Given the above, do you think you'd be okay with having:
Filesystem snapshot that will be active on next boot: <drop-down>
and
Create new whole-filesystem snapshot now: <label> <apply>
in a Btrfs-specific section in Palimpsest? That's all that's needed for the UI component of this feature.
As always, DKD and Palimpsest is supposed to be complementary to the command-line tools. So all this is only relevant for creating nice UIs for managing btrfs.
Yup, that's the case.
(Oh, and if it turns out that creating/destroying btrfs snaphots isn't a privileged operation (I can't remember at this point) it would probably make sense for Nautilus to just use the btrfs tools directly instead of going through a system daemon. There's just no need to overcomplicate things.)
Creating a new snapshot is unprivileged, but mounting an old one (which nautilus would need to do in order to show you the contents of a previous snapshot, so that you can decide which files you want to restore from it) requires a mount(8) call for each snapshot.
Thanks,
- Chris.
On Tue, 2009-11-17 at 23:34 -0500, Chris Ball wrote:
Given the above, do you think you'd be okay with having:
Filesystem snapshot that will be active on next boot: <drop-down>
Shouldn't it say "next time volume is mounted" instead of "next boot"? We can always special case rootfs to say "next boot" of course (since rootfs can't be unmounted until next boot).
Also, what is the mechanism to configure this? Just a simple command from btrfs-progs (best)? Or does it require surgery to /etc/fstab and/or the initramfs (bad)?
Create new whole-filesystem snapshot now: <label> <apply>
in a Btrfs-specific section in Palimpsest? That's all that's needed for the UI component of this feature.
From a 50,000 feet view all this sounds good to me.
(Oh, and if it turns out that creating/destroying btrfs snaphots isn't a privileged operation (I can't remember at this point) it would probably make sense for Nautilus to just use the btrfs tools directly instead of going through a system daemon. There's just no need to overcomplicate things.)
Creating a new snapshot is unprivileged, but mounting an old one (which nautilus would need to do in order to show you the contents of a previous snapshot, so that you can decide which files you want to restore from it) requires a mount(8) call for each snapshot.
OK. We need to decide how all this is going to work - maybe some of it will go into GIO and be part of an abstraction that also works for other filesystems, maybe it will be a Nautilus-only feature. I don't know yet, leaning towards the latter right now, but I guess we'll find out.
Thanks, David
Hi David,
Shouldn't it say "next time volume is mounted" instead of "next boot"? We can always special case rootfs to say "next boot" of course (since rootfs can't be unmounted until next boot).
Good point. That's fine.
Also, what is the mechanism to configure this? Just a simple command from btrfs-progs (best)? Or does it require surgery to /etc/fstab and/or the initramfs (bad)?
Josef's been thinking about exactly that -- the current situation requires you to add subvol=<snapshot-name> to the mount args, which indeed would require you to change fstab (for non-rootfs) or to add a rootflags= argument to the grub menu (for rootfs). He's considering changing the btrfs disk format to add a "default subvolume for this fs" field that would lead us to a simple btrfsctl command for setting that field instead. I agree that his solution's what we'd like.
OK. We need to decide how all this is going to work - maybe some of it will go into GIO and be part of an abstraction that also works for other filesystems, maybe it will be a Nautilus-only feature. I don't know yet, leaning towards the latter right now, but I guess we'll find out.
Makes sense.
Thanks again. I've updated the feature draft to include the Palimpsest UI suggestion.
- Chris.
On Wed, 2009-11-18 at 10:47 -0500, Chris Ball wrote:
Also, what is the mechanism to configure this? Just a simple command from btrfs-progs (best)? Or does it require surgery to /etc/fstab and/or the initramfs (bad)?
Josef's been thinking about exactly that -- the current situation requires you to add subvol=<snapshot-name> to the mount args, which indeed would require you to change fstab (for non-rootfs) or to add a rootflags= argument to the grub menu (for rootfs). He's considering changing the btrfs disk format to add a "default subvolume for this fs" field that would lead us to a simple btrfsctl command for setting that field instead. I agree that his solution's what we'd like.
OK, sounds good to me. I'm subscribed to the btrfs-list so I guess I'll just sit around and wait for Josef's patch to show up.
Thanks, David
On Wed, Nov 18, 2009 at 8:13 PM, Kevin Kofler kevin.kofler@chello.at wrote:
Chris Ball wrote:
Creating a new snapshot is unprivileged
Huh? Isn't that a license for any user to waste massive amounts of disk space, ignoring any per-user quota? Whole file system operations must be root only!
Snapshots are subject to the permissions of the root inode that we're snapshotting, so if the permissions are set such that the user has write permissions for that root, then they can create snapshots. An example of this would be if you created individual subvolumes for individual home directories. The users would have permissions to their respective roots and be allowed to snapshot them. This isn't a whole file system operation, its a per-root operation. Thanks,
Josef
tis 2009-11-17 klockan 09:52 -0500 skrev Chris Ball:
- Someone on the feature's Talk Page suggests that we should encourage people using anaconda to create a btrfs rootfs separate to their /home, so that they can rollback rootfs snapshots without affecting their homedir.
If you have a single btrfs filesystem with one snapshot from a while back and one "head", would it be possible for some hypothetical tool to create a new head which looks like the root from the snapshot but with the latest /home on top? I mean, without actually copying the files from the snapshot, just creating a "fake snapshot" that can the be mounted and writeable. Then you wouldn't need a separate /home.
/abo
On 11/16/2009 07:55 PM, Chris Ball wrote:
It'd be great to get feedback on whether this is the right idea
You might look into how the nexenta guys implemented this: http://www.nexenta.org/os/TransactionalZFSUpgrades to see if they have some thoughts worth borrowing.
With the caveat that I haven't yet created a btrfs (seriously waiting on a Seagate RMA), I think most of the complaints on this thread are due to thinking of filesystems in old terms, due to limitations of old storage concepts.
To do this right probably requires the careful use of subvolumes. One for rpm, one for logs, one for each package installed, one for each user's home, etc. From rpm you can know what needs rolling back. This probably implies foo-var, foo-etc, foo-log, foo-share, etc, which is inefficient, so maybe some refinement is needed there as well.
-Bill