stratis-devel

stratis-devel@lists.fedorahosted.org

78 discussions

Re: Options for dealing with devicemapper context
by Andy Grover 19 Mar '18

19 Mar '18

On 03/19/2018 09:56 AM, Tony Asleson wrote: > I was wondering if this discussion (and ones like it) should be done on > the public mailing list? To encourage community involvement. Yes please, in the future. For now, I'm CCing the list here, so our throngs of theoretical list subscribers :-) could catch up by reading the quoted conversation below. Thanks -- Andy > > > On 03/19/2018 11:02 AM, Tony Asleson wrote: >> My preference is what you refer to as class II, with placing much of the >> base functionality into devicemapper.rs and limiting or removing dm >> reference passing to functions and methods. >> >> Regards, >> -Tony >> >> >> On 03/16/2018 04:10 PM, Anne Mulhern wrote: >>> Hi! >>> >>> I have identified several options. Do you have a preference among them, >>> or have you another option in mind? Why? >>> >>> The options fall into two classes: >>> >>> I. Pass the DM context around as a parameter. >>> II. Instantiate it once and use a method to get a reference to the parameter where necessary. >>> >>> For (I) the question is where to instantiate the context. The two consistent options are >>> A. At the outermost level >>> B. Exactly where needed. >>> >>> A drawback of (A) is that it requires a bit of maintenance; a merit is that it is >>> informative, since the caller knows if the context is needed somewhere by any of the >>> method's callees. (A) is implemented here: https://github.com/stratis-storage/stratisd/pull/835. >>> It was mostly already done, this is the last little bit. >>> >>> A drawback of (B) is that it makes failure getting the context need to be handled wherever >>> the context is obtained. Mostly, this is fairly benign, since a method that needs a DM >>> context probably has a lot of other ways to fail, too, so usually the code should be as >>> simple as a "?". Another drawback is that it is an action that is repeated unnecessarily. >>> A merit is that fewer arguments are passed in many methods. >>> >>> For (II) there are three tasks: >>> 1. Write the necessary code that memoizes the DM after the first call and returns a reference to the DM. >>> This method is called get_dm_context. This is a bit tricky, there is an unsafe block. >>> 2. Write the code that calls get_dm_context() and correctly handles the possible error that may occur the first >>> time get_dm_context() is called (using catch_unwind, which is probably a bit risky, kind of like unsafe). >>> 3. Alter the other code accordingly (this is rather broad) >>> >>> and three possible options: >>> A. Do (1), (2), and (3) in stratisd. In this case (3) consists of calling get_dm_context() whenever it >>> is needed to pass to a devicemapper-rs method, e.g., LinearDev::setup() and eliminating where >>> the DM context is instantiated and passed down to callees. See https://github.com/stratis-storage/stratisd/pull/841 >>> for a partial implementation of this approach. >>> B. Do (1) in devicemapper-rs and (2) and (3) in stratisd. See https://github.com/stratis-storage/devicemapper-rs/pull/286 >>> for (1) and https://github.com/stratis-storage/stratisd/pull/843 for (2) and partial implementation of (3). >>> C. Do (1) in devicemapper-rs, (2) in stratisd, and (3) in both. For (3) in devicemapper-rs, eliminate all &DM parameters >>> in all methods. See https://github.com/stratis-storage/devicemapper-rs/pull/288 for rough devicemapper-rs implementation >>> of this part. In stratisd, call all devicemapper-rs methods w/out passing any DM parameter, since the methods no longer >>> require any such. Do not pass DM parameter anywhere. This part has not been implemented, but you get the idea. >>> >>> I tend to prefer (II), because it changes the places to fail from many to one, at least in theory. >>> >>> - mulhern >>> >> >

2 2

stratisd 0.1.2 tagged
by Andy Grover 31 Oct '17

31 Oct '17

Hi all, We've started tagging the Stratis daemon (stratisd) monthly, as we work towards the 0.5 and 1.0 releases, and I thought it would also be good to mention progress on the mailing list. There's also work happening in other repos, and I'll likely include those changelogs in the future as well. Also, this gives me a chance to plug the Stratis COPR repo! If you're running Fedora and want to try Stratis, or want to create a Fedora VM to try it (recommended!) then installing a recent version of Stratis is as easy as: dnf copr enable grover/stratis dnf install stratis This is an easy way to try things out while we wait to get into Fedora proper. Thanks! stratisd 0.1.2: Andy Grover <agrover(a)redhat.com> (5): Specify nix 0.9.x and update Make XFS be quiet when it makes a filesystem Implement Blockdev D-Bus API Change format_flex_name to take a uuid instead of a reference Version 0.1.2 Todd Gill <tgill(a)redhat.com> (11): suspend running loopback tests on Travis Update devnode_to_devno() to return None for NotFound errors update get_mount_point() to return EngineResult<Option<PathBuf>> set UUID for XFS to the UUID used by Stratis for filesystem and MDV Add snapshot support to filesystem Add a test to take a snapshot and verify contents Move create_fs(), set_uuid and xfs_growfs() to util.rs Update comments from /// to // if they are not used by rust doc removed un-necessary mut from the pool variable in test_filesystem_rename() Remove use of UUID references where possible dbus snapshot interface / specify a name for a snapshot Tony Asleson <tasleson(a)redhat.com> (3): README.md(s) Minor updates Reduce spurious test output dbus api: Handle deferred at start up mulhern <amulhern(a)redhat.com> (29): Add an outdated target to the Makefile Update D-Bus requirement to 0.5.3 Upgrade rand requirement to lowest version compiling for rustc 1.17.0 Replace temp-file-removing block of code with a function Update for new signature for new() and setup() methods Specify devicemapper v0.11.0 Have pool.check() return a Result Give blockdevmgr the responsibility for allocating necessary data Omit an unwrap() Have ThinPool store the UUID of its pool Have BlockDevMgr store the UUID of its owning pool Update to new version of clippy Fix clippy error identity_conversion Allow clippy implicit_hasher Allow cloning of BlkDevSegments Rewrite ThinPool::extend_data() to make use of set_data_segments() Have ThinPool::extend_data() take a &[BlkDevSegment] argument Specify devicemapper v0.12.0 Drop test_linear_device Make all integration tests into unit tests Make a few blockdevmgr methods private Use BlockDevMgr::initialize instead of internal function initialize Make extend_thinpool more general Explicitly extend the thinpool in ThinPool tests Change DevOwnership::Ours constructor to include device UUID If dev claims pool membership, but mgr doesn't agree, return error Remove test_thinpool_device Update to new version of clippy Omit 'static lifetime from a few constant definitions Regards -- Andy

1 0

`real_test_empty_pool` test fails
by Jacob Salzberg 02 Oct '17

02 Oct '17

Hello Stratis developers! I attempted to run the `make test-real` on a virtual machine of Fedora 26-1.5 Server. One of the tests failed, and I'm wondering if that is important enough to post as an issue. To be more specific, when I try to run `real_test_empty_pool,` it fails at one of the `unwrap()`s: > make test-real > sudo env > "PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/jsalzberg/.cargo/bin:/home/jsalzberg/Programming/stratisd/target/debug:/home/jsalzberg/.local/bin:/home/jsalzberg/bin" > RUSTFLAGS='-D warnings' RUST_BACKTRACE=1 cargo test -- --test > real_test_empty_pool > Compiling libstratis v0.1.1 > (file:///home/jsalzberg/Programming/stratisd) > Finished dev [unoptimized + debuginfo] target(s) in 4.55 secs > Running target/debug/deps/libstratis-a40f09f70f597433 > > running 0 tests > > test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 56 filtered out > > Running target/debug/deps/stratisd-29c4deddc9023466 > > running 0 tests > > test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out > > Running target/debug/deps/loopbacked_tests-3d242529a0f13830 > > running 0 tests > > test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 17 filtered out > > Running target/debug/deps/real_tests-aea58de6898d93b0 > > running 1 test > test real_test_empty_pool ... FAILED > > failures: > > ---- real_test_empty_pool stdout ---- > thread 'real_test_empty_pool' panicked at 'called `Result::unwrap()` > on an `Err` value: Error { repr: Os { code: 2, message: "No such file or > directory" } }', src/libcore/result.rs:860:4 > note: Some details are omitted, run with `RUST_BACKTRACE=full` for a > verbose backtrace. > stack backtrace: > 0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace > 1: std::sys_common::backtrace::_print > 2: std::panicking::default_hook::{{closure}} > 3: std::panicking::default_hook > 4: std::panicking::rust_panic_with_hook > 5: std::panicking::begin_panic_new > 6: std::panicking::begin_panic_fmt > 7: rust_begin_unwind > 8: core::panicking::panic_fmt > 9: core::result::unwrap_failed > at /builddir/build/BUILD/rustc-1.20.0-src/src/libcore/ > macros.rs:41 > 10: <core::result::Result<T, E>>::unwrap > at /builddir/build/BUILD/rustc-1.20.0-src/src/libcore/ > result.rs:738 > 11: real_tests::test_with_spec > at tests/real_tests.rs:116 > 12: real_tests::real_test_empty_pool > at tests/real_tests.rs:196 > 13: <F as test::FnBox<T>>::call_box > 14: __rust_maybe_catch_panic > > > failures: > real_test_empty_pool > > test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 16 > filtered out > PS: That test is written with a lot of `unwrap()`s, and it is hard to tell which of the `unwrap()`s causes the error. Should they all be changed to `expect("")`s?

2 2

Block checksum via MDRAID?
by Gionatan Danti 15 Aug '17

15 Aug '17

Hi list, I just learned about Stratis, and it is a very exciting project! The idea of ZFS-like features, while preserving proper layering in the I/O stack, in a single, simple package is really cool. I read the Stratis PDF documentation and, if I understand it properly, you plan to use dm-integrity to checksum blocks on the I/O device. I wonder if you considered extending current MDRAID code to integrate checksumming capability: in this manner, you not only "shave off" a I/O layer (no need of dm-integrity on the single block devices), but *any* new MD array become automatically checksum-protected. Some years ago, this very idea was heavily debated on MDRAID mailing list. At the time BTRFS looked "almost ready", so the MDADM maintainer decided that disk's own hardware ECC was enough to protect from the most likely problems. Maybe this can be right time to revisit this position? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 2

Stratis 0.1 released
by Andy Grover 03 Aug '17

03 Aug '17

Stratis 0.1 is now available. This is the initial alpha release. It is feature-complete for basic expected functionality, including creating pools from blockdevs, creating filesystems from a pool, renaming, listing, deletion, and recreating running state from on-disk metadata. It has not yet been tested extensively. * It it recommended to use a scratch system or VM for running Stratis, with no valuable data. * The use of other tools that use device-mapper, such as LVM, on a test system is best avoided. * The use of small blockdevs (total size under 5 GiB) to create a pool is best avoided, since Stratis doesn't yet gracefully handle running out of space. * Once stratisd is run and then exited (e.g. with a ctrl-C), a reboot is required before it can successfully run again. The easiest way to try Stratis is to install Fedora 25 or 26 (x86_64) and add our COPR test repo: `dnf copr enable grover/stratis` `dnf install stratis stratisd` Then, run 'stratisd' as root in one terminal, and interact with it using the 'stratis' command-line tool in another root terminal. See "man stratis' for more info. You may give feedback by: * Sending an email to this list * Opening an issue against either https://github.com/stratis-storage/stratisd or https://github.com/stratis-storage/stratis-cli * Joining the #stratis-storage irc channel on irc.freenode.net. The truly brave can find release tarballs and build instructions for the five Stratis components here: https://github.com/stratis-storage/stratisd/releases https://github.com/stratis-storage/stratis-cli/releases https://github.com/stratis-storage/into-dbus-python/releases https://github.com/stratis-storage/dbus-signature-pyparsing/releases https://github.com/stratis-storage/stratisd-client-dbus/releases Thanks -- Regards -- Andy

1 0

XFS info
by Todd Gill 12 Jun '17

12 Jun '17

This exchange made me think it might be wise for Stratis to take a "order of magnitude" parameter when we create a filesystem. <tgill> Hi <tgill> I’d like to run XFS over a dm-thin device. I’ve heard that XFS is best setup for future potential growth if the thin device has a size of 1 TiB. I think the idea was to avoid provisioning something small and keep extending it. Is this accurate? <sandeen> the maximum alllocation group (at least for otherwise default geometry) is 1T. So a 32T filesystem would have 32 1T allocation groups <sandeen> if you had started with, say, a 128g filesystem and grown to 32T, you would have had many, many more allocation groups than optimal <sandeen> so I imagine that's what "you've heard" <sandeen> does that make sense? <sandeen> so honestly, if you create a 1T filesystej, you'll get 4x256G AGs (modulo other geometries due to storage layout, but 4 AGs is the default), and if you later grow to 32G, you'll actually have 4x as many AGs as you would have if you'd directly mkfs'd 32T <sandeen> that's probably not a huge deal, but you do end up in a slightly different spot than if you'd started at 32T <sandeen> er, sorry "later grow to 32T" <sandeen> short answer is, don't grow several orders of magnitude. <tgill> I need to do a little more reading about how allocation groups work, but it makes sense. :-) <tgill> when I run mkfs.xfs on the thin 1T - 547 MiB is consumed by the pool This is XFS meta data? <sandeen> yes - the metadata itself will take up very little (log, as well) but it will be ampliefied by the thin chunk size

2 1

clean restart on stratisd crash
by Andy Grover 03 May '17

03 May '17

17:21 <mulhern> Because, when I do setup(), whether or not Stratis shut down cleanly is kind of important. If not shutdown, dm devices belonging to pool are still hanging around when Stratis starts back up. [using email since people aren't on irc atm] We can't assume the entire stack is present, but we should be able to assume that if we go to create a DM device with a given name and it already exists, then we are recovering from a crash and can reuse the existing device. This gets back to https://github.com/stratis-storage/stratis-docs/issues/33, about specifying the format of DM names. For this, the dm names for all the different devices we use need to be specified, non-overlapping, and stable. This was working in Froyo, see dmdevice.rs[1]. DmDevice::new() would only create/table_load a device with a given name after it checked it didn't exist. Otherwise it just returned the existing one. (Idempotency!...?) I'll start adding some early thoughts on this to that GH issue. Regards -- Andy [1] https://github.com/agrover/froyo/blob/master/src/dmdevice.rs#L22

1 0

Hi, test
by Andy Grover 14 Mar '17

14 Mar '17

Hi, this list is for discussion of the Stratis storage project, see https://stratis-storage.github.io/ or https://github.com/stratis-storage/ for more info. Thanks -- Regards -- Andy

1 0

2024

2023

2022

2021

2020

2019

2018

2017

stratis-devel