[blivet:master 00/26] RAID patches

[Anne Mulhern]

These patches will be vaguely remniscent of the last set of RAID related
patches I sent out, but not all that close.

The changes require an anaconda patch.

The first four are nice and simple.
The very last is only related to RAID because I felt
I had to understand the method in order to understand device populateKSData()
methods, and to understand it I had to think about, so I ended up rewriting it.

mulhern (26):
  Import name 'lvm' instead of names from lvm package.
  Comment mdraid.mdcreate() and update tests appropriately.
  Remove DMRaidArrayDevice.members property.
  Extract selection of members in completed() into a separate method.
  No longer use _standard_levels as the default set of RAID levels.
  Adjust RaidLevel hierarchy so that all raid level objects extend
    RAIDLevel
  Allow the RAID object itself to be a valid RAID descriptor for lookup.
  Move Container class to raid package and tidy it up
  Add Linear and Single to the RAID classes.
  Add a has_redundancy method that returns True if there is actual
    redundancy
  Add an is_uniform property to the RAID levels.
  Add a Dup class to the various descendants of RAIDLevel.
  Make createBitmap() a property and update tests appropriately.
  Use has_redundancy raid property when checking whether a device is
    removable
  Update the mdraid.mdadd comments
  Use has_redundancy property to decide how to add a member to an array.
  Check for required methods in MDRaidLevels.isRaidLevel.
  Remove getRaidLevel() from mdraid file and make RAID_levels public
  Add "linear" to mdraid's list of supported raid levels.
  Add lists of supported RAID levels for btrfs and lvm
  Make BTRFS devices use RAID objects instead of strings for levels
  Put get_supported_raid_levels in devicefactory.py
  Distribute devicefactory.get_raid_level within devices classes.
  Make all devicefactory classes uses RAID objects instead of strings.
  Move the code for getting a space requirement from devicefactory to
    raid.
  Simplify a small chunk of Blivet.updateKSData()

blivet/__init__.py                   |  12 +-
 blivet/devicefactory.py              | 137 ++++++++++------
 blivet/devicelibs/btrfs.py           |  10 +-
 blivet/devicelibs/lvm.py             |   3 +
 blivet/devicelibs/mdraid.py          |  64 +++-----
 blivet/devicelibs/raid.py            | 297 +++++++++++++++++++++++++----------
 blivet/devices.py                    | 186 +++++++++++++++++-----
 tests/devicefactory_test.py          |  26 +--
 tests/devicelibs_test/mdraid_test.py |  21 +--
 tests/devicelibs_test/raid_test.py   |  73 ++++-----
 tests/devices_test.py                |  58 +++++--
 tests/devicetree_test.py             |   2 +-
 12 files changed, 590 insertions(+), 299 deletions(-)

[blivet:master 01/26] Import name 'lvm' instead of names from lvm package.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicefactory.py | 30 ++++++++++++++----------------
 1 file changed, 14 insertions(+), 16 deletions(-)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index b30c55b..ee0c651 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -25,9 +25,7 @@ from .errors import DeviceFactoryError, StorageError
 from .devices import LUKSDevice
 from .formats import getFormat
 from .devicelibs import mdraid
-from .devicelibs.lvm import get_pv_space
-from .devicelibs.lvm import get_pool_padding
-from .devicelibs.lvm import LVM_PE_SIZE
+from .devicelibs import lvm
 from .partitioning import SameSizeSet
 from .partitioning import TotalSizeSet
 from .partitioning import doPartitioning
@@ -1109,7 +1107,7 @@ class LVMFactory(DeviceFactory):
             super(LVMFactory, self)._handle_no_size()

def _get_device_space(self):
-        return get_pv_space(self.size, len(self._get_member_devices()))
+        return lvm.get_pv_space(self.size, len(self._get_member_devices()))

def _get_device_size(self):
         size = self.size
@@ -1153,7 +1151,7 @@ class LVMFactory(DeviceFactory):
             log.debug("size bumped to %s to include free disk space", size)
         else:
             # container_size is a request for a fixed size for the container
-            size += get_pv_space(self.container_size, len(self.disks))
+            size += lvm.get_pv_space(self.container_size, len(self.disks))

# this does not apply if a specific container size was requested
         if self.container_size in [SIZE_POLICY_AUTO, SIZE_POLICY_MAX]:
@@ -1163,18 +1161,18 @@ class LVMFactory(DeviceFactory):
                 # The member count here uses the container's current member set
                 # since that's the basis for the current device's disk space
                 # usage.
-                size -= get_pv_space(self.device.size,
+                size -= lvm.get_pv_space(self.device.size,
                    len(self.container.parents))
                 log.debug("size cut to %s to omit old device space", size)

if self.container_raid_level:
             # add five extents per disk to account for md metadata
             # (it was originally one per disk but that wasn't enough for raid5)
-            size += LVM_PE_SIZE * len(self.disks) * 5
+            size += lvm.LVM_PE_SIZE * len(self.disks) * 5

if self.container_encrypted:
             # Add space for LUKS metadata, each parent will be encrypted
-            size += LVM_PE_SIZE * len(self.disks)
+            size += lvm.LVM_PE_SIZE * len(self.disks)

return size

@@ -1222,7 +1220,7 @@ class LVMFactory(DeviceFactory):
                                                                  SIZE_POLICY_MAX]:
             # container pushed to the limit, but we need some extra space for
             # metadata, so we need to make the LV smaller
-            extra_md_space = LVM_PE_SIZE * len(self.disks) * 5
+            extra_md_space = lvm.LVM_PE_SIZE * len(self.disks) * 5
             kwargs["size"] -= extra_md_space
         return self.storage.newLV(*args, **kwargs)

@@ -1348,7 +1346,7 @@ class LVMThinPFactory(LVMFactory):
     @property
     def _pesize(self):
         """ The extent size of our vg or the default if we have no vg. """
-        return getattr(self.container, "peSize", LVM_PE_SIZE)
+        return getattr(self.container, "peSize", lvm.LVM_PE_SIZE)

def _get_device_space(self):
         """ Calculate and return the total disk space needed for the device.
@@ -1357,7 +1355,7 @@ class LVMThinPFactory(LVMFactory):
         """
         space = super(LVMThinPFactory, self)._get_device_space()
         log.debug("calculated total disk space prior to padding: %s", space)
-        space += get_pool_padding(space, pesize=self._pesize)
+        space += lvm.get_pool_padding(space, pesize=self._pesize)
         log.debug("total disk space needed: %s", space)
         return space

@@ -1375,7 +1373,7 @@ class LVMThinPFactory(LVMFactory):
                 size -= self.pool.freeSpace
                 log.debug("size cut to %s to omit pool free space", size)

-                pad = get_pool_padding(self.pool.freeSpace,
+                pad = lvm.get_pool_padding(self.pool.freeSpace,
                                        pesize=self._pesize)
                 size -= pad
                 log.debug("size cut to %s to omit pool padding from free "
@@ -1387,7 +1385,7 @@ class LVMThinPFactory(LVMFactory):
                 # The member count here uses the container's current member set
                 # since that's the basis for the current device's disk space
                 # usage.
-                pad = get_pool_padding(self.device.size,
+                pad = lvm.get_pool_padding(self.device.size,
                                        pesize=self._pesize)
                 log.debug("old device size: %s ; old pad: %s", self.device.size, pad)
                 size -= pad
@@ -1452,7 +1450,7 @@ class LVMThinPFactory(LVMFactory):
                 size -= self.device.poolSpaceUsed   # don't count our device

# increase vg free space by the size of the current pool's pad
-            pad = get_pool_padding(self.pool.size,
+            pad = lvm.get_pool_padding(self.pool.size,
                                    pesize=self._pesize)
             log.debug("increasing free by current pool pad size (%s)", pad)
             free += pad
@@ -1461,11 +1459,11 @@ class LVMThinPFactory(LVMFactory):
         free = self.container.align(free + self.container.freeSpace)
         size = self.container.align(size, roundup=True)

-        pad = get_pool_padding(size, pesize=self._pesize)
+        pad = lvm.get_pool_padding(size, pesize=self._pesize)

log.debug("size is %s ; pad is %s ; free is %s", size, pad, free)
         if free < (size + pad):
-            pad = int(get_pool_padding(free, pesize=self._pesize, reverse=True))
+            pad = int(lvm.get_pool_padding(free, pesize=self._pesize, reverse=True))
             free = self.container.align(free - pad) # round down
             log.info("adjusting pool size from %s to %s so it fits "
                      "in container %s", size, free, self.container.name)

[blivet:master 02/26] Comment mdraid.mdcreate() and update tests appropriately.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py          | 11 +++++++++++
 tests/devicelibs_test/mdraid_test.py |  5 +++--
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index d07f7bf..aeb554c 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -114,6 +114,17 @@ def mdadm(args):
         raise MDRaidError("running mdadm " + " ".join(args) + " failed")

def mdcreate(device, level, disks, spares=0, metadataVer=None, bitmap=False):
+    """ Create an mdarray from a list of devices.
+
+        :param str device: the name for the array
+        :param level: the level of the array
+        :type level: :class:`~.devicelibs.raid.RAIDLevel`
+        :param disks: the members of the array
+        :type disks: list of str
+        :param int spares: the number of spares in the array
+        :param str metadataVer: one of the mdadm metadata versions
+        :param bool bitmap: whether to create an internal bitmap on the device
+    """
     argv = ["--create", device, "--run", "--level=%s" % level]
     raid_devs = len(disks) - spares
     argv.append("--raid-devices=%d" % raid_devs)
diff --git a/tests/devicelibs_test/mdraid_test.py b/tests/devicelibs_test/mdraid_test.py
index 8de5414..73d2ff1 100755
--- a/tests/devicelibs_test/mdraid_test.py
+++ b/tests/devicelibs_test/mdraid_test.py
@@ -3,6 +3,7 @@ import os
 import unittest
 import time

+import blivet.devicelibs.raid as raid
 import blivet.devicelibs.mdraid as mdraid
 from blivet.errors import MDRaidError
 from blivet.size import Size
@@ -75,12 +76,12 @@ class MDRaidAsRootTestCase(baseclass.DevicelibsTestCase):
         ## mdcreate
         ##
         # pass
-        self.assertEqual(mdraid.mdcreate(self._dev_name, 1, [_LOOP_DEV0, _LOOP_DEV1]), None)
+        self.assertEqual(mdraid.mdcreate(self._dev_name, raid.RAID1, [_LOOP_DEV0, _LOOP_DEV1]), None)
         # wait for raid to settle
         time.sleep(2)

# fail
-        self.assertRaises(MDRaidError, mdraid.mdcreate, "/dev/md1", 1, ["/not/existing/dev0", "/not/existing/dev1"])
+        self.assertRaises(MDRaidError, mdraid.mdcreate, "/dev/md1", raid.RAID1, ["/not/existing/dev0", "/not/existing/dev1"])

##
         ## mddeactivate

[blivet:master 02/26] Comment mdraid.mdcreate() and update tests appropriately.

[David Lehman]

On 05/29/2014 01:47 PM, mulhern wrote:
> Signed-off-by: mulhern
>
> blivet/devicelibs/mdraid.py          | 11 +++++++++++
>   tests/devicelibs_test/mdraid_test.py |  5 +++--
>   2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
> index d07f7bf..aeb554c 100644
> --- a/blivet/devicelibs/mdraid.py
> +++ b/blivet/devicelibs/mdraid.py
> @@ -114,6 +114,17 @@ def mdadm(args):
>           raise MDRaidError("running mdadm " + " ".join(args) + " failed")
>
> def mdcreate(device, level, disks, spares=0, metadataVer=None, bitmap=False):
> """ Create an mdarray from a list of devices.
>    :param str device: the name for the array

We should refer to this as a path to be clear.
>    :param level: the level of the array
>    :type level: :class:`~.devicelibs.raid.RAIDLevel`

Just thinking out loud: it might make sense to also allow raid level as 
a string, especially if people will eventually be using devicelibs 
directly. OTOH I realize we're using Size for sizes in devicelibs, so 
maybe using RAIDLevel is equivalent to that.

David

[blivet:master 03/26] Remove DMRaidArrayDevice.members property.

[Anne Mulhern]

It seems to be entirely unused since ContainerDevice class was introduced.

Signed-off-by: mulhern 
---
 blivet/devices.py | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/blivet/devices.py b/blivet/devices.py
index cdd0642..f13ba8a 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3814,10 +3814,6 @@ class DMRaidArrayDevice(DMDevice, ContainerDevice):
         return self._raidSet

@property
-    def members(self):
-        return self.parents
-
-    @property
     def devices(self):
         """ Return a list of this array's member device instances. """
         return self.parents

[blivet:master 04/26] Extract selection of members in completed() into a separate method.

[Anne Mulhern]

Use it in completed().

Signed-off-by: mulhern 
---
 blivet/devices.py | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/blivet/devices.py b/blivet/devices.py
index f13ba8a..e8004b7 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3605,13 +3605,22 @@ class MDRaidArrayDevice(ContainerDevice):
         return rc

@property
-    def complete(self):
+    def members(self):
+        """ The list of this md array's members.
+
+            If the array is a BIOS RAID array then its unique parent
+            is a container and its actual member devices are the
+            container's parents.
+        """
         if self.type == "mdbiosraidarray":
-            members = len(self.parents[0].parents)
+            members = self.parents[0].parents
         else:
-            members = len(self.parents)
+            members = self.parents
+        return members

-        return (self.memberDevices <= members) or not self.exists
+    @property
+    def complete(self):
+        return (self.memberDevices <= len(self.members)) or not self.exists

@property
     def devices(self):

[blivet:master 04/26] Extract selection of members in completed() into a separate method.

[Vratislav Podzimek]

On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> Use it in completed().
> Signed-off-by: mulhern
>
> blivet/devices.py | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/blivet/devices.py b/blivet/devices.py
> index f13ba8a..e8004b7 100644
> --- a/blivet/devices.py
> +++ b/blivet/devices.py
> @@ -3605,13 +3605,22 @@ class MDRaidArrayDevice(ContainerDevice):
>          return rc
>  @property
> def complete(self):
> def members(self):
>    """ The list of this md array's members.
>        If the array is a BIOS RAID array then its unique parent
>        is a container and its actual member devices are the
>        container's parents.
>    """
>    if self.type == "mdbiosraidarray":
>        members = len(self.parents[0].parents)
>        members = self.parents[0].parents
>    else:
>        members = len(self.parents)
>        members = self.parents
>    return members

I believe self.parents is an instance of the ParentList class and this
property is supposed to return a list. So something like
    members = list(self.parents)
would be better.

[blivet:master 04/26] Extract selection of members in completed() into a separate method.

[Anne Mulhern]

----- Original Message -----
> From: "Vratislav Podzimek" 
> To: "anaconda patch review" 
> Sent: Friday, May 30, 2014 3:07:19 AM
> Subject: Re: [blivet:master 04/26] Extract selection of members in	completed() into a separate method.
>
> On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> > Use it in completed().
> > Signed-off-by: mulhern
> >
> > blivet/devices.py | 17 +++++++++++++----
> >  1 file changed, 13 insertions(+), 4 deletions(-)
> >
> > diff --git a/blivet/devices.py b/blivet/devices.py
> > index f13ba8a..e8004b7 100644
> > --- a/blivet/devices.py
> > +++ b/blivet/devices.py
> > @@ -3605,13 +3605,22 @@ class MDRaidArrayDevice(ContainerDevice):
> >          return rc
> >  @property
> > def complete(self):
> > def members(self):
> >    """ The list of this md array's members.
> >        If the array is a BIOS RAID array then its unique parent
> >        is a container and its actual member devices are the
> >        container's parents.
> >    """
> >    if self.type == "mdbiosraidarray":
> >        members = len(self.parents[0].parents)
> >        members = self.parents[0].parents
> >    else:
> >        members = len(self.parents)
> >        members = self.parents
> >    return members
>
> I believe self.parents is an instance of the ParentList class and this
> property is supposed to return a list. So something like
>     members = list(self.parents)
> would be better.
>
> --
> Vratislav Podzimek
>
> Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic
>
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

Ok, sent a slightly revised patch.

- mulhern

[blivet:master 05/26] No longer use _standard_levels as the default set of RAID levels.

[Anne Mulhern]

The notion of a "standard" RAID level is being eroded too rapidly to make it
really useful. ALL_LEVELS is a better notion.
This change has the added benefit of simplifying the RAIDlevels constructor.
Also, make _raid_levels property a set instead of a list since no real
ordering make sense.

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py        |  2 +-
 blivet/devicelibs/raid.py          | 77 ++++++++++++--------------------------
 tests/devicelibs_test/raid_test.py | 13 ++++---
 3 files changed, 31 insertions(+), 61 deletions(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index aeb554c..2106548 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -44,7 +44,7 @@ class MDRaidLevels(raid.RAIDLevels):
         except AttributeError:
             return False

-_RAID_levels = MDRaidLevels()
+_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10"])

class Container(object):
     name = "container"
diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index bdc17f7..86b584e 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -245,38 +245,24 @@ class RAIDLevels(object):
        for its RAID levels.
     """

-    def __init__(self, levels=True):
+    def __init__(self, levels=None):
         """Add the specified standard levels to the levels in this object.

-           :param levels: the standard levels to be added to this object
-           :type levels: bool or a list of valid RAID level descriptors
+           :param levels: the levels to be added to this object
+           :type levels: list of valid RAID level descriptors

If levels is True, add all standard levels. Else, levels
            must be a list of valid level descriptors of standard levels.
            Duplicate descriptors are ignored.
         """
-        self._raid_levels = []
-        if levels is False:
-            pass
-        elif levels is True:
-            for l in self.standard_levels:
-                self.addRaidLevel(l)
-        else:
-            try:
-                for level in levels:
-                    matches = [l for l in self.standard_levels if level in l.names]
-                    if len(matches) != 1:
-                        raise RaidError("invalid standard RAID level descriptor %s" % level)
-                    else:
-                        self.addRaidLevel(matches[0])
-            except TypeError:
-                raise RaidError("levels must be a boolean or an iterable")
-
-
-    _standard_levels = []
-
-    standard_levels = property(lambda s: s._standard_levels,
-       doc="any standard RAID level classes defined in this package.")
+        levels = levels or []
+        self._raid_levels = set()
+        for level in levels:
+            matches = [l for l in ALL_LEVELS if level in l.names]
+            if len(matches) != 1:
+                raise RaidError("invalid standard RAID level descriptor %s" % level)
+            else:
+                self.addRaidLevel(matches[0])

@classmethod
     def isRaidLevel(cls, level):
@@ -291,12 +277,11 @@ class RAIDLevels(object):
            The name property must be defined; it should be one of the
            elements in the names list.

-           All RAID objects in standard_levels are guaranteed to pass these
+           All RAID objects that extend RAIDlevel are guaranteed to pass these
            minimum requirements.

This method should not be overridden in any subclass so that it
-           is so restrictive that a RAID object in standard_levels does
-           not satisfy it.
+           is so restrictive that a RAIDlevel object does not satisfy it.
         """
         try:
             name = level.names[0]
@@ -307,23 +292,6 @@ class RAIDLevels(object):
             return False
         return True

-    @classmethod
-    def addRAIDLevelToStandardLevels(cls, level):
-        """Adds this RAID level to the internal list of standard RAID levels
-           defined in this package.
-
-           :param level: an object representing a RAID level
-           :type level: object
-
-           Raises a RaidError if level is not a valid RAID level.
-
-           Does not allow duplicate level objects.
-        """
-        if not cls.isRaidLevel(level):
-            raise RaidError('level is not a valid RAID level')
-        if not level in cls._standard_levels:
-            cls._standard_levels.append(level)
-
     def raidLevel(self, descriptor):
         """Return RAID object corresponding to descriptor.

@@ -338,7 +306,7 @@ class RAIDLevels(object):
         raise RaidError("invalid RAID level descriptor %s" % descriptor)

def addRaidLevel(self, level):
-        """Adds level to the list of levels if it is not already there.
+        """Adds level to levels if it is not already there.

:param object level: an object representing a RAID level

@@ -348,12 +316,13 @@ class RAIDLevels(object):
         """
         if not self.isRaidLevel(level):
             raise RaidError("level is not a valid RAID level")
-        if not level in self._raid_levels:
-            self._raid_levels.append(level)
+        self._raid_levels.add(level)

def __iter__(self):
         return iter(self._raid_levels)

+ALL_LEVELS = RAIDLevels()
+
 class RAID0(RAIDLevel):

level = property(lambda s: "0")
@@ -376,7 +345,7 @@ class RAID0(RAIDLevel):
         return member_count * 16

RAID0 = RAID0()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID0)
+ALL_LEVELS.addRaidLevel(RAID0)

class RAID1(RAIDLevel):
     level = property(lambda s: "1")
@@ -399,7 +368,7 @@ class RAID1(RAIDLevel):
         return None

RAID1 = RAID1()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID1)
+ALL_LEVELS.addRaidLevel(RAID1)

class RAID4(RAIDLevel):
     level = property(lambda s: "4")
@@ -422,7 +391,7 @@ class RAID4(RAIDLevel):
         return (member_count - 1) * 16

RAID4 = RAID4()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID4)
+ALL_LEVELS.addRaidLevel(RAID4)

class RAID5(RAIDLevel):
     level = property(lambda s: "5")
@@ -445,7 +414,7 @@ class RAID5(RAIDLevel):
         return (member_count - 1) * 16

RAID5 = RAID5()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID5)
+ALL_LEVELS.addRaidLevel(RAID5)

class RAID6(RAIDLevel):
     level = property(lambda s: "6")
@@ -468,7 +437,7 @@ class RAID6(RAIDLevel):
         return None

RAID6 = RAID6()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID6)
+ALL_LEVELS.addRaidLevel(RAID6)

class RAID10(RAIDLevel):
     level = property(lambda s: "10")
@@ -491,4 +460,4 @@ class RAID10(RAIDLevel):
         return None

RAID10 = RAID10()
-RAIDLevels.addRAIDLevelToStandardLevels(RAID10)
+ALL_LEVELS.addRaidLevel(RAID10)
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index 7c7501b..e192aea 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -1,6 +1,7 @@
 #!/usr/bin/python
 import unittest

+import blivet.devicelibs.mdraid as mdraid
 import blivet.devicelibs.raid as raid
 import blivet.errors as errors
 from blivet.size import Size
@@ -8,8 +9,8 @@ from blivet.size import Size
 class RaidTestCase(unittest.TestCase):

def setUp(self):
-        self.levels = raid.RAIDLevels()
-        self.levels_none = raid.RAIDLevels(False)
+        self.levels = raid.RAIDLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10"])
+        self.levels_none = raid.RAIDLevels([])
         self.levels_some = raid.RAIDLevels(["mirror", 6])

def testRaid(self):
@@ -99,7 +100,7 @@ class RaidTestCase(unittest.TestCase):
         ##
         ## size
         ##
-        for r in raid.RAIDLevels():
+        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -109,7 +110,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size(0)),
                r.get_net_array_size(4, Size("32MiB")))

-        for r in raid.RAIDLevels():
+        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -119,7 +120,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size(0)),
                r.get_net_array_size(5, Size("32MiB")))

-        for r in raid.RAIDLevels():
+        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -129,7 +130,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size("32MiB")),
                0)

-        for r in raid.RAIDLevels():
+        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),

[blivet:master 05/26] No longer use _standard_levels as the default set of RAID levels.

[Vratislav Podzimek]

On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> The notion of a "standard" RAID level is being eroded too rapidly to make it
> really useful. ALL_LEVELS is a better notion.
> This change has the added benefit of simplifying the RAIDlevels constructor.
> Also, make _raid_levels property a set instead of a list since no real
> ordering make sense.

I believe the ordering makes sense -- users would be quite confused if
we offered them to choose from the following RAID levels:
1. RAID 4
2. RAID 0
3. RAID 5
4. RAID 10
...

Maybe the RAID levels don't have to be ordered internally, but when
queried, they should be returned as a list from the lowest number to the
biggest one. Anaconda does the ordering by its Glade files, but that
doesn't have to be the case of other tools using blivet.
> Signed-off-by: mulhern
>
> blivet/devicelibs/mdraid.py        |  2 +-
>  blivet/devicelibs/raid.py          | 77 ++++++++++++--------------------------
>  tests/devicelibs_test/raid_test.py | 13 ++++---
>  3 files changed, 31 insertions(+), 61 deletions(-)
>
> diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
> index aeb554c..2106548 100644
> --- a/blivet/devicelibs/mdraid.py
> +++ b/blivet/devicelibs/mdraid.py
> @@ -44,7 +44,7 @@ class MDRaidLevels(raid.RAIDLevels):
>          except AttributeError:
>              return False
>
> -_RAID_levels = MDRaidLevels()
> +_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10"])
>
> class Container(object):
>      name = "container"
> diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
> index bdc17f7..86b584e 100644
> --- a/blivet/devicelibs/raid.py
> +++ b/blivet/devicelibs/raid.py
> @@ -245,38 +245,24 @@ class RAIDLevels(object):
>         for its RAID levels.
>      """
> def __init__(self, levels=True):
> def __init__(self, levels=None):
>     """Add the specified standard levels to the levels in this object.
>       :param levels: the standard levels to be added to this object
>       :type levels: bool or a list of valid RAID level descriptors
>       :param levels: the levels to be added to this object
>       :type levels: list of valid RAID level descriptors
>
>       If levels is True, add all standard levels. Else, levels
>       must be a list of valid level descriptors of standard levels.
>       Duplicate descriptors are ignored.
>    """
>    self._raid_levels = []
>    if levels is False:
>        pass
>    elif levels is True:
>        for l in self.standard_levels:
>            self.addRaidLevel(l)
>    else:
>        try:
>            for level in levels:
>                matches = [l for l in self.standard_levels if level in l.names]
>                if len(matches) != 1:
>                    raise RaidError("invalid standard RAID level descriptor %s" % level)
>                else:
>                    self.addRaidLevel(matches[0])
>        except TypeError:
>            raise RaidError("levels must be a boolean or an iterable")
> _standard_levels = []
> standard_levels = property(lambda s: s._standard_levels,
>   doc="any standard RAID level classes defined in this package.")
>    levels = levels or []
>    self._raid_levels = set()
>    for level in levels:
>        matches = [l for l in ALL_LEVELS if level in l.names]
>        if len(matches) != 1:
>            raise RaidError("invalid standard RAID level descriptor %s" % level)
>        else:
>            self.addRaidLevel(matches[0])
>
> @classmethod
>    def isRaidLevel(cls, level):
>
> @@ -291,12 +277,11 @@ class RAIDLevels(object):
>             The name property must be defined; it should be one of the
>             elements in the names list.
>       All RAID objects in standard_levels are guaranteed to pass these
>       All RAID objects that extend RAIDlevel are guaranteed to pass these
>       minimum requirements.
>
>       This method should not be overridden in any subclass so that it
>       is so restrictive that a RAID object in standard_levels does
>       not satisfy it.
>       is so restrictive that a RAIDlevel object does not satisfy it.
>    """
>    try:
>        name = level.names[0]
>
> @@ -307,23 +292,6 @@ class RAIDLevels(object):
>              return False
>          return True
> @classmethod
> def addRAIDLevelToStandardLevels(cls, level):
>    """Adds this RAID level to the internal list of standard RAID levels
>       defined in this package.
>       :param level: an object representing a RAID level
>       :type level: object
>       Raises a RaidError if level is not a valid RAID level.
>       Does not allow duplicate level objects.
>    """
>    if not cls.isRaidLevel(level):
>        raise RaidError('level is not a valid RAID level')
>    if not level in cls._standard_levels:
>        cls._standard_levels.append(level)
> def raidLevel(self, descriptor):
>        """Return RAID object corresponding to descriptor.
>
> @@ -338,7 +306,7 @@ class RAIDLevels(object):
>          raise RaidError("invalid RAID level descriptor %s" % descriptor)
>  def addRaidLevel(self, level):
>    """Adds level to the list of levels if it is not already there.
>    """Adds level to levels if it is not already there.
>
>       :param object level: an object representing a RAID level
>
> @@ -348,12 +316,13 @@ class RAIDLevels(object):
>          """
>          if not self.isRaidLevel(level):
>              raise RaidError("level is not a valid RAID level")
>    if not level in self._raid_levels:
>        self._raid_levels.append(level)
>    self._raid_levels.add(level)
>
> def __iter__(self):
>        return iter(self._raid_levels)
>
> +ALL_LEVELS = RAIDLevels()
>
> class RAID0(RAIDLevel):
>  level = property(lambda s: "0")
>
> @@ -376,7 +345,7 @@ class RAID0(RAIDLevel):
>          return member_count * 16
>
> RAID0 = RAID0()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID0)
> +ALL_LEVELS.addRaidLevel(RAID0)
>
> class RAID1(RAIDLevel):
>      level = property(lambda s: "1")
> @@ -399,7 +368,7 @@ class RAID1(RAIDLevel):
>          return None
>
> RAID1 = RAID1()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID1)
> +ALL_LEVELS.addRaidLevel(RAID1)
>
> class RAID4(RAIDLevel):
>      level = property(lambda s: "4")
> @@ -422,7 +391,7 @@ class RAID4(RAIDLevel):
>          return (member_count - 1) * 16
>
> RAID4 = RAID4()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID4)
> +ALL_LEVELS.addRaidLevel(RAID4)
>
> class RAID5(RAIDLevel):
>      level = property(lambda s: "5")
> @@ -445,7 +414,7 @@ class RAID5(RAIDLevel):
>          return (member_count - 1) * 16
>
> RAID5 = RAID5()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID5)
> +ALL_LEVELS.addRaidLevel(RAID5)
>
> class RAID6(RAIDLevel):
>      level = property(lambda s: "6")
> @@ -468,7 +437,7 @@ class RAID6(RAIDLevel):
>          return None
>
> RAID6 = RAID6()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID6)
> +ALL_LEVELS.addRaidLevel(RAID6)
>
> class RAID10(RAIDLevel):
>      level = property(lambda s: "10")
> @@ -491,4 +460,4 @@ class RAID10(RAIDLevel):
>          return None
>
> RAID10 = RAID10()
> -RAIDLevels.addRAIDLevelToStandardLevels(RAID10)
> +ALL_LEVELS.addRaidLevel(RAID10)
> diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
> index 7c7501b..e192aea 100644
> --- a/tests/devicelibs_test/raid_test.py
> +++ b/tests/devicelibs_test/raid_test.py
> @@ -1,6 +1,7 @@
>  #!/usr/bin/python
>  import unittest
>
> +import blivet.devicelibs.mdraid as mdraid
>  import blivet.devicelibs.raid as raid
>  import blivet.errors as errors
>  from blivet.size import Size
> @@ -8,8 +9,8 @@ from blivet.size import Size
>  class RaidTestCase(unittest.TestCase):
>  def setUp(self):
>    self.levels = raid.RAIDLevels()
>    self.levels_none = raid.RAIDLevels(False)
>    self.levels = raid.RAIDLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10"])
>    self.levels_none = raid.RAIDLevels([])
>    self.levels_some = raid.RAIDLevels(["mirror", 6])
>
> def testRaid(self):
>
> @@ -99,7 +100,7 @@ class RaidTestCase(unittest.TestCase):
>          ##
>          ## size
>          ##
>    for r in raid.RAIDLevels():
>    for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:

This and all the other similar lines should use generator comprehension
instead of list comprehension.
>          self.assertEqual(r.get_size([ Size("32MiB"),
>                                    Size("128MiB"),
>                                    Size("128MiB"),
>
> @@ -109,7 +110,7 @@ class RaidTestCase(unittest.TestCase):
>                 lambda x: Size(0)),
>                 r.get_net_array_size(4, Size("32MiB")))
>    for r in raid.RAIDLevels():
>    for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
>        self.assertEqual(r.get_size([ Size("32MiB"),
>                                  Size("128MiB"),
>                                  Size("128MiB"),
>
> @@ -119,7 +120,7 @@ class RaidTestCase(unittest.TestCase):
>                 lambda x: Size(0)),
>                 r.get_net_array_size(5, Size("32MiB")))
>    for r in raid.RAIDLevels():
>    for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
>        self.assertEqual(r.get_size([ Size("32MiB"),
>                                  Size("128MiB"),
>                                  Size("128MiB"),
>
> @@ -129,7 +130,7 @@ class RaidTestCase(unittest.TestCase):
>                 lambda x: Size("32MiB")),
>                 0)
>    for r in raid.RAIDLevels():
>    for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
>        self.assertEqual(r.get_size([ Size("32MiB"),
>                                  Size("128MiB"),
>                                  Size("128MiB"),

[blivet:master 05/26] No longer use _standard_levels as the default set of RAID levels.

[Anne Mulhern]

----- Original Message -----
> From: "Vratislav Podzimek" 
> To: "anaconda patch review" 
> Sent: Friday, May 30, 2014 3:14:29 AM
> Subject: Re: [blivet:master 05/26] No longer use _standard_levels as the	default set of RAID levels.
>
> On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> > The notion of a "standard" RAID level is being eroded too rapidly to make
> > it
> > really useful. ALL_LEVELS is a better notion.
> > This change has the added benefit of simplifying the RAIDlevels
> > constructor.
> > Also, make _raid_levels property a set instead of a list since no real
> > ordering make sense.
>
> I believe the ordering makes sense -- users would be quite confused if
> we offered them to choose from the following RAID levels:
> RAID 4
> RAID 0
> RAID 5
> RAID 10
>
> ...
>
> Maybe the RAID levels don't have to be ordered internally, but when
> queried, they should be returned as a list from the lowest number to the
> biggest one. Anaconda does the ordering by its Glade files, but that
> doesn't have to be the case of other tools using blivet.

Where do Linear, Single, Container, RAID01, Dup, etc. fit in your ordering?
I'ld rather have clients decide what is best for them then impose an ordering
that will surely be wrong for some.
> > Signed-off-by: mulhern
> >
> > blivet/devicelibs/mdraid.py        |  2 +-
> >  blivet/devicelibs/raid.py          | 77
> >  ++++++++++++--------------------------
> >  tests/devicelibs_test/raid_test.py | 13 ++++---
> >  3 files changed, 31 insertions(+), 61 deletions(-)
> >
> > diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
> > index aeb554c..2106548 100644
> > --- a/blivet/devicelibs/mdraid.py
> > +++ b/blivet/devicelibs/mdraid.py
> > @@ -44,7 +44,7 @@ class MDRaidLevels(raid.RAIDLevels):
> >          except AttributeError:
> >              return False
> >
> > -_RAID_levels = MDRaidLevels()
> > +_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6",
> > "raid10"])
> >
> > class Container(object):
> >      name = "container"
> > diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
> > index bdc17f7..86b584e 100644
> > --- a/blivet/devicelibs/raid.py
> > +++ b/blivet/devicelibs/raid.py
> > @@ -245,38 +245,24 @@ class RAIDLevels(object):
> >         for its RAID levels.
> >      """
> > def __init__(self, levels=True):
> > def __init__(self, levels=None):
> >     """Add the specified standard levels to the levels in this object.
> >       :param levels: the standard levels to be added to this object
> >       :type levels: bool or a list of valid RAID level descriptors
> >       :param levels: the levels to be added to this object
> >       :type levels: list of valid RAID level descriptors
> >
> >       If levels is True, add all standard levels. Else, levels
> >       must be a list of valid level descriptors of standard levels.
> >       Duplicate descriptors are ignored.
> >    """
> >    self._raid_levels = []
> >    if levels is False:
> >        pass
> >    elif levels is True:
> >        for l in self.standard_levels:
> >            self.addRaidLevel(l)
> >    else:
> >        try:
> >            for level in levels:
> >                matches = [l for l in self.standard_levels if level in
> >
> > l.names]
> >                if len(matches) != 1:
> >                    raise RaidError("invalid standard RAID level
> >
> > descriptor %s" % level)
> >                else:
> >                    self.addRaidLevel(matches[0])
> >        except TypeError:
> >            raise RaidError("levels must be a boolean or an iterable")
> > _standard_levels = []
> > standard_levels = property(lambda s: s._standard_levels,
> >   doc="any standard RAID level classes defined in this package.")
> >    levels = levels or []
> >    self._raid_levels = set()
> >    for level in levels:
> >        matches = [l for l in ALL_LEVELS if level in l.names]
> >        if len(matches) != 1:
> >            raise RaidError("invalid standard RAID level descriptor
> >
> > %s" % level)
> >        else:
> >            self.addRaidLevel(matches[0])
> >
> > @classmethod
> >    def isRaidLevel(cls, level):
> >
> > @@ -291,12 +277,11 @@ class RAIDLevels(object):
> >             The name property must be defined; it should be one of the
> >             elements in the names list.
> >       All RAID objects in standard_levels are guaranteed to pass
> >
> > these
> >       All RAID objects that extend RAIDlevel are guaranteed to pass
> >
> > these
> >             minimum requirements.
> >         This method should not be overridden in any subclass so that it
> >       is so restrictive that a RAID object in standard_levels does
> >       not satisfy it.
> >       is so restrictive that a RAIDlevel object does not satisfy it.
> >    """
> >    try:
> >        name = level.names[0]
> >
> > @@ -307,23 +292,6 @@ class RAIDLevels(object):
> >              return False
> >          return True
> > @classmethod
> > def addRAIDLevelToStandardLevels(cls, level):
> >    """Adds this RAID level to the internal list of standard RAID
> >
> > levels
> >       defined in this package.
> >       :param level: an object representing a RAID level
> >       :type level: object
> >       Raises a RaidError if level is not a valid RAID level.
> >       Does not allow duplicate level objects.
> >    """
> >    if not cls.isRaidLevel(level):
> >        raise RaidError('level is not a valid RAID level')
> >    if not level in cls._standard_levels:
> >        cls._standard_levels.append(level)
> > def raidLevel(self, descriptor):
> >        """Return RAID object corresponding to descriptor.
> >
> > @@ -338,7 +306,7 @@ class RAIDLevels(object):
> >          raise RaidError("invalid RAID level descriptor %s" % descriptor)
> >  def addRaidLevel(self, level):
> >    """Adds level to the list of levels if it is not already there.
> >    """Adds level to levels if it is not already there.
> >
> >       :param object level: an object representing a RAID level
> >
> > @@ -348,12 +316,13 @@ class RAIDLevels(object):
> >          """
> >          if not self.isRaidLevel(level):
> >              raise RaidError("level is not a valid RAID level")
> >    if not level in self._raid_levels:
> >        self._raid_levels.append(level)
> >    self._raid_levels.add(level)
> >
> > def __iter__(self):
> >        return iter(self._raid_levels)
> >
> > +ALL_LEVELS = RAIDLevels()
> >
> > class RAID0(RAIDLevel):
> >  level = property(lambda s: "0")
> >
> > @@ -376,7 +345,7 @@ class RAID0(RAIDLevel):
> >          return member_count * 16
> >
> > RAID0 = RAID0()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID0)
> > +ALL_LEVELS.addRaidLevel(RAID0)
> >
> > class RAID1(RAIDLevel):
> >      level = property(lambda s: "1")
> > @@ -399,7 +368,7 @@ class RAID1(RAIDLevel):
> >          return None
> >
> > RAID1 = RAID1()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID1)
> > +ALL_LEVELS.addRaidLevel(RAID1)
> >
> > class RAID4(RAIDLevel):
> >      level = property(lambda s: "4")
> > @@ -422,7 +391,7 @@ class RAID4(RAIDLevel):
> >          return (member_count - 1) * 16
> >
> > RAID4 = RAID4()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID4)
> > +ALL_LEVELS.addRaidLevel(RAID4)
> >
> > class RAID5(RAIDLevel):
> >      level = property(lambda s: "5")
> > @@ -445,7 +414,7 @@ class RAID5(RAIDLevel):
> >          return (member_count - 1) * 16
> >
> > RAID5 = RAID5()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID5)
> > +ALL_LEVELS.addRaidLevel(RAID5)
> >
> > class RAID6(RAIDLevel):
> >      level = property(lambda s: "6")
> > @@ -468,7 +437,7 @@ class RAID6(RAIDLevel):
> >          return None
> >
> > RAID6 = RAID6()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID6)
> > +ALL_LEVELS.addRaidLevel(RAID6)
> >
> > class RAID10(RAIDLevel):
> >      level = property(lambda s: "10")
> > @@ -491,4 +460,4 @@ class RAID10(RAIDLevel):
> >          return None
> >
> > RAID10 = RAID10()
> > -RAIDLevels.addRAIDLevelToStandardLevels(RAID10)
> > +ALL_LEVELS.addRaidLevel(RAID10)
> > diff --git a/tests/devicelibs_test/raid_test.py
> > b/tests/devicelibs_test/raid_test.py
> > index 7c7501b..e192aea 100644
> > --- a/tests/devicelibs_test/raid_test.py
> > +++ b/tests/devicelibs_test/raid_test.py
> > @@ -1,6 +1,7 @@
> >  #!/usr/bin/python
> >  import unittest
> >
> > +import blivet.devicelibs.mdraid as mdraid
> >  import blivet.devicelibs.raid as raid
> >  import blivet.errors as errors
> >  from blivet.size import Size
> > @@ -8,8 +9,8 @@ from blivet.size import Size
> >  class RaidTestCase(unittest.TestCase):
> >  def setUp(self):
> >    self.levels = raid.RAIDLevels()
> >    self.levels_none = raid.RAIDLevels(False)
> >    self.levels = raid.RAIDLevels(["raid0", "raid1", "raid4", "raid5",
> >
> > "raid6", "raid10"])
> >    self.levels_none = raid.RAIDLevels([])
> >    self.levels_some = raid.RAIDLevels(["mirror", 6])
> >
> > def testRaid(self):
> >
> > @@ -99,7 +100,7 @@ class RaidTestCase(unittest.TestCase):
> >          ##
> >          ## size
> >          ##
> >    for r in raid.RAIDLevels():
> >    for r in [l for l in raid.ALL_LEVELS if l is not
> >
> > mdraid.Container]:
>
> This and all the other similar lines should use generator comprehension
> instead of list comprehension.

Done.
> >          self.assertEqual(r.get_size([ Size("32MiB"),
> >                                    Size("128MiB"),
> >                                    Size("128MiB"),
> >
> > @@ -109,7 +110,7 @@ class RaidTestCase(unittest.TestCase):
> >                 lambda x: Size(0)),
> >                 r.get_net_array_size(4, Size("32MiB")))
> >    for r in raid.RAIDLevels():
> >    for r in [l for l in raid.ALL_LEVELS if l is not
> >
> > mdraid.Container]:
> >              self.assertEqual(r.get_size([ Size("32MiB"),
> >                                        Size("128MiB"),
> >                                        Size("128MiB"),
> > @@ -119,7 +120,7 @@ class RaidTestCase(unittest.TestCase):
> >                 lambda x: Size(0)),
> >                 r.get_net_array_size(5, Size("32MiB")))
> >    for r in raid.RAIDLevels():
> >    for r in [l for l in raid.ALL_LEVELS if l is not
> >
> > mdraid.Container]:
> >              self.assertEqual(r.get_size([ Size("32MiB"),
> >                                        Size("128MiB"),
> >                                        Size("128MiB"),
> > @@ -129,7 +130,7 @@ class RaidTestCase(unittest.TestCase):
> >                 lambda x: Size("32MiB")),
> >                 0)
> >    for r in raid.RAIDLevels():
> >    for r in [l for l in raid.ALL_LEVELS if l is not
> >
> > mdraid.Container]:
> >              self.assertEqual(r.get_size([ Size("32MiB"),
> >                                        Size("128MiB"),
> >                                        Size("128MiB"),
>
> --
> Vratislav Podzimek
>
> Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic
>
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

[blivet:master 05/26] No longer use _standard_levels as the default set of RAID levels.

[Vratislav Podzimek]

On Fri, 2014-05-30 at 09:07 -0400, Anne Mulhern wrote:
> ----- Original Message -----
> > From: "Vratislav Podzimek" 
> > To: "anaconda patch review" 
> > Sent: Friday, May 30, 2014 3:14:29 AM
> > Subject: Re: [blivet:master 05/26] No longer use _standard_levels as the	default set of RAID levels.
> >
> > On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> > > The notion of a "standard" RAID level is being eroded too rapidly to make
> > > it
> > > really useful. ALL_LEVELS is a better notion.
> > > This change has the added benefit of simplifying the RAIDlevels
> > > constructor.
> > > Also, make _raid_levels property a set instead of a list since no real
> > > ordering make sense.
> >
> > I believe the ordering makes sense -- users would be quite confused if
> > we offered them to choose from the following RAID levels:
> > RAID 4
> > RAID 0
> > RAID 5
> > RAID 10
> >
> > ...
> >
> > Maybe the RAID levels don't have to be ordered internally, but when
> > queried, they should be returned as a list from the lowest number to the
> > biggest one. Anaconda does the ordering by its Glade files, but that
> > doesn't have to be the case of other tools using blivet.
>
> Where do Linear, Single, Container, RAID01, Dup, etc. fit in your ordering?
> I'ld rather have clients decide what is best for them then impose an ordering
> that will surely be wrong for some.

Ah, that makes sense. I forgot to take these into account. Using set is
okay then.

[blivet:master 06/26] Adjust RaidLevel hierarchy so that all raid level objects extend RAIDLevel

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py | 58 ++++++++++++++++++++++++++---------------------
 1 file changed, 32 insertions(+), 26 deletions(-)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 86b584e..84f2aa1 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -34,18 +34,39 @@ def div_up(a,b):
     return (a + (b - 1))//b

class RAIDLevel(object):
+    """An abstract class which is the parent of all classes which represent
+       a RAID level.
+
+       It ensures that RAIDLevel objects will really be singleton objects
+       by overriding copy methods.
+    """
+    __metaclass__ = abc.ABCMeta
+
+    name = abc.abstractproperty(doc="The canonical name for this level")
+    names = abc.abstractproperty(doc="List of recognized names for this level.")
+    min_members = abc.abstractproperty(doc=
+       "The minimum number of members required to make a fully functioning array.")
+
+    def __str__(self):
+        return self.name
+
+    def __copy__(self):
+        return self
+
+    def __deepcopy__(self, memo):
+        # pylint: disable=unused-argument
+        return self
+
+class RAIDn(RAIDLevel):

"""An abstract class which is the parent of classes which represent a
-       RAID level. A better word would be classification, since 'level'
+       numeric RAID level. A better word would be classification, since 'level'
        implies an ordering, but level is the canonical word.

The abstract properties of the class are:

- level: A string containing the number that designates this level

-       - min_members: The minimum number of members required for this
-           level to be sensibly used.
-
        - nick: A single nickname for this level, may be None

All methods in this class fall into these categories:
@@ -66,10 +87,6 @@ class RAIDLevel(object):

# ABSTRACT PROPERTIES
     level = abc.abstractproperty(doc="A code representing the level")
-
-    min_members = abc.abstractproperty(
-       doc="The minimum number of members required for this level")
-
     nick = abc.abstractproperty(doc="A nickname for this level")

# PROPERTIES
@@ -234,10 +251,6 @@ class RAIDLevel(object):
         min_data_size = self._trim(min_size - superblock_size, chunk_size)
         return self.get_net_array_size(num_members, min_data_size)

-    def __str__(self):
-        return self.name
-
-
 class RAIDLevels(object):
     """A class which keeps track of registered RAID levels. This class
        may be extended, overriding the isRaid method to include any
@@ -283,14 +296,7 @@ class RAIDLevels(object):
            This method should not be overridden in any subclass so that it
            is so restrictive that a RAIDlevel object does not satisfy it.
         """
-        try:
-            name = level.names[0]
-            name = level.name
-            if name not in level.names:
-                return False
-        except (TypeError, AttributeError, IndexError):
-            return False
-        return True
+        return len(level.names) > 0 and level.name in level.names

def raidLevel(self, descriptor):
         """Return RAID object corresponding to descriptor.
@@ -323,7 +329,7 @@ class RAIDLevels(object):

ALL_LEVELS = RAIDLevels()

-class RAID0(RAIDLevel):
+class RAID0(RAIDn):

level = property(lambda s: "0")
     min_members = property(lambda s: 2)
@@ -347,7 +353,7 @@ class RAID0(RAIDLevel):
 RAID0 = RAID0()
 ALL_LEVELS.addRaidLevel(RAID0)

-class RAID1(RAIDLevel):
+class RAID1(RAIDn):
     level = property(lambda s: "1")
     min_members = property(lambda s: 2)
     nick = property(lambda s: "mirror")
@@ -370,7 +376,7 @@ class RAID1(RAIDLevel):
 RAID1 = RAID1()
 ALL_LEVELS.addRaidLevel(RAID1)

-class RAID4(RAIDLevel):
+class RAID4(RAIDn):
     level = property(lambda s: "4")
     min_members = property(lambda s: 3)
     nick = property(lambda s: None)
@@ -393,7 +399,7 @@ class RAID4(RAIDLevel):
 RAID4 = RAID4()
 ALL_LEVELS.addRaidLevel(RAID4)

-class RAID5(RAIDLevel):
+class RAID5(RAIDn):
     level = property(lambda s: "5")
     min_members = property(lambda s: 3)
     nick = property(lambda s: None)
@@ -416,7 +422,7 @@ class RAID5(RAIDLevel):
 RAID5 = RAID5()
 ALL_LEVELS.addRaidLevel(RAID5)

-class RAID6(RAIDLevel):
+class RAID6(RAIDn):
     level = property(lambda s: "6")
     min_members = property(lambda s: 4)
     nick = property(lambda s: None)
@@ -439,7 +445,7 @@ class RAID6(RAIDLevel):
 RAID6 = RAID6()
 ALL_LEVELS.addRaidLevel(RAID6)

-class RAID10(RAIDLevel):
+class RAID10(RAIDn):
     level = property(lambda s: "10")
     min_members = property(lambda s: 4)
     nick = property(lambda s: None)

[blivet:master 07/26] Allow the RAID object itself to be a valid RAID descriptor for lookup.

[Anne Mulhern]

Update tests accordingly.

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py          | 5 ++++-
 blivet/devices.py                  | 2 +-
 tests/devicelibs_test/raid_test.py | 1 +
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 84f2aa1..33af02f 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -303,11 +303,14 @@ class RAIDLevels(object):

:param object descriptor: a RAID level descriptor

+           Note that descriptor may be any object that identifies a
+           RAID level, including the RAID object itself.
+
            Raises a RaidError if no RAID object can be found for this
            descriptor.
         """
         for level in self._raid_levels:
-            if descriptor in level.names:
+            if descriptor in level.names or descriptor == level:
                 return level
         raise RaidError("invalid RAID level descriptor %s" % descriptor)

diff --git a/blivet/devices.py b/blivet/devices.py
index e8004b7..9480bae 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3277,7 +3277,7 @@ class MDRaidArrayDevice(ContainerDevice):
             :type uuid: str

:keyword level: the device's RAID level
-            :type level: str (eg: "raid1")
+            :type level: any valid RAID level descriptor
             :keyword int memberDevices: the number of active member devices
             :keyword int totalDevices: the total number of member devices
             :keyword metadataVersion: the version of the device's md metadata
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index e192aea..f32c59e 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -45,6 +45,7 @@ class RaidTestCase(unittest.TestCase):
         self.assertIs(self.levels.raidLevel("raid4"), raid.RAID4)
         self.assertIs(self.levels.raidLevel("mirror"), raid.RAID1)
         self.assertIs(self.levels.raidLevel("stripe"), raid.RAID0)
+        self.assertIs(self.levels.raidLevel(raid.RAID0), raid.RAID0)

self.assertRaises(errors.RaidError, self.levels.raidLevel, "bogus")

[blivet:master 07/26] Allow the RAID object itself to be a valid RAID descriptor for lookup.

[Vratislav Podzimek]

On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> Update tests accordingly.
> Signed-off-by: mulhern
>
> blivet/devicelibs/raid.py          | 5 ++++-
>  blivet/devices.py                  | 2 +-
>  tests/devicelibs_test/raid_test.py | 1 +
>  3 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
> index 84f2aa1..33af02f 100644
> --- a/blivet/devicelibs/raid.py
> +++ b/blivet/devicelibs/raid.py
> @@ -303,11 +303,14 @@ class RAIDLevels(object):
>         :param object descriptor: a RAID level descriptor
>
>       Note that descriptor may be any object that identifies a
>       RAID level, including the RAID object itself.
>       Raises a RaidError if no RAID object can be found for this
>       descriptor.
>    """
>    for level in self._raid_levels:
>        if descriptor in level.names:
>        if descriptor in level.names or descriptor == level:

How is the __eq__ defined for RAIDLevel? Or do we want to use 'is'
instead of '==' here?

[blivet:master 07/26] Allow the RAID object itself to be a valid RAID descriptor for lookup.

[Anne Mulhern]

----- Original Message -----
> From: "Vratislav Podzimek" 
> To: "anaconda patch review" 
> Sent: Friday, May 30, 2014 3:16:12 AM
> Subject: Re: [blivet:master 07/26] Allow the RAID object itself to be a	valid RAID descriptor for lookup.
>
> On Thu, 2014-05-29 at 14:47 -0400, mulhern wrote:
> > Update tests accordingly.
> > Signed-off-by: mulhern
> >
> > blivet/devicelibs/raid.py          | 5 ++++-
> >  blivet/devices.py                  | 2 +-
> >  tests/devicelibs_test/raid_test.py | 1 +
> >  3 files changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
> > index 84f2aa1..33af02f 100644
> > --- a/blivet/devicelibs/raid.py
> > +++ b/blivet/devicelibs/raid.py
> > @@ -303,11 +303,14 @@ class RAIDLevels(object):
> >         :param object descriptor: a RAID level descriptor
> >
> >       Note that descriptor may be any object that identifies a
> >       RAID level, including the RAID object itself.
> >       Raises a RaidError if no RAID object can be found for this
> >       descriptor.
> >    """
> >    for level in self._raid_levels:
> >        if descriptor in level.names:
> >        if descriptor in level.names or descriptor == level:
>
> How is the __eq__ defined for RAIDLevel? Or do we want to use 'is'
> instead of '==' here?
>
> --
> Vratislav Podzimek
>
> Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic
>
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

We want 'is', since we've taken
all that trouble to make each RAID level object unique.
Thanks for catching that, fixed in current copy.

- mulhern

[blivet:master 08/26] Move Container class to raid package and tidy it up

[Anne Mulhern]

Remove dead attribute nick and get_raw_array_size which is really
only appropriate for RAIDn classes and was renamed to get_net_array_size
in a previous commit.

Make Container.get_recommended_stride() raise an exception as the
notion of a recommended stride is meaningless for containers.
Do not catch the exception because if there's a situation where a Container
is being formatted, the only reason why get_recommended_stride() would be
called, something has gone wrong.

Add get_max_spares() and get_base_member_size() which both raise an exception.

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py        | 23 +----------------------
 blivet/devicelibs/raid.py          | 20 ++++++++++++++++++++
 tests/devicelibs_test/raid_test.py |  9 ++++-----
 3 files changed, 25 insertions(+), 27 deletions(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index 2106548..41ef859 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -44,28 +44,7 @@ class MDRaidLevels(raid.RAIDLevels):
         except AttributeError:
             return False

-_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10"])
-
-class Container(object):
-    name = "container"
-    names = [name]
-    nick = None
-    min_members = 1
-    def get_recommended_stride(self, member_devices):
-        # pylint: disable=unused-argument
-        return None
-    def get_raw_array_size(self, member_count, smallest_member_size):
-        # pylint: disable=unused-argument
-        raise MDRaidError("get_raw_array_size is not defined for level container.")
-    def get_size(self, member_sizes, num_members=None, chunk_size=None, superblock_size_func=None):
-        # pylint: disable=unused-argument
-        return sum(member_sizes)
-
-    def __str__(self):
-        return self.name
-
-Container = Container()
-_RAID_levels.addRaidLevel(Container)
+_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container"])

def getRaidLevel(descriptor):
     """Return an object for this raid level descriptor.
diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 33af02f..bfe28cb 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -470,3 +470,23 @@ class RAID10(RAIDn):

RAID10 = RAID10()
 ALL_LEVELS.addRaidLevel(RAID10)
+
+class Container(RAIDLevel):
+    name = "container"
+    names = [name]
+    min_members = 1
+    def get_max_spares(self, member_count):
+        # pylint: disable=unused-argument
+        raise RaidError("get_max_spares is not defined for level container")
+    def get_base_member_size(self, size, member_count):
+        # pylint: disable=unused-argument
+        raise RaidError("get_base_member_size is not defined for level container")
+    def get_recommended_stride(self, member_count):
+        # pylint: disable=unused-argument
+        raise RaidError("get_recommended_stride is not defined for level container")
+    def get_size(self, member_sizes, num_members=None, chunk_size=None, superblock_size_func=None):
+        # pylint: disable=unused-argument
+        return sum(member_sizes)
+
+Container = Container()
+ALL_LEVELS.addRaidLevel(Container)
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index f32c59e..7d4f22b 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -1,7 +1,6 @@
 #!/usr/bin/python
 import unittest

-import blivet.devicelibs.mdraid as mdraid
 import blivet.devicelibs.raid as raid
 import blivet.errors as errors
 from blivet.size import Size
@@ -101,7 +100,7 @@ class RaidTestCase(unittest.TestCase):
         ##
         ## size
         ##
-        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -111,7 +110,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size(0)),
                r.get_net_array_size(4, Size("32MiB")))

-        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -121,7 +120,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size(0)),
                r.get_net_array_size(5, Size("32MiB")))

-        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),
@@ -131,7 +130,7 @@ class RaidTestCase(unittest.TestCase):
                lambda x: Size("32MiB")),
                0)

-        for r in [l for l in raid.ALL_LEVELS if l is not mdraid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
             self.assertEqual(r.get_size([ Size("32MiB"),
                                       Size("128MiB"),
                                       Size("128MiB"),

[blivet:master 09/26] Add Linear and Single to the RAID classes.

[Anne Mulhern]

Let them share an abstract super-class with an implementation, they just
handle the names.

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py          | 49 ++++++++++++++++++++++++++++++++
 tests/devicelibs_test/raid_test.py | 58 +++++++++++++++++---------------------
 2 files changed, 75 insertions(+), 32 deletions(-)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index bfe28cb..0964b92 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -490,3 +490,52 @@ class Container(RAIDLevel):

Container = Container()
 ALL_LEVELS.addRaidLevel(Container)
+
+class ErsatzRAID(RAIDLevel):
+    """ A superclass for a raid level which is not really a raid level at
+        all, just a bunch of block devices of possibly differing sizes
+        thrown together. This concept has different names depending on where
+        it crops up. btrfs's name is single, lvm's is linear. Consequently,
+        this abstract class implements all the functionality, but there are
+        distinct subclasses which have different names.
+    """
+    min_members = 1
+
+    def get_max_spares(self, member_count):
+        return member_count - self.min_members
+
+    def get_base_member_size(self, size, member_count):
+        # pylint: disable=unused-argument
+        raise RaidError("get_base_member_size is not defined for level linear")
+
+    def get_recommended_stride(self, member_count):
+        # pylint: disable=unused-argument
+        return None
+
+    def get_size(self, member_sizes, num_members=None, chunk_size=None, superblock_size_func=None):
+        # pylint: disable=unused-argument
+        if not member_sizes:
+            return 0
+
+        if superblock_size_func is None:
+            raise RaidError("superblock_size_func value of None is not acceptable")
+
+        total_space = sum(member_sizes)
+        superblock_size = superblock_size_func(total_space)
+        return total_space - len(member_sizes) * superblock_size
+
+class Linear(ErsatzRAID):
+    """ subclass with canonical lvm name """
+    name = 'linear'
+    names = [name]
+
+Linear = Linear()
+ALL_LEVELS.addRaidLevel(Linear)
+
+class Single(ErsatzRAID):
+    """ subclass with canonical btrfs name. """
+    name = 'single'
+    names = [name]
+
+Single = Single()
+ALL_LEVELS.addRaidLevel(Single)
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index 7d4f22b..4d65b39 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -14,6 +14,10 @@ class RaidTestCase(unittest.TestCase):

def testRaid(self):

+        self.assertRaisesRegexp(TypeError,
+           "Can't instantiate abstract class",
+           raid.ErsatzRAID)
+
         ##
         ## get_min_members
         ##
@@ -23,6 +27,7 @@ class RaidTestCase(unittest.TestCase):
         self.assertEqual(raid.RAID5.min_members, 3)
         self.assertEqual(raid.RAID6.min_members, 4)
         self.assertEqual(raid.RAID10.min_members, 4)
+        self.assertEqual(raid.Linear.min_members, 1)

##
         ## get_max_spares
@@ -33,6 +38,8 @@ class RaidTestCase(unittest.TestCase):
         self.assertEqual(raid.RAID5.get_max_spares(5), 2)
         self.assertEqual(raid.RAID6.get_max_spares(5), 1)
         self.assertEqual(raid.RAID10.get_max_spares(5), 1)
+        self.assertEqual(raid.Linear.get_max_spares(5), 4)
+        self.assertEqual(raid.Single.get_max_spares(5), 4)

##
         ## raidLevel
@@ -72,6 +79,10 @@ class RaidTestCase(unittest.TestCase):
                           4, 3)
         self.assertRaises(errors.RaidError, raid.RAID10.get_base_member_size,
                           -4, 4)
+        self.assertRaisesRegexp(errors.RaidError,
+           "not defined",
+           raid.Linear.get_base_member_size,
+           4, 3)

##
         ## get_net_array_size
@@ -94,51 +105,34 @@ class RaidTestCase(unittest.TestCase):
         self.assertEqual(raid.RAID0.get_recommended_stride(4), 64)
         self.assertEqual(raid.RAID4.get_recommended_stride(4), 48)
         self.assertEqual(raid.RAID5.get_recommended_stride(4), 48)
+        self.assertIsNone(raid.Linear.get_recommended_stride(4))

self.assertRaises(errors.RaidError, raid.RAID10.get_recommended_stride, 1)

##
         ## size
         ##
+        sizes = [Size("32MiB"), Size("128MiB"), Size("128MiB"), Size("64MiB")]
         for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
-            self.assertEqual(r.get_size([ Size("32MiB"),
-                                      Size("128MiB"),
-                                      Size("128MiB"),
-                                      Size("64MiB") ],
-               4,
-               Size("1MiB"),
-               lambda x: Size(0)),
-               r.get_net_array_size(4, Size("32MiB")))
+            self.assertEqual(r.get_size(sizes, 4, Size("1MiB"), lambda x: Size(0)),
+               r.get_net_array_size(4, Size("32MiB")) if isinstance(r, raid.RAIDn) else sum(sizes))

for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
-            self.assertEqual(r.get_size([ Size("32MiB"),
-                                      Size("128MiB"),
-                                      Size("128MiB"),
-                                      Size("64MiB") ],
-               5,
-               Size("1MiB"),
-               lambda x: Size(0)),
-               r.get_net_array_size(5, Size("32MiB")))
+            self.assertEqual(r.get_size(sizes, 5, Size("1MiB"), lambda x: Size(0)),
+               r.get_net_array_size(5, Size("32MiB")) if isinstance(r, raid.RAIDn) else sum(sizes))

for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
-            self.assertEqual(r.get_size([ Size("32MiB"),
-                                      Size("128MiB"),
-                                      Size("128MiB"),
-                                      Size("64MiB") ],
-               4,
-               Size("1MiB"),
-               lambda x: Size("32MiB")),
-               0)
+            self.assertEqual(r.get_size(sizes, 4, Size("1MiB"), lambda x: Size("32MiB")),
+               0 if isinstance(r, raid.RAIDn) else (sum(sizes) - 4 * Size("32MiB")))

for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
-            self.assertEqual(r.get_size([ Size("32MiB"),
-                                      Size("128MiB"),
-                                      Size("128MiB"),
-                                      Size("64MiB") ],
-               4,
-               Size("2MiB"),
-               lambda x: Size("31MiB")),
-               0 if r not in (raid.RAID1, raid.RAID10) else r.get_net_array_size(4, Size("1MiB")))
+            if isinstance(r, raid.RAIDn):
+                if r not in (raid.RAID1, raid.RAID10):
+                    self.assertEqual(r.get_size(sizes, 4, Size("2MiB"), lambda x: Size("31MiB")), 0)
+                else:
+                    self.assertEqual(r.get_size(sizes, 4, Size("2MiB"), lambda x: Size("31MiB")), r.get_net_array_size(4, Size("1MiB")))
+            else:
+                self.assertEqual(r.get_size(sizes, 4, Size("2MiB"), lambda x: Size("31MiB")), sum(sizes) - 4 * Size("31MiB"))

##
         ## names

[blivet:master 10/26] Add a has_redundancy method that returns True if there is actual redundancy

[Anne Mulhern]

Some RAID levels, e.g., 0, don't actually have any redundancy.

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 0964b92..2a9561f 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -46,6 +46,8 @@ class RAIDLevel(object):
     names = abc.abstractproperty(doc="List of recognized names for this level.")
     min_members = abc.abstractproperty(doc=
        "The minimum number of members required to make a fully functioning array.")
+    has_redundancy = abc.abstractproperty(doc=
+       "Whether this RAID level incorporates inherent redundancy.")

def __str__(self):
         return self.name
@@ -337,6 +339,7 @@ class RAID0(RAIDn):
     level = property(lambda s: "0")
     min_members = property(lambda s: 2)
     nick = property(lambda s: "stripe")
+    has_redundancy = property(lambda s: False)

def _get_max_spares(self, member_count):
         return 0
@@ -360,6 +363,7 @@ class RAID1(RAIDn):
     level = property(lambda s: "1")
     min_members = property(lambda s: 2)
     nick = property(lambda s: "mirror")
+    has_redundancy = property(lambda s: True)

def _get_max_spares(self, member_count):
         return member_count - self.min_members
@@ -383,6 +387,7 @@ class RAID4(RAIDn):
     level = property(lambda s: "4")
     min_members = property(lambda s: 3)
     nick = property(lambda s: None)
+    has_redundancy = property(lambda s: True)

def _get_max_spares(self, member_count):
         return member_count - self.min_members
@@ -406,6 +411,7 @@ class RAID5(RAIDn):
     level = property(lambda s: "5")
     min_members = property(lambda s: 3)
     nick = property(lambda s: None)
+    has_redundancy = property(lambda s: True)

def _get_max_spares(self, member_count):
         return member_count - self.min_members
@@ -429,6 +435,7 @@ class RAID6(RAIDn):
     level = property(lambda s: "6")
     min_members = property(lambda s: 4)
     nick = property(lambda s: None)
+    has_redundancy = property(lambda s: True)

def _get_max_spares(self, member_count):
         return member_count - self.min_members
@@ -452,6 +459,7 @@ class RAID10(RAIDn):
     level = property(lambda s: "10")
     min_members = property(lambda s: 4)
     nick = property(lambda s: None)
+    has_redundancy = property(lambda s: True)

def _get_max_spares(self, member_count):
         return member_count - self.min_members
@@ -475,6 +483,8 @@ class Container(RAIDLevel):
     name = "container"
     names = [name]
     min_members = 1
+    has_redundancy = property(lambda s: False)
+
     def get_max_spares(self, member_count):
         # pylint: disable=unused-argument
         raise RaidError("get_max_spares is not defined for level container")
@@ -500,6 +510,7 @@ class ErsatzRAID(RAIDLevel):
         distinct subclasses which have different names.
     """
     min_members = 1
+    has_redundancy = property(lambda s: False)

def get_max_spares(self, member_count):
         return member_count - self.min_members

[blivet:master 11/26] Add an is_uniform property to the RAID levels.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 2a9561f..36d2acc 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -49,6 +49,9 @@ class RAIDLevel(object):
     has_redundancy = abc.abstractproperty(doc=
        "Whether this RAID level incorporates inherent redundancy.")

+    is_uniform = abc.abstractproperty(doc=
+       "Whether data is uniformly distributed across all devices.")
+
     def __str__(self):
         return self.name

@@ -92,6 +95,8 @@ class RAIDn(RAIDLevel):
     nick = abc.abstractproperty(doc="A nickname for this level")

# PROPERTIES
+    is_uniform = property(lambda s: True)
+
     number = property(lambda s : int(s.level),
        doc="A numeric code for this level")

@@ -484,6 +489,7 @@ class Container(RAIDLevel):
     names = [name]
     min_members = 1
     has_redundancy = property(lambda s: False)
+    is_uniform = property(lambda s: False)

def get_max_spares(self, member_count):
         # pylint: disable=unused-argument
@@ -511,6 +517,7 @@ class ErsatzRAID(RAIDLevel):
     """
     min_members = 1
     has_redundancy = property(lambda s: False)
+    is_uniform = property(lambda s: False)

def get_max_spares(self, member_count):
         return member_count - self.min_members

[blivet:master 12/26] Add a Dup class to the various descendants of RAIDLevel.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/raid.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 36d2acc..6b05122 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -557,3 +557,17 @@ class Single(ErsatzRAID):

Single = Single()
 ALL_LEVELS.addRaidLevel(Single)
+
+class Dup(RAIDLevel):
+    """ A RAID level which expresses one way btrfs metadata may be distributed.
+
+        For this RAID level, duplication occurs within a single block device.
+    """
+    name = 'dup'
+    names = [name]
+    min_members = 1
+    has_redundancy = property(lambda s: True)
+    is_uniform = property(lambda s: False)
+
+Dup = Dup()
+ALL_LEVELS.addRaidLevel(Dup)

[blivet:master 13/26] Make createBitmap() a property and update tests appropriately.

[Anne Mulhern]

createBitmap() seems to be a simple function of the device properties.
All of these properties are relevant when its value is used in calling
mdcreate(), so they should all be taken into account at that time.
---
 blivet/devices.py     | 17 +++++++++--------
 tests/devices_test.py |  7 ++++++-
 2 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/blivet/devices.py b/blivet/devices.py
index 9480bae..868d48f 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3349,13 +3349,18 @@ class MDRaidArrayDevice(ContainerDevice):
             :param value: new raid level
             :param type:  a valid raid level descriptor
             :returns:     None
-
-            Sets createBitmap True unless level is 0
         """
         self._level = mdraid.getRaidLevel(value) # pylint: disable=attribute-defined-outside-init

-        # bitmaps are not meaningful on raid0 according to mdadm-3.0.3
-        self.createBitmap = self._level.name != "raid0" # pylint: disable=attribute-defined-outside-init
+    @property
+    def createBitmap(self):
+        """ Whether or not a bitmap should be created on the array.
+
+            If the the array is sufficiently small, a bitmap yields no benefit.
+
+            If the array has no redundancy, a bitmap is just pointless.
+        """
+        return self.level.has_redundancy and self.size >= 1000 and  self.format.type != "swap"

def getSuperBlockSize(self, raw_array_size):
         """Estimate the superblock size for a member of an array,
@@ -3689,10 +3694,6 @@ class MDRaidArrayDevice(ContainerDevice):
            self.format.type == "prepboot":
             self.metadataVersion = "1.0"

-        # Bitmaps are not useful for swap and small partitions
-        if self.size < 1000 or self.format.type == "swap":
-            self.createBitmap = False # pylint: disable=attribute-defined-outside-init
-
     def _postCreate(self):
         # this is critical since our status method requires a valid sysfs path
         md_node = mdraid.md_node_from_name(self.name)
diff --git a/tests/devices_test.py b/tests/devices_test.py
index 6598eba..3d6ec5e 100644
--- a/tests/devices_test.py
+++ b/tests/devices_test.py
@@ -62,7 +62,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):

def setUp(self):
         self._state_functions = {
-           "createBitmap" : self.assertTrue,
+           "createBitmap" : self.assertFalse,
            "currentSize" : lambda x, m: self.assertEqual(x, Size(0), m),
            "description" : self.assertIsNotNone,
            "devices" : lambda x, m: self.assertEqual(len(x), 0, m) and
@@ -320,6 +320,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))

self.stateCheck(self.dev14,
+                        createBitmap=self.assertTrue,
                         devices=lambda x, m: self.assertEqual(len(x), 3, m),
                         level=lambda x, m: self.assertEqual(x.number, 4, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 3, m),
@@ -328,6 +329,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))

self.stateCheck(self.dev15,
+                        createBitmap=self.assertTrue,
                         devices=lambda x, m: self.assertEqual(len(x), 3, m),
                         level=lambda x, m: self.assertEqual(x.number, 5, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 3, m),
@@ -336,6 +338,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))

self.stateCheck(self.dev16,
+                        createBitmap=self.assertTrue,
                         devices=lambda x, m: self.assertEqual(len(x), 4, m),
                         level=lambda x, m: self.assertEqual(x.number, 6, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),
@@ -344,6 +347,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         totalDevices=lambda x, m: self.assertEqual(x, 4, m))

self.stateCheck(self.dev17,
+                        createBitmap=self.assertTrue,
                         devices=lambda x, m: self.assertEqual(len(x), 4, m),
                         level=lambda x, m: self.assertEqual(x.number, 10, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),
@@ -352,6 +356,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         totalDevices=lambda x, m: self.assertEqual(x, 4, m))

self.stateCheck(self.dev18,
+                        createBitmap=self.assertTrue,
                         devices=lambda x, m: self.assertEqual(len(x), 4, m),
                         level=lambda x, m: self.assertEqual(x.number, 10, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),

[blivet:master 14/26] Use has_redundancy raid property when checking whether a device is removable

[Anne Mulhern]

There is a semantic change here, an error will be raised in the case of
every non-redundant raid, which in this case includes "container" as well
as "raid0".

Signed-off-by: mulhern 
---
 blivet/devices.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/blivet/devices.py b/blivet/devices.py
index 868d48f..c68be31 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3539,7 +3539,11 @@ class MDRaidArrayDevice(ContainerDevice):
             self.memberDevices += 1

def _removeParent(self, member):
-        if self.level.name == "raid0" and self.exists and member.format.exists:
+        """ If this is a raid array that is not actually redundant and it
+            appears to have formatting and therefore probably data on it,
+            removing one of its devices is a bad idea.
+        """
+        if not self.level.has_redundancy and self.exists and member.format.exists:
             raise errors.DeviceError("cannot remove members from existing raid0")

super(MDRaidArrayDevice, self)._removeParent(member)

[blivet:master 15/26] Update the mdraid.mdadd comments

[Anne Mulhern]

They were restricted to be about the raid levels that were supported when
the comment was first written.

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index 41ef859..b17a25a 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -136,8 +136,10 @@ def mdadd(array, device, incremental=False, raid_devices=None):
         :keyword bool incremental: add the device incrementally (see note below)
         :keyword int raid_devices: the number of active member devices

-        The raid_devices parameter is used when adding devices to a raid0 array
-        since raid0 does not allow spares.
+        The raid_devices parameter is used when adding devices to a raid
+        array that has no actual redundancy. In this case it is necessary
+        to explicitly grow the array all at once rather than manage it in
+        the sense of adding spares.

Whether the new device will be added as a spare or an active member is
         decided by mdadm.

[blivet:master 16/26] Use has_redundancy property to decide how to add a member to an array.

[Anne Mulhern]

Better than using explicit names.

Signed-off-by: mulhern 
---
 blivet/devices.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/blivet/devices.py b/blivet/devices.py
index c68be31..2958faa 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3733,10 +3733,10 @@ class MDRaidArrayDevice(ContainerDevice):

def _add(self, member):
         self.setup()
-        if self.level.name == "raid0":
-            raid_devices = self.memberDevices
-        else:
+        if self.level.has_redundancy:
             raid_devices = None
+        else:
+            raid_devices = self.memberDevices

mdraid.mdadd(self.path, member.path, raid_devices=raid_devices)

[blivet:master 17/26] Check for required methods in MDRaidLevels.isRaidLevel.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index b17a25a..75f7ffe 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -37,12 +37,11 @@ MD_CHUNK_SIZE = Size("512 KiB")
 class MDRaidLevels(raid.RAIDLevels):
     @classmethod
     def isRaidLevel(cls, level):
-        """Every mdraid level must define min_members."""
-        try:
-            min_members = level.min_members
-            return super(MDRaidLevels, cls).isRaidLevel(level) and min_members > 0
-        except AttributeError:
-            return False
+        return super(MDRaidLevels, cls).isRaidLevel(level) and \
+           hasattr(level, 'get_max_spares') and \
+           hasattr(level, 'get_base_member_size') and \
+           hasattr(level, 'get_recommended_stride') and \
+           hasattr(level, 'get_size')

_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container"])

[blivet:master 18/26] Remove getRaidLevel() from mdraid file and make RAID_levels public

[Anne Mulhern]

devicelibs.btrfs and devicelibs.lvm will be the same. Keeping getRaidLevel()
will just require making copies of it for each of these, in order to remain
consistent.

Update some tests as appropriate, where RaidError is thrown instead of
MDRaidError.

Signed-off-by: mulhern 
---
 blivet/devicefactory.py              |  7 +++----
 blivet/devicelibs/mdraid.py          | 13 ++-----------
 blivet/devices.py                    |  2 +-
 tests/devicefactory_test.py          |  8 ++++----
 tests/devicelibs_test/mdraid_test.py | 16 ++++++++--------
 tests/devices_test.py                | 14 +++++++-------
 tests/devicetree_test.py             |  2 +-
 7 files changed, 26 insertions(+), 36 deletions(-)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index ee0c651..96e189c 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -758,7 +758,7 @@ class DeviceFactory(object):
             if level in [None, "single"]:
                 continue

-            min_disks = mdraid.getRaidLevel(level).min_members
+            min_disks = mdraid.RAID_levels.raidLevel(level).min_members
             disks = set(d for m in self._get_member_devices() for d in m.disks)
             if len(disks) < min_disks:
                 raise DeviceFactoryError("Not enough disks for %s" % level)
@@ -1535,9 +1535,8 @@ class MDFactory(DeviceFactory):

def _get_device_space(self):
         member_count = len(self._get_member_devices())
-        size_per_member = mdraid.getRaidLevel(self.raid_level).get_base_member_size(
-           self.size,
-           member_count)
+        raid_level = mdraid.RAID_levels.raidLevel(self.raid_level)
+        size_per_member = raid_level.get_base_member_size(self.size, member_count)
         size_per_member += mdraid.get_raid_superblock_size(self.size)
         return size_per_member * member_count

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index 75f7ffe..0bfbbe8 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -23,7 +23,7 @@
 import os

from .. import util
-from ..errors import MDRaidError, RaidError
+from ..errors import MDRaidError
 from ..size import Size
 from . import raid

@@ -43,16 +43,7 @@ class MDRaidLevels(raid.RAIDLevels):
            hasattr(level, 'get_recommended_stride') and \
            hasattr(level, 'get_size')

-_RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container"])
-
-def getRaidLevel(descriptor):
-    """Return an object for this raid level descriptor.
-       Raises an MDRaidError if the descriptor is not valid.
-    """
-    try:
-        return _RAID_levels.raidLevel(descriptor)
-    except RaidError as e:
-        raise MDRaidError(e.message)
+RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container"])

def get_raid_superblock_size(size, version=None):
     """ mdadm has different amounts of space reserved for its use depending
diff --git a/blivet/devices.py b/blivet/devices.py
index 2958faa..6993ded 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -3350,7 +3350,7 @@ class MDRaidArrayDevice(ContainerDevice):
             :param type:  a valid raid level descriptor
             :returns:     None
         """
-        self._level = mdraid.getRaidLevel(value) # pylint: disable=attribute-defined-outside-init
+        self._level = mdraid.RAID_levels.raidLevel(value) # pylint: disable=attribute-defined-outside-init

@property
     def createBitmap(self):
diff --git a/tests/devicefactory_test.py b/tests/devicefactory_test.py
index 28fa079..d4746dd 100644
--- a/tests/devicefactory_test.py
+++ b/tests/devicefactory_test.py
@@ -5,7 +5,7 @@ import unittest
 import blivet

from blivet import devicefactory
-from blivet.errors import MDRaidError, RaidError
+from blivet.errors import RaidError
 from blivet.size import Size

class MDFactoryTestCase(unittest.TestCase):
@@ -26,11 +26,11 @@ class MDFactoryTestCase(unittest.TestCase):
            raid_level=0)

def testMDFactory(self):
-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
            "invalid RAID level",
            self.factory1._get_device_space)

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
            "invalid RAID level",
            self.factory1._configure)

@@ -38,7 +38,7 @@ class MDFactoryTestCase(unittest.TestCase):

self.assertIsNone(self.factory1.get_container())

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
            "invalid RAID level",
            self.factory1._get_new_device,
            parents=[])
diff --git a/tests/devicelibs_test/mdraid_test.py b/tests/devicelibs_test/mdraid_test.py
index 73d2ff1..5abffea 100755
--- a/tests/devicelibs_test/mdraid_test.py
+++ b/tests/devicelibs_test/mdraid_test.py
@@ -15,15 +15,15 @@ class MDRaidTestCase(unittest.TestCase):
     def testMDRaid(self):

##
-        ## getRaidLevel
+        ## level lookup
         ##
-        self.assertEqual(mdraid.getRaidLevel("container").name, "container")
-        self.assertEqual(mdraid.getRaidLevel("stripe").name, "raid0")
-        self.assertEqual(mdraid.getRaidLevel("mirror").name, "raid1")
-        self.assertEqual(mdraid.getRaidLevel("4").name, "raid4")
-        self.assertEqual(mdraid.getRaidLevel(5).name, "raid5")
-        self.assertEqual(mdraid.getRaidLevel("RAID6").name, "raid6")
-        self.assertEqual(mdraid.getRaidLevel("raid10").name, "raid10")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("container").name, "container")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("stripe").name, "raid0")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("mirror").name, "raid1")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("4").name, "raid4")
+        self.assertEqual(mdraid.RAID_levels.raidLevel(5).name, "raid5")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("RAID6").name, "raid6")
+        self.assertEqual(mdraid.RAID_levels.raidLevel("raid10").name, "raid10")

##
         ## get_raid_superblock_size
diff --git a/tests/devices_test.py b/tests/devices_test.py
index 3d6ec5e..3f8c4fc 100644
--- a/tests/devices_test.py
+++ b/tests/devices_test.py
@@ -8,7 +8,7 @@ from mock import Mock
 import blivet

from blivet.errors import DeviceError
-from blivet.errors import MDRaidError
+from blivet.errors import RaidError

from blivet.devices import BTRFSSubVolumeDevice
 from blivet.devices import BTRFSVolumeDevice
@@ -365,18 +365,18 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         spares=lambda x, m: self.assertEqual(x, 1, m),
                         totalDevices=lambda x, m: self.assertEqual(x, 5, m))

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
                                 "invalid RAID level",
                                 MDRaidArrayDevice,
                                 "dev")

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
                                 "invalid RAID level",
                                 MDRaidArrayDevice,
                                 "dev",
                                 level="raid2")

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
                                 "invalid RAID level",
                                 MDRaidArrayDevice,
                                 "dev",
@@ -391,7 +391,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                                 parents=[StorageDevice("parent",
                                                 fmt=getFormat("mdmember"))])

-        self.assertRaisesRegexp(MDRaidError,
+        self.assertRaisesRegexp(RaidError,
                                 "invalid RAID level descriptor junk",
                                 MDRaidArrayDevice,
                                 "dev",
@@ -407,10 +407,10 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):

def testMDRaidArrayDeviceMethods(self):
         """Test for method calls on initialized MDRaidDevices."""
-        with self.assertRaisesRegexp(MDRaidError, "invalid RAID level" ):
+        with self.assertRaisesRegexp(RaidError, "invalid RAID level" ):
             self.dev7.level = "junk"

-        with self.assertRaisesRegexp(MDRaidError, "invalid RAID level" ):
+        with self.assertRaisesRegexp(RaidError, "invalid RAID level" ):
             self.dev7.level = None

class BTRFSDeviceTestCase(DeviceStateTestCase):
diff --git a/tests/devicetree_test.py b/tests/devicetree_test.py
index b0105f9..56ce17f 100644
--- a/tests/devicetree_test.py
+++ b/tests/devicetree_test.py
@@ -171,7 +171,7 @@ class MDRaid0TestCase(BlivetResetTestCase):
     _validate_attrs = BlivetResetTestCase._validate_attrs + ["level", "spares"]

def set_up_disks(self):
-        level = devicelibs.mdraid.getRaidLevel(self.level)
+        level = devicelibs.mdraid.RAID_levels.raidLevel(self.level)
         disk_count = level.min_members
         self.disks = dict()
         for i in range(disk_count):

[blivet:master 19/26] Add "linear" to mdraid's list of supported raid levels.

[Anne Mulhern]

Signed-off-by: mulhern 
---
 blivet/devicelibs/mdraid.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index 0bfbbe8..f0d8162 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -43,7 +43,7 @@ class MDRaidLevels(raid.RAIDLevels):
            hasattr(level, 'get_recommended_stride') and \
            hasattr(level, 'get_size')

-RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container"])
+RAID_levels = MDRaidLevels(["raid0", "raid1", "raid4", "raid5", "raid6", "raid10", "container", "linear"])

def get_raid_superblock_size(size, version=None):
     """ mdadm has different amounts of space reserved for its use depending

[blivet:master 20/26] Add lists of supported RAID levels for btrfs and lvm

[Anne Mulhern]

Update raid level code in BTRFSDevice appropriately so that it no longer
get raid.RAIDLevels(), instead gets the list of BTRFS supported RAID levels.

Note that lvm does not itself actually support linear. Linear is an option
when making an lvm filesystem on top of RAID, so linear really belongs
in mdraid's supported RAID levels.

Signed-off-by: mulhern 
---
 blivet/devicelibs/btrfs.py | 3 +++
 blivet/devicelibs/lvm.py   | 3 +++
 blivet/devices.py          | 3 +--
 3 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/blivet/devicelibs/btrfs.py b/blivet/devicelibs/btrfs.py
index 61697ba..4d2eb6c 100644
--- a/blivet/devicelibs/btrfs.py
+++ b/blivet/devicelibs/btrfs.py
@@ -23,6 +23,7 @@
 import os
 import re

+from . import raid
 from .. import util
 from ..errors import BTRFSError

@@ -32,6 +33,8 @@ log = logging.getLogger("blivet")
 # this is the volume id btrfs always assigns to the top-level volume/tree
 MAIN_VOLUME_ID = 5

+RAID_levels = raid.RAIDLevels(["raid0", "raid1", "raid10", "single"])
+
 def btrfs(args, capture=False):
     if capture:
         exec_func = util.capture_output
diff --git a/blivet/devicelibs/lvm.py b/blivet/devicelibs/lvm.py
index 67fc1e5..3c55adb 100644
--- a/blivet/devicelibs/lvm.py
+++ b/blivet/devicelibs/lvm.py
@@ -26,6 +26,7 @@ from decimal import Decimal
 import logging
 log = logging.getLogger("blivet")

+from . import raid
 from ..size import Size
 from .. import util
 from .. import arch
@@ -44,6 +45,8 @@ LVM_THINP_MAX_METADATA_SIZE = Size("16 GiB")
 LVM_THINP_MIN_CHUNK_SIZE = Size("64 KiB")
 LVM_THINP_MAX_CHUNK_SIZE = Size("1 GiB")

+RAID_levels = raid.RAIDLevels(["raid0", "raid1"])
+
 def has_lvm():
     if util.find_program_in_path("lvm"):
         for line in open("/proc/devices").readlines():
diff --git a/blivet/devices.py b/blivet/devices.py
index 6993ded..5513b2d 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -29,7 +29,6 @@ from decimal import Decimal
 import re

# device backend modules
-from .devicelibs import raid
 from .devicelibs import mdraid
 from .devicelibs import lvm
 from .devicelibs import dm
@@ -4755,7 +4754,7 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):
     def _removeParent(self, member):
         # btrfs won't let you degrade it
         limits = []
-        levels = raid.RAIDLevels()
+        levels = btrfs.RAID_levels
         if self.dataLevel and self.dataLevel != "single":
             limits.append(levels.raidLevel(self.dataLevel).min_members)

[blivet:master 20/26] Add lists of supported RAID levels for btrfs and lvm

[David Lehman]

On 05/29/2014 01:47 PM, mulhern wrote:
> Update raid level code in BTRFSDevice appropriately so that it no longer
> get raid.RAIDLevels(), instead gets the list of BTRFS supported RAID levels.
>
> Note that lvm does not itself actually support linear. Linear is an option
> when making an lvm filesystem on top of RAID, so linear really belongs
> in mdraid's supported RAID levels.

LVM's default segment type (aka raid level) is linear, so it does 
support linear to the same extent that it supports any other.

[blivet:master 20/26] Add lists of supported RAID levels for btrfs and lvm

[Anne Mulhern]

----- Original Message -----
> From: "David Lehman" 
> To: [e-mail]
> Sent: Thursday, May 29, 2014 6:18:22 PM
> Subject: Re: [blivet:master 20/26] Add lists of supported RAID levels for	btrfs and lvm
>
> On 05/29/2014 01:47 PM, mulhern wrote:
> > Update raid level code in BTRFSDevice appropriately so that it no longer
> > get raid.RAIDLevels(), instead gets the list of BTRFS supported RAID
> > levels.
> >
> > Note that lvm does not itself actually support linear. Linear is an option
> > when making an lvm filesystem on top of RAID, so linear really belongs
> > in mdraid's supported RAID levels.
>
> LVM's default segment type (aka raid level) is linear, so it does
> support linear to the same extent that it supports any other.
> _______________________________________________
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

"linear" is back for lvm, incorrect note removed from commit message.

- mulhern

[blivet:master 21/26] Make BTRFS devices use RAID objects instead of strings for levels

[Anne Mulhern]

Update some comments on btrfs.create_volume() to match.

Also, refine BTRFSVolumeDevice._removeParent() method.
There is a change in behavior, if there is any RAIDlevel then the minimum
number of parents can never be less than 1. Previously, in the case of
single raid, it could be 0.

Make a separate list for btrfs's notion of a metadata RAID level.

Signed-off-by: mulhern 
---
 blivet/devicelibs/btrfs.py |   7 +++-
 blivet/devices.py          | 100 ++++++++++++++++++++++++++++++++++++---------
 2 files changed, 86 insertions(+), 21 deletions(-)

diff --git a/blivet/devicelibs/btrfs.py b/blivet/devicelibs/btrfs.py
index 4d2eb6c..72c0b71 100644
--- a/blivet/devicelibs/btrfs.py
+++ b/blivet/devicelibs/btrfs.py
@@ -35,6 +35,8 @@ MAIN_VOLUME_ID = 5

RAID_levels = raid.RAIDLevels(["raid0", "raid1", "raid10", "single"])

+metadata_levels = raid.RAIDLevels(["raid0", "raid1", "raid10", "single", "dup"])
+
 def btrfs(args, capture=False):
     if capture:
         exec_func = util.capture_output
@@ -54,7 +56,10 @@ def create_volume(devices, label=None, data=None, metadata=None):
        Optional arguments label, data, metadata corresponds to flags
        accepted by mkfs.btrfs.

-       For now, data and metadata must be strings mkfs.btrfs understands.
+       :param data: a raid level for data
+       :type data: :class:`~.devicelibs.raid.RAIDLevel`
+       :param metadata: a raid level for metadata
+       :type metadata: :class:`~.devicelibs.raid.RAIDLevel`
     """
     if not devices:
         raise ValueError("no devices specified")
diff --git a/blivet/devices.py b/blivet/devices.py
index 5513b2d..346261f 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -35,6 +35,7 @@ from .devicelibs import dm
 from .devicelibs import loop
 from .devicelibs import btrfs
 from .devicelibs import crypto
+from .devicelibs import raid
 import parted
 import _ped
 import block
@@ -4716,11 +4717,18 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):
     _formatUUIDAttr = property(lambda s: "volUUID")

def __init__(self, *args, **kwargs):
-        self.dataLevel = kwargs.pop("dataLevel", None)
-        self.metaDataLevel = kwargs.pop("metaDataLevel", None)
+
+        # pop these arguments before the constructor call to avoid
+        # unrecognized keyword error in superclass constructor
+        dataLevel = kwargs.pop("dataLevel", None)
+        metaDataLevel = kwargs.pop("metaDataLevel", None)

super(BTRFSVolumeDevice, self).__init__(*args, **kwargs)

+        # assign after constructor to avoid AttributeErrors in setter functions
+        self.dataLevel = dataLevel
+        self.metaDataLevel = metaDataLevel
+
         self.subvolumes = []
         self.size_policy = self.size

@@ -4736,6 +4744,62 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):

self._defaultSubVolumeID = None

+    @property
+    def dataLevel(self):
+        """ Return the RAID level for data.
+
+            :returns: raid level
+            :rtype: an object that represents a raid level
+        """
+        return self._dataLevel
+
+    @dataLevel.setter
+    def dataLevel(self, value):
+        """ Set the RAID level for data.
+
+            :param value: new raid level
+            :param type:  a valid raid level descriptor
+            :returns:     None
+
+            If no raid level is specified then the best choice is single
+            since it can accomodate devices of different sizes.
+        """
+        # pylint: disable=attribute-defined-outside-init
+        if value is None:
+            self._dataLevel = btrfs.RAID_levels.raidLevel("single")
+        else:
+            self._dataLevel = btrfs.RAID_levels.raidLevel(value)
+
+    @property
+    def metaDataLevel(self):
+        """ Return the RAID level for metadata.
+
+            :returns: raid level
+            :rtype: an object that represents a raid level
+        """
+        return self._metaDataLevel
+
+    @metaDataLevel.setter
+    def metaDataLevel(self, value):
+        """ Set the RAID level for metadata.
+
+            :param value: new raid level
+            :param type:  a valid raid level descriptor
+            :returns:     None
+
+            If no raid level is specified and there are multiple devices,
+            best choice is the default that mkfs.btrfs chooses, raid1.
+            If there is only one device, the best default is dup.
+        """
+        # pylint: disable=attribute-defined-outside-init
+        if value is None:
+            if len(self.parents) > 1:
+                self._metaDataLevel = btrfs.metadata_levels.raidLevel("raid1")
+            else:
+                self._metaDataLevel = btrfs.metadata_levels.raidLevel("dup")
+        else:
+            self._metaDataLevel = btrfs.RAID_levels.raidLevel(value)
+
     def _setFormat(self, fmt):
         """ Set the Device's format. """
         super(BTRFSVolumeDevice, self)._setFormat(fmt)
@@ -4746,28 +4810,24 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):

def _getSize(self):
         size = sum([d.size for d in self.parents])
-        if self.dataLevel in ("raid1", "raid10"):
+        if self.dataLevel in (raid.RAID1, raid.RAID10):
             size /= len(self.parents)

return size

def _removeParent(self, member):
-        # btrfs won't let you degrade it
-        limits = []
-        levels = btrfs.RAID_levels
-        if self.dataLevel and self.dataLevel != "single":
-            limits.append(levels.raidLevel(self.dataLevel).min_members)
-
-        if self.metaDataLevel and self.metaDataLevel != "single":
-            limits.append(levels.raidLevel(self.metaDataLevel).min_members)
-
-        if limits:
-            min_members = min(limits)
-
-        if limits and len(self.parents) - 1 < min_members:
-            raise errors.DeviceError("cannot remove member due to raid level "
-                              "constraints")
+        """ Raises a DeviceError if the device has a raid level and the
+            resulting number of parents would be fewer than the minimum
+            number required by the raid level.

+            Note: btrfs does not permit degrading an array.
+        """
+        levels = [l for l in [self.dataLevel, self.metaDataLevel] if l]
+        if levels:
+            min_level = min(levels, key=lambda l: l.min_members)
+            min_members = min_level.min_members
+            if len(self.parents) - 1 < min_members:
+                raise errors.DeviceError("device %s requires at least %d membersfor raid level %s" % (self.name, min_members, min_level))
         super(BTRFSVolumeDevice, self)._removeParent(member)

def _addSubVolume(self, vol):
@@ -4893,8 +4953,8 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):

def populateKSData(self, data):
         super(BTRFSVolumeDevice, self).populateKSData(data)
-        data.dataLevel = self.dataLevel
-        data.metaDataLevel = self.metaDataLevel
+        data.dataLevel = self.dataLevel.name if self.dataLevel else None
+        data.metaDataLevel = self.metaDataLevel.name if self.metaDataLevel else None
         data.devices = ["btrfs.%d" % p.id for p in self.parents]
         data.preexist = self.exists

[blivet:master 22/26] Put get_supported_raid_levels in devicefactory.py

[Anne Mulhern]

Change return type to a set of RAIDLevel objects.
---
 blivet/devicefactory.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index 96e189c..d425b40 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -24,6 +24,7 @@ from .storage_log import log_method_call
 from .errors import DeviceFactoryError, StorageError
 from .devices import LUKSDevice
 from .formats import getFormat
+from .devicelibs import btrfs
 from .devicelibs import mdraid
 from .devicelibs import lvm
 from .partitioning import SameSizeSet
@@ -69,6 +70,25 @@ def get_device_type(device):

return device_type

+def get_supported_raid_levels(device_type):
+    """ Return the supported raid levels for this device type.
+
+        :param device_type: an enumeration indicating the device type
+        :type device_type: int
+
+        :returns: a set of supported raid levels for this device type
+        :rtype: set of :class:`~.devicelibs.raid.RAIDLevel`
+    """
+    pkg = None
+    if device_type == DEVICE_TYPE_BTRFS:
+        pkg = btrfs
+    elif device_type in (DEVICE_TYPE_LVM, DEVICE_TYPE_LVM_THINP):
+        pkg = lvm
+    elif device_type == DEVICE_TYPE_MD:
+        pkg = mdraid
+
+    return set(pkg.RAID_levels) if pkg else set()
+
 def get_raid_level(device):
     # TODO: move this into StorageDevice
     use_dev = device

[blivet:master 23/26] Distribute devicefactory.get_raid_level within devices classes.

[Anne Mulhern]

Change the name to raidLevel, make it a property, and change the
return type to a RAIDLevel object.

Signed-off-by: mulhern 
---
 blivet/devicefactory.py | 19 -------------------
 blivet/devices.py       | 31 +++++++++++++++++++++++++++++++
 tests/devices_test.py   | 37 ++++++++++++++++++++++++++++++-------
 3 files changed, 61 insertions(+), 26 deletions(-)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index d425b40..5dcebd3 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -89,25 +89,6 @@ def get_supported_raid_levels(device_type):

return set(pkg.RAID_levels) if pkg else set()

-def get_raid_level(device):
-    # TODO: move this into StorageDevice
-    use_dev = device
-    if isinstance(device, LUKSDevice):
-        use_dev = device.slave
-
-    # TODO: lvm and perhaps pulling raid level from md pvs
-    raid_level = None
-    if hasattr(use_dev, "level"):
-        raid_level = use_dev.level.name
-    elif hasattr(use_dev, "dataLevel"):
-        raid_level = use_dev.dataLevel or "single"
-    elif hasattr(use_dev, "volume"):
-        raid_level = use_dev.volume.dataLevel or "single"
-    elif hasattr(use_dev, "lvs") and len(use_dev.parents) == 1:
-        raid_level = get_raid_level(use_dev.parents[0])
-
-    return raid_level
-
 def get_device_factory(blivet, device_type, size, **kwargs):
     """ Return a suitable DeviceFactory instance for device_type. """
     disks = kwargs.pop("disks", [])
diff --git a/blivet/devices.py b/blivet/devices.py
index 346261f..80b4d47 100644
--- a/blivet/devices.py
+++ b/blivet/devices.py
@@ -1102,6 +1102,17 @@ class StorageDevice(Device):
             return -1
         return 0

+    @property
+    def raidLevel(self):
+        """ A RAID level for this device, or sometimes for its component
+            devices. Where this is a distinction, returns the RAID level
+            for the data, not the metadata.
+
+            :returns: the raid level for this device
+            :rtype: :class:`~.devicelibs.raid.RAIDLevel`
+        """
+        return None
+
     def populateKSData(self, data):
         # the common pieces are basically the formatting
         self.format.populateKSData(data)
@@ -2195,6 +2206,10 @@ class LUKSDevice(DMCryptDevice):
     def dracutSetupArgs(self):
         return set(["rd.luks.uuid=luks-%s" % self.slave.format.uuid])

+    @property
+    def raidLevel(self):
+        return self.slave.raidLevel
+
     def populateKSData(self, data):
         self.slave.populateKSData(data)
         data.encrypted = True
@@ -3037,6 +3052,10 @@ class LVMLogicalVolumeDevice(DMDevice):
             return -1
         return 0

+    @property
+    def raidLevel(self):
+        return None if not self.parents else self.parents[0].raidLevel
+
     def populateKSData(self, data):
         super(LVMLogicalVolumeDevice, self).populateKSData(data)
         data.vgname = self.vg.name
@@ -3778,6 +3797,10 @@ class MDRaidArrayDevice(ContainerDevice):
     def dracutSetupArgs(self):
         return set(["rd.md.uuid=%s" % self.uuid])

+    @property
+    def raidLevel(self):
+        return self.level
+
     def populateKSData(self, data):
         if self.isDisk:
             return
@@ -4951,6 +4974,10 @@ class BTRFSVolumeDevice(BTRFSDevice, ContainerDevice):
         finally:
             self._undo_temp_mount()

+    @property
+    def raidLevel(self):
+        return self.dataLevel
+
     def populateKSData(self, data):
         super(BTRFSVolumeDevice, self).populateKSData(data)
         data.dataLevel = self.dataLevel.name if self.dataLevel else None
@@ -5029,6 +5056,10 @@ class BTRFSSubVolumeDevice(BTRFSDevice):
         btrfs.delete_subvolume(mountpoint, self.name)
         self.volume._undo_temp_mount()

+    @property
+    def raidLevel(self):
+        return self.volume.raidLevel
+
     def populateKSData(self, data):
         super(BTRFSSubVolumeDevice, self).populateKSData(data)
         data.subvol = True
diff --git a/tests/devices_test.py b/tests/devices_test.py
index 3f8c4fc..8613e22 100644
--- a/tests/devices_test.py
+++ b/tests/devices_test.py
@@ -17,6 +17,7 @@ from blivet.devices import OpticalDevice
 from blivet.devices import StorageDevice
 from blivet.devices import ParentList
 from blivet.devicelibs import btrfs
+from blivet.devicelibs import raid
 from blivet.size import Size

from blivet.formats import getFormat
@@ -82,6 +83,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                                     self.assertIsInstance(x, ParentList, m),
            "path" : lambda x, m: self.assertRegexpMatches(x, "^/dev", m),
            "partitionable" : self.assertFalse,
+           "raidLevel" : self.assertIsNone,
            "resizable" : self.assertFalse,
            "size" : lambda x, m: self.assertEqual(x, Size(0), m),
            "spares" : lambda x, m: self.assertEqual(x, 0, m),
@@ -238,20 +240,27 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
         ##
         self.stateCheck(self.dev1,
                         level=lambda x, m: self.assertEqual(x.name, "container", m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.Container, m),
                         type=lambda x, m: self.assertEqual(x, "mdcontainer", m))
         self.stateCheck(self.dev2,
                         createBitmap=self.assertFalse,
-                        level=lambda x, m: self.assertEqual(x.number, 0, m))
+                        level=lambda x, m: self.assertEqual(x.number, 0, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID0, m))
         self.stateCheck(self.dev3,
-                        level=lambda x, m: self.assertEqual(x.number, 1, m))
+                        level=lambda x, m: self.assertEqual(x.number, 1, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID1, m))
         self.stateCheck(self.dev4,
-                        level=lambda x, m: self.assertEqual(x.number, 4, m))
+                        level=lambda x, m: self.assertEqual(x.number, 4, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID4, m))
         self.stateCheck(self.dev5,
-                        level=lambda x, m: self.assertEqual(x.number, 5, m))
+                        level=lambda x, m: self.assertEqual(x.number, 5, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID5, m))
         self.stateCheck(self.dev6,
-                        level=lambda x, m: self.assertEqual(x.number, 6, m))
+                        level=lambda x, m: self.assertEqual(x.number, 6, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID6, m))
         self.stateCheck(self.dev7,
-                        level=lambda x, m: self.assertEqual(x.number, 10, m))
+                        level=lambda x, m: self.assertEqual(x.number, 10, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID10, m))

##
         ## existing device tests
@@ -259,7 +268,8 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
         self.stateCheck(self.dev8,
                         exists=self.assertTrue,
                         level=lambda x, m: self.assertEqual(x.number, 1, m),
-                        metadataVersion=self.assertIsNone)
+                        metadataVersion=self.assertIsNone,
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID1, m))

##
@@ -274,6 +284,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         memberDevices=lambda x, m: self.assertEqual(x, 2, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
                         partitionable=self.assertTrue,
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID0, m),
                         totalDevices=lambda x, m: self.assertEqual(x, 2, m),
                         type = lambda x, m: self.assertEqual(x, "mdbiosraidarray", m))

@@ -283,6 +294,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
         self.stateCheck(self.dev10,
                         createBitmap=self.assertFalse,
                         level=lambda x, m: self.assertEqual(x.number, 0, m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID0, m),
                         targetSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m))

self.stateCheck(self.dev11,
@@ -293,6 +305,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         memberDevices=lambda x, m: self.assertEqual(x, 2, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
                         partitionable=self.assertTrue,
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID1, m),
                         targetSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 2, m),
                         type=lambda x, m: self.assertEqual(x, "mdbiosraidarray", m))
@@ -305,6 +318,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         memberDevices=lambda x, m: self.assertEqual(x, 2, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
                         partitionable=self.assertTrue,
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID1, m),
                         targetSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 2, m),
                         type = lambda x, m: self.assertEqual(x, "mdbiosraidarray", m))
@@ -315,6 +329,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 0, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 3, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID0, m),
                         size=lambda x, m: self.assertEqual(x, Size("3 MiB"), m),
                         targetSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))
@@ -325,6 +340,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 4, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 3, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID4, m),
                         size=lambda x, m: self.assertEqual(x, Size("2 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))

@@ -334,6 +350,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 5, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 3, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID5, m),
                         size=lambda x, m: self.assertEqual(x, Size("2 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 3, m))

@@ -343,6 +360,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 6, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID6, m),
                         size=lambda x, m: self.assertEqual(x, Size("2 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 4, m))

@@ -352,6 +370,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 10, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID10, m),
                         size=lambda x, m: self.assertEqual(x, Size("2 MiB"), m),
                         totalDevices=lambda x, m: self.assertEqual(x, 4, m))

@@ -361,6 +380,7 @@ class MDRaidArrayDeviceTestCase(DeviceStateTestCase):
                         level=lambda x, m: self.assertEqual(x.number, 10, m),
                         memberDevices=lambda x, m: self.assertEqual(x, 4, m),
                         parents=lambda x, m: self.assertNotEqual(x, [], m),
+                        raidLevel=lambda x, m: self.assertEqual(x, raid.RAID10, m),
                         size=lambda x, m: self.assertEqual(x, Size("2 MiB"), m),
                         spares=lambda x, m: self.assertEqual(x, 1, m),
                         totalDevices=lambda x, m: self.assertEqual(x, 5, m))
@@ -437,6 +457,7 @@ class BTRFSDeviceTestCase(DeviceStateTestCase):
            "partitionable" : self.assertFalse,
            "path" : lambda x, m: self.assertRegexpMatches(x, "^/dev", m),
            "resizable" : lambda x, m: self.assertFalse,
+           "raidLevel" : self.assertIsNone,
            "size" : lambda x, m: self.assertEqual(x, Size(0), m),
            "status" : self.assertFalse,
            "sysfsPath" : lambda x, m: self.assertEqual(x, "", m),
@@ -465,12 +486,14 @@ class BTRFSDeviceTestCase(DeviceStateTestCase):

self.stateCheck(self.dev1,
            parents=lambda x, m: self.assertEqual(len(x), 1, m),
+           raidLevel=lambda x, m: self.assertEqual(x, raid.Single, m),
            type=lambda x, m: self.assertEqual(x, "btrfs volume", m))

self.stateCheck(self.dev3,
            currentSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
            maxSize=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
            parents=lambda x, m: self.assertEqual(len(x), 1, m),
+           raidLevel=lambda x, m: self.assertEqual(x, raid.Single, m),
            size=lambda x, m: self.assertEqual(x, Size("32 MiB"), m),
            type=lambda x, m: self.assertEqual(x, "btrfs volume", m))

[blivet:master 23/26] Distribute devicefactory.get_raid_level within devices classes.

[David Lehman]

On 05/29/2014 01:47 PM, mulhern wrote:
> Change the name to raidLevel, make it a property, and change the
> return type to a RAIDLevel object.

The way this was defined in devicefactory is fairly specific to 
anaconda's GUI.  I have doubts as to whether it is general enough to 
live in the device classes. In fact, I think it would make more sense in 
anaconda's custom_storage_helpers than in the device classes themselves. 
in the device classes it is somewhat confusing how some devices get 
their raidLevel from another device in the stack, while others do not. 
In a GUI helper function such an inconsistency is easily understood.

David

[blivet:master 23/26] Distribute devicefactory.get_raid_level within devices classes.

[Anne Mulhern]

----- Original Message -----
> From: "David Lehman" 
> To: [e-mail]
> Sent: Thursday, May 29, 2014 6:34:32 PM
> Subject: Re: [blivet:master 23/26] Distribute devicefactory.get_raid_level	within devices classes.
>
> On 05/29/2014 01:47 PM, mulhern wrote:
> > Change the name to raidLevel, make it a property, and change the
> > return type to a RAIDLevel object.
>
> The way this was defined in devicefactory is fairly specific to
> anaconda's GUI.  I have doubts as to whether it is general enough to
> live in the device classes. In fact, I think it would make more sense in
> anaconda's custom_storage_helpers than in the device classes themselves.
> in the device classes it is somewhat confusing how some devices get
> their raidLevel from another device in the stack, while others do not.
> In a GUI helper function such an inconsistency is easily understood.
>
> David
> _______________________________________________
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

Revised and resent. I'm not sending the anaconda side, because
that still needs a lot of unrelated revisions.

- mulhern

[blivet:master 24/26] Make all devicefactory classes uses RAID objects instead of strings.

[Anne Mulhern]

Also, since LinearRAID now exists, do not have to special case 'single',
do not have to special case it in DeviceFactory.configure().

Add a constructor for MDFactory that raises an error if there is no raid level
specified.

Fix up associated tests.

Signed-off-by: mulhern 
---
 blivet/devicefactory.py            | 67 ++++++++++++++++++++++++++++++--------
 tests/devicefactory_test.py        | 20 ++++++++----
 tests/devicelibs_test/raid_test.py |  8 ++---
 3 files changed, 70 insertions(+), 25 deletions(-)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index 5dcebd3..a42622e 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -27,6 +27,7 @@ from .formats import getFormat
 from .devicelibs import btrfs
 from .devicelibs import mdraid
 from .devicelibs import lvm
+from .devicelibs import raid
 from .partitioning import SameSizeSet
 from .partitioning import TotalSizeSet
 from .partitioning import doPartitioning
@@ -234,8 +235,8 @@ class DeviceFactory(object):
             :keyword label: filesystem label text
             :type label: str

-            :keyword raid_level: raid level string, eg: "raid1"
-            :type raid_level: str
+            :keyword raid_level: raid level descriptor
+            :type raid_level: any valid RAID level descriptor
             :keyword encrypted: whether to encrypt (boolean)
             :type encrypted: bool
             :keyword name: name of requested device
@@ -253,7 +254,7 @@ class DeviceFactory(object):
             :keyword container_name: name of requested container
             :type container_name: str
             :keyword container_raid_level: raid level for container
-            :type container_raid_level: str
+            :type container_raid_level: any valid RAID level descriptor
             :keyword container_encrypted: whether to encrypt the container
             :type container_encrypted: bool
             :keyword container_size: requested container size
@@ -304,6 +305,41 @@ class DeviceFactory(object):
         self.__names = []
         self.__roots = []

+    @property
+    def raid_level(self):
+        return self._raid_level
+
+    @raid_level.setter
+    def raid_level(self, value):
+        """ Sets the RAID level for the factory.
+
+            :param value: new RAID level
+            :param type: a valid RAID level descriptor
+            :returns: None
+        """
+        # pylint: disable=attribute-defined-outside-init
+        if value is None:
+            self._raid_level = None
+        else:
+            self._raid_level = raid.ALL_LEVELS.raidLevel(value)
+
+    @property
+    def container_raid_level(self):
+        return self._container_raid_level
+
+    @container_raid_level.setter
+    def container_raid_level(self, value):
+        """ Sets the RAID level for the factory.
+
+            :param value: new RAID level
+            :param type: a valid RAID level descriptor
+            :returns: None
+        """
+        # pylint: disable=attribute-defined-outside-init
+        if value is None:
+            self._container_raid_level = None
+        else:
+            self._container_raid_level = raid.ALL_LEVELS.raidLevel(value)
     #
     # methods related to device size and disk space requirements
     #
@@ -756,12 +792,11 @@ class DeviceFactory(object):
         # device or container raid level.
         for level_attr in ["raid_level", "container_raid_level"]:
             level = getattr(self, level_attr, None)
-            if level in [None, "single"]:
+            if level is None:
                 continue

-            min_disks = mdraid.RAID_levels.raidLevel(level).min_members
             disks = set(d for m in self._get_member_devices() for d in m.disks)
-            if len(disks) < min_disks:
+            if len(disks) < level.min_members:
                 raise DeviceFactoryError("Not enough disks for %s" % level)

# Configure any type-specific container device. The obvious example of
@@ -1534,10 +1569,14 @@ class MDFactory(DeviceFactory):
     child_factory_fstype = "mdmember"
     size_set_class = SameSizeSet

+    def __init__(self, storage, size, disks, **kwargs):
+        super(MDFactory, self).__init__(storage, size, disks, **kwargs)
+        if not self.raid_level:
+            raise DeviceFactoryError("MDFactory class must have some RAID level.")
+
     def _get_device_space(self):
         member_count = len(self._get_member_devices())
-        raid_level = mdraid.RAID_levels.raidLevel(self.raid_level)
-        size_per_member = raid_level.get_base_member_size(self.size, member_count)
+        size_per_member = self.raid_level.get_base_member_size(self.size, member_count)
         size_per_member += mdraid.get_raid_superblock_size(self.size)
         return size_per_member * member_count

@@ -1579,11 +1618,11 @@ class BTRFSFactory(DeviceFactory):
             log.info("overriding encryption setting for btrfs factory")
             self.encrypted = False

-        self.container_raid_level = self.container_raid_level or "single"
-        if self.container_raid_level == "single":
-            self.size_set_class = TotalSizeSet
-        else:
+        self.container_raid_level = self.container_raid_level or btrfs.RAID_levels.raidLevel("single")
+        if self.container_raid_level.is_uniform:
             self.size_set_class = SameSizeSet
+        else:
+            self.size_set_class = TotalSizeSet

def _handle_no_size(self):
         """ Set device size so that it grows to the largest size possible. """
@@ -1618,9 +1657,9 @@ class BTRFSFactory(DeviceFactory):

def _get_device_space(self):
         # until we get/need something better
-        if self.container_raid_level in ("single", "raid0"):
+        if self.container_raid_level in (raid.Single, raid.RAID0):
             return self.size
-        elif self.container_raid_level in ("raid1", "raid10"):
+        elif self.container_raid_level in (raid.RAID1, raid.RAID10):
             return self.size * len(self._get_member_devices())

@property
diff --git a/tests/devicefactory_test.py b/tests/devicefactory_test.py
index d4746dd..4b0e315 100644
--- a/tests/devicefactory_test.py
+++ b/tests/devicefactory_test.py
@@ -5,6 +5,7 @@ import unittest
 import blivet

from blivet import devicefactory
+from blivet.devicelibs import raid
 from blivet.errors import RaidError
 from blivet.size import Size

@@ -18,7 +19,8 @@ class MDFactoryTestCase(unittest.TestCase):
         self.b = blivet.Blivet()
         self.factory1 = devicefactory.get_device_factory(self.b,
            devicefactory.DEVICE_TYPE_MD,
-           Size("1 GiB"))
+           Size("1 GiB"),
+           raid_level=raid.RAID1)

self.factory2 = devicefactory.get_device_factory(self.b,
            devicefactory.DEVICE_TYPE_MD,
@@ -26,22 +28,26 @@ class MDFactoryTestCase(unittest.TestCase):
            raid_level=0)

def testMDFactory(self):
+        self.assertRaisesRegexp(devicefactory.DeviceFactoryError,
+           "must have some RAID level",
+           devicefactory.get_device_factory,
+           self.b,
+           devicefactory.DEVICE_TYPE_MD,
+           Size("1 GiB"))
+
         self.assertRaisesRegexp(RaidError,
-           "invalid RAID level",
+           "requires at least",
            self.factory1._get_device_space)

self.assertRaisesRegexp(RaidError,
-           "invalid RAID level",
+           "requires at least",
            self.factory1._configure)

self.assertEqual(self.factory1.container_list, [])

self.assertIsNone(self.factory1.get_container())

-        self.assertRaisesRegexp(RaidError,
-           "invalid RAID level",
-           self.factory1._get_new_device,
-           parents=[])
+        self.assertIsNotNone(self.factory1._get_new_device(parents=[]))

self.assertRaisesRegexp(RaidError,
            "requires at least",
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index 4d65b39..35c912a 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -113,19 +113,19 @@ class RaidTestCase(unittest.TestCase):
         ## size
         ##
         sizes = [Size("32MiB"), Size("128MiB"), Size("128MiB"), Size("64MiB")]
-        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l not in (raid.Container, raid.Dup)]:
             self.assertEqual(r.get_size(sizes, 4, Size("1MiB"), lambda x: Size(0)),
                r.get_net_array_size(4, Size("32MiB")) if isinstance(r, raid.RAIDn) else sum(sizes))

-        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l not in (raid.Container, raid.Dup)]:
             self.assertEqual(r.get_size(sizes, 5, Size("1MiB"), lambda x: Size(0)),
                r.get_net_array_size(5, Size("32MiB")) if isinstance(r, raid.RAIDn) else sum(sizes))

-        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l not in (raid.Container, raid.Dup)]:
             self.assertEqual(r.get_size(sizes, 4, Size("1MiB"), lambda x: Size("32MiB")),
                0 if isinstance(r, raid.RAIDn) else (sum(sizes) - 4 * Size("32MiB")))

-        for r in [l for l in raid.ALL_LEVELS if l is not raid.Container]:
+        for r in [l for l in raid.ALL_LEVELS if l not in (raid.Container, raid.Dup)]:
             if isinstance(r, raid.RAIDn):
                 if r not in (raid.RAID1, raid.RAID10):
                     self.assertEqual(r.get_size(sizes, 4, Size("2MiB"), lambda x: Size("31MiB")), 0)

[blivet:master 25/26] Move the code for getting a space requirement from devicefactory to raid.

[Anne Mulhern]

In devicelibs/raid.py make a new function called get_space which estimates
the space required for data and metadata. Make a few auxiliary functions.

mdraid now requires that its RAIDLevel objects supply get_space, rather
than get_base_member_size (which only made sense for RAIDLevel objects for
which uniform is True).

Signed-off-by: mulhern 
---
 blivet/devicefactory.py            |  8 ++---
 blivet/devicelibs/mdraid.py        |  2 +-
 blivet/devicelibs/raid.py          | 68 +++++++++++++++++++++++++++++++++++---
 tests/devicelibs_test/raid_test.py |  4 ---
 4 files changed, 69 insertions(+), 13 deletions(-)

diff --git a/blivet/devicefactory.py b/blivet/devicefactory.py
index a42622e..434f94a 100644
--- a/blivet/devicefactory.py
+++ b/blivet/devicefactory.py
@@ -1575,10 +1575,10 @@ class MDFactory(DeviceFactory):
             raise DeviceFactoryError("MDFactory class must have some RAID level.")

def _get_device_space(self):
-        member_count = len(self._get_member_devices())
-        size_per_member = self.raid_level.get_base_member_size(self.size, member_count)
-        size_per_member += mdraid.get_raid_superblock_size(self.size)
-        return size_per_member * member_count
+        return self.raid_level.get_space(self.size,
+           len(self._get_member_devices()),
+           None,
+           mdraid.get_raid_superblock_size)

def _get_total_space(self):
         return self._get_device_space()
diff --git a/blivet/devicelibs/mdraid.py b/blivet/devicelibs/mdraid.py
index f0d8162..b82c43d 100644
--- a/blivet/devicelibs/mdraid.py
+++ b/blivet/devicelibs/mdraid.py
@@ -39,7 +39,7 @@ class MDRaidLevels(raid.RAIDLevels):
     def isRaidLevel(cls, level):
         return super(MDRaidLevels, cls).isRaidLevel(level) and \
            hasattr(level, 'get_max_spares') and \
-           hasattr(level, 'get_base_member_size') and \
+           hasattr(level, 'get_space') and \
            hasattr(level, 'get_recommended_stride') and \
            hasattr(level, 'get_size')

diff --git a/blivet/devicelibs/raid.py b/blivet/devicelibs/raid.py
index 6b05122..1ec4e07 100644
--- a/blivet/devicelibs/raid.py
+++ b/blivet/devicelibs/raid.py
@@ -200,6 +200,21 @@ class RAIDn(RAIDLevel):
         """
         raise NotImplementedError()

+    @abc.abstractmethod
+    def _pad(self, size, chunk_size):
+        """Helper function; not to be called directly.
+
+           Pads size to the smallest size greater than size that is in units
+           of chunk_size.
+
+           :param size: the size of the array
+           :type size: :class:`~.size.Size`
+           :param chunk_size: the smallest unit of size this array allows
+           :type chunk_size: :class:`~.size.Size`
+           :rtype: :class:`~.size.Size`
+        """
+        raise NotImplementedError()
+
     def get_recommended_stride(self, member_count):
         """Return a recommended stride size in blocks.

@@ -258,6 +273,31 @@ class RAIDn(RAIDLevel):
         min_data_size = self._trim(min_size - superblock_size, chunk_size)
         return self.get_net_array_size(num_members, min_data_size)

+    def get_space(self, size, num_members, chunk_size=None, superblock_size_func=None):
+        """Estimate the amount of memory required by this array, including
+           memory allocated for metadata.
+
+           :param size: the amount of data on this array
+           :type size: :class:`~.size.Size`
+           :param int num_members: the number of members in the array
+           :param chunk_size: the smallest unit of size for
+           :type chunk_size: :class:`~.size.Size`
+           :param superblock_size_func: a function that estimates the
+              superblock size for this array
+           :type superblock_size_func: a function from :class:`~.size.Size` to
+              :class:`~.size.Size`
+           :returns: an estimate of the memory required, including metadata
+           :rtype: :class:`~.size.Size`
+        """
+        if superblock_size_func is None:
+            raise RaidError("superblock_size_func value of None is not acceptable")
+
+        size_per_member = self.get_base_member_size(size, num_members)
+        size_per_member += superblock_size_func(size)
+        if chunk_size is not None:
+            size_per_member = self._pad(size_per_member, chunk_size)
+        return size_per_member * num_members
+
 class RAIDLevels(object):
     """A class which keeps track of registered RAID levels. This class
        may be extended, overriding the isRaid method to include any
@@ -358,6 +398,9 @@ class RAID0(RAIDn):
     def _trim(self, size, chunk_size):
         return size - size % chunk_size

+    def _pad(self, size, chunk_size):
+        return size + (chunk_size - (size % chunk_size)) % chunk_size
+
     def _get_recommended_stride(self, member_count):
         return member_count * 16

@@ -382,6 +425,9 @@ class RAID1(RAIDn):
     def _trim(self, size, chunk_size):
         return size

+    def _pad(self, size, chunk_size):
+        return size
+
     def _get_recommended_stride(self, member_count):
         return None

@@ -406,6 +452,9 @@ class RAID4(RAIDn):
     def _trim(self, size, chunk_size):
         return size - size % chunk_size

+    def _pad(self, size, chunk_size):
+        return size + (chunk_size - (size % chunk_size)) % chunk_size
+
     def _get_recommended_stride(self, member_count):
         return (member_count - 1) * 16

@@ -430,6 +479,9 @@ class RAID5(RAIDn):
     def _trim(self, size, chunk_size):
         return size - size % chunk_size

+    def _pad(self, size, chunk_size):
+        return size + (chunk_size - (size % chunk_size)) % chunk_size
+
     def _get_recommended_stride(self, member_count):
         return (member_count - 1) * 16

@@ -454,6 +506,9 @@ class RAID6(RAIDn):
     def _trim(self, size, chunk_size):
         return size - size % chunk_size

+    def _pad(self, size, chunk_size):
+        return size + (chunk_size - (size % chunk_size)) % chunk_size
+
     def _get_recommended_stride(self, member_count):
         return None

@@ -478,6 +533,9 @@ class RAID10(RAIDn):
     def _trim(self, size, chunk_size):
         return size

+    def _pad(self, size, chunk_size):
+        return size + (chunk_size - (size % chunk_size)) % chunk_size
+
     def _get_recommended_stride(self, member_count):
         return None

@@ -494,9 +552,9 @@ class Container(RAIDLevel):
     def get_max_spares(self, member_count):
         # pylint: disable=unused-argument
         raise RaidError("get_max_spares is not defined for level container")
-    def get_base_member_size(self, size, member_count):
+    def get_space(self, size, num_members, chunk_size=None, superblock_size_func=None):
         # pylint: disable=unused-argument
-        raise RaidError("get_base_member_size is not defined for level container")
+        return size
     def get_recommended_stride(self, member_count):
         # pylint: disable=unused-argument
         raise RaidError("get_recommended_stride is not defined for level container")
@@ -522,9 +580,11 @@ class ErsatzRAID(RAIDLevel):
     def get_max_spares(self, member_count):
         return member_count - self.min_members

-    def get_base_member_size(self, size, member_count):
+    def get_space(self, size, num_members, chunk_size=None, superblock_size_func=None):
         # pylint: disable=unused-argument
-        raise RaidError("get_base_member_size is not defined for level linear")
+        if superblock_size_func is None:
+            raise RaidError("superblock_size_func value of None is not acceptable")
+        return size + num_members * superblock_size_func(size)

def get_recommended_stride(self, member_count):
         # pylint: disable=unused-argument
diff --git a/tests/devicelibs_test/raid_test.py b/tests/devicelibs_test/raid_test.py
index 35c912a..4715268 100644
--- a/tests/devicelibs_test/raid_test.py
+++ b/tests/devicelibs_test/raid_test.py
@@ -79,10 +79,6 @@ class RaidTestCase(unittest.TestCase):
                           4, 3)
         self.assertRaises(errors.RaidError, raid.RAID10.get_base_member_size,
                           -4, 4)
-        self.assertRaisesRegexp(errors.RaidError,
-           "not defined",
-           raid.Linear.get_base_member_size,
-           4, 3)

##
         ## get_net_array_size

[blivet:master 25/26] Move the code for getting a space requirement from devicefactory to raid.

[Martin Kolman]

>       :param size: the amount of data on this array
>       :type size: :class:`~.size.Size`
>       :param int num_members: the number of members in the array
>       :param chunk_size: the smallest unit of size for

^ something missing ?
>       :type chunk_size: :class:`~.size.Size`
>       :param superblock_size_func: a function that estimates the
>          superblock size for this array
>       :type superblock_size_func: a function from :class:`~.size.Size`
>
> to
>          :class:`~.size.Size`
>       :returns: an estimate of the memory required, including metadata
>       :rtype: :class:`~.size.Size`
>    """

[blivet:master 25/26] Move the code for getting a space requirement from devicefactory to raid.

[Anne Mulhern]

----- Original Message -----
> From: "Martin Kolman" 
> To: "anaconda patch review" 
> Sent: Friday, May 30, 2014 10:01:08 AM
> Subject: Re: [blivet:master 25/26] Move the code for getting a space	requirement	from devicefactory to raid.
> >       :param size: the amount of data on this array
> >       :type size: :class:`~.size.Size`
> >       :param int num_members: the number of members in the array
> >       :param chunk_size: the smallest unit of size for
>                                                          ^ something
>                                                          missing ?
> >       :type chunk_size: :class:`~.size.Size`
> >       :param superblock_size_func: a function that estimates the
> >          superblock size for this array
> >       :type superblock_size_func: a function from
> >
> > :class:`~.size.Size`
> > to
> >          :class:`~.size.Size`
> >       :returns: an estimate of the memory required, including
> >
> > metadata
> >       :rtype: :class:`~.size.Size`
> >    """
>
> anaconda-patches mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches

Yup, fixed, also fixed in get_size.

- mulhern

[blivet:master 26/26] Simplify a small chunk of Blivet.updateKSData()

[Anne Mulhern]

---
 blivet/__init__.py | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/blivet/__init__.py b/blivet/__init__.py
index b77ff8b..d68c16c 100644
--- a/blivet/__init__.py
+++ b/blivet/__init__.py
@@ -2136,17 +2136,13 @@ class Blivet(object):
                                 for a in d.ancestors))
         devices.sort(key=lambda d: len(d.ancestors))
         for device in devices:
-            class_attr = None
-            list_attr = None
-            for cls in ksMap.iterkeys():
-                if isinstance(device, cls):
-                    class_attr, list_attr = ksMap[cls]
-                    break
-
-            if not class_attr or not list_attr:
+            cls = next((c for c in ksMap if isinstance(device, c)), None)
+            if cls is None:
                 log.info("omitting ksdata: %s", device)
                 continue

+            class_attr, list_attr = ksMap[cls]
+
             cls = getattr(self.ksdata, class_attr)
             data = cls()    # all defaults

[David Lehman]

On 05/29/2014 01:47 PM, mulhern wrote:
> These patches will be vaguely remniscent of the last set of RAID related
> patches I sent out, but not all that close.
>
> The changes require an anaconda patch.
>
> The first four are nice and simple.
> The very last is only related to RAID because I felt
> I had to understand the method in order to understand device populateKSData()
> methods, and to understand it I had to think about, so I ended up rewriting it.

I have to give these a more thorough read, but overall I definitely 
think this is headed in the right direction.

David
> mulhern (26):
>    Import name 'lvm' instead of names from lvm package.
>    Comment mdraid.mdcreate() and update tests appropriately.
>    Remove DMRaidArrayDevice.members property.
>    Extract selection of members in completed() into a separate method.
>    No longer use _standard_levels as the default set of RAID levels.
>    Adjust RaidLevel hierarchy so that all raid level objects extend
>      RAIDLevel
>    Allow the RAID object itself to be a valid RAID descriptor for lookup.
>    Move Container class to raid package and tidy it up
>    Add Linear and Single to the RAID classes.
>    Add a has_redundancy method that returns True if there is actual
>      redundancy
>    Add an is_uniform property to the RAID levels.
>    Add a Dup class to the various descendants of RAIDLevel.
>    Make createBitmap() a property and update tests appropriately.
>    Use has_redundancy raid property when checking whether a device is
>      removable
>    Update the mdraid.mdadd comments
>    Use has_redundancy property to decide how to add a member to an array.
>    Check for required methods in MDRaidLevels.isRaidLevel.
>    Remove getRaidLevel() from mdraid file and make RAID_levels public
>    Add "linear" to mdraid's list of supported raid levels.
>    Add lists of supported RAID levels for btrfs and lvm
>    Make BTRFS devices use RAID objects instead of strings for levels
>    Put get_supported_raid_levels in devicefactory.py
>    Distribute devicefactory.get_raid_level within devices classes.
>    Make all devicefactory classes uses RAID objects instead of strings.
>    Move the code for getting a space requirement from devicefactory to
>      raid.
>    Simplify a small chunk of Blivet.updateKSData()
>
> blivet/__init__.py                   |  12 +-
>   blivet/devicefactory.py              | 137 ++++++++++------
>   blivet/devicelibs/btrfs.py           |  10 +-
>   blivet/devicelibs/lvm.py             |   3 +
>   blivet/devicelibs/mdraid.py          |  64 +++-----
>   blivet/devicelibs/raid.py            | 297 +++++++++++++++++++++++++----------
>   blivet/devices.py                    | 186 +++++++++++++++++-----
>   tests/devicefactory_test.py          |  26 +--
>   tests/devicelibs_test/mdraid_test.py |  21 +--
>   tests/devicelibs_test/raid_test.py   |  73 ++++-----
>   tests/devices_test.py                |  58 +++++--
>   tests/devicetree_test.py             |   2 +-
>   12 files changed, 590 insertions(+), 299 deletions(-)

[David Lehman]

On 05/29/2014 01:47 PM, mulhern wrote:
> These patches will be vaguely remniscent of the last set of RAID related
> patches I sent out, but not all that close.
>
> The changes require an anaconda patch.
>
> The first four are nice and simple.
> The very last is only related to RAID because I felt
> I had to understand the method in order to understand device populateKSData()
> methods, and to understand it I had to think about, so I ended up rewriting it.
>
> mulhern (26):
>    Import name 'lvm' instead of names from lvm package.
>    Comment mdraid.mdcreate() and update tests appropriately.
>    Remove DMRaidArrayDevice.members property.
>    Extract selection of members in completed() into a separate method.
>    No longer use _standard_levels as the default set of RAID levels.
>    Adjust RaidLevel hierarchy so that all raid level objects extend
>      RAIDLevel
>    Allow the RAID object itself to be a valid RAID descriptor for lookup.
>    Move Container class to raid package and tidy it up
>    Add Linear and Single to the RAID classes.
>    Add a has_redundancy method that returns True if there is actual
>      redundancy
>    Add an is_uniform property to the RAID levels.
>    Add a Dup class to the various descendants of RAIDLevel.
>    Make createBitmap() a property and update tests appropriately.
>    Use has_redundancy raid property when checking whether a device is
>      removable
>    Update the mdraid.mdadd comments
>    Use has_redundancy property to decide how to add a member to an array.
>    Check for required methods in MDRaidLevels.isRaidLevel.
>    Remove getRaidLevel() from mdraid file and make RAID_levels public
>    Add "linear" to mdraid's list of supported raid levels.
>    Add lists of supported RAID levels for btrfs and lvm
>    Make BTRFS devices use RAID objects instead of strings for levels
>    Put get_supported_raid_levels in devicefactory.py
>    Distribute devicefactory.get_raid_level within devices classes.
>    Make all devicefactory classes uses RAID objects instead of strings.
>    Move the code for getting a space requirement from devicefactory to
>      raid.
>    Simplify a small chunk of Blivet.updateKSData()
>
> blivet/__init__.py                   |  12 +-
>   blivet/devicefactory.py              | 137 ++++++++++------
>   blivet/devicelibs/btrfs.py           |  10 +-
>   blivet/devicelibs/lvm.py             |   3 +
>   blivet/devicelibs/mdraid.py          |  64 +++-----
>   blivet/devicelibs/raid.py            | 297 +++++++++++++++++++++++++----------
>   blivet/devices.py                    | 186 +++++++++++++++++-----
>   tests/devicefactory_test.py          |  26 +--
>   tests/devicelibs_test/mdraid_test.py |  21 +--
>   tests/devicelibs_test/raid_test.py   |  73 ++++-----
>   tests/devices_test.py                |  58 +++++--
>   tests/devicetree_test.py             |   2 +-
>   12 files changed, 590 insertions(+), 299 deletions(-)

ACK