Patch 1 is from previous pyudev set, but since it grew a followup it seemed worth separating into its own set.
In testing I found that I no longer know how to hit the issue we're handling with patch 2 here, but it should be better with this change. Previously we would look up the correct device by UUID and use it, but would not update its name to match reality. (The array must be active -- otherwise we wouldn't be in addUdevMDDevice.)
Patch 3 is fairly self-explanatory and just walks back part of a slightly overzealous recent patch.
David Lehman (3): Remove an obsolete block related to unpredictable md device names. Update md name when lookup relied on UUID. Don't pass md array UUID as member format UUID.
blivet/devicetree.py | 41 ++++++----------------------------------- 1 file changed, 6 insertions(+), 35 deletions(-)
The functionality this block originally provided should be covered by the UUID lookup we do in addUdevMDDevice.
This block no longer works because it relies on md-specific data being in the udev database itself, which is no longer the case. It hasn't been since we removed anaconda's udev rules. --- blivet/devicetree.py | 36 ++---------------------------------- 1 file changed, 2 insertions(+), 34 deletions(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index a6b14d8..f8fc112 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1594,42 +1594,10 @@ class DeviceTree(object): log.warning("invalid data for %s: no RAID level", device.name) return
- md_metadata = None - md_name = None - - # check the list of devices udev knows about to see if the array - # this device belongs to is already active - for dev in udev.get_devices(): - if not udev.device_is_md(dev): - continue - - try: - dev_uuid = udev.device_get_md_uuid(dev) - dev_level = udev.device_get_md_level(dev) - except KeyError: - continue - - if dev_uuid is None or dev_level is None: - continue - - if dev_uuid == md_uuid and dev_level == md_level: - md_name = udev.device_get_md_name(dev) - md_metadata = udev.device_get_md_metadata(dev) - if not md_name: - # containers don't typically have names and they also - # don't have a symlink in /dev/md - md_name = udev.device_get_name(dev) - if md_level != "container" and \ - re.match(r'md\d+$', md_name): - # md0 -> 0 - md_name = md_name[2:] - - break - # mdexamine yields MD_METADATA only for metadata version > 0.90 # if MD_METADATA is missing, assume metadata version is 0.90 - md_metadata = md_metadata or udev.device_get_md_metadata(md_info) or "0.90" - + md_metadata = udev.device_get_md_metadata(md_info) or "0.90" + md_name = udev.device_get_md_name(md_info) if not md_name: md_path = md_info.get("DEVICE", "") if md_path:
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:02 AM Subject: [PATCH 1/3] Remove an obsolete block related to unpredictable md device names.
The functionality this block originally provided should be covered by the UUID lookup we do in addUdevMDDevice.
This block no longer works because it relies on md-specific data being in the udev database itself, which is no longer the case. It hasn't been since we removed anaconda's udev rules.
blivet/devicetree.py | 36 ++---------------------------------- 1 file changed, 2 insertions(+), 34 deletions(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index a6b14d8..f8fc112 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1594,42 +1594,10 @@ class DeviceTree(object): log.warning("invalid data for %s: no RAID level", device.name) return
md_metadata = Nonemd_name = None# check the list of devices udev knows about to see if the array# this device belongs to is already activefor dev in udev.get_devices():if not udev.device_is_md(dev):continuetry:dev_uuid = udev.device_get_md_uuid(dev)dev_level = udev.device_get_md_level(dev)except KeyError:continueif dev_uuid is None or dev_level is None:continueif dev_uuid == md_uuid and dev_level == md_level:md_name = udev.device_get_md_name(dev)md_metadata = udev.device_get_md_metadata(dev)if not md_name:# containers don't typically have names and theyalso
# don't have a symlink in /dev/mdmd_name = udev.device_get_name(dev)if md_level != "container" and \re.match(r'md\d+$', md_name):# md0 -> 0md_name = md_name[2:]break# mdexamine yields MD_METADATA only for metadata version > 0.90 # if MD_METADATA is missing, assume metadata version is 0.90md_metadata = md_metadata orudev.device_get_md_metadata(md_info) or "0.90"
md_metadata = udev.device_get_md_metadata(md_info) or "0.90"md_name = udev.device_get_md_name(md_info) if not md_name: md_path = md_info.get("DEVICE", "") if md_path:-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Ack.
- mulhern
----- Original Message -----
From: "Anne Mulhern" amulhern@redhat.com To: "anaconda patch review" anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 11:09:47 AM Subject: Re: [PATCH 1/3] Remove an obsolete block related to unpredictable md device names.
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:02 AM Subject: [PATCH 1/3] Remove an obsolete block related to unpredictable md device names.
The functionality this block originally provided should be covered by the UUID lookup we do in addUdevMDDevice.
This block no longer works because it relies on md-specific data being in the udev database itself, which is no longer the case. It hasn't been since we removed anaconda's udev rules.
blivet/devicetree.py | 36 ++---------------------------------- 1 file changed, 2 insertions(+), 34 deletions(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index a6b14d8..f8fc112 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1594,42 +1594,10 @@ class DeviceTree(object): log.warning("invalid data for %s: no RAID level", device.name) return
md_metadata = Nonemd_name = None# check the list of devices udev knows about to see if thearray
# this device belongs to is already activefor dev in udev.get_devices():if not udev.device_is_md(dev):continuetry:dev_uuid = udev.device_get_md_uuid(dev)dev_level = udev.device_get_md_level(dev)except KeyError:continueif dev_uuid is None or dev_level is None:continueif dev_uuid == md_uuid and dev_level == md_level:md_name = udev.device_get_md_name(dev)md_metadata = udev.device_get_md_metadata(dev)if not md_name:# containers don't typically have names and theyalso
# don't have a symlink in /dev/mdmd_name = udev.device_get_name(dev)if md_level != "container" and \re.match(r'md\d+$', md_name):# md0 -> 0md_name = md_name[2:]break# mdexamine yields MD_METADATA only for metadata version > 0.90 # if MD_METADATA is missing, assume metadata version is 0.90md_metadata = md_metadata orudev.device_get_md_metadata(md_info) or "0.90"
md_metadata = udev.device_get_md_metadata(md_info) or "0.90"md_name = udev.device_get_md_name(md_info) if not md_name: md_path = md_info.get("DEVICE", "") if md_path:-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Ack.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Except that the commit message is still a bit deceptive, since md does set some of those things with its own rules.
- mulhern
On 08/01/2014 04:11 AM, Anne Mulhern wrote:
----- Original Message -----
From: "Anne Mulhern" amulhern@redhat.com To: "anaconda patch review" anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 11:09:47 AM Subject: Re: [PATCH 1/3] Remove an obsolete block related to unpredictable md device names.
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:02 AM Subject: [PATCH 1/3] Remove an obsolete block related to unpredictable md device names.
The functionality this block originally provided should be covered by the UUID lookup we do in addUdevMDDevice.
This block no longer works because it relies on md-specific data being in the udev database itself, which is no longer the case. It hasn't been since we removed anaconda's udev rules.
blivet/devicetree.py | 36 ++---------------------------------- 1 file changed, 2 insertions(+), 34 deletions(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index a6b14d8..f8fc112 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1594,42 +1594,10 @@ class DeviceTree(object): log.warning("invalid data for %s: no RAID level", device.name) return
md_metadata = Nonemd_name = None# check the list of devices udev knows about to see if thearray
# this device belongs to is already activefor dev in udev.get_devices():if not udev.device_is_md(dev):continuetry:dev_uuid = udev.device_get_md_uuid(dev)dev_level = udev.device_get_md_level(dev)except KeyError:continueif dev_uuid is None or dev_level is None:continueif dev_uuid == md_uuid and dev_level == md_level:md_name = udev.device_get_md_name(dev)md_metadata = udev.device_get_md_metadata(dev)if not md_name:# containers don't typically have names and theyalso
# don't have a symlink in /dev/mdmd_name = udev.device_get_name(dev)if md_level != "container" and \re.match(r'md\d+$', md_name):# md0 -> 0md_name = md_name[2:]break# mdexamine yields MD_METADATA only for metadata version > 0.90 # if MD_METADATA is missing, assume metadata version is 0.90md_metadata = md_metadata orudev.device_get_md_metadata(md_info) or "0.90"
md_metadata = udev.device_get_md_metadata(md_info) or "0.90"md_name = udev.device_get_md_name(md_info) if not md_name: md_path = md_info.get("DEVICE", "") if md_path:-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Ack.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Except that the commit message is still a bit deceptive, since md does set some of those things with its own rules.
Good point. I will update the commit message before pushing.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
--- blivet/devicetree.py | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index f8fc112..73558df 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -910,6 +910,9 @@ class DeviceTree(object): log.warning("failed to obtain uuid for mdraid device") else: device = self.getDeviceByUuid(uuid, incomplete=flags.allow_degraded_mdraid) + if device: + # update the device instance with the real name + device.name = name
# if we get here, we found all of the slave devices and # something must be wrong -- if all of the slaves are in
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:03 AM Subject: [PATCH 2/3] Update md name when lookup relied on UUID.
blivet/devicetree.py | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index f8fc112..73558df 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -910,6 +910,9 @@ class DeviceTree(object): log.warning("failed to obtain uuid for mdraid device") else: device = self.getDeviceByUuid(uuid, incomplete=flags.allow_degraded_mdraid)
if device:# update the device instance with the real namedevice.name = name # if we get here, we found all of the slave devices and # something must be wrong -- if all of the slaves are in-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Please add
Related: rhbz#1090009
to commit message since this change should go in with the code just above it.
Also, it would make more general sense to move the assignment down and out a bit and have something like:
if device is not None: device.name = name else: # if we get here, we found all of the slave devices and # something must be wrong -- if all of the slaves etc.
In some cases, if we actually managed to get the device by name for instance, that assignment definitely is redundant, but there's always the chance that we'll add trying to get the device by yet another way somewhere up above, and then fail to set the name in the case that we got the device, which would be a pain to figure out later.
- mulhern
On 08/01/2014 03:56 AM, Anne Mulhern wrote:
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:03 AM Subject: [PATCH 2/3] Update md name when lookup relied on UUID.
blivet/devicetree.py | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index f8fc112..73558df 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -910,6 +910,9 @@ class DeviceTree(object): log.warning("failed to obtain uuid for mdraid device") else: device = self.getDeviceByUuid(uuid, incomplete=flags.allow_degraded_mdraid)
if device:# update the device instance with the real namedevice.name = name # if we get here, we found all of the slave devices and # something must be wrong -- if all of the slaves are in-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Please add
Related: rhbz#1090009
to commit message since this change should go in with the code just above it.
Will do.
Also, it would make more general sense to move the assignment down and out a bit and have something like:
if device is not None: device.name = name else: # if we get here, we found all of the slave devices and # something must be wrong -- if all of the slaves etc.
In some cases, if we actually managed to get the device by name for instance, that assignment definitely is redundant, but there's always the chance that we'll add trying to get the device by yet another way somewhere up above, and then fail to set the name in the case that we got the device, which would be a pain to figure out later.
Although it seems unlikely that we'll add another lookup key I went ahead and made this change. I also changed it to bypass the name setter and assign to device._name instead since changing the names of existing devices is generally not allowed.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
Related: rhbz#1070095 --- blivet/devicetree.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index 73558df..03da94f 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1806,7 +1806,7 @@ class DeviceTree(object):
# attempt to reset the uuid using mdexamine info # will succeed only if metadata version > 0.90 - kwargs["uuid"] = udev.device_get_md_device_uuid(info) or kwargs["uuid"] + kwargs["uuid"] = udev.device_get_md_device_uuid(info)
kwargs["biosraid"] = udev.device_is_biosraid_member(info) elif format_type == "LVM2_member":
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:04 AM Subject: [PATCH 3/3] Don't pass md array UUID as member format UUID.
Related: rhbz#1070095
blivet/devicetree.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index 73558df..03da94f 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1806,7 +1806,7 @@ class DeviceTree(object):
# attempt to reset the uuid using mdexamine info # will succeed only if metadata version > 0.90
kwargs["uuid"] = udev.device_get_md_device_uuid(info) orkwargs["uuid"]
kwargs["uuid"] = udev.device_get_md_device_uuid(info) kwargs["biosraid"] = udev.device_is_biosraid_member(info) elif format_type == "LVM2_member":-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
If it's really true that the value obtained from device_get_uuid() is certain to be completely irrelevant here, so that we can override it unconditionally, then device_get_uuid() can probably be simplified to a 1-liner and maybe a comment added about the nature of ID_FS_UUID (it's not clear to me what might be expected to set it.)
Also, the comment should be changed to something like:
reset the device UUID using mdexamine info UUID is None if metadata version <= 0.90
or something like that.
- mulhern
On 08/01/2014 03:35 AM, Anne Mulhern wrote:
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:04 AM Subject: [PATCH 3/3] Don't pass md array UUID as member format UUID.
Related: rhbz#1070095
blivet/devicetree.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index 73558df..03da94f 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1806,7 +1806,7 @@ class DeviceTree(object):
# attempt to reset the uuid using mdexamine info # will succeed only if metadata version > 0.90
kwargs["uuid"] = udev.device_get_md_device_uuid(info) orkwargs["uuid"]
kwargs["uuid"] = udev.device_get_md_device_uuid(info) kwargs["biosraid"] = udev.device_is_biosraid_member(info) elif format_type == "LVM2_member":-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
If it's really true that the value obtained from device_get_uuid() is certain to be completely irrelevant here, so that we can override it unconditionally, then device_get_uuid() can probably be simplified to a 1-liner and maybe a comment added about the nature of ID_FS_UUID (it's not clear to me what might be expected to set it.)
If MD_UUID was ever in the udev info for md members, it isn't anymore. It probably came from anaconda's udev rules.
ID_FS_UUID is the UUID of the filesystem/data on the current device. It only becomes complicated with md and btrfs because, instead of using ID_FS_UUID for the device-specific (and therefore unique, like a UUID should be) value, they use it for the container/array/volume and introduce ID_FS_UUID_SUB for the member-specific UUIDs. LVM (correctly) has the PV UUID as ID_FS_UUID of the members and then you have to look elsewhere for VG and/or LV UUIDs.
Anyway, I was able to reduce device_get_uuid to a single line and since I just found out that ID_FS_UUID_SUB contains the md member-specific UUID I changed device_get_md_device_uuid like this:
@@ -395,7 +387,9 @@ def device_get_md_device_uuid(info): # Value for MD_UUID known to be obtained from: # * pyudev/libudev # * mdraid/mdadm (only 1.x metadata versions) - return info.get('MD_DEV_UUID') + # Value for ID_FS_UUID_SUB known to be obtained from: + # * pyudev/libudev + return info.get('ID_FS_UUID_SUB') or info.get('MD_DEV_UUID')
Also, the comment should be changed to something like:
reset the device UUID using mdexamine info UUID is None if metadata version <= 0.90
or something like that.
Done.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 6:46:19 PM Subject: Re: [PATCH 3/3] Don't pass md array UUID as member format UUID.
On 08/01/2014 03:35 AM, Anne Mulhern wrote:
----- Original Message -----
From: "David Lehman" dlehman@redhat.com To: anaconda-patches@lists.fedorahosted.org Sent: Friday, August 1, 2014 12:16:04 AM Subject: [PATCH 3/3] Don't pass md array UUID as member format UUID.
Related: rhbz#1070095
blivet/devicetree.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blivet/devicetree.py b/blivet/devicetree.py index 73558df..03da94f 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -1806,7 +1806,7 @@ class DeviceTree(object):
# attempt to reset the uuid using mdexamine info # will succeed only if metadata version > 0.90
kwargs["uuid"] = udev.device_get_md_device_uuid(info) orkwargs["uuid"]
kwargs["uuid"] = udev.device_get_md_device_uuid(info) kwargs["biosraid"] = udev.device_is_biosraid_member(info) elif format_type == "LVM2_member":-- 1.9.3
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
If it's really true that the value obtained from device_get_uuid() is certain to be completely irrelevant here, so that we can override it unconditionally, then device_get_uuid() can probably be simplified to a 1-liner and maybe a comment added about the nature of ID_FS_UUID (it's not clear to me what might be expected to set it.)
If MD_UUID was ever in the udev info for md members, it isn't anymore. It probably came from anaconda's udev rules.
I believe that I actually saw it pop up as a result of running the previous pyudev code in the debugger. I can't for the life of me see any way that mdadm could be setting it, but I do think I saw it.
ID_FS_UUID is the UUID of the filesystem/data on the current device. It
only becomes complicated with md and btrfs because, instead of using ID_FS_UUID for the device-specific (and therefore unique, like a UUID should be) value, they use it for the container/array/volume and introduce ID_FS_UUID_SUB for the member-specific UUIDs. LVM (correctly) has the PV UUID as ID_FS_UUID of the members and then you have to look elsewhere for VG and/or LV UUIDs.
Anyway, I was able to reduce device_get_uuid to a single line and since I just found out that ID_FS_UUID_SUB contains the md member-specific UUID I changed device_get_md_device_uuid like this:
@@ -395,7 +387,9 @@ def device_get_md_device_uuid(info): # Value for MD_UUID known to be obtained from: # * pyudev/libudev # * mdraid/mdadm (only 1.x metadata versions)
- return info.get('MD_DEV_UUID')
- # Value for ID_FS_UUID_SUB known to be obtained from:
- # * pyudev/libudev
- return info.get('ID_FS_UUID_SUB') or info.get('MD_DEV_UUID')
I just realized that now that the infos are distinct that it works, but wierdly. If you pass udev info you may get value for ID_FS_UUID_SUB, but if md info then you may get value for MD_DEV_UUID. So, it might make sense to pass both infos.
I'm not sure how to handle this exactly, but it does make an argument for the notion of having a SuperCache of the kind I described in my previous email.
Also, the comment should be changed to something like:
reset the device UUID using mdexamine info UUID is None if metadata version <= 0.90
or something like that.
Done.
- mulhern
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
anaconda-patches mailing list anaconda-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
On Thu, Jul 31, 2014 at 05:16:01PM -0500, David Lehman wrote:
Patch 1 is from previous pyudev set, but since it grew a followup it seemed worth separating into its own set.
In testing I found that I no longer know how to hit the issue we're handling with patch 2 here, but it should be better with this change. Previously we would look up the correct device by UUID and use it, but would not update its name to match reality. (The array must be active -- otherwise we wouldn't be in addUdevMDDevice.)
Patch 3 is fairly self-explanatory and just walks back part of a slightly overzealous recent patch.
David Lehman (3): Remove an obsolete block related to unpredictable md device names. Update md name when lookup relied on UUID. Don't pass md array UUID as member format UUID.
blivet/devicetree.py | 41 ++++++----------------------------------- 1 file changed, 6 insertions(+), 35 deletions(-)
-- 1.9.3
Ack
anaconda-patches@lists.fedorahosted.org