anaconda-patches April 2016

anaconda-patches@lists.fedorahosted.org

9 participants
1157 discussions

[master 1/1] Don't use booleans in Requires
by dashea 01 Apr '16

01 Apr '16

From: David Shea <dshea(a)redhat.com> We're not allowed to use booleans in Requires because dnf-by-default did not include the tools used to construct a Fedora compose. This means that boot.iso and other situations will end up with both glibc-langpack-en and glibc-all-langpacks, but that's apparently the world we live in now. --- anaconda.spec.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/anaconda.spec.in b/anaconda.spec.in index 6a43f11..89c5d65 100644 --- a/anaconda.spec.in +++ b/anaconda.spec.in @@ -144,7 +144,7 @@ Requires: python3-coverage >= 4.0-0.12.b3 Requires: anaconda-tui = %{version}-%{release} # Make sure we get the en locale one way or another -Requires: (glibc-langpack-en or glibc-all-langpacks) +Requires: glibc-langpack-en Obsoletes: anaconda-images <= 10 Provides: anaconda-images = %{version}-%{release} -- To view this commit on github, visit https://github.com/rhinstaller/anaconda/commit/f5dff36501759923de7f23cbc4eb…

1 0

Re: [rhinstaller/anaconda/pulls/575 master] Reset invalid disk selection before proceeding.
by dashea 01 Apr '16

01 Apr '16

Pushed manually as e9d82e38 (master) and af0cfb2e (rhel7-branch). -- To view this pull request on github, visit https://github.com/rhinstaller/anaconda/pull/575

1 0

Re: [rhinstaller/anaconda/pulls/575 master] Reset invalid disk selection before proceeding.
by dashea 01 Apr '16

01 Apr '16

Closed. -- To view this pull request on github, visit https://github.com/rhinstaller/anaconda/pull/575

1 0

Re: [rhinstaller/blivet/pulls/352 2.0-devel] Fix PopulatorMixin.handle_format for anaconda (#1322943)
by dashea 01 Apr '16

01 Apr '16

Closed. -- To view this pull request on github, visit https://github.com/rhinstaller/blivet/pull/352

1 0

[3.0-devel 7/7] Update api to reflect changes from isolating installer-specific code.
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> Remove a few methods and attrs that were removed from Blivet class, and add "mountpoints" attr which was added to DeviceTreeBase. --- doc/api/blivet.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/api/blivet.rst b/doc/api/blivet.rst index 4c2d7c1..e0c547f 100644 --- a/doc/api/blivet.rst +++ b/doc/api/blivet.rst @@ -22,8 +22,6 @@ blivet * :mod:`blivet.blivet` * :class:`~blivet.blivet.Blivet` * :attr:`~blivet.blivet.Blivet.btrfs_volumes` - * :meth:`~blivet.blivet.Blivet.clear_partitions` - * :attr:`~blivet.blivet.Blivet.config` * :meth:`~blivet.blivet.Blivet.copy` * :meth:`~blivet.blivet.Blivet.create_device` * :meth:`~blivet.blivet.Blivet.default_fstype` @@ -67,7 +65,6 @@ blivet * :attr:`~blivet.blivet.Blivet.swaps` * :attr:`~blivet.blivet.Blivet.thinlvs` * :attr:`~blivet.blivet.Blivet.thinpools` - * :meth:`~blivet.blivet.Blivet.update_ksdata` * :attr:`~blivet.blivet.Blivet.vgs` * :mod:`blivet.deviceaction` @@ -120,6 +117,7 @@ blivet * :meth:`~blivet.devicetree.DeviceTreeBase.hide` * :attr:`~blivet.devicetree.DeviceTreeBase.labels` * :attr:`~blivet.devicetree.DeviceTreeBase.leaves` + * :attr:`~blivet.devicetree.DeviceTreeBase.mountpoints` * :meth:`~blivet.populator.populator.PopulatorMixin.populate` * :meth:`~blivet.devicetree.DeviceTreeBase.recursive_remove` * :meth:`~blivet.devicetree.DeviceTreeBase.resolve_device` -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/027af8df65b489fed17844b8351568…

1 0

[3.0-devel 6/7] Update examples and test cases for changes from isolating installer code.
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> --- examples/factory.py | 4 ++-- examples/lvm.py | 2 +- examples/lvm_cache.py | 4 ++-- examples/lvm_non_linear.py | 4 ++-- examples/partitioning.py | 4 ++-- tests/clearpart_test.py | 2 +- tests/imagebackedtestcase.py | 8 ++++---- tests/populator_test.py | 4 ++-- 8 files changed, 16 insertions(+), 16 deletions(-) diff --git a/examples/factory.py b/examples/factory.py index c580494..7c47746 100644 --- a/examples/factory.py +++ b/examples/factory.py @@ -9,9 +9,9 @@ # create two disk image files on which to create new devices disk1_file = create_sparse_tempfile("disk1", Size("100GiB")) -b.config.disk_images["disk1"] = disk1_file +b.disk_images["disk1"] = disk1_file disk2_file = create_sparse_tempfile("disk2", Size("100GiB")) -b.config.disk_images["disk2"] = disk2_file +b.disk_images["disk2"] = disk2_file b.reset() diff --git a/examples/lvm.py b/examples/lvm.py index 1c271f2..ac1f561 100644 --- a/examples/lvm.py +++ b/examples/lvm.py @@ -9,7 +9,7 @@ # create a disk image file on which to create new devices disk1_file = create_sparse_tempfile("disk1", Size("100GiB")) -b.config.disk_images["disk1"] = disk1_file +b.disk_images["disk1"] = disk1_file b.reset() diff --git a/examples/lvm_cache.py b/examples/lvm_cache.py index 6d49097..e8dd062 100644 --- a/examples/lvm_cache.py +++ b/examples/lvm_cache.py @@ -10,9 +10,9 @@ # create a disk image file on which to create new devices disk1_file = create_sparse_tempfile("disk1", Size("100GiB")) -b.config.disk_images["disk1"] = disk1_file +b.disk_images["disk1"] = disk1_file disk2_file = create_sparse_tempfile("disk2", Size("100GiB")) -b.config.disk_images["disk2"] = disk2_file +b.disk_images["disk2"] = disk2_file b.reset() diff --git a/examples/lvm_non_linear.py b/examples/lvm_non_linear.py index 186b523..08a484c 100644 --- a/examples/lvm_non_linear.py +++ b/examples/lvm_non_linear.py @@ -9,9 +9,9 @@ # create a disk image file on which to create new devices disk1_file = create_sparse_tempfile("disk1", Size("100GiB")) -b.config.disk_images["disk1"] = disk1_file +b.disk_images["disk1"] = disk1_file disk2_file = create_sparse_tempfile("disk2", Size("100GiB")) -b.config.disk_images["disk2"] = disk2_file +b.disk_images["disk2"] = disk2_file b.reset() diff --git a/examples/partitioning.py b/examples/partitioning.py index c002c2f..ee53fa8 100644 --- a/examples/partitioning.py +++ b/examples/partitioning.py @@ -9,9 +9,9 @@ # create two disk image files on which to create new devices disk1_file = create_sparse_tempfile("disk1", Size("100GiB")) -b.config.disk_images["disk1"] = disk1_file +b.disk_images["disk1"] = disk1_file disk2_file = create_sparse_tempfile("disk2", Size("100GiB")) -b.config.disk_images["disk2"] = disk2_file +b.disk_images["disk2"] = disk2_file b.reset() diff --git a/tests/clearpart_test.py b/tests/clearpart_test.py index 3ded684..d973f63 100644 --- a/tests/clearpart_test.py +++ b/tests/clearpart_test.py @@ -20,7 +20,7 @@ def setUp(self): def test_should_clear(self): """ Test the Blivet.should_clear method. """ - b = blivet.Blivet() + b = blivet.osinstall.InstallerStorage() DiskDevice = blivet.devices.DiskDevice PartitionDevice = blivet.devices.PartitionDevice diff --git a/tests/imagebackedtestcase.py b/tests/imagebackedtestcase.py index 01e2929..ae668cb 100644 --- a/tests/imagebackedtestcase.py +++ b/tests/imagebackedtestcase.py @@ -2,7 +2,7 @@ import os import unittest -from blivet import Blivet +from blivet.osinstall import InstallerStorage from blivet import util from blivet.size import Size from blivet.flags import flags @@ -44,7 +44,7 @@ def set_up_disks(self): """ for (name, size) in iter(self.disks.items()): path = util.create_sparse_tempfile(name, size) - self.blivet.config.disk_images[name] = path + self.blivet.disk_images[name] = path # # set up the disk images with a disklabel @@ -95,7 +95,7 @@ def set_up_storage(self): def setUp(self): """ Do any setup required prior to running a test. """ flags.image_install = True - self.blivet = Blivet() + self.blivet = InstallerStorage() self.addCleanup(self._clean_up) self.set_up_storage() @@ -104,7 +104,7 @@ def _clean_up(self): """ Clean up any resources that may have been set up for a test. """ self.blivet.reset() self.blivet.devicetree.teardown_disk_images() - for fn in self.blivet.config.disk_images.values(): + for fn in self.blivet.disk_images.values(): if os.path.exists(fn): os.unlink(fn) diff --git a/tests/populator_test.py b/tests/populator_test.py index 8975b0f..ccd4974 100644 --- a/tests/populator_test.py +++ b/tests/populator_test.py @@ -48,7 +48,7 @@ def setUp(self): # anaconda first configures disk images for (name, size) in iter(self.disks.items()): path = util.create_sparse_tempfile(name, size) - self.blivet.config.disk_images[name] = path + self.blivet.disk_images[name] = path # at this point the DMLinearDevice has correct size self.blivet.setup_disk_images() @@ -67,7 +67,7 @@ def setUp(self): def tearDown(self): self.blivet.reset() self.blivet.devicetree.teardown_disk_images() - for fn in self.blivet.config.disk_images.values(): + for fn in self.blivet.disk_images.values(): if os.path.exists(fn): os.unlink(fn) -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/a28ac859345919d00f2864d7a8c928…

1 0

[3.0-devel 5/7] Get rid of blivet.blivet's last deps on blivet.osinstall
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> The two dependencies we're eliminating here: blivet.osinstall.StorageDiscoveryConfig blivet.osinstall.find_existing_installations This has the added benefit of eliminating depedence on pykickstart.constants. Unfortunately this means changing a lot of stuff all at once, but these are mostly moves without any changes. Stuff that's now out of blivet.blivet.Blivet and now in blivet.osinstall.InstallerStorage: _free_space_snapshot (attr) free_space_snapshot (prop) create_free_space_snapshot Things which were changed in blivet.blivet.Blivet to eliminate dependence on anything in blivet.osinstall: reset get_free_space Those two methods exist as they originally were in blivet.osinstall.InstallerStorage now. InstallerStorage also handles protected devices now as well, so all of that logic has been moved out of blivet.populator.populator and into this class. Namely, three functions have been moved to InstallerStorage: _resolve_protected_device_specs _find_live_backing_device _mark_protected_device iSCSI, FCoE, and zFCP startup/config writes now take place in InstallerStorage. --- blivet/autopart.py | 4 +- blivet/blivet.py | 77 ++++--------------------------- blivet/osinstall.py | 104 +++++++++++++++++++++++++++++++++++++++++- blivet/populator/populator.py | 46 ------------------- 4 files changed, 114 insertions(+), 117 deletions(-) diff --git a/blivet/autopart.py b/blivet/autopart.py index a75c567..347c1bc 100644 --- a/blivet/autopart.py +++ b/blivet/autopart.py @@ -110,8 +110,8 @@ def _get_candidate_disks(storage): for a default-sized (500MiB) partition. They must also be in :attr:`StorageDiscoveryConfig.clear_part_disks` if it is non-empty. - :param storage: a Blivet instance - :type storage: :class:`~.Blivet` + :param storage: an InstallerStorage instance + :type storage: :class:`~.osinstall.InstallerStorage` :returns: a list of partitioned disks with at least 500MiB of free space :rtype: list of :class:`~.devices.StorageDevice` """ diff --git a/blivet/blivet.py b/blivet/blivet.py index 0fe0721..401056a 100644 --- a/blivet/blivet.py +++ b/blivet/blivet.py @@ -43,11 +43,7 @@ from .flags import flags from .platform import platform as _platform from .formats import get_format -from .osinstall import StorageDiscoveryConfig find_existing_installations from . import arch -from .iscsi import iscsi -from .fcoe import fcoe -from .zfcp import zfcp from . import devicefactory from . import get_bootloader, get_sysroot, short_product_name, __version__ from .threads import SynchronizedMeta @@ -68,8 +64,6 @@ def __init__(self, ksdata=None): self.ksdata = ksdata self._bootloader = None - self.config = StorageDiscoveryConfig() - # storage configuration variables self.do_autopart = False self.clear_part_choice = None @@ -102,7 +96,6 @@ def __init__(self, ksdata=None): disk_images=self.disk_images) self.roots = [] self.services = set() - self._free_space_snapshot = None def do_it(self, callbacks=None): """ @@ -145,20 +138,6 @@ def reset(self, cleanup_only=False): about the cleanup_only keyword argument. """ log.info("resetting Blivet (version %s) instance %s", __version__, self) - if flags.installer_mode: - # save passphrases for luks devices so we don't have to reprompt - self.encryption_passphrase = None - for device in self.devices: - if device.format.type == "luks" and device.format.exists: - self.save_passphrase(device) - - if self.ksdata: - self.config.update(self.ksdata) - - if flags.installer_mode and not flags.image_install: - iscsi.startup() - fcoe.startup() - zfcp.startup() self.devicetree.reset(passphrase=self.encryption_passphrase, luks_dict=self.__luks_devs, @@ -173,14 +152,6 @@ def reset(self, cleanup_only=False): # no longer in the tree self.bootloader.reset() - self.roots = [] - if flags.installer_mode: - self.roots = find_existing_installations(self.devicetree) - self.dump_state("initial") - - if not flags.installer_mode: - self.devicetree.handle_nodev_filesystems() - self.update_bootloader_disk_list() @property @@ -431,21 +402,20 @@ def remove_empty_extended_partitions(self): extended = self.devicetree.get_device_by_name(extended_name) self.destroy_device(extended) - def get_free_space(self, disks=None, clear_part_type=None): + def get_free_space(self, disks=None, partitions=None): """ Return a dict with free space info for each disk. The dict values are 2-tuples: (disk_free, fs_free). fs_free is space available by shrinking filesystems. disk_free is space not allocated to any partition. - disks and clear_part_type allow specifying a set of disks other than - self.disks and a clear_part_type value other than - self.config.clear_part_type. + disks and partitions allow specifying a set of disks other than + self.disks and partition values other than self.parttions. :keyword disks: overrides :attr:`disks` :type disks: list - :keyword clear_part_type: overrides :attr:`self.config.clear_part_type` - :type clear_part_type: int + :keyword partitions: overrides :attr:`partitions` + :type partitions: list :returns: dict with disk name keys and tuple (disk, fs) free values :rtype: dict @@ -457,31 +427,17 @@ def get_free_space(self, disks=None, clear_part_type=None): if disks is None: disks = self.disks - if clear_part_type is None: - clear_part_type = self.config.clear_part_type + if partitions is None: + partitions = self.partitions free = {} for disk in disks: - should_clear = self.should_clear(disk, clear_part_type=clear_part_type, - clear_part_disks=[disk.name]) - if should_clear: - free[disk.name] = (disk.size, Size(0)) - continue - disk_free = Size(0) fs_free = Size(0) if disk.partitioned: disk_free = disk.format.free - for partition in [p for p in self.partitions if p.disk == disk]: - # only check actual filesystems since lvm &c require a bunch of - # operations to translate free filesystem space into free disk - # space - should_clear = self.should_clear(partition, - clear_part_type=clear_part_type, - clear_part_disks=[disk.name]) - if should_clear: - disk_free += partition.size - elif hasattr(partition.format, "free"): + for partition in (p for p in partitions if p.disk == disk): + if hasattr(partition.format, "free"): fs_free += partition.format.free elif hasattr(disk.format, "free"): fs_free = disk.format.free @@ -1061,9 +1017,6 @@ def write(self): if not os.path.isdir("%s/etc" % get_sysroot()): os.mkdir("%s/etc" % get_sysroot()) - iscsi.write(get_sysroot(), self) - fcoe.write(get_sysroot()) - zfcp.write(get_sysroot()) self.write_dasd_conf(get_sysroot()) def write_dasd_conf(self, root): @@ -1363,15 +1316,3 @@ def copy(self): log.debug("finished Blivet copy") return new - - @property - def free_space_snapshot(self): - # if no snapshot is available, do it now and return it - self._free_space_snapshot = self._free_space_snapshot or self.get_free_space() - - return self._free_space_snapshot - - def create_free_space_snapshot(self): - self._free_space_snapshot = self.get_free_space() - - return self._free_space_snapshot diff --git a/blivet/osinstall.py b/blivet/osinstall.py index 6b47c04..7f561d5 100644 --- a/blivet/osinstall.py +++ b/blivet/osinstall.py @@ -33,7 +33,7 @@ from pykickstart.constants import AUTOPART_TYPE_LVM, CLEARPART_TYPE_NONE, CLEARPART_TYPE_LINUX, CLEARPART_TYPE_ALL, CLEARPART_TYPE_LIST -from . import util +from . import util, __version__ from . import get_sysroot, get_target_physical_root, error_handler, ERROR_RAISE from .blivet import Blivet @@ -43,6 +43,9 @@ from .formats import get_device_format_class from .formats import get_format from .flags import flags +from .iscsi import iscsi +from .fcoe import fcoe +from .zfcp import zfcp from .platform import platform as _platform from .platform import EFI from .size import Size @@ -1110,6 +1113,7 @@ def __init__(self, ksdata=None): self.__luks_devs = {} self.fsset = FSSet(self.devicetree) self._free_space_snapshot = None + self.live_backing_device = None def do_it(self, callbacks=None): """ @@ -1185,6 +1189,9 @@ def write(self): self.make_mtab() self.fsset.write() + iscsi.write(get_sysroot(), self) + fcoe.write(get_sysroot()) + zfcp.write(get_sysroot()) def mountpoints(self): return self.fsset.mountpoints @@ -1219,6 +1226,18 @@ def file_system_free_space(self): return free + @property + def free_space_snapshot(self): + # if no snapshot is available, do it now and return it + self._free_space_snapshot = self._free_space_snapshot or self.get_free_space() + + return self._free_space_snapshot + + def create_free_space_snapshot(self): + self._free_space_snapshot = self.get_free_space() + + return self._free_space_snapshot + def update_ksdata(self): """ Update ksdata to reflect the settings of this Blivet instance. """ if not self.ksdata or not self.mountpoints: @@ -1340,10 +1359,93 @@ def _update_custom_storage_ksdata(self): self.fsset = FSSet(self.devicetree) def reset(self, cleanup_only=False): + """ Reset storage configuration to reflect actual system state. + + This will cancel any queued actions and rescan from scratch but not + clobber user-obtained information like passphrases, iscsi config, &c + + :keyword cleanup_only: prepare the tree only to deactivate devices + :type cleanup_only: bool + + See :meth:`devicetree.Devicetree.populate` for more information + about the cleanup_only keyword argument. + """ + log.info("resetting Blivet (version %s) instance %s", __version__, self) + if flags.installer_mode: + # save passphrases for luks devices so we don't have to reprompt + self.encryption_passphrase = None + for device in self.devices: + if device.format.type == "luks" and device.format.exists: + self.save_passphrase(device) + + if self.ksdata: + self.config.update(self.ksdata) + + if flags.installer_mode and not flags.image_install: + iscsi.startup() + fcoe.startup() + zfcp.startup() + super().reset(cleanup_only=cleanup_only) self.fsset = FSSet(self.devicetree) + # protected device handling + self.protected_dev_names = [] + self._resolve_protected_device_specs() + self._find_live_backing_device() + for devname in self.protected_dev_names: + dev = self.devicetree.get_device_by_name(devname) + self._mark_protected_device(dev) + + self.roots = [] + if flags.installer_mode: + self.roots = find_existing_installations(self.devicetree) + self.dump_state("initial") + + if not flags.installer_mode: + self.devicetree.handle_nodev_filesystems() + + def _resolve_protected_device_specs(self): + """ Resolve the protected device specs to device names. """ + for spec in self.config.protected_dev_specs: + name = self.devicetree.resolve_device(spec) + log.debug("protected device spec %s resolved to %s", spec, name) + if name: + self.protected_dev_names.append(name) + + def _find_live_backing_device(self): + # FIXME: the backing dev for the live image can't be used as an + # install target. note that this is a little bit of a hack + # since we're assuming that /run/initramfs/live will exist + for mnt in open("/proc/mounts").readlines(): + if " /run/initramfs/live " not in mnt: + continue + + live_device_name = mnt.split()[0].split("/")[-1] + log.info("%s looks to be the live device; marking as protected", + live_device_name) + self.protected_dev_names.append(live_device_name) + self.live_backing_device = live_device_name + break + + def _mark_protected_device(self, device): + """ + If this device is protected, mark it as such now. Once the tree + has been populated, devices' protected attribute is how we will + identify protected devices. + + :param :class: `blivet.devices.storage.StorageDevice` device: device to + mark as protected + """ + if device.name in self.protected_dev_names: + device.protected = True + # if this is the live backing device we want to mark its parents + # as protected also + if device.name == self.live_backing_device: + for parent in device.parents: + parent.protected = True + def empty_device(self, device): empty = True if device.partitioned: diff --git a/blivet/populator/populator.py b/blivet/populator/populator.py index 6d4eedf..f4ec990 100644 --- a/blivet/populator/populator.py +++ b/blivet/populator/populator.py @@ -80,13 +80,6 @@ def reset(self, passphrase=None, luks_dict=None, disk_images=None): # this will overwrite self.exclusive_disks self.set_disk_images(disk_images) - # protected device specs as provided by the user - self.protected_dev_specs = getattr(conf, "protected_dev_specs", []) - self.live_backing_device = None - - # names of protected devices at the time of tree population - self.protected_dev_names = [] - self.__passphrases = [] if passphrase: self.__passphrases.append(passphrase) @@ -245,18 +238,6 @@ def _mark_readonly_device(self, info, device): util.get_sysfs_attr(udev.device_get_sysfs_path(info), 'ro') == '1': device.readonly = True - def _mark_protected_device(self, device): - # If this device is protected, mark it as such now. Once the tree - # has been populated, devices' protected attribute is how we will - # identify protected devices. - if device.name in self.protected_dev_names: - device.protected = True - # if this is the live backing device we want to mark its parents - # as protected also - if device.name == self.live_backing_device: - for parent in device.parents: - parent.protected = True - def _update_exclusive_disks(self, device): # If we just added a multipath or fwraid disk that is in exclusive_disks # we have to make sure all of its members are in the list too. @@ -316,7 +297,6 @@ def handle_device(self, info, update_orig_fmt=False): log.info("got device: %r", device) self._mark_readonly_device(info, device) - self._mark_protected_device(device) self._update_exclusive_disks(device) # now handle the device's formatting @@ -466,29 +446,6 @@ def populate(self, cleanup_only=False): if flags.installer_mode: self.teardown_all() - def _resolve_protected_device_specs(self): - # resolve the protected device specs to device names - for spec in self.protected_dev_specs: - name = udev.resolve_devspec(spec) - log.debug("protected device spec %s resolved to %s", spec, name) - if name: - self.protected_dev_names.append(name) - - def _find_live_backing_device(self): - # FIXME: the backing dev for the live image can't be used as an - # install target. note that this is a little bit of a hack - # since we're assuming that /run/initramfs/live will exist - for mnt in open("/proc/mounts").readlines(): - if " /run/initramfs/live " not in mnt: - continue - - live_device_name = mnt.split()[0].split("/")[-1] - log.info("%s looks to be the live device; marking as protected", - live_device_name) - self.protected_dev_names.append(live_device_name) - self.live_backing_device = live_device_name - break - def _populate(self): log.info("DeviceTree.populate: ignored_disks is %s ; exclusive_disks is %s", self.ignored_disks, self.exclusive_disks) @@ -500,9 +457,6 @@ def _populate(self): self.setup_disk_images() - self._resolve_protected_device_specs() - self._find_live_backing_device() - old_devices = {} # Now, loop and scan for devices that have appeared since the two above -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/bc57c9a3a4187d760b3a84ba209814…

1 0

[3.0-devel 4/7] Get rid of blivet.blivet's dependence on blivet.osinstall.FSSet
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> All of the FSSet stuff pertains to installation, so move everything from Blivet to InstallerStorage subclass. The methods and properties moved are: file_system_free_space root_device (prop) update_ksdata _update_custom_storage_ksdata These now only exist in InstallerStorage. This also creates the "mountpoints" property in blivet.devicetree This property fits pretty nicely in conjunction with other properties living inside of the devicetree. It also simplifies things in blivet.osinstall.FSSet. --- blivet/blivet.py | 166 ++------------------------------------------------ blivet/devicetree.py | 9 +++ blivet/osinstall.py | 169 ++++++++++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 177 insertions(+), 167 deletions(-) diff --git a/blivet/blivet.py b/blivet/blivet.py index 7f795b0..0fe0721 100644 --- a/blivet/blivet.py +++ b/blivet/blivet.py @@ -27,10 +27,8 @@ import time import functools -from pykickstart.constants import CLEARPART_TYPE_ALL, CLEARPART_TYPE_LIST, CLEARPART_TYPE_NONE - from .storage_log import log_method_call, log_exception_info -from .devices import BTRFSDevice, BTRFSSubVolumeDevice, BTRFSVolumeDevice +from .devices import BTRFSSubVolumeDevice, BTRFSVolumeDevice from .devices import LVMLogicalVolumeDevice, LVMVolumeGroupDevice from .devices import MDRaidArrayDevice, PartitionDevice, TmpFSDevice, device_path_to_name from .deviceaction import ActionCreateDevice, ActionCreateFormat, ActionDestroyDevice @@ -45,7 +43,7 @@ from .flags import flags from .platform import platform as _platform from .formats import get_format -from .osinstall import FSSet, StorageDiscoveryConfig find_existing_installations +from .osinstall import StorageDiscoveryConfig find_existing_installations from . import arch from .iscsi import iscsi from .fcoe import fcoe @@ -54,8 +52,6 @@ from . import get_bootloader, get_sysroot, short_product_name, __version__ from .threads import SynchronizedMeta -from .i18n import _ - import logging log = logging.getLogger("blivet") @@ -104,7 +100,6 @@ def __init__(self, ksdata=None): ignored_disks=self.ignored_disks, exclusive_disks=self.exclusive_disks, disk_images=self.disk_images) - self.fsset = FSSet(self.devicetree) self.roots = [] self.services = set() self._free_space_snapshot = None @@ -171,7 +166,6 @@ def reset(self, cleanup_only=False): exclusive_disks=self.exclusive_disks, disk_images=self.disk_images) self.devicetree.populate(cleanup_only=cleanup_only) - self.fsset = FSSet(self.devicetree) self.edd_dict = get_edd_dict(self.partitioned) self.devicetree.edd_dict = self.edd_dict if self.bootloader: @@ -1041,32 +1035,6 @@ def setup_disk_images(self): self.devicetree.set_disk_images(self.disk_images) self.devicetree.setup_disk_images() - @property - def file_system_free_space(self): - """ Combined free space in / and /usr as :class:`~.size.Size`. """ - mountpoints = ["/", "/usr"] - free = Size(0) - btrfs_volumes = [] - for mountpoint in mountpoints: - device = self.mountpoints.get(mountpoint) - if not device: - continue - - # don't count the size of btrfs volumes repeatedly when multiple - # subvolumes are present - if isinstance(device, BTRFSSubVolumeDevice): - if device.volume in btrfs_volumes: - continue - else: - btrfs_volumes.append(device.volume) - - if device.format.exists: - free += device.format.free - else: - free += device.format.free_space_estimate(device.size) - - return free - def dump_state(self, suffix): """ Dump the current device list to the storage shelf. """ key = "devices.%d.%s" % (time.time(), suffix) @@ -1093,7 +1061,6 @@ def write(self): if not os.path.isdir("%s/etc" % get_sysroot()): os.mkdir("%s/etc" % get_sysroot()) - self.fsset.write() iscsi.write(get_sysroot(), self) fcoe.write(get_sysroot()) zfcp.write(get_sysroot()) @@ -1165,8 +1132,9 @@ def boot_disk(self): @property def boot_device(self): dev = None - if self.fsset: - dev = self.mountpoints.get("/boot", self.root_device) + root_device = self.mountpoints.get("/") + + dev = self.mountpoints.get("/boot", root_device) return dev @property @@ -1234,11 +1202,7 @@ def set_default_fstype(self, newtype): @property def mountpoints(self): - return self.fsset.mountpoints - - @property - def root_device(self): - return self.fsset.root_device + return self.devicetree.mountpoints def compare_disks(self, first, second): if not isinstance(first, str): @@ -1400,124 +1364,6 @@ def copy(self): log.debug("finished Blivet copy") return new - def update_ksdata(self): - """ Update ksdata to reflect the settings of this Blivet instance. """ - if not self.ksdata or not self.mountpoints: - return - - # clear out whatever was there before - self.ksdata.partition.partitions = [] - self.ksdata.logvol.lvList = [] - self.ksdata.raid.raidList = [] - self.ksdata.volgroup.vgList = [] - self.ksdata.btrfs.btrfsList = [] - - # iscsi? - # fcoe? - # zfcp? - # dmraid? - - # bootloader - - # ignoredisk - if self.ignored_disks: - self.ksdata.ignoredisk.drives = self.ignored_disks[:] - elif self.exclusive_disks: - self.ksdata.ignoredisk.onlyuse = self.exclusive_disks[:] - - # autopart - self.ksdata.autopart.autopart = self.do_autopart - self.ksdata.autopart.type = self.autopart_type - self.ksdata.autopart.encrypted = self.encrypted_autopart - - # clearpart - self.ksdata.clearpart.type = self.config.clear_part_type - self.ksdata.clearpart.drives = self.config.clear_part_disks[:] - self.ksdata.clearpart.devices = self.config.clear_part_devices[:] - self.ksdata.clearpart.initAll = self.config.initialize_disks - if self.ksdata.clearpart.type == CLEARPART_TYPE_NONE: - # Make a list of initialized disks and of removed partitions. If any - # partitions were removed from disks that were not completely - # cleared we'll have to use CLEARPART_TYPE_LIST and provide a list - # of all removed partitions. If no partitions were removed from a - # disk that was not cleared/reinitialized we can use - # CLEARPART_TYPE_ALL. - self.ksdata.clearpart.devices = [] - self.ksdata.clearpart.drives = [] - fresh_disks = [d.name for d in self.disks if d.partitioned and - not d.format.exists] - - destroy_actions = self.devicetree.actions.find(action_type="destroy", - object_type="device") - - cleared_partitions = [] - partial = False - for action in destroy_actions: - if action.device.type == "partition": - if action.device.disk.name not in fresh_disks: - partial = True - - cleared_partitions.append(action.device.name) - - if not destroy_actions: - pass - elif partial: - # make a list of removed partitions - self.ksdata.clearpart.type = CLEARPART_TYPE_LIST - self.ksdata.clearpart.devices = cleared_partitions - else: - # if they didn't partially clear any disks, use the shorthand - self.ksdata.clearpart.type = CLEARPART_TYPE_ALL - self.ksdata.clearpart.drives = fresh_disks - - if self.do_autopart: - return - - self._update_custom_storage_ksdata() - - def _update_custom_storage_ksdata(self): - """ Update KSData for custom storage. """ - - # custom storage - ks_map = {PartitionDevice: ("PartData", "partition"), - TmpFSDevice: ("PartData", "partition"), - LVMLogicalVolumeDevice: ("LogVolData", "logvol"), - LVMVolumeGroupDevice: ("VolGroupData", "volgroup"), - MDRaidArrayDevice: ("RaidData", "raid"), - BTRFSDevice: ("BTRFSData", "btrfs")} - - # make a list of ancestors of all used devices - devices = list(set(a for d in list(self.mountpoints.values()) + self.swaps - for a in d.ancestors)) - - # devices which share information with their distinct raw device - complementary_devices = [d for d in devices if d.raw_device is not d] - - devices.sort(key=lambda d: len(d.ancestors)) - for device in devices: - cls = next((c for c in ks_map if isinstance(device, c)), None) - if cls is None: - log.info("omitting ksdata: %s", device) - continue - - class_attr, list_attr = ks_map[cls] - - cls = getattr(self.ksdata, class_attr) - data = cls() # all defaults - - complements = [d for d in complementary_devices if d.raw_device is device] - - if len(complements) > 1: - log.warning("omitting ksdata for %s, found too many (%d) complementary devices", device, len(complements)) - continue - - device = complements[0] if complements else device - - device.populate_ksdata(data) - - parent = getattr(self.ksdata, list_attr) - parent.dataList().append(data) - @property def free_space_snapshot(self): # if no snapshot is available, do it now and return it diff --git a/blivet/devicetree.py b/blivet/devicetree.py index c8db437..fabdc70 100644 --- a/blivet/devicetree.py +++ b/blivet/devicetree.py @@ -778,6 +778,15 @@ def labels(self): return labels + @property + def mountpoints(self): + """ Dict with mountpoint keys and Device values. """ + filesystems = {} + for device in self.devices: + if device.format.mountable and device.format.mountpoint: + filesystems[device.format.mountpoint] = device + return filesystems + # # Disk filter # diff --git a/blivet/osinstall.py b/blivet/osinstall.py index 463d138..6b47c04 100644 --- a/blivet/osinstall.py +++ b/blivet/osinstall.py @@ -38,20 +38,20 @@ from .blivet import Blivet from .storage_log import log_exception_info -from .devices import FileDevice, NFSDevice, NoDevice, OpticalDevice, NetworkStorageDevice, DirectoryDevice, MDRaidArrayDevice, PartitionDevice +from .devices import FileDevice, NFSDevice, NoDevice, OpticalDevice, NetworkStorageDevice, DirectoryDevice, MDRaidArrayDevice, PartitionDevice, BTRFSSubVolumeDevice, TmpFSDevice, LVMLogicalVolumeDevice, LVMVolumeGroupDevice, BTRFSDevice from .errors import FSTabTypeMismatchError, UnrecognizedFSTabEntryError, StorageError, FSResizeError, FormatResizeError, UnknownSourceDeviceError from .formats import get_device_format_class from .formats import get_format from .flags import flags from .platform import platform as _platform from .platform import EFI +from .size import Size from .i18n import _ import logging log = logging.getLogger("blivet") - def release_from_redhat_release(fn): """ Attempt to identify the installation of a Linux distribution via @@ -402,11 +402,7 @@ def devices(self): @property def mountpoints(self): - filesystems = {} - for device in self.devices: - if device.format.mountable and device.format.mountpoint: - filesystems[device.format.mountpoint] = device - return filesystems + return self.devicetree.mountpoints def _parse_one_line(self, devspec, mountpoint, fstype, options, _dump="0", _passno="0"): """Parse an fstab entry for a device, return the corresponding device. @@ -1188,6 +1184,165 @@ def write(self): Blivet.write(self) self.make_mtab() + self.fsset.write() + + def mountpoints(self): + return self.fsset.mountpoints + + @property + def root_device(self): + return self.fsset.root_device + + @property + def file_system_free_space(self): + """ Combined free space in / and /usr as :class:`~.size.Size`. """ + mountpoints = ["/", "/usr"] + free = Size(0) + btrfs_volumes = [] + for mountpoint in mountpoints: + device = self.mountpoints.get(mountpoint) + if not device: + continue + + # don't count the size of btrfs volumes repeatedly when multiple + # subvolumes are present + if isinstance(device, BTRFSSubVolumeDevice): + if device.volume in btrfs_volumes: + continue + else: + btrfs_volumes.append(device.volume) + + if device.format.exists: + free += device.format.free + else: + free += device.format.free_space_estimate(device.size) + + return free + + def update_ksdata(self): + """ Update ksdata to reflect the settings of this Blivet instance. """ + if not self.ksdata or not self.mountpoints: + return + + # clear out whatever was there before + self.ksdata.partition.partitions = [] + self.ksdata.logvol.lvList = [] + self.ksdata.raid.raidList = [] + self.ksdata.volgroup.vgList = [] + self.ksdata.btrfs.btrfsList = [] + + # iscsi? + # fcoe? + # zfcp? + # dmraid? + + # bootloader + + # ignoredisk + if self.ignored_disks: + self.ksdata.ignoredisk.drives = self.ignored_disks[:] + elif self.exclusive_disks: + self.ksdata.ignoredisk.onlyuse = self.exclusive_disks[:] + + # autopart + self.ksdata.autopart.autopart = self.do_autopart + self.ksdata.autopart.type = self.autopart_type + self.ksdata.autopart.encrypted = self.encrypted_autopart + + # clearpart + self.ksdata.clearpart.type = self.config.clear_part_type + self.ksdata.clearpart.drives = self.config.clear_part_disks[:] + self.ksdata.clearpart.devices = self.config.clear_part_devices[:] + self.ksdata.clearpart.initAll = self.config.initialize_disks + if self.ksdata.clearpart.type == CLEARPART_TYPE_NONE: + # Make a list of initialized disks and of removed partitions. If any + # partitions were removed from disks that were not completely + # cleared we'll have to use CLEARPART_TYPE_LIST and provide a list + # of all removed partitions. If no partitions were removed from a + # disk that was not cleared/reinitialized we can use + # CLEARPART_TYPE_ALL. + self.ksdata.clearpart.devices = [] + self.ksdata.clearpart.drives = [] + fresh_disks = [d.name for d in self.disks if d.partitioned and + not d.format.exists] + + destroy_actions = self.devicetree.actions.find(action_type="destroy", + object_type="device") + + cleared_partitions = [] + partial = False + for action in destroy_actions: + if action.device.type == "partition": + if action.device.disk.name not in fresh_disks: + partial = True + + cleared_partitions.append(action.device.name) + + if not destroy_actions: + pass + elif partial: + # make a list of removed partitions + self.ksdata.clearpart.type = CLEARPART_TYPE_LIST + self.ksdata.clearpart.devices = cleared_partitions + else: + # if they didn't partially clear any disks, use the shorthand + self.ksdata.clearpart.type = CLEARPART_TYPE_ALL + self.ksdata.clearpart.drives = fresh_disks + + if self.do_autopart: + return + + self._update_custom_storage_ksdata() + + def _update_custom_storage_ksdata(self): + """ Update KSData for custom storage. """ + + # custom storage + ks_map = {PartitionDevice: ("PartData", "partition"), + TmpFSDevice: ("PartData", "partition"), + LVMLogicalVolumeDevice: ("LogVolData", "logvol"), + LVMVolumeGroupDevice: ("VolGroupData", "volgroup"), + MDRaidArrayDevice: ("RaidData", "raid"), + BTRFSDevice: ("BTRFSData", "btrfs")} + + # make a list of ancestors of all used devices + devices = list(set(a for d in list(self.mountpoints.values()) + self.swaps + for a in d.ancestors)) + + # devices which share information with their distinct raw device + complementary_devices = [d for d in devices if d.raw_device is not d] + + devices.sort(key=lambda d: len(d.ancestors)) + for device in devices: + cls = next((c for c in ks_map if isinstance(device, c)), None) + if cls is None: + log.info("omitting ksdata: %s", device) + continue + + class_attr, list_attr = ks_map[cls] + + cls = getattr(self.ksdata, class_attr) + data = cls() # all defaults + + complements = [d for d in complementary_devices if d.raw_device is device] + + if len(complements) > 1: + log.warning("omitting ksdata for %s, found too many (%d) complementary devices", device, len(complements)) + continue + + device = complements[0] if complements else device + + device.populate_ksdata(data) + + parent = getattr(self.ksdata, list_attr) + parent.dataList().append(data) + + self.fsset = FSSet(self.devicetree) + + def reset(self, cleanup_only=False): + super().reset(cleanup_only=cleanup_only) + + self.fsset = FSSet(self.devicetree) def empty_device(self, device): empty = True -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/61fadc2ef88b36e399608243056b12…

1 0

[3.0-devel 3/7] Mv installer-specific functions to install. Create InstallerStorage.
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> Two major things happen here: * Create InstallerStorage is created in blivet.osinstall This is a subclass of blivet.blivet.Blivet, except it also contains some installer-specific attrs and functions. Speaking of which-- * Mv installer-specific functions and properties from Blivet -> InstallerStorage. That list is: empty_devices unused_devices (prop) should_clear clear_partitions format_by_default must_format turn_on_swap mount_filesystems umount_filesystems parse_fstab mk_dev_root create_swap_file make_mtab add_fstab_swap remove_fstab_swap set_fstab_swap autopart_type (attr) These were all removed from Blivet and are now in InstallerStorage. Blivet.do_it was very stripped down; all the installer-specific logic was moved into InstallerStorage.do_it. --- blivet/blivet.py | 337 +---------------------------------------------- blivet/osinstall.py | 371 +++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 371 insertions(+), 337 deletions(-) diff --git a/blivet/blivet.py b/blivet/blivet.py index 95fc2de..7f795b0 100644 --- a/blivet/blivet.py +++ b/blivet/blivet.py @@ -25,10 +25,9 @@ import shelve import contextlib import time -import parted import functools -from pykickstart.constants import AUTOPART_TYPE_LVM, CLEARPART_TYPE_ALL, CLEARPART_TYPE_LINUX, CLEARPART_TYPE_LIST, CLEARPART_TYPE_NONE +from pykickstart.constants import CLEARPART_TYPE_ALL, CLEARPART_TYPE_LIST, CLEARPART_TYPE_NONE from .storage_log import log_method_call, log_exception_info from .devices import BTRFSDevice, BTRFSSubVolumeDevice, BTRFSVolumeDevice @@ -61,17 +60,6 @@ log = logging.getLogger("blivet") -def empty_device(device): - empty = True - if device.partitioned: - partitions = device.children - empty = all([p.is_magic for p in partitions]) - else: - empty = (device.format.type is None) - - return empty - - class Blivet(object, metaclass=SynchronizedMeta): """ Top-level class for managing storage configuration. """ @@ -90,7 +78,6 @@ def __init__(self, ksdata=None): self.do_autopart = False self.clear_part_choice = None self.encrypted_autopart = False - self.autopart_type = AUTOPART_TYPE_LVM self.encryption_passphrase = None self.encryption_cipher = None self.escrow_certificates = {} @@ -132,64 +119,6 @@ def do_it(self, callbacks=None): """ self.devicetree.actions.process(callbacks=callbacks, devices=self.devices) - if not flags.installer_mode: - return - - # now set the boot partition's flag - if self.bootloader and not self.bootloader.skip_bootloader: - if self.bootloader.stage2_bootable: - boot = self.boot_device - else: - boot = self.bootloader_device - - if boot.type == "mdarray": - boot_devs = boot.parents - else: - boot_devs = [boot] - - for dev in boot_devs: - if not hasattr(dev, "bootable"): - log.info("Skipping %s, not bootable", dev) - continue - - # Dos labels can only have one partition marked as active - # and unmarking ie the windows partition is not a good idea - skip = False - if dev.disk.format.parted_disk.type == "msdos": - for p in dev.disk.format.parted_disk.partitions: - if p.type == parted.PARTITION_NORMAL and \ - p.getFlag(parted.PARTITION_BOOT): - skip = True - break - - # GPT labeled disks should only have bootable set on the - # EFI system partition (parted sets the EFI System GUID on - # GPT partitions with the boot flag) - if dev.disk.format.label_type == "gpt" and \ - dev.format.type not in ["efi", "macefi"]: - skip = True - - if skip: - log.info("Skipping %s", dev.name) - continue - - # hfs+ partitions on gpt can't be marked bootable via parted - if dev.disk.format.parted_disk.type != "gpt" or \ - dev.format.type not in ["hfs+", "macefi"]: - log.info("setting boot flag on %s", dev.name) - dev.bootable = True - - # Set the boot partition's name on disk labels that support it - if dev.parted_partition.disk.supportsFeature(parted.DISK_TYPE_PARTITION_NAME): - ped_partition = dev.parted_partition.getPedPartition() - ped_partition.setName(dev.format.name) - log.info("Setting label on %s to '%s'", dev, dev.format.name) - - dev.disk.setup() - dev.disk.format.commit_to_disk() - - if flags.installer_mode: - self.dump_state("final") @property def next_id(self): @@ -261,31 +190,6 @@ def reset(self, cleanup_only=False): self.update_bootloader_disk_list() @property - def unused_devices(self): - used_devices = [] - for root in self.roots: - for device in list(root.mounts.values()) + root.swaps: - if device not in self.devices: - continue - - used_devices.extend(device.ancestors) - - for new in [d for d in self.devicetree.leaves if not d.format.exists]: - if new.format.mountable and not new.format.mountpoint: - continue - - used_devices.extend(new.ancestors) - - for device in self.partitions: - if getattr(device, "is_logical", False): - extended = device.disk.format.extended_partition.path - used_devices.append(self.devicetree.get_device_by_path(extended)) - - used = set(used_devices) - _all = set(self.devices) - return list(_all.difference(used)) - - @property def devices(self): """ A list of all the devices in the device tree. """ devices = self.devicetree.devices @@ -467,100 +371,6 @@ def swaps(self): swaps.sort(key=lambda d: d.name) return swaps - def should_clear(self, device, **kwargs): - """ Return True if a clearpart settings say a device should be cleared. - - :param device: the device (required) - :type device: :class:`~.devices.StorageDevice` - :keyword clear_part_type: overrides :attr:`self.config.clear_part_type` - :type clear_part_type: int - :keyword clear_part_disks: overrides - :attr:`self.config.clear_part_disks` - :type clear_part_disks: list - :keyword clear_part_devices: overrides - :attr:`self.config.clear_part_devices` - :type clear_part_devices: list - :returns: whether or not clear_partitions should remove this device - :rtype: bool - """ - clear_part_type = kwargs.get("clear_part_type", self.config.clear_part_type) - clear_part_disks = kwargs.get("clear_part_disks", - self.config.clear_part_disks) - clear_part_devices = kwargs.get("clear_part_devices", - self.config.clear_part_devices) - - for disk in device.disks: - # this will not include disks with hidden formats like multipath - # and firmware raid member disks - if clear_part_disks and disk.name not in clear_part_disks: - return False - - if not self.config.clear_non_existent: - if (device.is_disk and not device.format.exists) or \ - (not device.is_disk and not device.exists): - return False - - # the only devices we want to clear when clear_part_type is - # CLEARPART_TYPE_NONE are uninitialized disks, or disks with no - # partitions, in clear_part_disks, and then only when we have been asked - # to initialize disks as needed - if clear_part_type in [CLEARPART_TYPE_NONE, None]: - if not self.config.initialize_disks or not device.is_disk: - return False - - if not empty_device(device): - return False - - if isinstance(device, PartitionDevice): - # Never clear the special first partition on a Mac disk label, as - # that holds the partition table itself. - # Something similar for the third partition on a Sun disklabel. - if device.is_magic: - return False - - # We don't want to fool with extended partitions, freespace, &c - if not device.is_primary and not device.is_logical: - return False - - if clear_part_type == CLEARPART_TYPE_LINUX and \ - not device.format.linux_native and \ - not device.get_flag(parted.PARTITION_LVM) and \ - not device.get_flag(parted.PARTITION_RAID) and \ - not device.get_flag(parted.PARTITION_SWAP): - return False - elif device.is_disk: - if device.partitioned and clear_part_type != CLEARPART_TYPE_ALL: - # if clear_part_type is not CLEARPART_TYPE_ALL but we'll still be - # removing every partition from the disk, return True since we - # will want to be able to create a new disklabel on this disk - if not empty_device(device): - return False - - # Never clear disks with hidden formats - if device.format.hidden: - return False - - # When clear_part_type is CLEARPART_TYPE_LINUX and a disk has non- - # linux whole-disk formatting, do not clear it. The exception is - # the case of an uninitialized disk when we've been asked to - # initialize disks as needed - if (clear_part_type == CLEARPART_TYPE_LINUX and - not ((self.config.initialize_disks and - empty_device(device)) or - (not device.partitioned and device.format.linux_native))): - return False - - # Don't clear devices holding install media. - descendants = self.devicetree.get_dependent_devices(device) - if device.protected or any(d.protected for d in descendants): - return False - - if clear_part_type == CLEARPART_TYPE_LIST and \ - device.name not in clear_part_devices: - return False - - return True - def recursive_remove(self, device): """ Remove a device after removing its dependent devices. @@ -572,42 +382,6 @@ def recursive_remove(self, device): """ self.devicetree.recursive_remove(device) - def clear_partitions(self): - """ Clear partitions and dependent devices from disks. - - This is also where zerombr is handled. - """ - # Sort partitions by descending partition number to minimize confusing - # things like multiple "destroy sda5" actions due to parted renumbering - # partitions. This can still happen through the UI but it makes sense to - # avoid it where possible. - partitions = sorted(self.partitions, - key=lambda p: p.parted_partition.number, - reverse=True) - for part in partitions: - log.debug("clearpart: looking at %s", part.name) - if not self.should_clear(part): - continue - - self.recursive_remove(part) - log.debug("partitions: %s", [p.getDeviceNodeName() for p in part.parted_partition.disk.partitions]) - - # now remove any empty extended partitions - self.remove_empty_extended_partitions() - - # ensure all disks have appropriate disklabels - for disk in self.disks: - zerombr = (self.config.zero_mbr and disk.format.type is None) - should_clear = self.should_clear(disk) - if should_clear: - self.recursive_remove(disk) - - if zerombr or should_clear: - log.debug("clearpart: initializing %s", disk.name) - self.initialize_disk(disk) - - self.update_bootloader_disk_list() - def initialize_disk(self, disk): """ (Re)initialize a disk by creating a disklabel on it. @@ -1136,44 +910,6 @@ def resize_device(self, device, new_size): for action in actions: self.devicetree.actions.add(action) - def format_by_default(self, device): - """Return whether the device should be reformatted by default.""" - formatlist = ['/boot', '/var', '/tmp', '/usr'] - exceptlist = ['/home', '/usr/local', '/opt', '/var/www'] - - if not device.format.linux_native: - return False - - if device.format.mountable: - if not device.format.mountpoint: - return False - - if device.format.mountpoint == "/" or \ - device.format.mountpoint in formatlist: - return True - - for p in formatlist: - if device.format.mountpoint.startswith(p): - for q in exceptlist: - if device.format.mountpoint.startswith(q): - return False - return True - elif device.format.type == "swap": - return True - - # be safe for anything else and default to off - return False - - def must_format(self, device): - """ Return a string explaining why the device must be reformatted. - - Return None if the device need not be reformatted. - """ - if device.format.mountable and device.format.mountpoint == "/": - return _("You must create a new filesystem on the root device.") - - return None - def safe_device_name(self, name): """ Convert a device name to something safe and return that. @@ -1358,7 +1094,6 @@ def write(self): os.mkdir("%s/etc" % get_sysroot()) self.fsset.write() - self.make_mtab() iscsi.write(get_sysroot(), self) fcoe.write(get_sysroot()) zfcp.write(get_sysroot()) @@ -1378,25 +1113,6 @@ def write_dasd_conf(self, root): fields = [dasd.busid] + dasd.get_opts() f.write("%s\n" % " ".join(fields),) - def turn_on_swap(self): - self.fsset.turn_on_swap(root_path=get_sysroot()) - - def mount_filesystems(self, read_only=None, skip_root=False): - self.fsset.mount_filesystems(root_path=get_sysroot(), - read_only=read_only, skip_root=skip_root) - - def umount_filesystems(self, swapoff=True): - self.fsset.umount_filesystems(swapoff=swapoff) - - def parse_fstab(self, chroot=None): - self.fsset.parse_fstab(chroot=chroot) - - def mk_dev_root(self): - self.fsset.mk_dev_root() - - def create_swap_file(self, device, size): - self.fsset.create_swap_file(device, size) - @property def bootloader(self): if self._bootloader is None and flags.installer_mode: @@ -1524,23 +1240,6 @@ def mountpoints(self): def root_device(self): return self.fsset.root_device - def make_mtab(self): - path = "/etc/mtab" - target = "/proc/self/mounts" - path = os.path.normpath("%s/%s" % (get_sysroot(), path)) - - if os.path.islink(path): - # return early if the mtab symlink is already how we like it - current_target = os.path.normpath(os.path.dirname(path) + - "/" + os.readlink(path)) - if current_target == target: - return - - if os.path.exists(path): - os.unlink(path) - - os.symlink(target, path) - def compare_disks(self, first, second): if not isinstance(first, str): first = first.name @@ -1830,37 +1529,3 @@ def create_free_space_snapshot(self): self._free_space_snapshot = self.get_free_space() return self._free_space_snapshot - - def add_fstab_swap(self, device): - """ - Add swap device to the list of swaps that should appear in the fstab. - - :param device: swap device that should be added to the list - :type device: blivet.devices.StorageDevice instance holding a swap format - - """ - - self.fsset.add_fstab_swap(device) - - def remove_fstab_swap(self, device): - """ - Remove swap device from the list of swaps that should appear in the fstab. - - :param device: swap device that should be removed from the list - :type device: blivet.devices.StorageDevice instance holding a swap format - - """ - - self.fsset.remove_fstab_swap(device) - - def set_fstab_swaps(self, devices): - """ - Set swap devices that should appear in the fstab. - - :param devices: iterable providing devices that should appear in the fstab - :type devices: iterable providing blivet.devices.StorageDevice instances holding - a swap format - - """ - - self.fsset.set_fstab_swaps(devices) diff --git a/blivet/osinstall.py b/blivet/osinstall.py index 2640955..463d138 100644 --- a/blivet/osinstall.py +++ b/blivet/osinstall.py @@ -24,17 +24,21 @@ import os import stat import time +import parted import gi gi.require_version("BlockDev", "1.0") from gi.repository import BlockDev as blockdev +from pykickstart.constants import AUTOPART_TYPE_LVM, CLEARPART_TYPE_NONE, CLEARPART_TYPE_LINUX, CLEARPART_TYPE_ALL, CLEARPART_TYPE_LIST + from . import util from . import get_sysroot, get_target_physical_root, error_handler, ERROR_RAISE +from .blivet import Blivet from .storage_log import log_exception_info -from .devices import FileDevice, NFSDevice, NoDevice, OpticalDevice, NetworkStorageDevice, DirectoryDevice, MDRaidArrayDevice +from .devices import FileDevice, NFSDevice, NoDevice, OpticalDevice, NetworkStorageDevice, DirectoryDevice, MDRaidArrayDevice, PartitionDevice from .errors import FSTabTypeMismatchError, UnrecognizedFSTabEntryError, StorageError, FSResizeError, FormatResizeError, UnknownSourceDeviceError from .formats import get_device_format_class from .formats import get_format @@ -283,6 +287,7 @@ class StorageDiscoveryConfig(object): """ Class to encapsulate various detection/initialization parameters. """ def __init__(self): + # storage configuration variables self.ignore_disk_interactive = False self.clear_part_type = None @@ -1094,6 +1099,370 @@ def get(self, key, default=None): return self.mappings.get(key, default) +class InstallerStorage(Blivet): + """ Top-level class for managing installer-related storage configuration. """ + def __init__(self, ksdata=None): + """ + :keyword ksdata: kickstart data store + :type ksdata: :class:`pykickstart.Handler` + """ + super().__init__(ksdata=ksdata) + + self.config = StorageDiscoveryConfig() + self.autopart_type = AUTOPART_TYPE_LVM + + self.__luks_devs = {} + self.fsset = FSSet(self.devicetree) + self._free_space_snapshot = None + + def do_it(self, callbacks=None): + """ + Commit queued changes to disk. + + :param callbacks: callbacks to be invoked when actions are executed + :type callbacks: return value of the :func:`~.callbacks.create_new_callbacks_ + + """ + super().do_it(callbacks=callbacks) + + if not flags.installer_mode: + return + + # now set the boot partition's flag + if self.bootloader and not self.bootloader.skip_bootloader: + if self.bootloader.stage2_bootable: + boot = self.boot_device + else: + boot = self.bootloader_device + + if boot.type == "mdarray": + boot_devs = boot.parents + else: + boot_devs = [boot] + + for dev in boot_devs: + if not hasattr(dev, "bootable"): + log.info("Skipping %s, not bootable", dev) + continue + + # Dos labels can only have one partition marked as active + # and unmarking ie the windows partition is not a good idea + skip = False + if dev.disk.format.parted_disk.type == "msdos": + for p in dev.disk.format.parted_disk.partitions: + if p.type == parted.PARTITION_NORMAL and \ + p.getFlag(parted.PARTITION_BOOT): + skip = True + break + + # GPT labeled disks should only have bootable set on the + # EFI system partition (parted sets the EFI System GUID on + # GPT partitions with the boot flag) + if dev.disk.format.label_type == "gpt" and \ + dev.format.type not in ["efi", "macefi"]: + skip = True + + if skip: + log.info("Skipping %s", dev.name) + continue + + # hfs+ partitions on gpt can't be marked bootable via parted + if dev.disk.format.parted_disk.type != "gpt" or \ + dev.format.type not in ["hfs+", "macefi"]: + log.info("setting boot flag on %s", dev.name) + dev.bootable = True + + # Set the boot partition's name on disk labels that support it + if dev.parted_partition.disk.supportsFeature(parted.DISK_TYPE_PARTITION_NAME): + ped_partition = dev.parted_partition.getPedPartition() + ped_partition.setName(dev.format.name) + log.info("Setting label on %s to '%s'", dev, dev.format.name) + + dev.disk.setup() + dev.disk.format.commit_to_disk() + + if flags.installer_mode: + self.dump_state("final") + + def write(self): + Blivet.write(self) + + self.make_mtab() + + def empty_device(self, device): + empty = True + if device.partitioned: + partitions = device.children + empty = all([p.is_magic for p in partitions]) + else: + empty = (device.format.type is None) + + return empty + + @property + def unused_devices(self): + used_devices = [] + for root in self.roots: + for device in list(root.mounts.values()) + root.swaps: + if device not in self.devices: + continue + + used_devices.extend(device.ancestors) + + for new in [d for d in self.devicetree.leaves if not d.format.exists]: + if new.format.mountable and not new.format.mountpoint: + continue + + used_devices.extend(new.ancestors) + + for device in self.partitions: + if getattr(device, "is_logical", False): + extended = device.disk.format.extended_partition.path + used_devices.append(self.devicetree.get_device_by_path(extended)) + + used = set(used_devices) + _all = set(self.devices) + return list(_all.difference(used)) + + def should_clear(self, device, **kwargs): + """ Return True if a clearpart settings say a device should be cleared. + + :param device: the device (required) + :type device: :class:`~.devices.StorageDevice` + :keyword clear_part_type: overrides :attr:`self.config.clear_part_type` + :type clear_part_type: int + :keyword clear_part_disks: overrides + :attr:`self.config.clear_part_disks` + :type clear_part_disks: list + :keyword clear_part_devices: overrides + :attr:`self.config.clear_part_devices` + :type clear_part_devices: list + :returns: whether or not clear_partitions should remove this device + :rtype: bool + """ + clear_part_type = kwargs.get("clear_part_type", self.config.clear_part_type) + clear_part_disks = kwargs.get("clear_part_disks", + self.config.clear_part_disks) + clear_part_devices = kwargs.get("clear_part_devices", + self.config.clear_part_devices) + + for disk in device.disks: + # this will not include disks with hidden formats like multipath + # and firmware raid member disks + if clear_part_disks and disk.name not in clear_part_disks: + return False + + if not self.config.clear_non_existent: + if (device.is_disk and not device.format.exists) or \ + (not device.is_disk and not device.exists): + return False + + # the only devices we want to clear when clear_part_type is + # CLEARPART_TYPE_NONE are uninitialized disks, or disks with no + # partitions, in clear_part_disks, and then only when we have been asked + # to initialize disks as needed + if clear_part_type in [CLEARPART_TYPE_NONE, None]: + if not self.config.initialize_disks or not device.is_disk: + return False + + if not self.empty_device(device): + return False + + if isinstance(device, PartitionDevice): + # Never clear the special first partition on a Mac disk label, as + # that holds the partition table itself. + # Something similar for the third partition on a Sun disklabel. + if device.is_magic: + return False + + # We don't want to fool with extended partitions, freespace, &c + if not device.is_primary and not device.is_logical: + return False + + if clear_part_type == CLEARPART_TYPE_LINUX and \ + not device.format.linux_native and \ + not device.get_flag(parted.PARTITION_LVM) and \ + not device.get_flag(parted.PARTITION_RAID) and \ + not device.get_flag(parted.PARTITION_SWAP): + return False + elif device.is_disk: + if device.partitioned and clear_part_type != CLEARPART_TYPE_ALL: + # if clear_part_type is not CLEARPART_TYPE_ALL but we'll still be + # removing every partition from the disk, return True since we + # will want to be able to create a new disklabel on this disk + if not self.empty_device(device): + return False + + # Never clear disks with hidden formats + if device.format.hidden: + return False + + # When clear_part_type is CLEARPART_TYPE_LINUX and a disk has non- + # linux whole-disk formatting, do not clear it. The exception is + # the case of an uninitialized disk when we've been asked to + # initialize disks as needed + if (clear_part_type == CLEARPART_TYPE_LINUX and + not ((self.config.initialize_disks and + self.empty_device(device)) or + (not device.partitioned and device.format.linux_native))): + return False + + # Don't clear devices holding install media. + descendants = self.devicetree.get_dependent_devices(device) + if device.protected or any(d.protected for d in descendants): + return False + + if clear_part_type == CLEARPART_TYPE_LIST and \ + device.name not in clear_part_devices: + return False + + return True + + def clear_partitions(self): + """ Clear partitions and dependent devices from disks. + + This is also where zerombr is handled. + """ + # Sort partitions by descending partition number to minimize confusing + # things like multiple "destroy sda5" actions due to parted renumbering + # partitions. This can still happen through the UI but it makes sense to + # avoid it where possible. + partitions = sorted(self.partitions, + key=lambda p: p.parted_partition.number, + reverse=True) + for part in partitions: + log.debug("clearpart: looking at %s", part.name) + if not self.should_clear(part): + continue + + self.recursive_remove(part) + log.debug("partitions: %s", [p.getDeviceNodeName() for p in part.parted_partition.disk.partitions]) + + # now remove any empty extended partitions + self.remove_empty_extended_partitions() + + # ensure all disks have appropriate disklabels + for disk in self.disks: + zerombr = (self.config.zero_mbr and disk.format.type is None) + should_clear = self.should_clear(disk) + if should_clear: + self.recursive_remove(disk) + + if zerombr or should_clear: + log.debug("clearpart: initializing %s", disk.name) + self.initialize_disk(disk) + + self.update_bootloader_disk_list() + + def format_by_default(self, device): + """Return whether the device should be reformatted by default.""" + formatlist = ['/boot', '/var', '/tmp', '/usr'] + exceptlist = ['/home', '/usr/local', '/opt', '/var/www'] + + if not device.format.linux_native: + return False + + if device.format.mountable: + if not device.format.mountpoint: + return False + + if device.format.mountpoint == "/" or \ + device.format.mountpoint in formatlist: + return True + + for p in formatlist: + if device.format.mountpoint.startswith(p): + for q in exceptlist: + if device.format.mountpoint.startswith(q): + return False + return True + elif device.format.type == "swap": + return True + + # be safe for anything else and default to off + return False + + def must_format(self, device): + """ Return a string explaining why the device must be reformatted. + + Return None if the device need not be reformatted. + """ + if device.format.mountable and device.format.mountpoint == "/": + return _("You must create a new filesystem on the root device.") + + return None + + def turn_on_swap(self): + self.fsset.turn_on_swap(root_path=get_sysroot()) + + def mount_filesystems(self, read_only=None, skip_root=False): + self.fsset.mount_filesystems(root_path=get_sysroot(), + read_only=read_only, skip_root=skip_root) + + def umount_filesystems(self, swapoff=True): + self.fsset.umount_filesystems(swapoff=swapoff) + + def parse_fstab(self, chroot=None): + self.fsset.parse_fstab(chroot=chroot) + + def mk_dev_root(self): + self.fsset.mk_dev_root() + + def create_swap_file(self, device, size): + self.fsset.create_swap_file(device, size) + + def make_mtab(self): + path = "/etc/mtab" + target = "/proc/self/mounts" + path = os.path.normpath("%s/%s" % (get_sysroot(), path)) + + if os.path.islink(path): + # return early if the mtab symlink is already how we like it + current_target = os.path.normpath(os.path.dirname(path) + + "/" + os.readlink(path)) + if current_target == target: + return + + if os.path.exists(path): + os.unlink(path) + + os.symlink(target, path) + + def add_fstab_swap(self, device): + """ + Add swap device to the list of swaps that should appear in the fstab. + + :param device: swap device that should be added to the list + :type device: blivet.devices.StorageDevice instance holding a swap format + + """ + + self.fsset.add_fstab_swap(device) + + def remove_fstab_swap(self, device): + """ + Remove swap device from the list of swaps that should appear in the fstab. + + :param device: swap device that should be removed from the list + :type device: blivet.devices.StorageDevice instance holding a swap format + + """ + + self.fsset.remove_fstab_swap(device) + + def set_fstab_swaps(self, devices): + """ + Set swap devices that should appear in the fstab. + + :param devices: iterable providing devices that should appear in the fstab + :type devices: iterable providing blivet.devices.StorageDevice instances holding + a swap format + + """ + + self.fsset.set_fstab_swaps(devices) + + def get_containing_device(path, devicetree): """ Return the device that a path resides on. """ if not os.path.exists(path): -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/fe733937f228ad6d1add2475b86b9b…

1 0

[3.0-devel 2/7] Move StorageDiscoveryConfig from blivet.blivet to blivet.osinstall
by dashea 01 Apr '16

01 Apr '16

From: "Samantha N. Bueno" <sbueno+anaconda(a)redhat.com> As this is mostly pertaining to the installer storage configuration, move this into osinstall. In the interest of not breaking things between commits, I've placed a temporary import of blivet.osinstall.StorageDiscoveryConfig in blivet.blivet, since, for now, blivet still references it. This'll go away shortly once those functions get moved into their rightful place. --- blivet/blivet.py | 33 +-------------------------------- blivet/osinstall.py | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 32 deletions(-) diff --git a/blivet/blivet.py b/blivet/blivet.py index aefb410..95fc2de 100644 --- a/blivet/blivet.py +++ b/blivet/blivet.py @@ -46,7 +46,7 @@ from .flags import flags from .platform import platform as _platform from .formats import get_format -from .osinstall import FSSet, find_existing_installations +from .osinstall import FSSet, StorageDiscoveryConfig find_existing_installations from . import arch from .iscsi import iscsi from .fcoe import fcoe @@ -72,37 +72,6 @@ def empty_device(device): return empty -class StorageDiscoveryConfig(object): - - """ Class to encapsulate various detection/initialization parameters. """ - - def __init__(self): - # storage configuration variables - self.ignore_disk_interactive = False - self.clear_part_type = None - self.clear_part_disks = [] - self.clear_part_devices = [] - self.initialize_disks = False - self.protected_dev_specs = [] - self.zero_mbr = False - - # Whether clear_partitions removes scheduled/non-existent devices and - # disklabels depends on this flag. - self.clear_non_existent = False - - def update(self, ksdata): - """ Update configuration from ksdata source. - - :param ksdata: kickstart data used as data source - :type ksdata: :class:`pykickstart.Handler` - """ - self.clear_part_type = ksdata.clearpart.type - self.clear_part_disks = ksdata.clearpart.drives[:] - self.clear_part_devices = ksdata.clearpart.devices[:] - self.initialize_disks = ksdata.clearpart.initAll - self.zero_mbr = ksdata.zerombr.zerombr - - class Blivet(object, metaclass=SynchronizedMeta): """ Top-level class for managing storage configuration. """ diff --git a/blivet/osinstall.py b/blivet/osinstall.py index 96fc4a5..2640955 100644 --- a/blivet/osinstall.py +++ b/blivet/osinstall.py @@ -278,6 +278,37 @@ def _find_existing_installations(devicetree): return roots +class StorageDiscoveryConfig(object): + + """ Class to encapsulate various detection/initialization parameters. """ + + def __init__(self): + # storage configuration variables + self.ignore_disk_interactive = False + self.clear_part_type = None + self.clear_part_disks = [] + self.clear_part_devices = [] + self.initialize_disks = False + self.protected_dev_specs = [] + self.zero_mbr = False + + # Whether clear_partitions removes scheduled/non-existent devices and + # disklabels depends on this flag. + self.clear_non_existent = False + + def update(self, ksdata): + """ Update configuration from ksdata source. + + :param ksdata: kickstart data used as data source + :type ksdata: :class:`pykickstart.Handler` + """ + self.clear_part_type = ksdata.clearpart.type + self.clear_part_disks = ksdata.clearpart.drives[:] + self.clear_part_devices = ksdata.clearpart.devices[:] + self.initialize_disks = ksdata.clearpart.initAll + self.zero_mbr = ksdata.zerombr.zerombr + + class FSSet(object): """ A class to represent a set of filesystems. """ -- To view this commit on github, visit https://github.com/rhinstaller/blivet/commit/3f16f14a6a2c356bad2e2a476bb874…

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

anaconda-patches April 2016