https://fedoraproject.org/wiki/Changes/ModernizeLiveMedia
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary == Modernize the live media by switching to the "new" live environment setup scripts provided by {{package|livesys-scripts}} and leverage new functionality in {{package|dracut}} to enable support for automatically enabling persistent overlays when flashed to USB sticks.
== Owner == * Name: [[User:Ngompa|Neal Gompa]], [[User:Mcoleman|Matt Coleman]] * Email: ngompa13@gmail.com, matt@1eanda.com
== Detailed Description == As part of preparing to change our tooling around producing images, we need to update the way we produce working live media. This has been done in two parts: the live environment setup scripts have been reworked to run properly through systemd and are packaged in {{package|livesys-scripts}} and new functionality in {{package|dracut}} has been added to enable support for automatically enabling persistent overlays when flashed to USB sticks so that <code>livecd-iso-to-disk.sh</code> can be retired.
== Benefit to Fedora == Since we introduced [[Releases/FeatureLiveCD|live media in Fedora Linux 7]], the actual mechanism in which the live environment sets itself up has been complex and intricately tied to the method in which we produce the media (using Kickstarts). The nature of the implementation of those scripts means that they are hard to understand and debug, which has caused problems in the past whenever we've needed to update them.
As we look forward to new and better tooling for producing images (such as {{package|kiwi}} and {{package|osbuild}}), we cannot continue to rely on kickstart-driven image builds that construct shell scripts on the fly to embed in the image as we do now. With {{package|livesys-scripts}}, those scripts have been simplified and turned into systemd services that activate only in live environments.
This also gives us the opportunity to introduce new functionality for live media. [https://github.com/dracutdevs/dracut/pull/1991 New functionality was added to dracut] and [https://src.fedoraproject.org/rpms/dracut/pull-request/26 backported to Fedora] so that we can retire the remaining usage of <code>livecd-iso-to-disk.sh</code> and provide a better experience with our live media, particularly for portable backup and rescue environments by introducing the ability to automatically setup persistence on boot when unpartitioned space is detected on a USB stick on boot.
== Scope == * Proposal owners: ** Modify [https://pagure.io/fedora-kickstarts fedora-kickstarts] to drop embedded livesys setup scripts and use {{package|livesys-scripts}} ** Add <code>livesys.service</code> and <code>livesys-late.service</code> to the systemd presets in {{package|fedora-release}} ** [https://src.fedoraproject.org/rpms/dracut/pull-request/26 Backport <code>dmsquash-live-autooverlay</code> module] to {{package|dracut}} ** Modify [https://github.com/weldr/lorax/tree/master/share/templates.d/99-generic lorax-generic-templates] to offer menu entries for configuring the persistent overlay
* Other developers: ** Anaconda may need adaptations to correctly filter out more live environment specific boot arguments when configuring the installed environment.
* Release engineering: [https://pagure.io/releng/issue/11091 #11091]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact == There should be no impact here, as this affects only the live environment.
== How To Test == Once the changes are merged, simply try to boot produced live media.
There should be new options for resetting the persistent overlay and booting with no persistence. The default options should boot with persistence and setup of persistence should work.
Testing persistence behavior is simple:
# Boot normally # Create a file in the home directory # Reboot into the environment # See the file is there still
Testing the reset behavior would go similar to the regular test, except you should expect the file to not show up in step 4. Same for booting with no persistence.
Otherwise, the media should work like before, and things like live installation should work as expected.
== User Experience == When booting live media, two new menu options will become available: one to reset the persistent overlay while booting, and one to boot without persistence. The default boot options will create a persistent overlay if it doesn't exist and can be written (e.g. when booting from a USB stick).
== Dependencies == * [https://pagure.io/fedora-kickstarts fedora-kickstarts] * {{package|anaconda}} * {{package|dracut}} * {{package|livesys-scripts}} * {{package|lorax}}
== Contingency Plan == * Contingency mechanism: Revert changes to fedora-kickstarts and lorax to go back to previous behavior * Contingency deadline: Final Freeze * Blocks release? Yes
== Documentation == Information on the persistent overlay functionality is included in the Dracut documentation.
== Release Notes == By default, Fedora Linux live environments flashed to writable USB media with sufficient free space will maintain user changes to the environment. This "persistence" can be reset at boot time through the boot menu.
On Tue, Oct 18, 2022 at 10:43 PM Ben Cotton bcotton@redhat.com wrote:
There should be new options for resetting the persistent overlay and booting with no persistence. The default options should boot with persistence and setup of persistence should work.
Neal, Matt, what is the rationale for enabling persistence for the default boot option? I have mixed opinions about this. One of the benefits of a Live image, as we use it today, is that it's always the same/fresh. If I use it and then hand it over to a friend/colleague, I don't need to worry that some personal files (like a browser history) were left behind. I also don't need to worry that I (or the previous user) made some changes which would negatively impact the installation process or my user environment (like configuring a different keymap, or installing some updates). It's always as the creators intended. With the proposed Change, suddenly I need to care and need to worry.
My impression is that currently persistence use is basically non-existent. Our well-advertised tools like Fedora Media Writer don't support it. Even if we flip this to make it easily available (which is probably a good thing), how many users do you estimate would actually want to make use of it? Who would want to work from a Live image regularly? I'm sure there are some use cases, but they seem so niche to me, that making it a non-default boot option wouldn't be a problem at all.
I wonder if you've thought about this and why you decided to propose making it enabled by default. Thanks.
On Wed, Oct 19, 2022 at 2:57 AM Kamil Paral kparal@redhat.com wrote:
On Tue, Oct 18, 2022 at 10:43 PM Ben Cotton bcotton@redhat.com wrote:
There should be new options for resetting the persistent overlay and booting with no persistence. The default options should boot with persistence and setup of persistence should work.
Neal, Matt, what is the rationale for enabling persistence for the default boot option? I have mixed opinions about this. One of the benefits of a Live image, as we use it today, is that it's always the same/fresh. If I use it and then hand it over to a friend/colleague, I don't need to worry that some personal files (like a browser history) were left behind. I also don't need to worry that I (or the previous user) made some changes which would negatively impact the installation process or my user environment (like configuring a different keymap, or installing some updates). It's always as the creators intended. With the proposed Change, suddenly I need to care and need to worry.
Hmm, that makes sense. The main reason I was thinking of it was to make the live media useful in portable and rescue environments. Maybe it would make sense to make the default boot option activate persistence if it exists, but use a temporary overlay if it doesn't (which is what we technically do today) and create menu options to trigger overlay creation.
That said, SoaS would probably benefit from default persistence, since I believe it's supposed to work from a USB stick.
My impression is that currently persistence use is basically non-existent. Our well-advertised tools like Fedora Media Writer don't support it. Even if we flip this to make it easily available (which is probably a good thing), how many users do you estimate would actually want to make use of it? Who would want to work from a Live image regularly? I'm sure there are some use cases, but they seem so niche to me, that making it a non-default boot option wouldn't be a problem at all.
I wonder if you've thought about this and why you decided to propose making it enabled by default. Thanks.
I know that a big part of why persistence use is non-existent is that support for integrating it into FMW has been deferred for years. Over that time, the script has see-sawed from working to not working. Having persistence means people can have Fedora environments they can carry around, and if we advertised the capability, I think people *would* use it.
Enabling persistence is pretty much a matter of adding a boot option to the grub menu item. If we could modify the grub menu configuration from FMW, then we could probably have a checkbox there to turn on/off the capability.
On 18/10/2022 22:35, Ben Cotton wrote:
There should be new options for resetting the persistent overlay and booting with no persistence. The default options should boot with persistence and setup of persistence should work.
I don't think that persistence by default is a good idea, because modern USB-flash drives are very unreliable and don't have a wear-cell balancer, so it will wear out very quickly for some frequently modified files.
Persistence is a good option, but should only be enabled at the user's choice.
On Wed, Oct 19, 2022 at 09:22:23AM +0200, Vitaly Zaitsev via devel wrote:
I don't think that persistence by default is a good idea, because modern USB-flash drives are very unreliable and don't have a wear-cell balancer, so it will wear out very quickly for some frequently modified files.
Yeah — lots of very-low-quality flash drives out there. Ran into this just trying to get decent ones for give-away swag.
I expect this to get worse and worse, as more and more people just use dropbox and other cloud storage for file transfer and random storage. Once again we're increasingly a niche case.
(It is, on the other hand, easier and easier to get really good MicroSD cards for relatively cheap. SanDisk High Endurance for ten bucks. But not quite cheap enough for giveways. Maybe the giveway item should be a branded card reader...)
On Tue, Oct 18, 2022 at 10:43 PM Ben Cotton <bcotton(a)redhat.com> wrote:
Neal, Matt, what is the rationale for enabling persistence for the default boot option? I have mixed opinions about this. One of the benefits of a Live image, as we use it today, is that it's always the same/fresh. If I use it and then hand it over to a friend/colleague, I don't need to worry that some personal files (like a browser history) were left behind. I also don't need to worry that I (or the previous user) made some changes which would negatively impact the installation process or my user environment (like configuring a different keymap, or installing some updates). It's always as the creators intended. With the proposed Change, suddenly I need to care and need to worry.
Yes, default should be the pristine image, at least as long no persistent overlay exists.
My impression is that currently persistence use is basically non-existent. Our well-advertised tools like Fedora Media Writer don't support it. Even
And that is exactly why.
I used to use it quite a alot - and be it just selecting the right keyboard layout for trying out an RC or such on various boxes which all happened to have the same keyboard layout, being in the same country.
Another use case is as a handy rescue option, where - again - I might want to set up my environment (home overlay), or even trying out selective updates between RCs, say for QA purposes (system overlay).
With {{package|livesys-scripts}}, those scripts have been simplified and turned into systemd services that activate only in live environments.
Just to confirm, will live-media-only these systemd services all be contained within a package (either livesys-scripts, or some other RPM), and will that package be kept dependent-less so it can be uninstalled at any point if it's not needed?
I'm just picturing all of these new "systemd services that activate only in live environments" possibly ending up on _installed_ systems. (Particularly since the supported desktop installation method is to install from the live media.)
Not a problem if they can just be `dnf remove`-d, but less great if they clutter up `systemctl list-unit-files` with a bunch of useless, perma-disabled units that can't (easily) be removed because some core package either contains or Requires: them. systemd _already_ comes with a **large** default collection of unit files.
On Tue, Oct 18, 2022, at 4:35 PM, Ben Cotton wrote:
Just for reference, today Fedora CoreOS uses a different implementation of this: https://github.com/coreos/fedora-coreos-config/tree/testing-devel/overlay.d/... See specifically https://github.com/coreos/fedora-coreos-config/blob/testing-devel/overlay.d/...
-
Ben Cotton -
Colin Walters -
Frank R Dana Jr. -
Kamil Paral -
Matthew Miller -
Michael J Gruber -
Neal Gompa -
Vitaly Zaitsev