Adventurous yet Safety-Minded

Thu Mar 11 10:53:18 UTC 2010

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2010 09:24 AM, yersinia wrote:
> On Wed, Mar 10, 2010 at 1:54 PM, Alexander Kahl <e-user at fsfe.org> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 03/10/2010 01:06 PM, Steven I Usdansky wrote:
>> There is no real recovery for traditional package systems as each change
>> on a package (install, update, remove etc.) changes the state of the
>> system as a whole, i.e. the system relies on side effects (mostly
>> writing files into shared/global locations) and provides no referential
>> transparency for such actions, same output for same input is never
>> guaranteed.
>>
> My (silly perhaps) personal opinion that the rollback part of a package
> management system depends very much on QA which are made the same packages
> in first place. The way you write  seems that the problem is insoluble for a
> package manager: if so it did seem strange that several rpm developers have
> lost so time to implement it in the first place and to extend it over time.
Because they've never seemed to question the foundations of a system
that relies on global state made up by interdependence between packages
during and after build and install time. What we have now is basically
an automaton that works deterministic in theory but is unpredictable in
practice. Using a mechanism like filesystem snapshots could enable real
rollbacks to a state that did actually exist but will make you lose
every other change happened in the meantime which might not be what you
want. Implementing and using RPM rollbacks, including filesystem-level
rollbacks or not, is like "guessing" a lost (full rollback of all
actions) or new (partly rollback, some actions) state by interpolating
stuff, like "rollback of B after installation of {A,B,C} at state `e'
could yield a state approximate to the one after installation of {A,C}
at state `e'". This can turn out as a lucky day in Las Vegas at best.
Running C after B has been installed could have modified files belonging
to A or C, installation of C might behave different if B was installed
during %post (currently we work around the latter case by always
requiring all otherwise optional dependencies being present).
Better but still not perfect would be "filesystem rollback to `e',
installation of {A,C}" to get the "real" desired state without B.

There are of course cases where package maintainer work and QA can
improve rollback quality:
Imagine mysql-server would run an unconditional database format upgrade
upon update in %post (which it luckily doesn't!) ruining your day after
an RPM rollback. Here, the maintainers could increase the atomicity of
the package by creating a snapshot of the currently running database(s),
upgrading the snapshot and storing the old data to an offside storage
that is brought back into place upon rollback.
To my knowledge right now though, we neither have a mechanism to help
with that nor any guideline encouraging such behavior. Same goes for
configuration files with something better than those lame .rpm{save,new}
backup files.

> But I think this has already been discussed in the past, many time. For
> example http://lists.rpm.org/pipermail/rpm-maint/2008-February/001912.html

Quoting from that post:
> - Most scriptlets in RedHat packages are fairly simple so their
> typically is nothing to really undo.
But if they're not simple (which is the case too often), the user is
screwed.

> - In house scriptlets were always written with the view that they
> were going to be in rolled back so we made them work.
In house scriptlets, yeah, but we're a community with many contributors.
And there *are* update scriptlets as noted above without proper offside
storage and rollbacks for that.

> - Often we could work around an upstream scriptlets issuses in
> rollback via some other rpm, or some code external to rpm.
Which will, with current technology in place, again give an
unpredictable state `e3' instead of install (e, {A,C}) -> e2.

> ...
> That said it will never be as reliable and simple as a rollback of a
> filesystem snapshot (however that is implemented). OTOH, that solution
> is outside the rpm problem space.
But it's in scope of the packaging and filesystem layout spaces, read on.

> and
> 
> http://lists.rpm.org/pipermail/rpm-list/2009-April/000227.html

Quoting again:
> The problem with rollback is that scripts with in the rpm can execute
> arbitrary shell commands.
Correct.

> This means that RPM cannot foresee which files are
> affected by a package and therefore cannot backup all files.
Correct, we're doing changes without properly recording former state.

> http://lists.rpm.org/pipermail/rpm-list/2009-April/000231.html
> Is it a implementation problem - difficult, sure, no dubt - or a
> universal law ?  I think the first but JMHO.
No, it's a logical problem of global state and referential transparency.
There are several scopes of this where rolling back is affected, most of
the actually covered and solved by Nix in elegant ways:
- - Non-atomic, global state of the runtime system configuration comprised
by all files residing in /etc at a given state. This is indeed to a
large degree outside of RPM's scope as users and tools executing during
runtime can change these files, making the problem one of filesystem
layout: Nix provides mechanisms to create snapshots of a set of
configuration files affecting one or more programs' behaviors both
during install and runtime so rollback to any former state is possible
without any low-level filesystem implementation magic.
- - Non-atomic global state of files not changing during runtime comprised
by all files residing in /bin, /sbin, /lib, /lib64, /boot, /usr and
usually only changed by RPM during transactions. While this is the
easiest to cover by implementing RPM rollbacks it's still not
referential transparent that way due to filesystems' hierarchical
natures. Nix solves this by having the physical files reside in a unique
place identified and referenced by a unique hash per package *and* per
package version allowing for an arbitrary number of versions installed
in parallel; a set of libraries and binaries each of a given version in
use for one user or system-wide as the default is, again, identified by
a hash and implemented by something as simple as symlinks giving the
admin and users the power of rollbacks by just tweaking a symlink
location and even enabling package installations, updates and rollbacks
of unprivileged users (!) without affecting real system state.
- - Non-atomic global state of files that are mutable at any time, usually
residing in /var and /home. This is the most difficult, unpredictable
area. One can, however, provide rollbacks by using a mechanism like
drafted above, creating snapshots during transactions, and, again, using
Nix' hash/symlink mechanisms to allow for rollbacks even without writing
legions of code files. /home however is - of course - permanently
outside all scope of packaging :)

So what you get is an omnipresent, healthy and minimal level of
redundancy that allows for >0.9 (just a guess) clean rollbacks without
using magic, predictable states through referential transparency /
atomicity of package installations, full support of *real* parallel
package version installations without any further packager addo, total
invulnerability from ABI breakage.. and the kitchen sink.

> But the problem is always relevant
> http://www.mancoosi.org/work/#wp3
Most of what is described here is already covered by Nix (leaving out
the hardware driver part).

This is why I endorse dumping yum, RPM *and* the stupid FHS in favor of
Nix, focusing all development on something that is clearly superior by
substantiating the synthesis of DOS (*cough*) and POSIX style
installations: Minimal redundancy, maximum compatibility, (mostly)
predictable rollbacks, atomicity, referential transparency: Purely
functional package management.

Oh, and by the way, we could leave behind all those discussions
regarding dynamic linking: RPATH for everything and everyone. If you've
linked against libfoo-4.2-2 during build time, libfoo-4.2-2 will be
present during runtime, same location, same file. Period. :)

Regards

- -- 
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkuYy54ACgkQVTRddCFHw121KQCgpXdw41osprZoRXjdB48sg6B9
Ww4AnAqSIkrK0uFeL0VeJUFpCkLgVWQD
=RTmW
-----END PGP SIGNATURE-----