Thanks Rob

Replies to questions interposed below.

Harry G Coin via FreeIPA-users wrote:

What's the 'current best practice' for what you might call a 'fully
deployed' freeipa install (meaning one that uses DNSSEC and all the
documented capability subsections)?

From what I can tell, there are two approaches:

Approach 1: Run it in a VM, then from time to time shut it down,
snapshot the 'known good' image, then bring the VM back up'. When
there's a problem that's not 'trivial', just revert to the 'known good'
snapshot and pay the price of re-entry of lost data.

 The reasoning here is:  Freeipa is much too interwoven among all the
sub-packages and dependency-hell, the time required to re-enter data
necessary to bring a backup current is at least known, compared to the
almost impossible-to-guess time and multi-package expertise required to
diagnosis much less cure problems in anything like the time available in
production environments.

Yes, this can help back out of failed upgrades, for example. Note that
this and any backup and restore procedure can wreak havoc on
replication. If you effectively take one server back in time then you
should re-init it from another server.

Could you flesh out 'the official' command sequence for that?  For example:

1. Restore the VM image to a 'known good' image (known-good means ipa-healthcheck reports no errors).

2. Boot the VM.

3. If there is no replication involved, re-enter data lost during the downtime and you're done.  Otherwise:

4. If there is replication involved, especially multi-master replication: then run these commands in this order on these systems: <Rob will fill this in?  Please?>

I'm not sure why you bring up dependencies in the context of backup and
restore.

'security' and 'other' seemingly 'unrelated'  'upgrades' to packages n levels deep but whose previously un-noticed freeipa killing race-condition or other bug manifests after the upgrade.  I find myself obligated to prevent any security or other change from happening until the lowest possible usage times.  For example today's 'random freeipa bother' is:

Problem: cannot install both protobuf-3.5.0-15.el8.x86_64 and protobuf-3.19.0-2.el8s.x86_64
 - package liborc1-1.7.9-1.el8.x86_64 requires libprotobuf.so.15()(64bit), but none of the providers can be installed
 - cannot install the best update candidate for package protobuf-3.19.0-2.el8s.x86_64
 - cannot install the best update candidate for package liborc1-1.7.5-1.el8s.x86_64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Also, there's a bit of a blurring of what comes to mind when the words 'multi-master' are adverstised, then the 'reality hair'  (or at least AFAIK) in the kasp db and perhaps other places (SOA), where one system 'is more master than others even with m-m agreements'.   There is 'lore' to do with 'manual steps' to 'move' 'special' files among what are otherwise thought of as 'master' systems.  This is not a complaint, I'm glad there is at least some way, but it generates worry there are other sub-systems that also require 'as yet unlearned lore' to actually provide 'a whole system' from a backup.

Approach 2: Use the ipa-backup, ipa-restore commands.   Just keep a
fresh install of the unconfigured pacakges, do ipa-restore and it will
get every single detail correct  across all the available documented
functionality, databases, including the dnssec ones and the other
'trust' addons, etc.

The reasoning here is:  It's better than approach 1 in that you don't
have to take down the system even for a moment to get a 'known good'
snapshot.   However, you have to trust there is testing of 'everything
the ipa-backup ipa-restore' process does actually does test the whole of
the documented functionality.

For production reasons we'd prefer approach 2, as taking the system down
and bringing it back up as in approach 1 introduces a momentary loss of
capability but has the benefit of 'total confidence'. But is it really
'the best approach' in the production world?

Backups are a good thing to have, but restoration should be due to
catastrophe. When doing a restore recommend deleting all other servers
and re-installing. You could conceivably re-init from the restored
server as well, but it really is a nuke from orbit operation.

This is really helpful, thank you.  The point then of 'ipa-backup' is-- an offline file meant to be used if everything burns to the ground, not to recover this or that failed master or replica.  

In either situation there will be down time.

Some level of backup is essentially online with MMR in 389-ds. This
can't prevent undesired loss or modification of data but it otherwise
provides the uptime you seem to be looking for.

All about uptime!  Thanks Rob

Harry

rob