Upgrading from 4.2.4 (FC23)

[Roberto Cornacchia]

Dear all,

Upgrading is always scary, I will appreciate any comment on the following.

Our freeIPA is serving a small number of FC desktops and users (< 10), and
is running on a FC 23 server, with packages for ipa versioned 4.2.4-2.fc23.

The simplest thing I can do is of course to upgrade the FC system until the
latest, one version at the time. What I probably want to do is actually
move to CentOS - I'm fed up with running after FC releases.

In both cases (especially in the second case), I thought it may be wise to
make a replica of the ipa server before starting the upgrade.

My plan would be:
- Have an up-to-date CentOS system (IPA-B), enroll it and promote it to
replica of the existing one (IPA-A)
- [ Question: is it better to have IPA-B on a recent version or on the same
version as IPA-A? ]
- Shut down IPA-A
- Verify that IPA-B works
- Wipe out IPA-A, install recent CentOS.
- Enroll IPA-A
- Promote it to replica.
- Enjoy

Am I overlooking something? Could I do something more prudently?

Thanks for your input!
Roberto

[Brian Topping]

Have you considered starting to use the docker image? I knew that there was upgrade and it worked very well, completely automated.

Maybe start separate test cluster with older image, then experience the upgrade. When you are satisfied, use the docker image that is the same version as your current cluster, then upgrade it.

Anyway, if that is uncomfortable, try to learn how to verify that the replicas are synchronized with CLI. When you can confirm that the replicas are synchronized, your plan is easier because you can also confirm at the end that they are still synchronized.

I would also do this with three replicas, not two. Then when you shut down A replica, you can confirm B and C are properly replicating as you upgrade before destroying A. If you get stuck, you can still safely start over.

Lastly, if you are using a system that supports snapshots, it’s the best way to keep the original copy of a replica before making changes. In the worst case, you can reset all the snapshots and reboot.
> On Dec 17, 2018, at 6:17 PM, Roberto Cornacchia via FreeIPA-users  wrote:
>
> Dear all,
>
> Upgrading is always scary, I will appreciate any comment on the following.
>
> Our freeIPA is serving a small number of FC desktops and users (< 10), and is running on a FC 23 server, with packages for ipa versioned 4.2.4-2.fc23.
>
> The simplest thing I can do is of course to upgrade the FC system until the latest, one version at the time. What I probably want to do is actually move to CentOS - I'm fed up with running after FC releases.
>
> In both cases (especially in the second case), I thought it may be wise to make a replica of the ipa server before starting the upgrade.
>
> My plan would be:
> Have an up-to-date CentOS system (IPA-B), enroll it and promote it to replica of the existing one (IPA-A)
> [ Question: is it better to have IPA-B on a recent version or on the same version as IPA-A? ]
> Shut down IPA-A
> Verify that IPA-B works
> Wipe out IPA-A, install recent CentOS.
> Enroll IPA-A
> Promote it to replica.
> Enjoy
>
> Am I overlooking something? Could I do something more prudently?
>
> Thanks for your input!
> Roberto
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/[e-mail]...

[Roberto Cornacchia]

Brian,

Thanks for your suggestions.
Actually using the Docker image seems very attractive, if it is close to
production quality. I can experiment anyway.

What I don't really know is whether I can experiment with creating replicas
of the current server. I mean, can creating replicas harm the existing
server in any way, in case something goes wrong?

On Mon, 17 Dec 2018 at 13:17 Brian Topping  wrote:
> Have you considered starting to use the docker image? I knew that there
> was upgrade and it worked very well, completely automated.
>
> Maybe start separate test cluster with older image, then experience the
> upgrade. When you are satisfied, use the docker image that is the same
> version as your current cluster, then upgrade it.
>
> Anyway, if that is uncomfortable, try to learn how to verify that the
> replicas are synchronized with CLI. When you can confirm that the replicas
> are synchronized, your plan is easier because you can also confirm at the
> end that they are still synchronized.
>
> I would also do this with three replicas, not two. Then when you shut down
> A replica, you can confirm B and C are properly replicating as you upgrade
> before destroying A. If you get stuck, you can still safely start over.
>
> Lastly, if you are using a system that supports snapshots, it’s the best
> way to keep the original copy of a replica before making changes. In the
> worst case, you can reset all the snapshots and reboot.
> > On Dec 17, 2018, at 6:17 PM, Roberto Cornacchia via FreeIPA-users <
>
> [e-mail]> wrote:
> > Dear all,
> >
> > Upgrading is always scary, I will appreciate any comment on the
>
> following.
> > Our freeIPA is serving a small number of FC desktops and users (< 10),
>
> and is running on a FC 23 server, with packages for ipa versioned
> 4.2.4-2.fc23.
> > The simplest thing I can do is of course to upgrade the FC system until
>
> the latest, one version at the time. What I probably want to do is actually
> move to CentOS - I'm fed up with running after FC releases.
> > In both cases (especially in the second case), I thought it may be wise
>
> to make a replica of the ipa server before starting the upgrade.
> > My plan would be:
> > Have an up-to-date CentOS system (IPA-B), enroll it and promote it to
>
> replica of the existing one (IPA-A)
> > [ Question: is it better to have IPA-B on a recent version or on the
>
> same version as IPA-A? ]
> > Shut down IPA-A
> > Verify that IPA-B works
> > Wipe out IPA-A, install recent CentOS.
> > Enroll IPA-A
> > Promote it to replica.
> > Enjoy
> >
> > Am I overlooking something? Could I do something more prudently?
> >
> > Thanks for your input!
> > Roberto
> >
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to
>
> [e-mail]
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
>
> https://lists.fedorahosted.org/archives/list/[e-mail]...

[Brian Topping]

Hi Roberto, my skills here are weaker than the actual team here but they are busy so I thought I might be able kick in a little.

Please do be careful. I recently had a situation where I had a machine crash during initial replication due to a bad CPU. It caused a lot of problems. I did not make a snapshot of the replication target before I did it, and the previously stable master became locked to replication changes, including removing the failed install node. Since I am converting everything to Kubernetes anyway, I just decided to start over. It was a lesson learned.

Proper snapshots would have meant I could just shut down, roll back, boot up and try again.

Sent from my iPhone
> On Dec 17, 2018, at 20:49, Roberto Cornacchia  wrote:
>
> Brian,
>
> Thanks for your suggestions. 
> Actually using the Docker image seems very attractive, if it is close to production quality. I can experiment anyway.
>
> What I don't really know is whether I can experiment with creating replicas of the current server. I mean, can creating replicas harm the existing server in any way, in case something goes wrong?
> > On Mon, 17 Dec 2018 at 13:17 Brian Topping  wrote:
> > Have you considered starting to use the docker image? I knew that there was upgrade and it worked very well, completely automated.
> >
> > Maybe start separate test cluster with older image, then experience the upgrade. When you are satisfied, use the docker image that is the same version as your current cluster, then upgrade it.
> >
> > Anyway, if that is uncomfortable, try to learn how to verify that the replicas are synchronized with CLI. When you can confirm that the replicas are synchronized, your plan is easier because you can also confirm at the end that they are still synchronized.
> >
> > I would also do this with three replicas, not two. Then when you shut down A replica, you can confirm B and C are properly replicating as you upgrade before destroying A. If you get stuck, you can still safely start over.
> >
> > Lastly, if you are using a system that supports snapshots, it’s the best way to keep the original copy of a replica before making changes. In the worst case, you can reset all the snapshots and reboot.
> > > On Dec 17, 2018, at 6:17 PM, Roberto Cornacchia via FreeIPA-users  wrote:
> > >
> > > Dear all,
> > >
> > > Upgrading is always scary, I will appreciate any comment on the following.
> > >
> > > Our freeIPA is serving a small number of FC desktops and users (< 10), and is running on a FC 23 server, with packages for ipa versioned 4.2.4-2.fc23.
> > >
> > > The simplest thing I can do is of course to upgrade the FC system until the latest, one version at the time. What I probably want to do is actually move to CentOS - I'm fed up with running after FC releases.
> > >
> > > In both cases (especially in the second case), I thought it may be wise to make a replica of the ipa server before starting the upgrade.
> > >
> > > My plan would be:
> > > Have an up-to-date CentOS system (IPA-B), enroll it and promote it to replica of the existing one (IPA-A)
> > > [ Question: is it better to have IPA-B on a recent version or on the same version as IPA-A? ]
> > > Shut down IPA-A
> > > Verify that IPA-B works
> > > Wipe out IPA-A, install recent CentOS.
> > > Enroll IPA-A
> > > Promote it to replica.
> > > Enjoy
> > >
> > > Am I overlooking something? Could I do something more prudently?
> > >
> > > Thanks for your input!
> > > Roberto
> > >
> > > FreeIPA-users mailing list -- [e-mail]
> > > To unsubscribe send an email to [e-mail]
> > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives: https://lists.fedorahosted.org/archives/list/[e-mail]...

[Rob Crittenden]

Brian Topping via FreeIPA-users wrote:
> Hi Roberto, my skills here are weaker than the actual team here but they
> are busy so I thought I might be able kick in a little.
>
> Please do be careful. I recently had a situation where I had a machine
> crash during initial replication due to a bad CPU. It caused a lot of
> problems. I did not make a snapshot of the replication target before I
> did it, and the previously stable master became locked to replication
> changes, including removing the failed install node. Since I am
> converting everything to Kubernetes anyway, I just decided to start
> over. It was a lesson learned.
>
> Proper snapshots would have meant I could just shut down, roll back,
> boot up and try again.

Your best bet IMHO is standing up a new server as a replica with all
optional services (CA, DNS, etc) and set it as the renewal master,
ensure it has a DNA config (by adding a user or group) and then
decommissioning the old server.

I mean, you *could* do what you suggested and apply incremental updates
but you're looking at upgrading 5 versions minimum (to F28).

At a minimum yes, you should setup a replica to ensure that the upgrade
doesn't blow your environment up.

Be sure to ensure your certificates are in good order before starting
(getcert list).

We generally recommend at least 2 masters with a CA. I realize this may
be a bit overkill in such a small environment, you could easily suffer
from single point of failure.

rob
> Sent from my iPhone
>
> On Dec 17, 2018, at 20:49, Roberto Cornacchia
> <[e-mail] > wrote:
> > Brian,
> >
> > Thanks for your suggestions. 
> > Actually using the Docker image seems very attractive, if it is close
> > to production quality. I can experiment anyway.
> >
> > What I don't really know is whether I can experiment with creating
> > replicas of the current server. I mean, can creating replicas harm the
> > existing server in any way, in case something goes wrong?
> >
> > On Mon, 17 Dec 2018 at 13:17 Brian Topping <[e-mail]
> > > wrote:
> > Have you considered starting to use the docker image? I knew that
> > there was upgrade and it worked very well, completely automated.
> >
> > Maybe start separate test cluster with older image, then
> > experience the upgrade. When you are satisfied, use the docker
> > image that is the same version as your current cluster, then
> > upgrade it.
> >
> > Anyway, if that is uncomfortable, try to learn how to verify that
> > the replicas are synchronized with CLI. When you can confirm that
> > the replicas are synchronized, your plan is easier because you can
> > also confirm at the end that they are still synchronized.
> >
> > I would also do this with three replicas, not two. Then when you
> > shut down A replica, you can confirm B and C are properly
> > replicating as you upgrade before destroying A. If you get stuck,
> > you can still safely start over.
> >
> > Lastly, if you are using a system that supports snapshots, it’s
> > the best way to keep the original copy of a replica before making
> > changes. In the worst case, you can reset all the snapshots and
> > reboot.
> >
> > > On Dec 17, 2018, at 6:17 PM, Roberto Cornacchia via
> > FreeIPA-users <[e-mail]
> > <mailto:[e-mail]>> wrote:
> > >
> > > Dear all,
> > >
> > > Upgrading is always scary, I will appreciate any comment on the
> > following.
> > >
> > > Our freeIPA is serving a small number of FC desktops and users
> > (< 10), and is running on a FC 23 server, with packages for ipa
> > versioned 4.2.4-2.fc23.
> > >
> > > The simplest thing I can do is of course to upgrade the FC
> > system until the latest, one version at the time. What I probably
> > want to do is actually move to CentOS - I'm fed up with running
> > after FC releases.
> > >
> > > In both cases (especially in the second case), I thought it may
> > be wise to make a replica of the ipa server before starting the
> > upgrade.
> > >
> > > My plan would be:
> > > - Have an up-to-date CentOS system (IPA-B), enroll it and
> > promote it to replica of the existing one (IPA-A)
> > > - [ Question: is it better to have IPA-B on a recent version or
> > on the same version as IPA-A? ]
> > > - Shut down IPA-A
> > > - Verify that IPA-B works
> > > - Wipe out IPA-A, install recent CentOS.
> > > - Enroll IPA-A
> > > - Promote it to replica.
> > > - Enjoy
> > >
> > > Am I overlooking something? Could I do something more prudently?
> > >
> > > Thanks for your input!
> > > Roberto
> > >
> > >
> > > _______________________________________________
> > > FreeIPA-users mailing list --
> > [e-mail]
> > <mailto:[e-mail]>
> > > To unsubscribe send an email to
> > [e-mail]
> > <mailto:[e-mail]>
> > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > > List Guidelines:
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> > https://lists.fedorahosted.org/archives/list/[e-mail]
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/[e-mail]...