FreeIPA wiki migration to OpenShift v3

[Martin Kosek]

Hello all,

I would like to start a discussion regarding the migration of current
FreeIPA services that are running on OpenShift v2 that was obsoleted [1]
and will go soon EOL (the ultimate cut-off date is Dec 31, 2017).

After a short discussion I had with several FreeIPA developers, the
preference remained with keeping this application on OpenShift (v3
generation), as it will let us easily maintain it on a PaaS, without
having to care about maintaining our own infra. It will be also easy to
delegate maintenance powers to more people.

Given above, I have now set up a Pro account with OpenShift v3 and
migrated the base FreeIPA wiki as an application there, with today
snapshot of data and images. When the POC deployment is ready and
approved on this list, I can switch the current wiki to readonly and
request change of "www.freeipa.org" DNS records to get it to production.

The POC wiki is running in [2], with OpenShift application sources being
stored in a public git repo [3]. Eventually, the OpenShift could be
configured to rebuild the wiki after a git push to [3], to enable easy
changes to wiki to it's maintainers. Let me know if there are any
concerns about having the wiki sources public. The secrets and keys are
of course not in the repo, but configured via OpenShift environment
variable.

The POC now runs pretty well, the only issue I found so far is linking
the wiki user authentication with Fedora auth. The problem is that the
current OpenID plugin [4] is deprecated and does not run with modern PHP
version and I could not get the new OpenID Connect one [5] to work
reliably with our wiki and Fedora OIDC service. I either received
authentication errors or later problems with linking the authenticated
user to current account. So for now I gave up and enabled simple
password auth by password again.

Feedback welcome!

Thanks,
Martin

[1] https://blog.openshift.com/migrate-to-v3-v2-eol/
[2] https://freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
[3] https://github.com/freeipa/freeipa-wiki
[4] https://www.mediawiki.org/wiki/Extension:OpenID
[5] https://www.mediawiki.org/wiki/Extension:OpenID_Connect

-- 
Martin Kosek <mkosek(a)redhat.com&gt;
Manager, Software Engineering - Identity Management Team
Red Hat, Inc.

[Martin Kosek]

On 09/15/2017 12:54 PM, Martin Kosek wrote:
>  Hello all,
>  
>  I would like to start a discussion regarding the migration of current
>  FreeIPA services that are running on OpenShift v2 that was obsoleted [1]
>  and will go soon EOL (the ultimate cut-off date is Dec 31, 2017).
>  
>  After a short discussion I had with several FreeIPA developers, the
>  preference remained with keeping this application on OpenShift (v3
>  generation), as it will let us easily maintain it on a PaaS, without
>  having to care about maintaining our own infra. It will be also easy to
>  delegate maintenance powers to more people.
>  
>  Given above, I have now set up a Pro account with OpenShift v3 and
>  migrated the base FreeIPA wiki as an application there, with today
>  snapshot of data and images. When the POC deployment is ready and
>  approved on this list, I can switch the current wiki to readonly and
>  request change of "www.freeipa.org" DNS records to get it to production.
>  
>  The POC wiki is running in [2], with OpenShift application sources being
>  stored in a public git repo [3]. Eventually, the OpenShift could be
>  configured to rebuild the wiki after a git push to [3], to enable easy
>  changes to wiki to it's maintainers. Let me know if there are any
>  concerns about having the wiki sources public. The secrets and keys are
>  of course not in the repo, but configured via OpenShift environment
>  variable.
>  
>  The POC now runs pretty well, the only issue I found so far is linking
>  the wiki user authentication with Fedora auth. The problem is that the
>  current OpenID plugin [4] is deprecated and does not run with modern PHP
>  version and I could not get the new OpenID Connect one [5] to work
>  reliably with our wiki and Fedora OIDC service. I either received
>  authentication errors or later problems with linking the authenticated
>  user to current account. So for now I gave up and enabled simple
>  password auth by password again.
>  
>  Feedback welcome!
>  
>  Thanks,
>  Martin
>  
>  [1] https://blog.openshift.com/migrate-to-v3-v2-eol/
>  [2] https://freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
>  [3] https://github.com/freeipa/freeipa-wiki
>  [4] https://www.mediawiki.org/wiki/Extension:OpenID
>  [5] https://www.mediawiki.org/wiki/Extension:OpenID_Connect 
Hello all,

I did not see any discussion on this topic, so I assume that people
either missed my message are are fine with the progress so far.

I worked on the new FreeIPA.org wiki over the weekend and did couple fixes:
- configured the site to expose the old guides that we still keep
referenced in https://www.freeipa.org/page/Upstream_User_Guide. That
should be all the missing external content needed on the site I am aware
off.
- added new secured routes for the application, to respond on
"www.freeipa.org" and "freeipa.org"

Thanks to the second step, you can now test the deployment by simply
adding an alias to /etc/hosts:
sudo echo "52.203.52.40  www.freeipa.org" >> /etc/hosts
and then going to www.freeipa.org (delete it after the testing)

You can tell that the alias is working when
https://www.freeipa.org/page/Special:Version
shows you the 1.29.1 Mediawiki version.

So what is missing to let us migrate?

1) As mentioned above, OpenID authentication is no longer working, so
unless someone can help and make it working, we will start
authenticating with plain passwords again.

2) I need to also make the mails working (useful for password resets or
other notifications). For that, I would need an SMTP server. Unless
someone has an SMTP server I could easily use from Mediawiki:
https://www.mediawiki.org/wiki/Manual:$wgSMTP
I would need to register us with something like
https://www.mailgun.com/
that would let me send emails from "freeipa.org" domain.

Martin

[Rob Crittenden]

Martin Kosek via FreeIPA-devel wrote:
>  On 09/15/2017 12:54 PM, Martin Kosek wrote:
> > Hello all,
> >
> > I would like to start a discussion regarding the migration of current
> > FreeIPA services that are running on OpenShift v2 that was obsoleted [1]
> > and will go soon EOL (the ultimate cut-off date is Dec 31, 2017).
> >
> > After a short discussion I had with several FreeIPA developers, the
> > preference remained with keeping this application on OpenShift (v3
> > generation), as it will let us easily maintain it on a PaaS, without
> > having to care about maintaining our own infra. It will be also easy to
> > delegate maintenance powers to more people.
> >
> > Given above, I have now set up a Pro account with OpenShift v3 and
> > migrated the base FreeIPA wiki as an application there, with today
> > snapshot of data and images. When the POC deployment is ready and
> > approved on this list, I can switch the current wiki to readonly and
> > request change of "www.freeipa.org" DNS records to get it to production.
> >
> > The POC wiki is running in [2], with OpenShift application sources being
> > stored in a public git repo [3]. Eventually, the OpenShift could be
> > configured to rebuild the wiki after a git push to [3], to enable easy
> > changes to wiki to it's maintainers. Let me know if there are any
> > concerns about having the wiki sources public. The secrets and keys are
> > of course not in the repo, but configured via OpenShift environment
> > variable.
> >
> > The POC now runs pretty well, the only issue I found so far is linking
> > the wiki user authentication with Fedora auth. The problem is that the
> > current OpenID plugin [4] is deprecated and does not run with modern PHP
> > version and I could not get the new OpenID Connect one [5] to work
> > reliably with our wiki and Fedora OIDC service. I either received
> > authentication errors or later problems with linking the authenticated
> > user to current account. So for now I gave up and enabled simple
> > password auth by password again.
> >
> > Feedback welcome!
> >
> > Thanks,
> > Martin
> >
> > [1] https://blog.openshift.com/migrate-to-v3-v2-eol/
> > [2] https://freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
> > [3] https://github.com/freeipa/freeipa-wiki
> > [4] https://www.mediawiki.org/wiki/Extension:OpenID
> > [5] https://www.mediawiki.org/wiki/Extension:OpenID_Connect
>  
>  Hello all,
>  
>  I did not see any discussion on this topic, so I assume that people
>  either missed my message are are fine with the progress so far.
>  
>  I worked on the new FreeIPA.org wiki over the weekend and did couple fixes:
>  - configured the site to expose the old guides that we still keep
>  referenced in https://www.freeipa.org/page/Upstream_User_Guide. That
>  should be all the missing external content needed on the site I am aware
>  off.
>  - added new secured routes for the application, to respond on
>  "www.freeipa.org" and "freeipa.org"
>  
>  Thanks to the second step, you can now test the deployment by simply
>  adding an alias to /etc/hosts:
>  sudo echo "52.203.52.40  www.freeipa.org" >> /etc/hosts
>  and then going to www.freeipa.org (delete it after the testing)
>  
>  You can tell that the alias is working when
>  https://www.freeipa.org/page/Special:Version
>  shows you the 1.29.1 Mediawiki version.
>  
>  So what is missing to let us migrate?
>  
>  1) As mentioned above, OpenID authentication is no longer working, so
>  unless someone can help and make it working, we will start
>  authenticating with plain passwords again.
>  
>  2) I need to also make the mails working (useful for password resets or
>  other notifications). For that, I would need an SMTP server. Unless
>  someone has an SMTP server I could easily use from Mediawiki:
>  https://www.mediawiki.org/wiki/Manual:$wgSMTP
>  I would need to register us with something like
>  https://www.mailgun.com/
>  that would let me send emails from "freeipa.org" domain. 
Do you have any information on why OpenID is failing? Is this something
we can work with the Ipsilon guys on?

rob

[Martin Kosek]

On 11/15/2017 04:53 PM, Rob Crittenden wrote:
>  Martin Kosek via FreeIPA-devel wrote:
> > On 09/15/2017 12:54 PM, Martin Kosek wrote:
> >> Hello all,
> >>
> >> I would like to start a discussion regarding the migration of current
> >> FreeIPA services that are running on OpenShift v2 that was obsoleted [1]
> >> and will go soon EOL (the ultimate cut-off date is Dec 31, 2017).
> >>
> >> After a short discussion I had with several FreeIPA developers, the
> >> preference remained with keeping this application on OpenShift (v3
> >> generation), as it will let us easily maintain it on a PaaS, without
> >> having to care about maintaining our own infra. It will be also easy to
> >> delegate maintenance powers to more people.
> >>
> >> Given above, I have now set up a Pro account with OpenShift v3 and
> >> migrated the base FreeIPA wiki as an application there, with today
> >> snapshot of data and images. When the POC deployment is ready and
> >> approved on this list, I can switch the current wiki to readonly and
> >> request change of "www.freeipa.org" DNS records to get it to
> production.
> >>
> >> The POC wiki is running in [2], with OpenShift application sources being
> >> stored in a public git repo [3]. Eventually, the OpenShift could be
> >> configured to rebuild the wiki after a git push to [3], to enable easy
> >> changes to wiki to it's maintainers. Let me know if there are any
> >> concerns about having the wiki sources public. The secrets and keys are
> >> of course not in the repo, but configured via OpenShift environment
> >> variable.
> >>
> >> The POC now runs pretty well, the only issue I found so far is linking
> >> the wiki user authentication with Fedora auth. The problem is that the
> >> current OpenID plugin [4] is deprecated and does not run with modern PHP
> >> version and I could not get the new OpenID Connect one [5] to work
> >> reliably with our wiki and Fedora OIDC service. I either received
> >> authentication errors or later problems with linking the authenticated
> >> user to current account. So for now I gave up and enabled simple
> >> password auth by password again.
> >>
> >> Feedback welcome!
> >>
> >> Thanks,
> >> Martin
> >>
> >> [1] https://blog.openshift.com/migrate-to-v3-v2-eol/
> >> [2] https://freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
> >> [3] https://github.com/freeipa/freeipa-wiki
> >> [4] https://www.mediawiki.org/wiki/Extension:OpenID
> >> [5] https://www.mediawiki.org/wiki/Extension:OpenID_Connect
> >
> > Hello all,
> >
> > I did not see any discussion on this topic, so I assume that people
> > either missed my message are are fine with the progress so far.
> >
> > I worked on the new FreeIPA.org wiki over the weekend and did couple fixes:
> > - configured the site to expose the old guides that we still keep
> > referenced in https://www.freeipa.org/page/Upstream_User_Guide. That
> > should be all the missing external content needed on the site I am aware
> > off.
> > - added new secured routes for the application, to respond on
> > "www.freeipa.org" and "freeipa.org"
> >
> > Thanks to the second step, you can now test the deployment by simply
> > adding an alias to /etc/hosts:
> > sudo echo "52.203.52.40  www.freeipa.org" >> /etc/hosts
> > and then going to www.freeipa.org (delete it after the testing)
> >
> > You can tell that the alias is working when
> > https://www.freeipa.org/page/Special:Version
> > shows you the 1.29.1 Mediawiki version.
> >
> > So what is missing to let us migrate?
> >
> > 1) As mentioned above, OpenID authentication is no longer working, so
> > unless someone can help and make it working, we will start
> > authenticating with plain passwords again.
> >
> > 2) I need to also make the mails working (useful for password resets or
> > other notifications). For that, I would need an SMTP server. Unless
> > someone has an SMTP server I could easily use from Mediawiki:
> > https://www.mediawiki.org/wiki/Manual:$wgSMTP
> > I would need to register us with something like
> > https://www.mailgun.com/
> > that would let me send emails from "freeipa.org" domain.
>  
>  Do you have any information on why OpenID is failing? Is this something
>  we can work with the Ipsilon guys on? 
I actually did work with Patrick Uiterwijk on debugging the OpenID Connect
authentication from FreeIPA wiki and managed to progress! Thanks to Patrick
configuring Ipsilon to use computable issuer&subject pair, I was able to add
issuer&subject pair to FreeIPA user database from old OpenID anchors and thus
have the OIDC - Mediawiki user pairing properly set.

Details in
https://pagure.io/fedora-infrastructure/issue/6318

This unblocked the OIDC authentication for the new wiki. The only remaining bug
I am aware of is new wiki users not having the right user name ("User1"):
https://pagure.io/fedora-infrastructure/issue/6318#comment-478597
But that should not be a blocker for migration, we do not have a big amount of
new users and they could be renamed manually until the bug is fixed.

I would actually prefer to migrate soon, after I get some validation from the
team that this staging wiki is "good enough" as I cannot update the current
wiki on OpenShift v2 anymore and keep it sufficiently up to date.

To test the test wiki, one just needs to update /etc/hosts as advised above.
You just need to use the current IP address of wiki node. That can be get from:

$ host freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com is an alias for
pro-us-east-1-infra-211691592.us-east-1.elb.amazonaws.com.
pro-us-east-1-infra-211691592.us-east-1.elb.amazonaws.com has address 54.82.169.234

i.e. in this case:

sudo echo "54.82.169.234  www.freeipa.org" >> /etc/hosts

and then just going to https://www.freeipa.org (SSL cert already set) and doing
basic OIDC login sanity test. I just tested and worked for me.

Martin

[Simo Sorce]

On Wed, 2017-11-15 at 22:03 +0100, Martin Kosek via FreeIPA-devel
wrote:
>  On 11/15/2017 04:53 PM, Rob Crittenden wrote:
>  > Martin Kosek via FreeIPA-devel wrote:
>  > > On 09/15/2017 12:54 PM, Martin Kosek wrote:
>  > > > Hello all,
>  > > > 
>  > > > I would like to start a discussion regarding the migration of
>  > > > current
>  > > > FreeIPA services that are running on OpenShift v2 that was
>  > > > obsoleted [1]
>  > > > and will go soon EOL (the ultimate cut-off date is Dec 31,
>  > > > 2017).
>  > > > 
>  > > > After a short discussion I had with several FreeIPA developers,
>  > > > the
>  > > > preference remained with keeping this application on OpenShift
>  > > > (v3
>  > > > generation), as it will let us easily maintain it on a PaaS,
>  > > > without
>  > > > having to care about maintaining our own infra. It will be also
>  > > > easy to
>  > > > delegate maintenance powers to more people.
>  > > > 
>  > > > Given above, I have now set up a Pro account with OpenShift v3
>  > > > and
>  > > > migrated the base FreeIPA wiki as an application there, with
>  > > > today
>  > > > snapshot of data and images. When the POC deployment is ready
>  > > > and
>  > > > approved on this list, I can switch the current wiki to
>  > > > readonly and
>  > > > request change of "www.freeipa.org" DNS records to get it to
>  > > > production.
>  > > > 
>  > > > The POC wiki is running in [2], with OpenShift application
>  > > > sources being
>  > > > stored in a public git repo [3]. Eventually, the OpenShift
>  > > > could be
>  > > > configured to rebuild the wiki after a git push to [3], to
>  > > > enable easy
>  > > > changes to wiki to it's maintainers. Let me know if there are
>  > > > any
>  > > > concerns about having the wiki sources public. The secrets and
>  > > > keys are
>  > > > of course not in the repo, but configured via OpenShift
>  > > > environment
>  > > > variable.
>  > > > 
>  > > > The POC now runs pretty well, the only issue I found so far is
>  > > > linking
>  > > > the wiki user authentication with Fedora auth. The problem is
>  > > > that the
>  > > > current OpenID plugin [4] is deprecated and does not run with
>  > > > modern PHP
>  > > > version and I could not get the new OpenID Connect one [5] to
>  > > > work
>  > > > reliably with our wiki and Fedora OIDC service. I either
>  > > > received
>  > > > authentication errors or later problems with linking the
>  > > > authenticated
>  > > > user to current account. So for now I gave up and enabled
>  > > > simple
>  > > > password auth by password again.
>  > > > 
>  > > > Feedback welcome!
>  > > > 
>  > > > Thanks,
>  > > > Martin
>  > > > 
>  > > > [1] https://blog.openshift.com/migrate-to-v3-v2-eol/
>  > > > [2] https://freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshi
>  > > > ftapps.com
>  > > > [3] https://github.com/freeipa/freeipa-wiki
>  > > > [4] https://www.mediawiki.org/wiki/Extension:OpenID
>  > > > [5] https://www.mediawiki.org/wiki/Extension:OpenID_Connect
>  > > 
>  > > Hello all,
>  > > 
>  > > I did not see any discussion on this topic, so I assume that
>  > > people
>  > > either missed my message are are fine with the progress so far.
>  > > 
>  > > I worked on the new FreeIPA.org wiki over the weekend and did
>  > > couple fixes:
>  > > - configured the site to expose the old guides that we still keep
>  > > referenced in https://www.freeipa.org/page/Upstream_User_Guide.
>  > > That
>  > > should be all the missing external content needed on the site I
>  > > am aware
>  > > off.
>  > > - added new secured routes for the application, to respond on
>  > > "www.freeipa.org" and "freeipa.org"
>  > > 
>  > > Thanks to the second step, you can now test the deployment by
>  > > simply
>  > > adding an alias to /etc/hosts:
>  > > sudo echo "52.203.52.40  www.freeipa.org" >> /etc/hosts
>  > > and then going to www.freeipa.org (delete it after the testing)
>  > > 
>  > > You can tell that the alias is working when
>  > > https://www.freeipa.org/page/Special:Version
>  > > shows you the 1.29.1 Mediawiki version.
>  > > 
>  > > So what is missing to let us migrate?
>  > > 
>  > > 1) As mentioned above, OpenID authentication is no longer
>  > > working, so
>  > > unless someone can help and make it working, we will start
>  > > authenticating with plain passwords again.
>  > > 
>  > > 2) I need to also make the mails working (useful for password
>  > > resets or
>  > > other notifications). For that, I would need an SMTP server.
>  > > Unless
>  > > someone has an SMTP server I could easily use from Mediawiki:
>  > > https://www.mediawiki.org/wiki/Manual:$wgSMTP
>  > > I would need to register us with something like
>  > > https://www.mailgun.com/
>  > > that would let me send emails from "freeipa.org" domain.
>  > 
>  > Do you have any information on why OpenID is failing? Is this
>  > something
>  > we can work with the Ipsilon guys on?
>  
>  I actually did work with Patrick Uiterwijk on debugging the OpenID
>  Connect
>  authentication from FreeIPA wiki and managed to progress! Thanks to
>  Patrick
>  configuring Ipsilon to use computable issuer&subject pair, I was able
>  to add
>  issuer&subject pair to FreeIPA user database from old OpenID anchors
>  and thus
>  have the OIDC - Mediawiki user pairing properly set.
>  
>  Details in
>  https://pagure.io/fedora-infrastructure/issue/6318
>  
>  This unblocked the OIDC authentication for the new wiki. The only
>  remaining bug
>  I am aware of is new wiki users not having the right user name
>  ("User1"):
>  https://pagure.io/fedora-infrastructure/issue/6318#comment-478597
>  But that should not be a blocker for migration, we do not have a big
>  amount of
>  new users and they could be renamed manually until the bug is fixed.
>  
>  I would actually prefer to migrate soon, after I get some validation
>  from the
>  team that this staging wiki is "good enough" as I cannot update the
>  current
>  wiki on OpenShift v2 anymore and keep it sufficiently up to date.
>  
>  To test the test wiki, one just needs to update /etc/hosts as advised
>  above.
>  You just need to use the current IP address of wiki node. That can be
>  get from:
>  
>  $ host freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com
>  freeipa-org-wiki-freeipa.b9ad.pro-us-east-1.openshiftapps.com is an
>  alias for
>  pro-us-east-1-infra-211691592.us-east-1.elb.amazonaws.com.
>  pro-us-east-1-infra-211691592.us-east-1.elb.amazonaws.com has address
>  54.82.169.234
>  
>  i.e. in this case:
>  
>  sudo echo "54.82.169.234  www.freeipa.org" >> /etc/hosts
>  
>  and then just going to https://www.freeipa.org (SSL cert already set)
>  and doing
>  basic OIDC login sanity test. I just tested and worked for me. 
Works great, I would just migrate asap as well.

Btw, note that your sudo command wont work, this is the correct
command:
sudo sh -c 'echo "54.82.169.234  www.freeipa.org" >> /etc/hosts'

To the next tester, remember to remove that line for /etc/hosts once you are done with the
testing :-)

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc