MongoDB in EPEL7

[Marek Skalický]

Hi,
current situation:
EPEL6 - MongoDB 2.4.x
EPEL7 - MongoDB 2.6.x

Upstream supports only upgrade to next major version. So from 2.4 it is supported only to
2.6.
Therefore I kept MongoDB 2.6 in EPEL7 (even two next major versions are released).

But MongoDB 2.6 is going to EOL (probably this week), so MongoDB in EPEL7 will be
unsupported.

How to solve this - what EPEL/Fedora guidelines says about upgrades? Upgrade EPEL6 to
EPEL7? Or keep unsupported version in EPEL7? 

I've post same question to devel(a)lists.fp.o [1] - there I was noticed that epel-devel
would be better.

Upstream says that there might be some minor incompatibilities between major releases and
it suggests users to check and fix these incompatibilities before upgrade. So these
upgrade might break some users instances.

I like answer of Peter Robinson [2] - upgrade to next major version. Left some time for
users to upgrade to this new version and after that prepare major upgrade again.
Does this comply to guidelines? (If I send some announcement to this listbefore)

Thanks,
Marek

[1]
https://lists.fedoraproject.org/archives/list/[e-mail].o...
[2]
https://lists.fedoraproject.org/archives/list/[e-mail].o...

[Tom Boutell]

There is no version 2.8 of Mongo, they renamed that to 3.0 before release of 3.0.

Just to provide a sense of the thought process other EPEL7 users might be going through
here:

I started researching this when I became aware that 2.6 was nearing upstream end-of-life
(that is now happening today).

I was aware of the recent major version upgrade of nodejs in EPEL7, so I assumed something
like that was imminent. However, when I looked here:

http://koji.fedoraproject.org/koji/buildinfo?buildID=802214

I saw that there have been several patches to EPEL6 MongoDB 2.4 since it reached its
upstream end-of-life in March.

So based on that evidence of prior commitment to maintaining MongoDB past end-of-life, I
assumed that the intention was to keep on backporting patches to both 2.4 and 2.6. And I
cancelled my red alert and stopped worrying about transitioning to the official MongoDB
repositories.

However it sounds like Marek, as the maintainer, is saying he doesn't have time to
maintain 2.4 or 2.6. 

That's fair of course. Free is a very good price, and all that. (:

For EPEL7, the upgrade seems fairly straightforward. 3.0 was really a marketing switch in
the name of 2.8. I have experienced few problems moving projects between 3.0 and 2.6.
Swapping the binaries and restarting is officially supported according to the
documentation. The compatibility break pages are pretty short and there isn't much
that would be in typical queries.

For EPEL6, the upgrade is harder because you must first bump them to 2.6 and then to 3.0,
there is no support for moving directly from 2.4 to 3.0. We could push out 2.6, wait a few
weeks, and push out 3.0, but this would have a very unsatisfactory result for people who
just don't happen to upgrade during those few weeks. They would get moved directly
from 2.4 to 3.0 and the process would not work.

There are also more bc breaks moving from 2.4, 2.6 and above are "pickier,"
notably a $set with no properties in the object will fail.
https://docs.mongodb.com/v2.6/release-notes/2.6-compatibility/#driver-com...

In both cases, a proper announcement is necessary.

A fair question is whether we should move to 3.0 or 3.2. The folks maintaining nodejs for
EPEL chose to move from 0.10 to 6.x, skipping 4.x, because one bc break is better than
two.

So what I would suggest is this:

FOR EPEL 6

* Announce our intentions.
* IN THE MEANTIME: If any really major security fails occur between now and when we get
this done, we grit our teeth and patch 2.4.
* Publish a 3.0 package that refuses to install with an appropriate error message if it
sees 2.4 (but is fine with seeing 2.6).
* Simultaneously publish 2.6-for-upgrading package; the instructions for those with 2.4
point you to that package, and to this URL:

https://docs.mongodb.com/v3.2/release-notes/2.6-upgrade/

* Users must manually install the 2.6-for-upgrading package, since the possible
complexities of a 2.4 to 2.6 upgrade are beyond an automated post-install script IMHO.
Notably:

https://docs.mongodb.com/v3.2/release-notes/2.6-upgrade/#upgrade-auth-prereq

* Once a user succeeds in that process, they can "yum update" painlessly to 3.0
like the EPEL7 people (see below).

FOR EPEL 7

* Announce our intentions. I don't know how much notice is thought appropriate. The
nodejs maintainers gave a lot of warning, but they also knew what was up way before end of
life (:
* Prepare a package for 3.0. It's a forced upgrade from 2.6.
* Release it. If possible, print this URL after install: 
https://docs.mongodb.com/v3.2/release-notes/3.0-compatibility/
* Done (:

Just my suggestion — I realize I just blew into town.

Hope we can figure this out before the next CVE!

[Marek Skalický]

To note, I've forgotten that MongoDB 2.6 is in RHSCL, so it should be supported till
April 2018.

Also I have no problem to backport easy fixes for EPEL6 too, but if problem is harder and
it is not possible create relatively small patch, I can't manage big patches.

Could some user want to still use old 2.6 version?

How much to care about EPEL6? RHEL6 is in Production 2 phase... so no software
enhancements. What is EPEL policy?

Other option how to provide newer versions of MongoDB could be to prepare copr
repositories witch each version. One pros for this option could be easier managing newer
dependencies (I am afraid that newer versions requires packages that are not in EPEL - and
adding new version specific packages to Fedora/EPEL requires package review... so it is
harder:-).

So other option how to solve this:
Try to prepare copr repositories for each version to allow users to easily install and use
new version. Keep 2.4 and 2.6 in EPEL6/7 and backport CVE fixes (it seems to me, that is
not so often in MongoDB). And if some CVE appear that would not be so easy to fix (rewrite
a lot of code), do upgrade to newer versions (~ build packages from copr in EPEL).

Does someone know if there is some EPEL guideline which describes what is better solution?

[Tom Boutell]

I think that if a CVE arrives that we can't easily address through a patch, we have to
be prepared to force an upgrade. Potentially "abandoning" a package that has
CVEs in the wild, in the hope people will read about an optional upgrade, sounds like a
policy we could regret.

Is there any history of EPEL just abandoning a package? What should happen in that
situation? Perhaps it's been necessary at some point (no support upstream, no one
available downstream either...).

[Tom Boutell]

Sorry, I just saw the part about RHSCL. But that page says:

"Community Project: Maintained by upstream communities of developers. The software is
cared for, but the developers make no commitments to update the repositories in a timely
manner."

Soooo... is there actually a commitment from Red Hat to patch that at all now that the
upstream is done with it? Are you the upstream they are referring to? (:

[Stephen John Smoogen]

On 1 November 2016 at 13:32, Tom Boutell <tom(a)punkave.com&gt; wrote:
>  I think that if a CVE arrives that we can't easily address
> through a patch, we have to be prepared to force an upgrade. Potentially
> "abandoning" a package that has CVEs in the wild, in the hope people will read
> about an optional upgrade, sounds like a policy we could regret.
>
>  Is there any history of EPEL just abandoning a package? What should happen in that
> situation? Perhaps it's been necessary at some point (no support upstream, no one
> available downstream either...). 
There is an incredibly long history of EPEL abandoning packages for
the above reasons all the time. It has been done pretty much from the
get-go. The standard practice has been that when a package no longer
is workable that it is withdrawn.

Yes it sucks all around but in many cases this is the path that has been taken.


>  _______________________________________________
>  epel-devel mailing list -- epel-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to epel-devel-leave(a)lists.fedoraproject.org 


-- 
Stephen J Smoogen.

[Stephen John Smoogen]

On 1 November 2016 at 13:33, Tom Boutell <tom(a)punkave.com&gt; wrote:
>  Sorry, I just saw the part about RHSCL. But that page says:
>
>  "Community Project: Maintained by upstream communities of developers. The software
> is cared for, but the developers make no commitments to update the repositories in a
> timely manner."
>  
Do you have a link to that? There is softwarecollections.org which is
the equivalent to EPEL. There is Red Hat Software Collections which is
not a community product but a paid for serivce from Red Hat.

>  Soooo... is there actually a commitment from Red Hat to patch that at
> all now that the upstream is done with it? Are you the upstream they are referring to? (:
>  _______________________________________________
>  epel-devel mailing list -- epel-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to epel-devel-leave(a)lists.fedoraproject.org 


-- 
Stephen J Smoogen.

[Tom Boutell]

Stephen, good question... I was looking here:

https://www.softwarecollections.org/en/scls/rhscl/rh-mongodb26/

Which came up first in Google for rhsc mongodb, but that's not right. I should have
been looking here:

https://access.redhat.com/support/policy/updates/rhscl

The Software Collections pages are so much more SEO-friendly it's not surprising I
wound up in the wrong place.

OK, so does this "RHSC will maintain 2.6 until October 2018" policy mean that
it's reasonable to expect the EPEL package to just leverage that work and keep on
trucking until then?

[Stephen John Smoogen]

On 1 November 2016 at 16:50, Tom Boutell <tom(a)punkave.com&gt; wrote:
>  Stephen, good question... I was looking here:
>
>  https://www.softwarecollections.org/en/scls/rhscl/rh-mongodb26/
>
>  Which came up first in Google for rhsc mongodb, but that's not right. I should have
> been looking here:
>
>  https://access.redhat.com/support/policy/updates/rhscl
>
>  The Software Collections pages are so much more SEO-friendly it's not surprising I
> wound up in the wrong place.
>
>  OK, so does this "RHSC will maintain 2.6 until October 2018" policy mean that
> it's reasonable to expect the EPEL package to just leverage that work and keep on
> trucking until then? 
I don't know if it is reasonable for several reasons.

1. We have no access (no pun intended) to those packages. [Yes the
person packages mongodb in Fedora works for Red Hat but that does not
mean they have access to the bits that another group is doing for SCL
work.]
2. SCL packages use their own form of magic and have their own
solutions to security issues. This can mean what they do for a
particular package won't work outside of an SCL environment. [This may
not be the case for mongodb.. I don't know enough to say yes/no.]







-- 
Stephen J Smoogen.

[Tom Boutell]

I see. Since MongoDB is under a GNU license, I assume you do not literally mean you have
zero access to the changes being made to it in SCL. My assumption is that you actually
mean there's no advanced or privileged access. So if some bad juju goes down, and we
want to look to SCL For help, Marek or whoever is maintaining 2.6 for EPEL at the time
would have to wait for those packages to appear in SCL before the process of porting them
to EPEL can even begin, time during which 2.6 is still vulnerable. Yes?

As for other solutions to security issues, is there any history of these packages
resolving security issues with mongodb with external OS-level features rather than via
patches to the code? It seems unlikely, in that a hack like firewalling it would be too
unsubtle by half and break functionality outright.

[Stephen John Smoogen]

On 2 November 2016 at 10:24, Tom Boutell <tom(a)punkave.com&gt; wrote:
>  I see. Since MongoDB is under a GNU license, I assume you do not
> literally
>  mean you have zero access to the changes being made to it in SCL. 
Just because something is under a GNU license does not mean that
anyone has access to the file. The GNU license only covers the rights
of a person who got the executable from the 'vendor' that they have
access to the source code.

>  My assumption is that you actually mean there's no advanced or
> privileged
>  access. So if some bad juju goes down, and we want to look to SCL For help,
>  Marek or whoever is maintaining 2.6 for EPEL at the time would have to wait
>  for those packages to appear in SCL before the process of porting them to
>  EPEL can even begin, time during which 2.6 is still vulnerable. Yes? 
What I am saying is that EPEL is made up of volunteers. If you are
volunteering to do this work then great. If you are expecting that
someone else is going to do this work for you.. then not so great.

>  As for other solutions to security issues, is there any history of these packages
> resolving security issues with mongodb with external OS-level features rather than via
> patches to the code? It seems unlikely, in that a hack like firewalling it would be too
> unsubtle by half and break functionality outright.
>  _______________________________________________
>  epel-devel mailing list -- epel-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to epel-devel-leave(a)lists.fedoraproject.org 


-- 
Stephen J Smoogen.

[Tom Boutell]

"What I am saying is that EPEL is made up of volunteers. If you are
volunteering to do this work then great. If you are expecting that
someone else is going to do this work for you.. then not so great."

Oh, absolutely. I'm not under the slightest illusion that anybody owes me maintenance
of an EPEL package. You can understand my desire to find out if somebody might be doing
it. But I'm perfectly OK with the response that they probably won't.

It's starting to look like the official MongoDB repositories might make a whole lot
more sense for the average admin currently using EPEL MongoDB packages:

https://docs.mongodb.com/v3.0/tutorial/install-mongodb-on-red-hat/

If I find out they aren't doing such a hot job with these, then the picture might
change.

[Marek Skalický]

>  1. We have no access (no pun intended) to those packages.[Yes the
>  person packages mongodb in Fedora works for Red Hat but that does not
>  mean they have access to the bits that another group is doing for SCL
>  work.] 
First, everyone should have an access to those packages - available source code is
required by most open source licenses. That is why CentOS can rebuild all RH stuff.

Also person (currently me) packaging MongoDB in RedHat for Fedora and SCL is the same.

>  2. SCL packages use their own form of magic and have their own
>  solutions to security issues. This can mean what they do for a
>  particular package won't work outside of an SCL environment. [This may
>  not be the case for mongodb.. I don't know enough to say yes/no.] 
MongoDB in collection is the same, only installed in different prefix. So patches for same
MongoDB version inside SCL and in Fedora/EPEL are identical.

[Marek Skalický]

Even there should be security patches for 2.6 for some time, I am not against upgrade in
EPEL7. I am not sure that upgrading in EPEL6 worth the time - I am afraid that a lot of
dependencies is outdated in EPEL6 and people could use newer RHEL version...

So is everyone fine with announcing this upgrade and preparing it? How long should take
period between upgrade steps (2.6->3.0, time period, 3.0->3.2) - two months?

Also I am not sure if I will upgrade to the latest version - it depends on how much
dependencies would have to bring to EPEL. Volunteers welcomed!

[Marek Skalický]

>  It's starting to look like the official MongoDB repositories
> might make a whole lot
>  more sense for the average admin currently using EPEL MongoDB packages:
>  
>  https://docs.mongodb.com/v3.0/tutorial/install-mongodb-on-red-hat/
>  
>  If I find out they aren't doing such a hot job with these, then the picture might
>  change. 
It may serve good. Everyone have to decide what fits better for him...

Some points:
1. I am not sure what is the current status of integration with systemctl (used on
RHEL7).
2. Even upstream won't prepare CVE fixes after version EOL.
3. Upstream packages are bundling all dependent packages (~10 for 2.6) - so upstream have
to handle also all (security) fixes for those packages. Not sure what is their policy.
3. SELinux support/integration is better for EPEL packages.

[Tom Boutell]

Thanks Marek, that does clear up a lot of confusion.

I think you're saying that because the maintainer is the same and the packages are
essentially the same (in this case), it's reasonably likely maintenance of 2.6 for
EPEL7 will continue until 2.6 reaches its end date in SCL (April 2018). Not certain, but
signs point to yes.

Is that a reasonable summary?

Thanks!