PostgreSQL 13 - Fedora 33 Self-Contained Change proposal

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/PostgreSQL_13

== Summary ==
Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora
from version 12 to version 13 in the non-modular (main) builds.

== Owner ==
* Name: [[User:panovotn| Patrik Novotny]]
* Email: panovotn(a)redhat.com

== Detailed Description ==
Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora
from version 12 to version 13 in the non-modular (main) builds.

This also involves updating and rebuilding the PostgreSQL plugins that
depend on postgresql server.

== Benefit to Fedora ==

Latest stable software is used by Fedora users.

== Scope ==
* Proposal owners:
<!-- What work do the feature owners have to accomplish to complete
the feature in time for release?  Is it a large change affecting many
parts of the distribution or is it a very isolated change? What are
those changes?-->
**Prepare PostgreSQL 13 as a module for Rawhide and at least one
stable Fedora release (done)
**Prepare PostgreSQL 12 as a module for Rawhide, so there would be a
failover in case of problems
**Build PostgreSQL 13 to Rawhide
**Check software that requires or depends on `postgresql-server` or
`libpq` packages for incompatibilities
**Gather user input on the changes between PostgreSQL 12 and PostgreSQL 13

* Other developers: N/A (not a System Wide Change)
* Policies and guidelines: N/A (not a System Wide Change)
* Trademark approval: N/A (not needed for this Change)


== Upgrade/compatibility impact ==
The PostgreSQL client library (libpq component) is compatible, so
there shouldn't be any issues, but rebuild of the depended components
is better to be sure. There is a
[https://copr.fedorainfracloud.org/coprs/panovotn/postgresql-13beta2/
COPR build] available for testing.
Server plugins might require a newer version update, because they
sometimes have explicit server requirements. PostgreSQL maintainer
will help fixing/rebuilding any issues in the plugins.

== How To Test ==

Usual testing as when upgrading between major PostgreSQL versions,
running `postgresql-setup --upgrade` is necessary between major
versions.

Test that all other software runs well with PostgreSQL 13.

== User Experience ==

The users will have to upgrade their databases the same way as between
major PostgreSQL versions, aka `postgresql-setup --upgrade`.

If users want to stick with PostgreSQL 12 for a little longer, there
should be PostgreSQL 12 module.
If users want to test it before it reaches Fedora 33, there is a
[https://copr.fedorainfracloud.org/coprs/panovotn/postgresql-13beta2/
COPR build] available.

== Dependencies ==

There are some packages (mostly server plugins), that build on top of
PostgreSQL. Since the separation of PostgreSQL client library (libpq
component), only packages that build server plugins should use
postgresql package in BuildRequires, others should use libpq. In both
the cases, rebuild should be done to make sure all potential binary
incompatibilities are handled.

== Contingency Plan ==

Modules will provide the functional version of PostgreSQL 12,
available to all users.

== Documentation ==

Upgrade startegy: https://www.postgresql.org/docs/13/upgrading.html

== Release Notes ==

Release notes for PostgreSQL 13 release:
https://www.postgresql.org/docs/13/index.html

Overall overview of the changes and improvements:
https://www.postgresql.org/docs/13/release-13.html


-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Miro Hrončok]

On 07. 07. 20 21:16, Ben Cotton wrote:
>  == Scope ==
>  * Proposal owners:
>  <!-- What work do the feature owners have to accomplish to complete
>  the feature in time for release?  Is it a large change affecting many
>  parts of the distribution or is it a very isolated change? What are
>  those changes?-->
>  **Prepare PostgreSQL 13 as a module for Rawhide and at least one
>  stable Fedora release (done)
>  **Prepare PostgreSQL 12 as a module for Rawhide, so there would be a
>  failover in case of problems
>  **Build PostgreSQL 13 to Rawhide
>  **Check software that requires or depends on `postgresql-server` or
>  `libpq` packages for incompatibilities
>  **Gather user input on the changes between PostgreSQL 12 and PostgreSQL 13 
When we updated from PostgreSQL 11 to PostgreSQL 12 in Fedora 32, there was no 
targeted rebuild of the dependent packages and a dozen of packages failed to 
install. We were firefighting this between beta and final in:

https://bugzilla.redhat.com/show_bug.cgi?id=1811800

I would very much like to see a targeted rebuild of dependent packages in a side 
tag in the scope (who does it? when?) and a contingency mechanism that reverts 
to PostgreSQL 12 if many packages cannot build.

>  == Contingency Plan ==
>  
>  Modules will provide the functional version of PostgreSQL 12,
>  available to all users. 
If the 13 dependent packages don't install, a module with PostgreSQL 12 is 
**not** a contingency plan.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Honza Horak]

On 7/8/20 12:24 PM, Miro Hrončok wrote:
>  
>  When we updated from PostgreSQL 11 to PostgreSQL 12 in Fedora 32, there 
>  was no targeted rebuild of the dependent packages and a dozen of 
>  packages failed to install. We were firefighting this between beta and 
>  final in:
>  
>  https://bugzilla.redhat.com/show_bug.cgi?id=1811800
>  
>  I would very much like to see a targeted rebuild of dependent packages 
>  in a side tag in the scope (who does it? when?) and a contingency 
>  mechanism that reverts to PostgreSQL 12 if many packages cannot build. 
+1 for better plan for rebuilding depended packages, although not all 
packages might necessarily be prepared for v13 early enough upstream -- 
for that reason I'd like to also see a list of depended packages that we 
consider crucial (postgis, pgpool, ...), and rest that would not block 
the postgresql update if they fail to build or install.

Honza

[Bruno Wolff III]

On Tue, Jul 07, 2020 at 15:16:22 -0400,
  Ben Cotton <bcotton(a)redhat.com&gt; wrote:
> https://fedoraproject.org/wiki/Changes/PostgreSQL_13
>
> == Summary ==
> Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora
> from version 12 to version 13 in the non-modular (main) builds. 
While I like to have the latest postgresql available for my use, 
it seems pretty aggressive to try to get postgresql 13 into 
F33. Using the beta versions is a pain because the catalog will often 
change right up to release and you need to dump and reload or run 
a conversion on your data with each update. So this won't be practical 
to put in rawhide until it is released. While the release might 
be done soon, postgresql releases often end up happening in October.

[Fabio Valentini]

On Tue, Jul 7, 2020 at 9:17 PM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  == Upgrade/compatibility impact ==
>  The PostgreSQL client library (libpq component) is compatible, so
>  there shouldn't be any issues, but rebuild of the depended components
>  is better to be sure. There is a
>  [https://copr.fedorainfracloud.org/coprs/panovotn/postgresql-13beta2/
>  COPR build] available for testing. 
What does "is compatible but rebuild of dependencies is better to be
sure" mean here?
Is there an ABI / SONAME involved in the libpq update?

>  Server plugins might require a newer version update, because they
>  sometimes have explicit server requirements. PostgreSQL maintainer
>  will help fixing/rebuilding any issues in the plugins. 
I'd rather see an organized rebuild of dependent packages, preferably
in a koji side tag.
During the fedora 32 development cycle, this didn't happen at all and
postgresql plugins needed to be "rescued" shortly before the fedora 32
GA.

>  == Dependencies ==
>
>  There are some packages (mostly server plugins), that build on top of
>  PostgreSQL. Since the separation of PostgreSQL client library (libpq
>  component), only packages that build server plugins should use
>  postgresql package in BuildRequires, others should use libpq. In both
>  the cases, rebuild should be done to make sure all potential binary
>  incompatibilities are handled. 
Do you expect binary incompatibilities between libpq 12 and 13? I
would hope that not to be the case if there's not an SONAME bump ...
but if there *is* an ABI change, a targeted rebuild of those
dependencies should happen.

>  == Contingency Plan ==
>
>  Modules will provide the functional version of PostgreSQL 12,
>  available to all users. 
As Miro said - this is not a contingency plan.
We can't just ship a broken version of postgresql by default.

>  == Release Notes ==
>
>  Release notes for PostgreSQL 13 release:
>  https://www.postgresql.org/docs/13/index.html
>
>  Overall overview of the changes and improvements:
>  https://www.postgresql.org/docs/13/release-13.html 
It looks like the final release of PostgreSQL 13 will happen not long
before the fedora 33 release date (sometime in "Q3 2020" according to
upstream).

I would suggest to wait with the PostgreSQL 13 update until the final
release, and then do the necessary builds and rebuilds of dependent
packages in a side tag, and check the results. If they're bad, hold
off on the PostgreSQL 13 update until fedora 34 (or fix broken
dependencies). If the results are good, merge the side tag and ship
PostgreSQL 13 with fedora 33 (preferably before the beta release).
Everything else is asking for trouble (particularly looking back at
the corresponding f32 Change).

Fabio

[Petr Pisar]

On Tue, Jul 14, 2020 at 08:09:22PM +0200, Fabio Valentini wrote:
>  On Tue, Jul 7, 2020 at 9:17 PM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
>  > == Contingency Plan ==
>  >
>  > Modules will provide the functional version of PostgreSQL 12,
>  > available to all users.
>  
>  As Miro said - this is not a contingency plan.
>  We can't just ship a broken version of postgresql by default.
>   
Default streams are not allowed in Fedora. Thus whatever content is shipped as
a module, it won't affect users by default. I cannot see a problem in shipping
a broken PostgreSQL in a module, since it's always an explicit user's choice
to enable, and install it. I understand the proposed contingency plan as
providing the broken server as an experimental stream.

-- Petr

[Miro Hrončok]

On 15. 07. 20 9:03, Petr Pisar wrote:
>  On Tue, Jul 14, 2020 at 08:09:22PM +0200, Fabio Valentini wrote:
> > On Tue, Jul 7, 2020 at 9:17 PM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
> >> == Contingency Plan ==
> >>
> >> Modules will provide the functional version of PostgreSQL 12,
> >> available to all users.
> >
> > As Miro said - this is not a contingency plan.
> > We can't just ship a broken version of postgresql by default.
> >
>  Default streams are not allowed in Fedora. Thus whatever content is shipped as
>  a module, it won't affect users by default. I cannot see a problem in shipping
>  a broken PostgreSQL in a module, since it's always an explicit user's choice
>  to enable, and install it. I understand the proposed contingency plan as
>  providing the broken server as an experimental stream. 
The problem is not with shipping broken PostgreSQL in a module, but shipping 
broken PostgreSQL as non-modular default and having a contingency that says: 
"use modules".

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Bruno Wolff III]

On Tue, Jul 07, 2020 at 15:16:22 -0400,
  Ben Cotton <bcotton(a)redhat.com&gt; wrote:
> https://fedoraproject.org/wiki/Changes/PostgreSQL_13
>
> == Summary ==
> Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora
> from version 12 to version 13 in the non-modular (main) builds. 
They are going to release 13beta3 on Thursday, so we aren't going to see 
a 13 release until very close to Fedora 33's release.

[Miro Hrončok]

On 07. 07. 20 21:16, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/PostgreSQL_13
>  
>  == Summary ==
>  Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora
>  from version 12 to version 13 in the non-modular (main) builds. 
Hello Patrik.

Is PostgreSQL 13 considered for Fedora 34?

Would you like some provenpackager help?

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok