https://fedoraproject.org/wiki/Changes/PostgreSQL_13
== Summary == Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora from version 12 to version 13 in the non-modular (main) builds.
Also, there will be a design change in postgresql modular packaging regarding the external libpq library ensuring future build time compatibility. More information in the Detailed Description section.
== Owner == * Name: [[User:panovotn| Patrik Novotny]] * Email: panovotn@redhat.com
== Detailed Description == Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora from version 12 to version 13 in the non-modular (main) builds.
This also involves updating and rebuilding the PostgreSQL plugins that depend on postgresql server.
There will also be a change to the packaging of postgresql modules, as with those, the external libpq library can be potentially problematic upon upstream releases.
As postgresql packages are built against the separated libpq package, an incompatibility between major versions can occur with new minor upstream releases. For example, when building postgresql version 12.4 against libpq version 13.1.
To keep the benefits of having separately maintained libpq library, libpq stays as a separate package.
However, to solve the potential issues with the postgresql builds, we will bundle libpq with a downstream changed SONAME within each postgresql release. The postgresql will be built against this libpq.
This bundled libpq will always be the same version as the corresponding postgresql as they will share the same codebase. As the libpq SONAME will be changed downstream to a non-standard SONAME, and separate libpq package with standard SONAME will be provided, this change won't affect the user experience for those components.
=== Plan ===
* Prepare Libpq 13, PostgreSQL 13 (both rawhide and module) and PostgreSQL 12 module * Rebuild dependencies in Copr ** By 2021-01-15
* Debug and fix compatibility issues found in dependencies (reasonable amount of non-critical in FTBFS state might be tolerable) ** By 2021-01-22
* Prepare builds in a side-tag * Merge side-tag into Rawhide ** By 2021-01-29
== Benefit to Fedora ==
Latest stable software is used by Fedora users.
== Scope == * Proposal owners: <!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> **Prepare PostgreSQL 13 as a module for Rawhide and at least one stable Fedora release (including the design change and symbol versioning fix) **Prepare PostgreSQL 12 as a module for Rawhide **Check software that requires or depends on `postgresql-server` or `libpq` packages for incompatibilities **Build PostgreSQL 13 (postgresql and libpq) to Rawhide in a side-tag (https://fedoraproject.org/wiki/Package_update_HOWTO#Creating_a_side-tag) **Rebuild depended packages against PostgreSQL 13 in the side-tag (see Dependencies) **Merge the side-tag to Rawhide **Gather user input on the changes between PostgreSQL 12 and PostgreSQL 13
* Other developers: N/A (not a System Wide Change) * Policies and guidelines: N/A (not a System Wide Change) * Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact == The PostgreSQL client library (libpq component) is compatible, but there is an issue with symbol versioning (originally reported https://bugzilla.redhat.com/show_bug.cgi?id=1893324 was reverted, but there is a plan to fix this properly together with this PostgreSQL update tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1908268). So, there shouldn't be any issues with API compatibility, but rebuild of the depended components is necessary.
Server plugins might require a newer version update, because they sometimes have explicit server requirements. PostgreSQL maintainer will help fixing/rebuilding any issues in the plugins.
== How To Test ==
Usual testing as when upgrading between major PostgreSQL versions, running `postgresql-setup --upgrade` is necessary between major versions.
Test that all other software runs well with PostgreSQL 13.
== User Experience ==
The users will have to upgrade their databases the same way as between major PostgreSQL versions, aka `postgresql-setup --upgrade` after installing PostgreSQL 13 server packages.
If users want to stick with PostgreSQL 12 for a little longer, there will be PostgreSQL 12 module
== Dependencies ==
There are some packages (mostly server plugins), that build on top of PostgreSQL. Since the separation of PostgreSQL client library (libpq component), only packages that build server plugins should use postgresql package in BuildRequires, others should use libpq. In both the cases, rebuild should be done to make sure all potential binary incompatibilities are handled.
* Critical dependecies - components with higher importance, will be prioritised during the rebase ** apr-util ** bacula ** bind ** cyrus-imapd ** cyrus-sasl ** dovecot ** freeradius ** gdal ** libecpg ** nagios-plugins ** opendbx ** perl-DBD-Pg ** php ** pgaudit ** postfix ** qt ** qt3 ** qt5-qtbase ** redland ** rsyslog ** rubygem-pg ** postgres-decoderbufs
The rest of the dependencies is not considered critical. Though, those will need to be rebuild as well.
* libpq ** EMBOSS ** Io-language ** PDAL ** PyGreSQL ** asterisk ** biboumi ** clisp ** collectd ** dpsearch ** exim ** flow-tools ** freight-tools ** gambas3 ** gammu ** gawk-pgsql ** gearmand ** gnatcoll-db ** gnokii ** grass ** hydra ** jabberd ** kdb ** kea ** libdbi-drivers ** libgda ** libnss-pgsql ** libpqxx ** libpreludedb ** libzdb ** lua-dbi ** lua-sql ** mapnik ** mapserver ** medusa ** minetestmapper ** ocaml-postgresql ** opensips ** openvas-manager ** osm2pgsql ** pdns ** perl-pgsql_perl5 ** pg_top ** pgadmin3 ** pgcenter ** pgmodeler ** php-pecl-pq ** player ** postgis ** postgresql ** postgresql-pgpool-II ** proftpd ** pspp ** pure-ftpd ** rdkit ** rhdb-utils ** root ** saga ** soci ** spatialite-gui ** sphinx ** sysbench ** tcl-pgtcl ** unicornscan ** virtualpg ** vtk ** zabbix
* postgresql-server ** gambas3 ** kdb ** kea ** libpqxx ** openvas-manager ** orafce ** pg-semver ** pgRouting ** pgadmin3 ** pgsphere ** postgis ** postgresql-ip4r ** postgresql-pgpool-II ** qt3 ** rdkit ** rhdb-utils ** timescaledb
== Contingency Plan ==
Revert changes in the non-modular packages and provide PostgreSQL 13 as a module stream only.
Soft contingency plan deadline: Before Mass Rebuild (scheduled on Wed 2021-01-20)
Hard contingency plan deadline: Before Branch Fedora 34 from Rawhide (scheduled on Tue 2021-02-09)
== Documentation ==
Upgrade startegy: https://www.postgresql.org/docs/13/upgrading.html
== Release Notes ==
Release notes for PostgreSQL 13 release: https://www.postgresql.org/docs/13/index.html
Overall overview of the changes and improvements: https://www.postgresql.org/docs/13/release-13.html
On Tue, Jan 12, 2021 at 03:57:02PM -0500, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/PostgreSQL_13
== Summary == Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora from version 12 to version 13 in the non-modular (main) builds.
Also, there will be a design change in postgresql modular packaging regarding the external libpq library ensuring future build time compatibility. More information in the Detailed Description section.
== Owner ==
- Name: [[User:panovotn| Patrik Novotny]]
- Email: panovotn@redhat.com
== Detailed Description == Update of PostgreSQL (`postgresql` and `libpq` components) in Fedora from version 12 to version 13 in the non-modular (main) builds.
This also involves updating and rebuilding the PostgreSQL plugins that depend on postgresql server.
There will also be a change to the packaging of postgresql modules, as with those, the external libpq library can be potentially problematic upon upstream releases.
As postgresql packages are built against the separated libpq package, an incompatibility between major versions can occur with new minor upstream releases. For example, when building postgresql version 12.4 against libpq version 13.1.
To keep the benefits of having separately maintained libpq library, libpq stays as a separate package.
However, to solve the potential issues with the postgresql builds, we will bundle libpq with a downstream changed SONAME within each postgresql release. The postgresql will be built against this libpq.
This bundled libpq will always be the same version as the corresponding postgresql as they will share the same codebase. As the libpq SONAME will be changed downstream to a non-standard SONAME, and separate libpq package with standard SONAME will be provided, this change won't affect the user experience for those components.
Hmm, this sounds rather complicated and risky. Do I get this right that postgresql will bundle a copy of libpq, and a separate unbundled libpq will be provided?
Why not just include a specific Requires on a specific version of libpq? (Maybe something like Requires:libpq(%_isa)>=x.y.z and libpq(%_isa)<x.y.z^ or whatever the best syntax is).
There are plenty of packages which require some specific version of a separately-packages library. We don't normally use bundling in such cases. Since both packages are under the same maintainership, it should be easy enough to keep them in sync.
== Upgrade/compatibility impact == The PostgreSQL client library (libpq component) is compatible, but there is an issue with symbol versioning (originally reported https://bugzilla.redhat.com/show_bug.cgi?id=1893324 was reverted, but there is a plan to fix this properly together with this PostgreSQL update tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1908268). So, there shouldn't be any issues with API compatibility, but rebuild of the depended components is necessary.
Those bugs have a lot of detail... It seems that the only real solution is to retroactively bump the so version. I couldn't figure out if that is what happened upstream.
Zbyszek
On Tuesday, January 12, 2021 9:57:02 PM CET Ben Cotton wrote:
As postgresql packages are built against the separated libpq package, an incompatibility between major versions can occur with new minor upstream releases. For example, when building postgresql version 12.4 against libpq version 13.1.
Last time when I checked with upstream, this shouldn't actually happen. We observed a few small problems from what I remember, but those were immediately fixed by follow-up patch upstream (to fix compatibility).
Also, Debian used to _run_ ¹ all the PostgreSQL server variants against one, the most up2date major version of libpq library (even though all libpq variants are provided at the same time).
Pavel
¹ The approach in Fedora modules was better, because we also built the package against the system-wide library, so we actually unit-tested the server+lib pair.
Hmm, this sounds rather complicated and risky. Do I get this right that postgresql will bundle a copy of libpq, and a separate unbundled libpq will be provided?
Why not just include a specific Requires on a specific version of libpq? (Maybe something like Requires:libpq(%_isa)>=x.y.z and libpq(%_isa)<x.y.z^ or whatever the best syntax is).
There are plenty of packages which require some specific version of a separately-packages library. We don't normally use bundling in such cases. Since both packages are under the same maintainership, it should be easy enough to keep them in sync.
The libpq library is part of the postgresql codebase. We have previously decided to separate it downstream and package it as a separate component. This means that different versions of postgresql are built modularly against a single non-modular libpq library.
Upstream releases minor updates for all supported major version at once like this:
13.0 -> 13.1 12.4 -> 12.5 11.9 -> 11.10 10.14 -> 10.15 9.6.19 -> 9.6.20 9.5.23 -> 9.5.24
This means that to be able to rebase all postgresql streams (13,12,11,10,9.6) to their latest minor release versions, first we need to rebase the libpq library to the (in this case) 13.1 version. In that scenario, all streams except postgresql:13 are being built against different version of libpq, even though there is the correct libpq version in each postgresql source tarball for each stream, as libpq is part of its codebase.
Historically, this way of packaging postgresql with separated libpq worked. However, upstream stated that they do not guarantee such compatibility, and postgresql was never intended to be built against external libpq.
With the release version 13.1, we encountered some (thankfully) minor incompatibilities, causing us to patch downstream all streams up to postgresql:13.
This new solution is not a classic bundle. It's a return to how postgresql is supposed to be built, while keeping a separate libpq package for users.
Those bugs have a lot of detail... It seems that the only real solution
is to retroactively bump the so version. I couldn't figure out if that is what happened upstream.
The only change here is the symbol version, as they are versioned downstream, and a mistake happened regarding that patch on rebase to the version 12.x. There is no upstream change, nor are any symbols actually being changed.
On Fri, Jan 29, 2021 at 04:17:02PM +0100, Patrik Novotny wrote:
Hmm, this sounds rather complicated and risky. Do I get this right that postgresql will bundle a copy of libpq, and a separate unbundled libpq will be provided?
Why not just include a specific Requires on a specific version of libpq? (Maybe something like Requires:libpq(%_isa)>=x.y.z and libpq(%_isa)<x.y.z^ or whatever the best syntax is).
There are plenty of packages which require some specific version of a separately-packages library. We don't normally use bundling in such cases. Since both packages are under the same maintainership, it should be easy enough to keep them in sync.
The libpq library is part of the postgresql codebase. We have previously decided to separate it downstream and package it as a separate component. This means that different versions of postgresql are built modularly against a single non-modular libpq library.
Upstream releases minor updates for all supported major version at once like this:
13.0 -> 13.1 12.4 -> 12.5 11.9 -> 11.10 10.14 -> 10.15 9.6.19 -> 9.6.20 9.5.23 -> 9.5.24
This means that to be able to rebase all postgresql streams (13,12,11,10,9.6) to their latest minor release versions, first we need to rebase the libpq library to the (in this case) 13.1 version. In that scenario, all streams except postgresql:13 are being built against different version of libpq, even though there is the correct libpq version in each postgresql source tarball for each stream, as libpq is part of its codebase.
Historically, this way of packaging postgresql with separated libpq worked. However, upstream stated that they do not guarantee such compatibility, and postgresql was never intended to be built against external libpq.
With the release version 13.1, we encountered some (thankfully) minor incompatibilities, causing us to patch downstream all streams up to postgresql:13.
This new solution is not a classic bundle. It's a return to how postgresql is supposed to be built, while keeping a separate libpq package for users.
Thank you for the explanation.
Zbyszek
Those bugs have a lot of detail... It seems that the only real solution
is to retroactively bump the so version. I couldn't figure out if that is what happened upstream.
The only change here is the symbol version, as they are versioned downstream, and a mistake happened regarding that patch on rebase to the version 12.x. There is no upstream change, nor are any symbols actually being changed.
-
Ben Cotton -
Patrik Novotny -
Pavel Raiskup -
Zbigniew Jędrzejewski-Szmek