F34 Change proposal: MariaDB 10.5 (Self-Contained Change)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/MariaDB_10.5

== Summary ==
Update of MariaDB ('mariadb' package) in Fedora from 10.4 to 10.5 version.
[[Category:Package MariaDB]]

== Owner ==
* Name: [[User:mschorm| Michal Schorm]]
* Email: [e-mail]

== Detailed Description ==
Update of MariaDB package in Fedora from 10.4 version to 10.5 version.

== Benefit to Fedora ==

I'm cooperating with the upstream to bring the latest stable software to
Fedora users.

10.5 series introduces number of enhancements, which cannot be found in
previous series.
Overview of the new features can be found here:
https://mariadb.com/kb/en/changes-improvements-in-mariadb-105/

== Scope ==
* Proposal owners:
**Prepare MariaDB 10.4 as a module for Rawhide and atleast one stable
Fedora release (done)<br />so users which want to stay on the current
release have the possibility.<br />This also serve as a failover mechanism
in case of issues with the 10.5.
**Prepare MariaDB 10.5 as a module for Rawhide and atleast one stable
Fedora release (done in Rawhide; the rest in BODHI)<br />so users can test
the 10.5 in advance. (installing 10.5 module on already stable release)<br
/>This also serve as a upgrade path - users can install 10.5 module on
Fedora release which have 10.4 in base; and then upgrade to 10.5 module on
a Fedora release which will have 10.5 in base.
**Release MariaDB 10.5 to Rawhide (blocked; 10.5 modules needs testing
first)
**Check software that requires or depends on 'mariadb' or 'galera' package
for incompatibilities<br />This shouldn't be an issue in general, as vast
majority of the software requires client library, provided by
"mariadb-connector-c" package, which won't change.
**Gather user input on the changes between MariaDB 10.4 and 10.5

* Other developers: N/A (not a System Wide Change)
* Release engineering:
* Policies and guidelines: N/A (not a System Wide Change)
* Trademark approval: N/A (not needed for this Change)

== Upgrade/compatibility impact ==

The MariaDB client library is compatible, so the shouldn't be any issues
and / or need for rebuild of dependent packages.

==='''UPDATE (10/2020)'''===
MariaDB 10.5 modules are now available for Fedora Rawhide and in BODHI for
the stable releases

== How To Test ==

Usual testing as when upgrading between major MariaDB versions.

Test that all other software runs well with MariaDB 10.5.
Report any issues, so I can reach the different upstreams and check if they
plan update their software to support MariaDB 10.5 and when.

== User Experience ==

The users will have to upgrade their databases the same way as between
major MariaDB versions.

If the users want to stick with MariaDB 10.4 for a little longer, the
MariaDB 10.4 module is available for them in all stable Fedora releases as
well as in Rawhide.
If the users want to test the 10.5 series beforehand, the MariaDB 10.5
module is available.

== Dependencies ==

Since the client library ('mariadb-connector-c') is not changing, dependent
software should work fine.

Only a rare cases builds against the server part of MariaDB. (e.g. building
a server plugin)

== Contingency Plan ==

Modules will provide the functional version of MariaDB 10.4, available to
all users.

* Contingency mechanism: Fedora Modules for 10.4 available
* Contingency deadline: already in place
* Blocks release? N/A (not a System Wide Change)
* Blocks product? N/A (not a System Wide Change)

== Documentation ==

Upgrade startegy:
https://mariadb.com/kb/en/library/upgrading-from-mariadb-104-to-mariadb-105/

Upgrading and incompatibilities:
https://mariadb.com/kb/en/library/upgrading-from-mariadb-104-to-mariadb-105/...

== Release Notes ==

Release notes for each release:
https://mariadb.com/kb/en/library/release-notes-mariadb-105-series/

Overall overview of the changes and improvements:
https://mariadb.com/kb/en/library/changes-improvements-in-mariadb-105/

[Michal Schorm]

Thanks Ben for the announcement.

The MariaDB modules are available for testing right now in Rawhide and
in BODHI. [1]
I'll be thankful for any testing of the modules, so I can look into
any issues that might appear, before getting the MariaDB 10.5 in the
Fedora Rawhide base.

MariaDB upstream aligned their release cycles to be more predictable.
A new series should come out every year.
With that said, I look forward to similar Self Contained Changes with
similar failover mechanisms (modules of the current and new version
available), so I'll also be glad for any tips for improvements of
handling this and similar changes.

[1] https://bodhi.fedoraproject.org/updates/?search=mariadb-10.5

--

Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat

[Petr Pisar]

On Mon, Oct 26, 2020 at 07:17:49PM +0100, Michal Schorm wrote:
> The MariaDB modules are available for testing right now in Rawhide and
> in BODHI. [1]
> I'll be thankful for any testing of the modules, so I can look into
> any issues that might appear, before getting the MariaDB 10.5 in the
> Fedora Rawhide base.

Thanks for the notice. I tested perl-DBD-MariaDB against mariadb:10.5 and
I confirm that all tests pass.

-- Petr

[Miro Hrončok]

On 10/26/20 5:20 PM, Ben Cotton wrote:
> **Check software that requires or depends on 'mariadb' or 'galera' package for 
> incompatibilities
> This shouldn't be an issue in general, as vast majority 
> of the software requires client library, provided by "mariadb-connector-c" 
> package, which won't change.

Are there packages that require rebuilding? E.g. I see that some packages 
require libmariadbd.so.19. What is the plan regarding them?

[Michal Schorm]

On Wed, Nov 4, 2020 at 10:55 AM Miro Hrončok  wrote:
> Are there packages that require rebuilding? E.g. I see that some packages
> require libmariadbd.so.19. What is the plan regarding them?

Packages that require the server (embedded) library
"libmariadbd.so.19" are the only one that might need rebuild.
Currently this is only a single package 'amarok', so I can cooperate
with its maintainer if rebuild would be needed.

The changes between the server embedded library between MariaDB 10.4
and 10.5 shouldn't be significant, other than added functionality.
I have to check the ABI compatibility to be sure. If the functionality
would be only extended, the dependent package might not need rebuild
at all; though it still would get one during some mass rebuild.

I added my above reply to the Self Contained Change.

--

Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat

[Miro Hrončok]

On 11/5/20 1:44 PM, Michal Schorm wrote:
> On Wed, Nov 4, 2020 at 10:55 AM Miro Hrončok  wrote:
> > Are there packages that require rebuilding? E.g. I see that some packages
> > require libmariadbd.so.19. What is the plan regarding them?
>
> Packages that require the server (embedded) library
> "libmariadbd.so.19" are the only one that might need rebuild.
> Currently this is only a single package 'amarok', so I can cooperate
> with its maintainer if rebuild would be needed.
>
> The changes between the server embedded library between MariaDB 10.4
> and 10.5 shouldn't be significant, other than added functionality.
> I have to check the ABI compatibility to be sure. If the functionality
> would be only extended, the dependent package might not need rebuild
> at all; though it still would get one during some mass rebuild.
>
> I added my above reply to the Self Contained Change.

Thank You.

My only remaining concern is the contingency mechanism (which is discussed 
elsewhere in this thread).

[Fabio Valentini]

On Mon, Oct 26, 2020 at 5:21 PM Ben Cotton  wrote:

(snip)
> == Contingency Plan ==
>
> Modules will provide the functional version of MariaDB 10.4, available to all users.
> Contingency mechanism: Fedora Modules for 10.4 available
> Contingency deadline: already in place
> Blocks release? N/A (not a System Wide Change)
> Blocks product? N/A (not a System Wide Change)

This is not a sufficient contingency plan. Leaving broken 10.5
non-modular packages in f34 is a non-starter.

Is there a realistic path to back out of the 10.5 update in rawhide /
F34 if there are problems?
It looks like the 10.4 -> 10.5 update requires database upgrades as
well, so would MariaDB 10.4 have problems with accessing databases
that have been migrated to 10.5?

Fabio

[Michal Schorm]

On Thu, Nov 5, 2020 at 12:21 AM Fabio Valentini  wrote:
> On Mon, Oct 26, 2020 at 5:21 PM Ben Cotton  wrote:
> > == Contingency Plan ==
> >
> > Modules will provide the functional version of MariaDB 10.4, available to all users.
> > Contingency mechanism: Fedora Modules for 10.4 available
>
> This is not a sufficient contingency plan. Leaving broken 10.5
> non-modular packages in f34 is a non-starter.
>
> Is there a realistic path to back out of the 10.5 update in rawhide /
> F34 if there are problems?
> It looks like the 10.4 -> 10.5 update requires database upgrades as
> well, so would MariaDB 10.4 have problems with accessing databases
> that have been migrated to 10.5?

In the worst case scenario, I would be forced to revert the change,
bump MariaDB 10.4 package epoch and release F34 with MariaDB 10.4
instead.

Database upgrades in general (this is not just about MariaDB or MySQL)
are very problematic.
Every sane DB upgrade *ever* should have a data backup prior and I
don't want, nor have any means to, solve the cases of corrupted DB
data which haven't got a backup.

What would be an issue however, if a significant number of users would
report the upgrade is problematic and they can't use the DB with the
new version.
The best thing both they and I can do is to file a BZ ticket (so we
are informed about it in the first place).
I will search the upstream JIRA ticket system for workarounds as a
part of the problem solving.
If any are found, I'd try to apply them or at least provide them to the users.

If the issues would be in place but no solution in sight, the revert
to MariaDB 10.4 in Rawhide (and F34 if already branched) is the way to
go.

If you will agree to this contingency mechanism, I will add it to the
Self-Contained Change wiki page.
Otherwise I'd ask you for a suggestion of what you picture as
sufficient contingency mechanism.

Michal
--

Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat

[Fabio Valentini]

On Thu, Nov 5, 2020 at 2:07 PM Michal Schorm  wrote:
> On Thu, Nov 5, 2020 at 12:21 AM Fabio Valentini  wrote:
> > On Mon, Oct 26, 2020 at 5:21 PM Ben Cotton  wrote:
> > > == Contingency Plan ==
> > >
> > > Modules will provide the functional version of MariaDB 10.4, available to all users.
> > > Contingency mechanism: Fedora Modules for 10.4 available
> >
> > This is not a sufficient contingency plan. Leaving broken 10.5
> > non-modular packages in f34 is a non-starter.
> >
> > Is there a realistic path to back out of the 10.5 update in rawhide /
> > F34 if there are problems?
> > It looks like the 10.4 -> 10.5 update requires database upgrades as
> > well, so would MariaDB 10.4 have problems with accessing databases
> > that have been migrated to 10.5?
>
> In the worst case scenario, I would be forced to revert the change,
> bump MariaDB 10.4 package epoch and release F34 with MariaDB 10.4
> instead.
>
> Database upgrades in general (this is not just about MariaDB or MySQL)
> are very problematic.
> Every sane DB upgrade *ever* should have a data backup prior and I
> don't want, nor have any means to, solve the cases of corrupted DB
> data which haven't got a backup.
>
> What would be an issue however, if a significant number of users would
> report the upgrade is problematic and they can't use the DB with the
> new version.
> The best thing both they and I can do is to file a BZ ticket (so we
> are informed about it in the first place).
> I will search the upstream JIRA ticket system for workarounds as a
> part of the problem solving.
> If any are found, I'd try to apply them or at least provide them to the users.
>
> If the issues would be in place but no solution in sight, the revert
> to MariaDB 10.4 in Rawhide (and F34 if already branched) is the way to
> go.
>
> If you will agree to this contingency mechanism, I will add it to the
> Self-Contained Change wiki page.
> Otherwise I'd ask you for a suggestion of what you picture as
> sufficient contingency mechanism.

Yes, this sounds like a good compromise. Thanks!

Fabio

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 05, 2020 at 02:06:37PM +0100, Michal Schorm wrote:
> On Thu, Nov 5, 2020 at 12:21 AM Fabio Valentini  wrote:
> > On Mon, Oct 26, 2020 at 5:21 PM Ben Cotton  wrote:
> > > == Contingency Plan ==
> > >
> > > Modules will provide the functional version of MariaDB 10.4, available to all users.
> > > Contingency mechanism: Fedora Modules for 10.4 available
> >
> > This is not a sufficient contingency plan. Leaving broken 10.5
> > non-modular packages in f34 is a non-starter.
> >
> > Is there a realistic path to back out of the 10.5 update in rawhide /
> > F34 if there are problems?
> > It looks like the 10.4 -> 10.5 update requires database upgrades as
> > well, so would MariaDB 10.4 have problems with accessing databases
> > that have been migrated to 10.5?
>
> In the worst case scenario, I would be forced to revert the change,
> bump MariaDB 10.4 package epoch and release F34 with MariaDB 10.4
> instead.
>
> Database upgrades in general (this is not just about MariaDB or MySQL)
> are very problematic.
> Every sane DB upgrade *ever* should have a data backup prior and I
> don't want, nor have any means to, solve the cases of corrupted DB
> data which haven't got a backup.
>
> What would be an issue however, if a significant number of users would
> report the upgrade is problematic and they can't use the DB with the
> new version.
> The best thing both they and I can do is to file a BZ ticket (so we
> are informed about it in the first place).
> I will search the upstream JIRA ticket system for workarounds as a
> part of the problem solving.
> If any are found, I'd try to apply them or at least provide them to the users.
>
> If the issues would be in place but no solution in sight, the revert
> to MariaDB 10.4 in Rawhide (and F34 if already branched) is the way to
> go.
>
> If you will agree to this contingency mechanism, I will add it to the
> Self-Contained Change wiki page.
> Otherwise I'd ask you for a suggestion of what you picture as
> sufficient contingency mechanism.

So, any progress on this?

Zbyszek

[Michal Schorm]

I update the Wiki page to state that the current contingency plan is a
revert of the change by bumping 'mariadb' package epoch.
I also added a note about the dependent packages that need rebuild.
That is a single package (amarok); I tested the rebuild in COPR and
discussed it with the 'amarok' package maintainer.

--

Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat

--

On Tue, Nov 17, 2020 at 8:12 PM Zbigniew Jędrzejewski-Szmek
 wrote:
> On Thu, Nov 05, 2020 at 02:06:37PM +0100, Michal Schorm wrote:
> > On Thu, Nov 5, 2020 at 12:21 AM Fabio Valentini  wrote:
> > > On Mon, Oct 26, 2020 at 5:21 PM Ben Cotton  wrote:
> > > > == Contingency Plan ==
> > > >
> > > > Modules will provide the functional version of MariaDB 10.4, available to all users.
> > > > Contingency mechanism: Fedora Modules for 10.4 available
> > >
> > > This is not a sufficient contingency plan. Leaving broken 10.5
> > > non-modular packages in f34 is a non-starter.
> > >
> > > Is there a realistic path to back out of the 10.5 update in rawhide /
> > > F34 if there are problems?
> > > It looks like the 10.4 -> 10.5 update requires database upgrades as
> > > well, so would MariaDB 10.4 have problems with accessing databases
> > > that have been migrated to 10.5?
> >
> > In the worst case scenario, I would be forced to revert the change,
> > bump MariaDB 10.4 package epoch and release F34 with MariaDB 10.4
> > instead.
> >
> > Database upgrades in general (this is not just about MariaDB or MySQL)
> > are very problematic.
> > Every sane DB upgrade *ever* should have a data backup prior and I
> > don't want, nor have any means to, solve the cases of corrupted DB
> > data which haven't got a backup.
> >
> > What would be an issue however, if a significant number of users would
> > report the upgrade is problematic and they can't use the DB with the
> > new version.
> > The best thing both they and I can do is to file a BZ ticket (so we
> > are informed about it in the first place).
> > I will search the upstream JIRA ticket system for workarounds as a
> > part of the problem solving.
> > If any are found, I'd try to apply them or at least provide them to the users.
> >
> > If the issues would be in place but no solution in sight, the revert
> > to MariaDB 10.4 in Rawhide (and F34 if already branched) is the way to
> > go.
> >
> > If you will agree to this contingency mechanism, I will add it to the
> > Self-Contained Change wiki page.
> > Otherwise I'd ask you for a suggestion of what you picture as
> > sufficient contingency mechanism.
>
> So, any progress on this?
>
> Zbyszek
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]