Heads Up: libmodulemd 2.0 coming soon to a Rawhide near you

[Stephen Gallagher]

The libmodulemd library, used by the Module Build Service, libdnf,
fedmod and other tools is going to be updated this week to version
2.0.0beta1. While this includes a soname bump, the Fedora packaging
will include a compat-libmodulemd1 (and python3-compat-libmodulemd1)
subpackage that is fully compatible with the 1.x API. I do not expect
any problematic fallout from this upgrade and existing tools should
continue to work fine until they can be ported to the newer API.

This will mean that anyone building against libmodulemd today will
need to change their spec files to indicate `BuildRequires:
compat-libmodulemd1-devel` to continue using the 1.0 API.

The 2.0 API is considerably more user-friendly and better documented
[1]. I strongly encourage any project that is consuming libmodulemd to
move to this new API as soon as possible. I will be maintaining the
1.0 compat library through Fedora 30, but I intend to retire it from
Fedora 31 soon after the branch occurs.

[1] https://sgallagh.fedorapeople.org/docs/libmodulemd/2.0/

[Fabio Valentini]

On Mon, Dec 10, 2018 at 5:01 PM Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  The libmodulemd library, used by the Module Build Service, libdnf,
>  fedmod and other tools is going to be updated this week to version
>  2.0.0beta1. While this includes a soname bump, the Fedora packaging
>  will include a compat-libmodulemd1 (and python3-compat-libmodulemd1)
>  subpackage that is fully compatible with the 1.x API. I do not expect
>  any problematic fallout from this upgrade and existing tools should
>  continue to work fine until they can be ported to the newer API. 
Hi,

Please don't use "compat-" package names, they are outdated.
The Packaging / Naming Guidelines have been amended some time ago, and
don't mention "compat-" anywhere anymore.

The relevant section for cases like this can be found here (Multiple
packages with the same base name):
https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#_multip...

So, according to these guidelines, the package names should instead be:

- libmodulemd (providing version 2.x)
- libmodulemd1 (providing version 1.x)
- python3-libmodulemd
- python3-libmodulemd1

>  This will mean that anyone building against libmodulemd today will
>  need to change their spec files to indicate `BuildRequires:
>  compat-libmodulemd1-devel` to continue using the 1.0 API. 
It's unfortunate that the pkgconfig for modulemd isn't versioned.
Now both packages (libmodulemd-devel, and libmodulemd1-devel) will
provide 'pkgconfig(modulemd)',
but the newer version will always take precedence due to the higher version.
Otherwise, packages could have explicitly requested the library with
the old API with BR'ing 'pkgconfig(modulemd-1)', and the new API with
BR'ing 'pkgconfig(modulemd-2)' (or something similar).

(BTW, do the old and new python packages' names contain the version?
Otherwise they'll probably conflict.)

Just my 2¢.

Fabio

> The 2.0 API is considerably more user-friendly and better documented
> [1]. I strongly encourage any project that is consuming libmodulemd to
> move to this new API as soon as possible. I will be maintaining the
> 1.0 compat library through Fedora 30, but I intend to retire it from
> Fedora 31 soon after the branch occurs.
>
> [1] https://sgallagh.fedorapeople.org/docs/libmodulemd/2.0/
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Stephen Gallagher]

On Mon, Dec 10, 2018 at 1:22 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  On Mon, Dec 10, 2018 at 5:01 PM Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  >
>  > The libmodulemd library, used by the Module Build Service, libdnf,
>  > fedmod and other tools is going to be updated this week to version
>  > 2.0.0beta1. While this includes a soname bump, the Fedora packaging
>  > will include a compat-libmodulemd1 (and python3-compat-libmodulemd1)
>  > subpackage that is fully compatible with the 1.x API. I do not expect
>  > any problematic fallout from this upgrade and existing tools should
>  > continue to work fine until they can be ported to the newer API.
>
>  Hi,
>
>  Please don't use "compat-" package names, they are outdated.
>  The Packaging / Naming Guidelines have been amended some time ago, and
>  don't mention "compat-" anywhere anymore.
>
>  The relevant section for cases like this can be found here (Multiple
>  packages with the same base name):
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Naming/#_multip...
>
>  So, according to these guidelines, the package names should instead be:
>
>  - libmodulemd (providing version 2.x)
>  - libmodulemd1 (providing version 1.x)
>  - python3-libmodulemd
>  - python3-libmodulemd1
>  
OK, that's easy enough to do. I haven't done the builds yet.

>  > This will mean that anyone building against libmodulemd today
> will
>  > need to change their spec files to indicate `BuildRequires:
>  > compat-libmodulemd1-devel` to continue using the 1.0 API.
>
>  It's unfortunate that the pkgconfig for modulemd isn't versioned.
>  Now both packages (libmodulemd-devel, and libmodulemd1-devel) will
>  provide 'pkgconfig(modulemd)', 
It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and 2.x
is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
two -devel subpackages is that they both want to own
/usr/lib64/libmodulemd.so, symlinked to different objects.

>  but the newer version will always take precedence due to the higher
> version.
>  Otherwise, packages could have explicitly requested the library with
>  the old API with BR'ing 'pkgconfig(modulemd-1)', and the new API with
>  BR'ing 'pkgconfig(modulemd-2)' (or something similar).
>  
They can do this today, see above.

>  (BTW, do the old and new python packages' names contain the
> version?
>  Otherwise they'll probably conflict.)
>  
The python packages are actually just metapackages that `Requires:
python3-gobject-base`. The content is handled by:
```
import gi
gi.require_version('Modulemd', '1.0')
from gi.repository import Modulemd
```
or
```
import gi
gi.require_version('Modulemd', '2.0')
from gi.repository import Modulemd
```

So they do not conflict either.

>  Just my 2¢.
>  
Thanks for the questions (and the suggestion about dropping the
compat-* naming).


> Fabio
>
> > The 2.0 API is considerably more user-friendly and better documented
> > [1]. I strongly encourage any project that is consuming libmodulemd to
> > move to this new API as soon as possible. I will be maintaining the
> > 1.0 compat library through Fedora 30, but I intend to retire it from
> > Fedora 31 soon after the branch occurs.
> >
> > [1] https://sgallagh.fedorapeople.org/docs/libmodulemd/2.0/
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Kalev Lember]

On 12/10/2018 07:30 PM, Stephen Gallagher wrote:
>  It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and
> 2.x
>  is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
>  two -devel subpackages is that they both want to own
>  /usr/lib64/libmodulemd.so, symlinked to different objects. 
Perhaps it would then make sense to rename libmodulemd.so to
libmodulemd-2.so in 2.x, so they don't conflict?

Kalev

[Fabio Valentini]

On Mon, Dec 10, 2018, 20:06 Kalev Lember <kalevlember(a)gmail.com wrote:

>  On 12/10/2018 07:30 PM, Stephen Gallagher wrote:
>  > It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and 2.x
>  > is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
>  > two -devel subpackages is that they both want to own
>  > /usr/lib64/libmodulemd.so, symlinked to different objects.
>
>  Perhaps it would then make sense to rename libmodulemd.so to
>  libmodulemd-2.so in 2.x, so they don't conflict?
>  
Why? The unversioned .so files are supposed to conflict between different
versions of the -devel packages (at least they usually do, see openssl and
it's compat package, for example). Installing both versions of the -devel
package at the same time isn't necessary (or supported) anyway.


>  Kalev
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Neal Gompa]

On Mon, Dec 10, 2018 at 2:06 PM Kalev Lember <kalevlember(a)gmail.com&gt; wrote:
>  On 12/10/2018 07:30 PM, Stephen Gallagher wrote:
>  > It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and 2.x
>  > is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
>  > two -devel subpackages is that they both want to own
>  > /usr/lib64/libmodulemd.so, symlinked to different objects.
>
>  Perhaps it would then make sense to rename libmodulemd.so to
>  libmodulemd-2.so in 2.x, so they don't conflict?
>  
No. It's better that the development subpackages conflict. There's
zero reason for them to be co-installable.

-- 
真実はいつも一つ！/ Always, there's only one truth!

[Pete Walter]

10.12.2018, 19:22, "Neal Gompa" <ngompa13(a)gmail.com&gt;:
>  On Mon, Dec 10, 2018 at 2:06 PM Kalev Lember
> <kalevlember(a)gmail.com&gt; wrote:
> >  On 12/10/2018 07:30 PM, Stephen Gallagher wrote:
> >  > It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and 2.x
> >  > is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
> >  > two -devel subpackages is that they both want to own
> >  > /usr/lib64/libmodulemd.so, symlinked to different objects.
> >
> >  Perhaps it would then make sense to rename libmodulemd.so to
> >  libmodulemd-2.so in 2.x, so they don't conflict?
>
>  No. It's better that the development subpackages conflict. There's
>  zero reason for them to be co-installable. 
Huh, better to conflict? That's just not true. Conflicting packages are a major hurdle
that we should try to avoid if at all possible. If it's still possible to still change
the design of the library (rename the .so file) then it certainly makes sense to do so.

Look at some well designed libraries, gtk2 and gtk3 for example that can exist in parallel
and have -devel packages that don't conflict.

Of course, you could make an argument that it is different there because gtk has longer
lifetime than libmodulemd, but it still makes sense to do things right if we can and not
make packages unnecessarily conflict. It's just good design that way.

Pete

[Fabio Valentini]

On Tue, Dec 11, 2018 at 11:05 AM Pete Walter <walter.pete(a)yandex.com&gt; wrote:
>  10.12.2018, 19:22, "Neal Gompa" <ngompa13(a)gmail.com&gt;:
>  > On Mon, Dec 10, 2018 at 2:06 PM Kalev Lember <kalevlember(a)gmail.com&gt; wrote:
>  >>  On 12/10/2018 07:30 PM, Stephen Gallagher wrote:
>  >>  > It is versioned, actually. The 1.x API is `pkconfig(modulemd)` and 2.x
>  >>  > is `pkgconfig(modulemd-2.0)`. The source of the conflict between the
>  >>  > two -devel subpackages is that they both want to own
>  >>  > /usr/lib64/libmodulemd.so, symlinked to different objects.
>  >>
>  >>  Perhaps it would then make sense to rename libmodulemd.so to
>  >>  libmodulemd-2.so in 2.x, so they don't conflict?
>  >
>  > No. It's better that the development subpackages conflict. There's
>  > zero reason for them to be co-installable. 
Exactly.

>  Huh, better to conflict? That's just not true. Conflicting
> packages are a major hurdle that we should try to avoid if at all possible. If it's
> still possible to still change the design of the library (rename the .so file) then it
> certainly makes sense to do so. 
That's nonsense. Compat packages for older versions of the same
library always conflict, on purpose. For an example, look at the
compat-openssl10-devel and openssl-devel packages. Packages are
developed and built against either one version or the other, and
*never* both - and even so, they can't be linked with both versions
simultaneously due to symbol conflicts.

>  Look at some well designed libraries, gtk2 and gtk3 for example that
> can exist in parallel and have -devel packages that don't conflict. 
gtk2/3 is a bad example. Those two packages provide two differently
named libraries, not different versions of a library with the same
name (libgtk-x11-2.0.so vs. libgtk-3.so). (Nevermind that the symbols
conflict anyway and nothing can link against both, see the webkit2gtk3
/ gtk2 plugin process mess.)

>  Of course, you could make an argument that it is different there
> because gtk has longer lifetime than libmodulemd, but it still makes sense to do things
> right if we can and not make packages unnecessarily conflict. It's just good design
> that way. 
It's good design to allow broken installations and development
environments? I'd argue not.

Fabio

> Pete
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Pete Walter]

11.12.2018, 10:29, "Fabio Valentini" <decathorpe(a)gmail.com&gt;:
>  On Tue, Dec 11, 2018 at 11:05 AM Pete Walter
> <walter.pete(a)yandex.com&gt; wrote:
> >  Huh, better to conflict? That's just not true. Conflicting packages are a major
> hurdle that we should try to avoid if at all possible. If it's still possible to still
> change the design of the library (rename the .so file) then it certainly makes sense to do
> so.
>
>  That's nonsense. Compat packages for older versions of the same
>  library always conflict, on purpose. For an example, look at the
>  compat-openssl10-devel and openssl-devel packages. Packages are
>  developed and built against either one version or the other, and
>  *never* both - and even so, they can't be linked with both versions
>  simultaneously due to symbol conflicts. 
Please don't call other people's opinions nonsense.
https://docs.fedoraproject.org/en-US/project/code-of-conduct/index.html

Anyway, I would say that openssl 1.1 upstream was never designed around distro packaging
in mind. They probably assumed that it's fine to have just one version of their
headers installed at one time ... which is not the case if you need to work on a piece of
software that uses the new API, and another piece of software that uses the old API.
Conflicting packages mean repeatedly installing and uninstalling things in that case. Not
what I'd call good packaging.

compat-openssl10-devel are just a distro hack to make it possible to _build_ stuff in koji
against the older API, but it makes it a pain for any developer to use the headers
(because of the conflict).

> >  Look at some well designed libraries, gtk2 and gtk3 for example
> that can exist in parallel and have -devel packages that don't conflict.
>
>  gtk2/3 is a bad example. Those two packages provide two differently
>  named libraries, not different versions of a library with the same
>  name (libgtk-x11-2.0.so vs. libgtk-3.so). (Nevermind that the symbols
>  conflict anyway and nothing can link against both, see the webkit2gtk3
>  / gtk2 plugin process mess.) 
That's unrelated that the runtime libraries have symbol conflicts. We were talking
about -devel parallel installability.

> >  Of course, you could make an argument that it is different there
> because gtk has longer lifetime than libmodulemd, but it still makes sense to do things
> right if we can and not make packages unnecessarily conflict. It's just good design
> that way.
>
>  It's good design to allow broken installations and development
>  environments? I'd argue not. 
Please don't move the goal posts. Symbol conflicts is a completely different issue and
conflicting -devel packages changes nothing there.

If the devel packages conflict, it just makes it so much harder to use Fedora to actually
develop software. It's fine for package building purposes, of course, but actually
developing stuff is what I enjoy doing on Fedora.

Pete

[Fabio Valentini]

On Tue, Dec 11, 2018 at 12:52 PM Pete Walter <walter.pete(a)yandex.com&gt; wrote:
>  11.12.2018, 10:29, "Fabio Valentini" <decathorpe(a)gmail.com&gt;:
>  > On Tue, Dec 11, 2018 at 11:05 AM Pete Walter <walter.pete(a)yandex.com&gt; wrote:
>  >>  Huh, better to conflict? That's just not true. Conflicting packages are a
> major hurdle that we should try to avoid if at all possible. If it's still possible to
> still change the design of the library (rename the .so file) then it certainly makes sense
> to do so.
>  >
>  > That's nonsense. Compat packages for older versions of the same
>  > library always conflict, on purpose. For an example, look at the
>  > compat-openssl10-devel and openssl-devel packages. Packages are
>  > developed and built against either one version or the other, and
>  > *never* both - and even so, they can't be linked with both versions
>  > simultaneously due to symbol conflicts.
>
>  Please don't call other people's opinions nonsense.
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/index.html 
I apologize. I didn't mean to be rude, or say anything other than
this: I think your statement doesn't reflect the way things currently
work.

>  Anyway, I would say that openssl 1.1 upstream was never designed
> around distro packaging in mind. They probably assumed that it's fine to have just one
> version of their headers installed at one time ... which is not the case if you need to
> work on a piece of software that uses the new API, and another piece of software that uses
> the old API. Conflicting packages mean repeatedly installing and uninstalling things in
> that case. Not what I'd call good packaging. 
That has nothing to do with "good" or "bad" packaging. It's an
upstream project's decision, and if the downstream packaging works
around it, and all dependent packages have to be patched to support
the downstream-patched version - now that's what I would call bad
packaging (or a hack).

>  compat-openssl10-devel are just a distro hack to make it possible to
> _build_ stuff in koji against the older API, but it makes it a pain for any developer to
> use the headers (because of the conflict). 
I agree. That's why the compat package will probably be retired once
all consumers have been ported to the newer version.

>  >>  Look at some well designed libraries, gtk2 and gtk3 for
> example that can exist in parallel and have -devel packages that don't conflict.
>  >
>  > gtk2/3 is a bad example. Those two packages provide two differently
>  > named libraries, not different versions of a library with the same
>  > name (libgtk-x11-2.0.so vs. libgtk-3.so). (Nevermind that the symbols
>  > conflict anyway and nothing can link against both, see the webkit2gtk3
>  > / gtk2 plugin process mess.)
>
>  That's unrelated that the runtime libraries have symbol conflicts. We were talking
> about -devel parallel installability. 
Unless 95% of all libraries change the way they are built, that's not
going to happen - currently most projects I watch do something like
this:

- The library API changes, and a major version is released with ABI
changes and an soname bump of the library.
- Dependent packages are *just rebuilt* against new library versions
if the APIs they called didn't change, and
- only packages that used changed APIs are adapted to the new API,
with no other changes.

However, renaming the base library name (including headers, pkgconfig
file, etc.) for every major release of a library would require code
changes in *every* downstream project using the library, whether the
APIs they called changed or not - and renders the (useful) concept
<soname bump -> just rebuild> completely obsolete, too.

>  >>  Of course, you could make an argument that it is different
> there because gtk has longer lifetime than libmodulemd, but it still makes sense to do
> things right if we can and not make packages unnecessarily conflict. It's just good
> design that way. 
I'm not arguing against "doing things right", but that's ultimately a
decision the upstream project has to make, and has nothing to do with
packaging.

>  > It's good design to allow broken installations and
> development
>  > environments? I'd argue not.
>
>  Please don't move the goal posts. Symbol conflicts is a completely different issue
> and conflicting -devel packages changes nothing there. 
If the library's base name doesn't change, the unversioned .so symlink
will conflict between the -devel packages - because it determines the
shared library the built project is linked with.
I don't see a way around that - other than to throw sonames out the
window, rename the library with every version, and have all consumers
change their code and build systems every time. (I don't think that's
sustainable, and probably library versions and sonames were originally
introduced to fix just that.)

> If the devel packages conflict, it just makes it so much harder to use Fedora to
actually develop software. It's fine for package building purposes, of course, but
actually developing stuff is what I enjoy doing on Fedora.
>
> Pete
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]