How to (better) deal with library major API changes

[Richard Shaw]

I'm still trying to figure out the best way to frame the problem and I
don't have a specific solution in mind, but there's got to be a better
solution for dealing with projects that make major API changes.

Pre-side-tag this was even more difficult:
1. Make sure the new version builds
2. Maybe do some testing with dependencies
3. Update the package and break Rawhide
4. Deal with the fallout

At least with side tags it buys us some time to port over packages that
don't build cleanly, but the longer these are open, the more likely other
people have touched them creating other issues.

I've actually started creating COPRs for this purpose and that helps check
things out "offline" from the main distro, but it's not a 100% solution.

What do we do with packages where SOME but not all of the dependent
packages support the new API?

Conservative option: Wait until they do and don't upgrade the package.

This is fine in the short term (3-6 months?) but many upstreams are less
than aggressive about updating dependencies and could potentially take a
year or more.

Middle of the road (but a lot of work) option:
1. Create a compat or SOVERSION appended package (requires new review)
2. Migrate the packages that won't build to it, build the new package and
working deps.

One big caveat is that in some cases it's all or nothing for the whole
"stack"

Contrived example:
OpenImageIO can build with OpenColorIO 2.0 but blender can't (has since
been fixed).

I can't rebuild OIIO with OCIO 2.0 and leave Blender at OCIO 1.X. Worse,
while it's a good thing I know about this dep chain, I'm unaware of any
systematic method to detect this so it completely relies on packager
knowledge, and we know how problematic that is with all the soname
breakages even though the guidelines are quite explicit there.

Aggressive option: Build it anyway and deal with the fallout the best we
can.

I don't actually think this is practical but included it for completeness.

Other recent examples:
OpenEXR 3 (ongoing saga)
vtk 9.0 (pretty much done, but was painful)

Thoughts?

Thanks,
Richard

[Miro Hrončok]

On 01. 07. 21 17:53, Richard Shaw wrote:
>  1. Create a compat or SOVERSION appended package (requires new
> review) 
Packages created so that multiple versions of the same package can coexist in 
the distribution does not require a new review:

https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...



-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Richard Shaw]

On Thu, Jul 1, 2021 at 11:11 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  On 01. 07. 21 17:53, Richard Shaw wrote:
>  > 1. Create a compat or SOVERSION appended package (requires new review)
>
>  Packages created so that multiple versions of the same package can coexist
>  in
>  the distribution does not require a new review:
>
>
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...

Ok, so does require a ticket though.

So the packager will need to make sure the libraries don't conflict or
provide the same files, but what about the -devel package? I actually think
you'd want it to conflict as it might bring to light the stack issues I
mentioned.

Thanks,
Richard

[Miro Hrončok]

On 01. 07. 21 18:20, Richard Shaw wrote:
>  On Thu, Jul 1, 2021 at 11:11 AM Miro Hrončok <mhroncok(a)redhat.com
>
>  <mailto:[e-mail]>> wrote:
>  
>      On 01. 07. 21 17:53, Richard Shaw wrote:
>       > 1. Create a compat or SOVERSION appended package (requires new review)
>  
>      Packages created so that multiple versions of the same package can coexist in
>      the distribution does not require a new review:
>  
>     
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>     
> <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  
>  
>  Ok, so does require a ticket though. 
No, it does not. To be exempted from the package review you can create a ticket 
OR you can create a "multiple versions of the same package" package.

>  So the packager will need to make sure the libraries don't
> conflict or provide 
>  the same files, but what about the -devel package? I actually think you'd want 
>  it to conflict as it might bring to light the stack issues I mentioned. 
Yeas, the devel packages might conflict.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Richard Shaw]

On Thu, Jul 1, 2021 at 12:25 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  On 01. 07. 21 18:20, Richard Shaw wrote:
>  > On Thu, Jul 1, 2021 at 11:11 AM Miro Hrončok <mhroncok(a)redhat.com
>  > <mailto:[e-mail]>> wrote:
>  >
>  >     On 01. 07. 21 17:53, Richard Shaw wrote:
>  >      > 1. Create a compat or SOVERSION appended package (requires new
>  review)
>  >
>  >     Packages created so that multiple versions of the same package can
>  coexist in
>  >     the distribution does not require a new review:
>  >
>  >
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  >     <
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  >
>  >
>  >
>  > Ok, so does require a ticket though.
>
>  No, it does not. To be exempted from the package review you can create a
>  ticket
>  OR you can create a "multiple versions of the same package" package.
>  
Ok, I re-read things a few times and now I get that part, but without a
review, how do I get the repo setup?

I have prepared an "openexr2" package. I chose just to include the 2
because it's possible upstream may release a new 2.x version so
"openexr2.5.5" doesn't make any sense.

Thanks,
Richard

[Miro Hrončok]

On 04. 07. 21 16:49, Richard Shaw wrote:
>  On Thu, Jul 1, 2021 at 12:25 PM Miro Hrončok <mhroncok(a)redhat.com
>
>  <mailto:[e-mail]>> wrote:
>  
>      On 01. 07. 21 18:20, Richard Shaw wrote:
>       > On Thu, Jul 1, 2021 at 11:11 AM Miro Hrončok <mhroncok(a)redhat.com
>      <mailto:[e-mail]>
>       > <mailto:[e-mail] <mailto:[e-mail]>>>
> wrote:
>       >
>       >     On 01. 07. 21 17:53, Richard Shaw wrote:
>       >      > 1. Create a compat or SOVERSION appended package (requires new
>      review)
>       >
>       >     Packages created so that multiple versions of the same package can
>      coexist in
>       >     the distribution does not require a new review:
>       >
>       >
>     
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>     
> <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>       >   
>      
>  <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuidelines/#_package_review_process
> <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>       >
>       >
>       > Ok, so does require a ticket though.
>  
>      No, it does not. To be exempted from the package review you can create a
>      ticket
>      OR you can create a "multiple versions of the same package" package.
>  
>  
>  Ok, I re-read things a few times and now I get that part, but without a review, 
>  how do I get the repo setup?
>  
>  I have prepared an "openexr2" package. I chose just to include the 2 because 
>  it's possible upstream may release a new 2.x version so "openexr2.5.5"
> doesn't 
>  make any sense. 

$ fedpkg request-repo openexr2 --exception --no-initial-commit


Technically, you only need --exception, but I also like to use 
--no-initial-commit, so the package can be imported with git history of openexr.


-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Zbigniew Jędrzejewski-Szmek]

On Sun, Jul 04, 2021 at 09:04:55PM +0200, Miro Hrončok wrote:
>  On 04. 07. 21 16:49, Richard Shaw wrote:
>  >On Thu, Jul 1, 2021 at 12:25 PM Miro Hrončok <mhroncok(a)redhat.com
>  ><mailto:[e-mail]>> wrote:
>  >
>  >    On 01. 07. 21 18:20, Richard Shaw wrote:
>  >     > On Thu, Jul 1, 2021 at 11:11 AM Miro Hrončok <mhroncok(a)redhat.com
>  >    <mailto:[e-mail]>
>  >     > <mailto:[e-mail] <mailto:[e-mail]>>>
> wrote:
>  >     >
>  >     >     On 01. 07. 21 17:53, Richard Shaw wrote:
>  >     >      > 1. Create a compat or SOVERSION appended package (requires new
>  >    review)
>  >     >
>  >     >     Packages created so that multiple versions of the same package can
>  >    coexist in
>  >     >     the distribution does not require a new review:
>  >     >
>  >     >
>  >   
> https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  >   
> <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  >     >       
>  <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuidelines/#_package_review_process
> <https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuideline...
>  >     >
>  >     >
>  >     > Ok, so does require a ticket though.
>  >
>  >    No, it does not. To be exempted from the package review you can create a
>  >    ticket
>  >    OR you can create a "multiple versions of the same package" package.
>  >
>  >
>  >Ok, I re-read things a few times and now I get that part, but
>  >without a review, how do I get the repo setup?
>  >
>  >I have prepared an "openexr2" package. I chose just to include the
>  >2 because it's possible upstream may release a new 2.x version so
>  >"openexr2.5.5" doesn't make any sense.
>  
>  
>  $ fedpkg request-repo openexr2 --exception --no-initial-commit
>  
>  
>  Technically, you only need --exception, but I also like to use
>  --no-initial-commit, so the package can be imported with git history
>  of openexr. 
This is "documented" in
https://fedoraproject.org/wiki/Package_Review_Process#Review_Purpose.
I put documented in quotes because the text is present, but it sure ain't
easy to find. All the wiki docs about initial package are fairly hard to
use: the information is split over multiple pages without a clear design
to the division, there should be lot more hyperlinking going on, etc.

Zbyszek

[Richard Shaw]

On Sun, Jul 4, 2021 at 2:05 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  Technically, you only need --exception, but I also like to use
>  --no-initial-commit, so the package can be imported with git history of
>  openexr.
>  
Ok, I did that, now how do I import the current openexr repo?

Thanks,
Richard

[Richard Shaw]

Never mind, not sure if it was the best way but I did:

$ fedpkg clone openexr openexr2

And then edited .git/config and updated the references and then:

$ fedpkg push

And it's all there. Just had to cope over my spec file modifications.

Thanks,
Richard

[Kaleb Keithley]

<top post>

Or, if I may be so bold—

Convince the developers to use versioned symbols when they break the
API/ABI.

This is the 21st Century, and we have solutions for this. It's not rocket
surgery.

</top post>


On Thu, Jul 1, 2021 at 11:55 AM Richard Shaw <hobbes1069(a)gmail.com&gt; wrote:

>  I'm still trying to figure out the best way to frame the problem
> and I
>  don't have a specific solution in mind, but there's got to be a better
>  solution for dealing with projects that make major API changes.
>
>  Pre-side-tag this was even more difficult:
>  1. Make sure the new version builds
>  2. Maybe do some testing with dependencies
>  3. Update the package and break Rawhide
>  4. Deal with the fallout
>
>  At least with side tags it buys us some time to port over packages that
>  don't build cleanly, but the longer these are open, the more likely other
>  people have touched them creating other issues.
>
>  I've actually started creating COPRs for this purpose and that helps check
>  things out "offline" from the main distro, but it's not a 100% solution.
>
>  What do we do with packages where SOME but not all of the dependent
>  packages support the new API?
>
>  Conservative option: Wait until they do and don't upgrade the package.
>
>  This is fine in the short term (3-6 months?) but many upstreams are less
>  than aggressive about updating dependencies and could potentially take a
>  year or more.
>
>  Middle of the road (but a lot of work) option:
>  1. Create a compat or SOVERSION appended package (requires new review)
>  2. Migrate the packages that won't build to it, build the new package and
>  working deps.
>
>  One big caveat is that in some cases it's all or nothing for the whole
>  "stack"
>
>  Contrived example:
>  OpenImageIO can build with OpenColorIO 2.0 but blender can't (has since
>  been fixed).
>
>  I can't rebuild OIIO with OCIO 2.0 and leave Blender at OCIO 1.X. Worse,
>  while it's a good thing I know about this dep chain, I'm unaware of any
>  systematic method to detect this so it completely relies on packager
>  knowledge, and we know how problematic that is with all the soname
>  breakages even though the guidelines are quite explicit there.
>
>  Aggressive option: Build it anyway and deal with the fallout the best we
>  can.
>
>  I don't actually think this is practical but included it for completeness.
>
>  Other recent examples:
>  OpenEXR 3 (ongoing saga)
>  vtk 9.0 (pretty much done, but was painful)
>
>  Thoughts?
>
>  Thanks,
>  Richard
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it:
>  https://pagure.io/fedora-infrastructure
>

[Richard Shaw]

On Thu, Jul 1, 2021 at 12:35 PM Kaleb Keithley <kkeithle(a)redhat.com&gt; wrote:

>  <top post>
>
>  Or, if I may be so bold—
>
>  Convince the developers to use versioned symbols when they break the
>  API/ABI.
>
>  This is the 21st Century, and we have solutions for this. It's not rocket
>  surgery.
>
>  </top post>
>  
That's actually not a problem most of the time. Some upstreams even
unnecessarily bump their soversion. But maintaining two (or potentially
more) versions of a library does create significant extra work for the
packager.  One example is the Coin package, currently Coin2/Coin3/Coin4
with all the alternatives gunk.

Thanks,
Richard

[Richard W.M. Jones]

On Thu, Jul 01, 2021 at 12:52:51PM -0500, Richard Shaw wrote:
>  On Thu, Jul 1, 2021 at 12:35 PM Kaleb Keithley
> <kkeithle(a)redhat.com&gt; wrote:
>  
>      <top post>
>  
>      Or, if I may be so bold—
>  
>      Convince the developers to use versioned symbols when they break the API/
>      ABI.
>  
>      This is the 21st Century, and we have solutions for this. It's not rocket
>      surgery.
>  
>      </top post>
>  
>  That's actually not a problem most of the time. Some upstreams even
>  unnecessarily bump their soversion. But maintaining two (or
>  potentially more) versions of a library does create significant
>  extra work for the packager.  One example is the Coin package,
>  currently Coin2/Coin3/Coin4 with all the alternatives gunk. 
I don't think that's what Kaleb was suggesting.  Coin2/3/4 packages
contain different SONAMEs:

# dnf repoquery -l Coin4 | grep /usr/lib64
/usr/lib64/libCoin.so.4.0.0
/usr/lib64/libCoin.so.80
# dnf repoquery -l Coin3 | grep /usr/lib64
/usr/lib64/libCoin.so.60
/usr/lib64/libCoin.so.60.1.3
# dnf repoquery -l Coin2 | grep /usr/lib64
/usr/lib64/libCoin.so.40
/usr/lib64/libCoin.so.40.5.0

Now I don't know if it would specifically work for Coin, but for many
C/C++ libraries you can use a single SONAME forever, and versioned
symbols for each symbol that has to change incompatibly.  Old binaries
will continue to use the old symbol transparently.  New binaries will
use the new symbol.  Both will be linked to the same SONAME.

glibc is the most famous/notorious user of this mechanism, but many
other libraries use it too.  Uli's documentation is a good place to
start:

https://akkadia.org/drepper/dsohowto.pdf

as well as:

https://gcc.gnu.org/wiki/SymbolVersioning

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Richard Shaw]

On Thu, Jul 1, 2021 at 1:52 PM Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:

>  I don't think that's what Kaleb was suggesting.  Coin2/3/4 packages
>  contain different SONAMEs:
>
>  # dnf repoquery -l Coin4 | grep /usr/lib64
>  /usr/lib64/libCoin.so.4.0.0
>  /usr/lib64/libCoin.so.80
>  # dnf repoquery -l Coin3 | grep /usr/lib64
>  /usr/lib64/libCoin.so.60
>  /usr/lib64/libCoin.so.60.1.3
>  # dnf repoquery -l Coin2 | grep /usr/lib64
>  /usr/lib64/libCoin.so.40
>  /usr/lib64/libCoin.so.40.5.0
>
>  Now I don't know if it would specifically work for Coin, but for many
>  C/C++ libraries you can use a single SONAME forever, and versioned
>  symbols for each symbol that has to change incompatibly.  Old binaries
>  will continue to use the old symbol transparently.  New binaries will
>  use the new symbol.  Both will be linked to the same SONAME.
>
>  glibc is the most famous/notorious user of this mechanism, but many
>  other libraries use it too.  Uli's documentation is a good place to
>  start:
>
>  https://akkadia.org/drepper/dsohowto.pdf
>
>  as well as:
>
>  https://gcc.gnu.org/wiki/SymbolVersioning 

Is that something that can be easily implemented at the packager level?

Also, in the case of OpenEXR upstream (ASWF) has reorganized the libraries
over the last couple of releases[1], merging some and splitting out others
creating a real mess. In fact, one of the other ASWF projects, OpenVDB
isn't even compatible with OpenEXR 3 and won't be until the next major
release.

So is my only real option to create a compat package and migrate all the
projects that can't use OpenEXR 3 to it?

For posterity here's my working COPR with the fairly long list of
dependencies:

https://copr.fedorainfracloud.org/coprs/hobbes1069/openexr/builds/

Thanks,
Richard

[1]
https://github.com/AcademySoftwareFoundation/Imath/blob/master/docs/Porti...

[Richard W.M. Jones]

On Sun, Jul 04, 2021 at 07:11:14AM -0500, Richard Shaw wrote:
>  On Thu, Jul 1, 2021 at 1:52 PM Richard W.M. Jones
> <rjones(a)redhat.com&gt; wrote:
>      https://akkadia.org/drepper/dsohowto.pdf
>  
>      as well as:
>  
>      https://gcc.gnu.org/wiki/SymbolVersioning
>  
>  
>  Is that something that can be easily implemented at the packager level? 
Unfortunately not, except in the sense that if upstream are doing it
right then RPM packaging will automatically do the right thing too.

Also I think I claimed this was possible/easy for C++ and IIRC there
are problems which make it difficult with C++.  It's certainly much
easier if you have a plain C API.

>  Also, in the case of OpenEXR upstream (ASWF) has reorganized the
>  libraries over the last couple of releases[1], merging some and
>  splitting out others creating a real mess. In fact, one of the other
>  ASWF projects, OpenVDB isn't even compatible with OpenEXR 3 and
>  won't be until the next major release.  
It sounds like they're not really taking the whole backwards
compatibility thing seriously.

>  So is my only real option to create a compat package and migrate all
> the
>  projects that can't use OpenEXR 3 to it?  
I think so :-(

Rich.

>  For posterity here's my working COPR with the fairly long list of
> dependencies:
>  
>  https://copr.fedorainfracloud.org/coprs/hobbes1069/openexr/builds/
>  
>  Thanks,
>  Richard 
>  
>  [1] https://github.com/AcademySoftwareFoundation/Imath/blob/master/docs/
>  PortingGuide2-3.md 
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW