Hello,

> Setting up a copr repo to find all other breakages sounds like a good idea.

I'm actually working on a tool to help with such use cases.
It may be a bit early since I only made an internal alpha release, and only planned to go public by the end of this month with it, but since that may help, feel free to give it a try and provide early feedback.

https://copr.fedorainfracloud.org/coprs/fberat/mass-prebuild/

Long story short, the tool aims to create for you mass rebuild around a given set of packages. It will first build the packages you feed it (like an SRPM), then try to calculate the direct reverse dependencies, and build them against your new package version.
If builds are failing, it will create another pristine build for state comparison, which should help reduce the investigation (even though it can't be perfect).

All of that is using COPR infrastructure, and COPR python interface.

You have more details in this description: https://gitlab.com/fberat/mass-prebuild/-/blob/dfcb3b1b5ae1384e963b56e4bcfbc73fea7b4c66/README.md

Fred.

On Tue, Jul 12, 2022 at 7:28 AM Aleksei Bavshin <alebastr@fedoraproject.org> wrote:
On 7/11/22 09:50, Ben Beasley wrote:
> Two of the three packages I tried to rebuild (bear and fmidi) failed
> with errors like:
>
> /usr/include/fmt/core.h:1733:7: error: static assertion failed: Cannot format an argument. To make type T formattable provide a formatter<T> specialization:https://fmt.dev/latest/api.html#udt
>   1733 |       formattable,
>        |       ^~~~~~~~~~~
>
> Links to failing scratch builds in the side tag:
>
>     https://koji.fedoraproject.org/koji/taskinfo?taskID=89380682
>
>     https://koji.fedoraproject.org/koji/taskinfo?taskID=89380736
>
> I found that there was already a bug report open upstream for Bear:
> https://github.com/rizsotto/Bear/issues/471

Thanks for bringing attention to that. I suspect the following fmt
changelog entry:

"Disabled automatic std::ostream insertion operator (operator<<)
discovery when fmt/ostream.h is included to prevent ODR violations. You
can get the old behavior by defining FMT_DEPRECATED_OSTREAM but this
will be removed in the next major release. Use fmt::streamed or
fmt::ostream_formatter to enable formatting via std::ostream instead."

That is a VERY breaking change and there will be more packages affected.
It's likely also affecting dependents of spdlog which use external fmt
and <spdlog/fmt/ostr.h>.

The fix is simple, but not backward-compatible:
    template <> struct fmt::formatter<type> : ostream_formatter {};
(a custom fmt::formatter for the type would work as well).

I already confirmed that `waybar` is broken and will submit a fix
upstream when I have time. Setting up a copr repo to find all other
breakages sounds like a good idea.
> If that FTBFS rate is typical of the full set of affected packages, then
> merging the side tag without resolving the API incompatibilities in
> dependent packages will be pretty disruptive to Rawhide as a whole.
>
> Would you consider setting up a COPR to figure out how many packages are
> affected by the API changes, and filing Bugzillas on the affected
> dependent packages? It might take a while for affected package
> maintainers to become aware of the problem, investigate, and perhaps
> work with upstreams to prepare the necessary patches.
>
> Link to fmt 9.0.0 release notes:
> https://github.com/fmtlib/fmt/releases/tag/9.0.0
>
> – Ben
>
> On 7/11/22 12:21, Ben Beasley wrote:
>>
>> I don’t believe this list is nearly complete. Two packages I maintain
>> that would be affected (fmidi and giada) are absent from the list.
>>
>> Using the following command:
>>
>>     mock -r fedora-rawhide-x86_64 --dnf-cmd -- repoquery
>> --whatrequires 'libfmt.so.8*'
>>
>> I found these additional dependent packages that were missing from the
>> original list:
>>
>>       * bear
>>       * cachelib
>>       * easyeffects
>>       * easyrpg-player
>>       * fb303
>>       * fbthrift
>>       * fizz
>>       * fmidi
>>       * freeopcua
>>       * gerbera
>>       * giada
>>       * gnuradio
>>       * gr-funcube
>>       * libsemigroups
>>       * luxcorerender
>>       * mcrouter
>>       * nheko
>>       * proxygen
>>       * rstudio
>>       * wangle
>>       * watchman
>>       * waybar
>>
>> I will go ahead and rebuild bear, fmidi, and giada into the side tag.
>>
>> On 7/10/22 13:43, Vitaly Zaitsev via devel wrote:
>>> Hello.
>>>
>>> fmt 9.0.x update will include a soversion bump from .8 to .9. It has
>>> both API and ABI changes.
>>>
>>> Changelog: https://github.com/fmtlib/fmt/releases/tag/9.0.0
>>>
>>> The list of affected packages in Rawhide:
>>> 0ad
>>> OpenImageIO
>>> cantera
>>> ceph
>>> dolphin-emu
>>> domoticz
>>> folly
>>> libsonata
>>> mkvtoolnix
>>> sdrpp
>>> spdlog
>>> vcpkg
>>> zswap-cli
>>>
>>> I can rebuild only spdlog, vcpkg and zswap-cli. For others, I will
>>> need proven-packagers assistance.
>>>
>>> Please use the f37-build-side-54844 side tag. I will merge it into
>>> Rawhide later.
>>>
>>
>> _______________________________________________
>> devel mailing list --devel@lists.fedoraproject.org
>> To unsubscribe send an email todevel-leave@lists.fedoraproject.org
>> Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>> Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure
>
> _______________________________________________
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-leave@lists.fedoraproject.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure