fedrq - new repoquerying tool

[Maxwell G]

Hi Fedorians,

I've been working on a repoquerying tool called fedrq [1] that I'd like
to share with you. Here's the elevator pitch: fedrq provides a friendly
interface to query the Fedora repositories. It makes it really easy to
query across Fedora and EPEL branches. It uses the dnf Python bindings
(libdnf5 backend is almost done) and doesn't shell out to dnf repoquery.
Amongst other things, fedrq allows querying for reverse dependencies,
packages that contain a certain Provide or file, subpackages of an SRPM,
and general package metadata. My favorite features are the easy branch
switching, `fedrq subpkgs` (there's no real equivalent in dnf
repoquery), and the ability to dump package metadata as JSON. The many
threads about how to properly query for dependencies when doing SO name
bump rebuilds and my own frustrations with dnf repoquery inspired this
tool.

You can install fedrq from PyPI or from gotmax23/fedrq [2] on Copr. The
EXAMPLES section of fedrq(1) [3] provides a good intro and demonstration
of fedrq's functionality. The tool is very much a WIP and any feedback
is appreciated.

For reverse dependency rebuilds, you probably want the following
command:

```
$ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
$ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
```

This accounts for any package that depends on any subpackage/binary RPM
produced by SRCNAME at buildtime and/or runtime, and spits out the names
of the source packages that need to be rebuilt. For simple packages that
output only one RPM, `fedrq whatrequires -F source PKGNAME` is
sufficient.

[1] https://sr.ht/~gotmax23/fedrq/
[2] https://copr.fedorainfracloud.org/coprs/gotmax23/fedrq/
[3] https://gotmax23.srht.site/fedrq/fedrq.1.html#EXAMPLES

--
Thanks,

Maxwell G (@gotmax23)
Pronouns: He/They

[Mattia Verga]

Il 12/02/23 06:31, Maxwell G via devel ha scritto:
> For reverse dependency rebuilds, you probably want the following
> command:
> $ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
> $ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
>
> This accounts for any package that depends on any subpackage/binary RPM
> produced by SRCNAME at buildtime and/or runtime, and spits out the names
> of the source packages that need to be rebuilt. For simple packages that
> output only one RPM, `fedrq whatrequires -F source PKGNAME` is
> sufficient.

Thanks for this! I never found a (easy) way to make dnf output the exact
reverse dependency of libindi, for example, and your tool does that well
with an easy to understand output:

$ fedrq whatrequires -F source $(fedrq subpkgs libindi)
indi-3rdparty-drivers
indi-3rdparty-libraries
kstars
libindi
phd2
stellarium

Though, it would be nice to have an easier subcommand or option just for
that, as it will be, I think, the most required use case. Maybe a 'fedrq
whatrequires --soname-bump SRCNAME'?

Mattia

[Maxwell G]

On Sun Feb 12, 2023 at 08:12 +0000, Mattia Verga via devel wrote:
> Il 12/02/23 06:31, Maxwell G via devel ha scritto:
> > For reverse dependency rebuilds, you probably want the following
> > command:
> > $ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
> > $ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
> >
> > This accounts for any package that depends on any subpackage/binary RPM
> > produced by SRCNAME at buildtime and/or runtime, and spits out the names
> > of the source packages that need to be rebuilt. For simple packages that
> > output only one RPM, `fedrq whatrequires -F source PKGNAME` is
> > sufficient.
>
> Thanks for this! I never found a (easy) way to make dnf output the exact
> reverse dependency of libindi, for example, and your tool does that well
> with an easy to understand output:
>
> $ fedrq whatrequires -F source $(fedrq subpkgs libindi)
> indi-3rdparty-drivers
> indi-3rdparty-libraries
> kstars
> libindi
> phd2
> stellarium
>
> Though, it would be nice to have an easier subcommand or option just for
> that, as it will be, I think, the most required use case. Maybe a 'fedrq
> whatrequires --soname-bump SRCNAME'?

Thanks for trying it out and for the feedback! I don't want to tack this
on to `whatrequires`, but a separate subcommand might be warranted. Name
suggestions are welcome. subpkgs-whatrequires is accurate but way too
long :). I created an issue for this in fedrq's issue tracker [1].

[1] https://todo.sr.ht/~gotmax23/fedrq/13

--
Maxwell G (@gotmax23)
Pronouns: He/They

[Mattia Verga]

Il 12/02/23 14:40, Maxwell G ha scritto:
> On Sun Feb 12, 2023 at 08:12 +0000, Mattia Verga via devel wrote:
> > Il 12/02/23 06:31, Maxwell G via devel ha scritto:
> > > For reverse dependency rebuilds, you probably want the following
> > > command:
> > > $ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
> > > $ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
> > >
> > > This accounts for any package that depends on any subpackage/binary RPM
> > > produced by SRCNAME at buildtime and/or runtime, and spits out the names
> > > of the source packages that need to be rebuilt. For simple packages that
> > > output only one RPM, `fedrq whatrequires -F source PKGNAME` is
> > > sufficient.
> >
> > Thanks for this! I never found a (easy) way to make dnf output the exact
> > reverse dependency of libindi, for example, and your tool does that well
> > with an easy to understand output:
> >
> > $ fedrq whatrequires -F source $(fedrq subpkgs libindi)
> > indi-3rdparty-drivers
> > indi-3rdparty-libraries
> > kstars
> > libindi
> > phd2
> > stellarium
> >
> > Though, it would be nice to have an easier subcommand or option just for
> > that, as it will be, I think, the most required use case. Maybe a 'fedrq
> > whatrequires --soname-bump SRCNAME'?
>
> Thanks for trying it out and for the feedback! I don't want to tack this
> on to `whatrequires`, but a separate subcommand might be warranted. Name
> suggestions are welcome. subpkgs-whatrequires is accurate but way too
> long :). I created an issue for this in fedrq's issue tracker [1].
>
> [1] https://todo.sr.ht/~gotmax23/fedrq/13

My 2 cents: 'fedrq reverse-builddep' (inspired from 'dnf builddep') or
'fedrq whatbuildrequire'... the former gets my preference, since it is
like the dnf command, thus more easy to remember.

Mattia

[Maxwell G]

On Sun Feb 12, 2023 at 13:40 +0000, Maxwell G via devel wrote:
> > Though, it would be nice to have an easier subcommand or option just for
> > that, as it will be, I think, the most required use case. Maybe a 'fedrq
> > whatrequires --soname-bump SRCNAME'?
>
> Thanks for trying it out and for the feedback! I don't want to tack this
> on to `whatrequires`, but a separate subcommand might be warranted. Name
> suggestions are welcome. subpkgs-whatrequires is accurate but way too
> long :). I created an issue for this in fedrq's issue tracker [1].
>
> [1] https://todo.sr.ht/~gotmax23/fedrq/13

This is now available on the main branch.
$ fedrq whatrequires -F source $(fedrq subpkgs SRCNAME)
can be replaced with
$ fedrq whatrequires-src -F source SRCNAME
or
$ fedrq wrsrc -F source SRCNAME
wrsrc is an alias for whatrequires-src.

If you want, you can install
https://copr.fedorainfracloud.org/coprs/gotmax23/fedrq-dev/build/5519594/
or
$ pip install 'git+https://git.sr.ht/~gotmax23/fedrq#main'
to try it out. I'll cut a release later this week.

--
Maxwell G (@gotmax23)
Pronouns: He/They

[Michel Alexandre Salim]

The subpackages command looks interesting, and potentially something I can use in ebranch!

Can you use rpmdistro-repoquery's repo definitions? That would allow dropping the embedded repo configs

Cheers,

Michel

[Maxwell G]

Hi Michel,

Thanks for the feedback!

On Mon Feb 13, 2023 at 08:04 CST, Michel Alexandre Salim wrote:
> The subpackages command looks interesting, and potentially something I
> can use in ebranch!

I like it too :).
> Can you use rpmdistro-repoquery's repo definitions? That would allow
> dropping the embedded repo configs

I'm not interested in adding a dependency on another repoquerying tool
just to use its repository defs. If you want extra repository defs
available in fedrq by default, I'd be happy to help you contribute them
or do it myself if you let me know which ones.
You can also configure other releases on your local system as explained
in fedrq(5) [1]. I should probably flesh that manpage out a little
more...

[1] https://gotmax23.srht.site/fedrq/fedrq.5.html

--
Best,

Maxwell G (@gotmax23)
Pronouns: He/They
NEW EMAIL: [e-mail]

[Michel Alexandre Salim]

Ah, in that case I think my best course of action is see if I can use fedrq as a library and possibly extend it to be able to override dnf options (like where repos are).

Would such PRs be acceptable?

Thanks,

Michel

[Maxwell G]

On Mon Feb 13, 2023 at 15:30 CST, Michel Alexandre Salim wrote:
> Ah, in that case I think my best course of action is see if I can use
> fedrq as a library and possibly extend it to be able to override dnf
> options (like where repos are).

There's no need to do that. fedrq can load any .repo file from any location or
from the system configuration. You just need to configure a release in fedrq's
configuration. I created a ticket [1] to track improving the documentation
around this. Feel free to ping me seperately if you have questions.

[1] https://todo.sr.ht/~gotmax23/fedrq/16

--
Thanks,

Maxwell G (@gotmax23)
Pronouns: He/They

[Kevin Kofler]

Maxwell G via devel wrote:
> $ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
> $ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent

Since your mail does not document what the -X flag does:

Quoting:
https://git.sr.ht/~gotmax23/fedrq/tree/e752163cdfb632f2fef4d63e3a61c480c83fc...

```
*-X*, *--exclude-subpackages*
    Filter out reverse dependencies that are built from the same source
    package as _any_ of the input packages. See the whatrequires EXAMPLES.
```

Kevin Kofler

[Maxwell G]

On Mon Feb 13, 2023 at 17:05 +0100, Kevin Kofler via devel wrote:
> Maxwell G via devel wrote:
> > $ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
> > $ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
>
> Since your mail does not document what the -X flag does:
>
> Quoting:
> https://git.sr.ht/~gotmax23/fedrq/tree/e752163cdfb632f2fef4d63e3a61c480c83fc...
> *-X*, *--exclude-subpackages*
>    Filter out reverse dependencies that are built from the same source
>    package as _any_ of the input packages. See the whatrequires EXAMPLES.

Thanks :). Here's an example of why this is useful:

```
$ fedrq whatrequires podman.x86_64
buildah-tests-1.29.0-1.fc38.x86_64
[...]
podman-compose-1.0.3-7.fc38.noarch
podman-docker-4:4.4.0~rc2-2.fc38.noarch
podman-quadlet-4:4.4.0~rc2-2.fc38.x86_64
podman-tests-4:4.4.0~rc2-2.fc38.x86_64
python3-mrack-podman-1.12.3-5.fc38.noarch
quadlet-0.2.4-4.fc38.x86_64
retrace-server-1.24.2-2.fc38.src
[...]
```

If you're trying to query for what requires podman, you might
not want other podman subpackages in the output.

--
Maxwell G (@gotmax23)
Pronouns: He/They
NEW EMAIL: [e-mail]

[Maxwell G]

On Sun Feb 12, 2023 at 05:31 +0000, Maxwell G wrote:
> Hi Fedorians,
>
> I've been working on a repoquerying tool called fedrq [1] that I'd
> like to share with you. Here's the elevator pitch: fedrq provides a
> friendly interface to query the Fedora repositories. It makes it
> really easy to query across Fedora and EPEL branches.
>
> [1] https://fedrq.gtmx.me

I've submitted fedrq for Fedora inclusion and it was pushed to updates a
couple days ago [1]. Thanks to Benson for reviewing it!

Since last time, there's been quite a few changes. fedrq now supports both dnf
and libdnf5. Commands produce (mostly) the same output between the two
backends. I reimplemented hawkey's Package sort algorithm in fedrq's libdnf5
backend. fedrq's API [3] exposes its inbuilt repository definitions and
abstraction/compatibility layer over the libdnf5 and dnf package Query APIs.

fedrq's CLI interface has also gained new functionality. There's new
whatrequires-src and whatobsoletes subco
mmands, there's --forcearch
support, there's more output formatting options, and there more flexible
repository selection. I also added a doc explaining the differences
between fedrq and dnf repoquery [4].

[1] https://bodhi.fedoraproject.org/updates/?search=fedrq-0.5.0
[2] https://pagure.io/GoSIG/go-leaves
[3] https://fedrq.gtmx.me/API/Summary
[4] https://fedrq.gtmx.me/dnf-repoquery-diff

--
Best,

Maxwell G (@gotmax23)
Pronouns: He/They

[Maxwell G]

Hi everyone,

On 2/11/23 23:31, Maxwell G via devel wrote:
> I've been working on a repoquerying tool called fedrq [1] that I'd 
> like to share with you. Here's the elevator pitch: fedrq provides a 
> friendly interface to query the Fedora repositories. It makes it 
> really easy to query across Fedora and EPEL branches. It uses the dnf 
> Python bindings (libdnf5 backend is almost done) and doesn't shell out 
> to dnf repoquery. Amongst other things, fedrq allows querying for 
> reverse dependencies, packages that contain a certain Provide or file, 
> subpackages of an SRPM, and general package metadata. My favorite 
> features are the easy branch switching, `fedrq subpkgs` (there's no 
> real equivalent in dnf repoquery), and the ability to dump package 
> metadata as JSON. The many threads about how to properly query for 
> dependencies when doing SO name bump rebuilds and my own frustrations 
> with dnf repoquery inspired this tool.
> [1] https://sr.ht/~gotmax23/fedrq/
> [2] https://copr.fedorainfracloud.org/coprs/gotmax23/fedrq/
> [3] https://gotmax23.srht.site/fedrq/fedrq.1.html#EXAMPLES

I have been actively developing fedrq for over a year now. Since last 
February, there have been many changes, improvements, and new features 
introduced[1]. fedrq has new commands including the download, 
download-spec, changelogs, and make-cache subcommands, new output 
formatting options, and many built-in release configurations to make it 
easy to query other RPM-based distributions. I also fleshed out the 
public API to provide a strong compatibility layer between the dnf and 
libdnf5 (Package)Query APIs. It's been heartening to see the tool 
adopted by Fedora developers and receive feedback and questions.

fedrq has been under beta (0.Y.Z releases) up until now, but development 
is nearing the 1.0.0 release milestone. Before that, I wanted to reach 
out again to see if anyone had additional feedback, suggestions, 
questions, or any other commentary. Feel free to respond here or hop 
over to fedrq's mailing list[2]!

[1] https://fedrq.gtmx.me/News/

[2] https://lists.sr.ht/~gotmax23/fedrq

Best,

Maxwell

[Jonny Heggheim]

Thanks for the tool, I have never found dnf repoquery to be user
friendly. I will keep it in mind next time I need to query the
repositories.

Very nice that you included man pages and tab completion!

Jonny