Execute RPM dependency generators on the .spec file which ships them

[Vít Ondruch]

I am still playing with idea of executing RPM dependency generators on 
the .spec file which ships them. I have created RPM RFE ticket quite a 
while ago here:

https://github.com/rpm-software-management/rpm/issues/782

So for no luck with having this implemented.

However, I think there is reasonable way. It would be enough to have 
some generic attr file around. I have create small test package 
exploring this idea and you can find it in my Copr repo:

https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/

https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...

On the same place, you can also find package leveraging this concept:

https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...

Would somebody else benefit from this idea/package or can somebody spot 
some issue with it? Thx for feedback.

Vít

[Petr Pisar]

V Fri, Oct 06, 2023 at 06:06:14PM +0200, Vít Ondruch napsal(a):
> I am still playing with idea of executing RPM dependency generators on the
> .spec file which ships them. I have created RPM RFE ticket quite a while ago
> here:
>
> https://github.com/rpm-software-management/rpm/issues/782
>
> So for no luck with having this implemented.
>
> However, I think there is reasonable way. It would be enough to have some
> generic attr file around. I have create small test package exploring this
> idea and you can find it in my Copr repo:
>
> https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> On the same place, you can also find package leveraging this concept:
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> Would somebody else benefit from this idea/package or can somebody spot some
> issue with it? Thx for feedback.

That's hilarious because it's completely out of specification for the genarators
https://rpm-software-management.github.io/rpm/manual/dependency_generators.html#generators:

A generator is just an executable that reads file name(s) from stdin

While your rpm-local-generator-test.spec redefines it to a function reading
an argument:

%global __local_generator_provides() local-generator-provide(%%{basename:%{1}})

I'd like to see comments from RPM maintainers.

Does it work for multiple files packaged in rpm-local-generator-test.spec?

-- Petr

[Vít Ondruch]

Dne 09. 10. 23 v 9:21 Petr Pisar napsal(a):
> V Fri, Oct 06, 2023 at 06:06:14PM +0200, Vít Ondruch napsal(a):
> > I am still playing with idea of executing RPM dependency generators on the
> > .spec file which ships them. I have created RPM RFE ticket quite a while ago
> > here:
> >
> > https://github.com/rpm-software-management/rpm/issues/782
> >
> > So for no luck with having this implemented.
> >
> > However, I think there is reasonable way. It would be enough to have some
> > generic attr file around. I have create small test package exploring this
> > idea and you can find it in my Copr repo:
> >
> > https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/
> >
> > https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
> >
> > On the same place, you can also find package leveraging this concept:
> >
> > https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
> >
> > Would somebody else benefit from this idea/package or can somebody spot some
> > issue with it? Thx for feedback.
>
> That's hilarious because it's completely out of specification for the genarators
> https://rpm-software-management.github.io/rpm/manual/dependency_generators.html#generators:
>  A generator is just an executable that reads file name(s) from stdin
>
> While your rpm-local-generator-test.spec redefines it to a function reading
> an argument:
>  %global __local_generator_provides() local-generator-provide(%%{basename:%{1}})
>
> I'd like to see comments from RPM maintainers.

This is documented:

https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...

I have used the function just because of simplicity, nothing else. You 
can see real life usage of the function generators here:

https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/pyth...
> Does it work for multiple files packaged in rpm-local-generator-test.spec?

The test example includes two files and there are generated provides for 
both of them So yes, it works for multiple files if I understand your 
question.

Vít
> -- Petr
>
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Maxwell G]

On Mon Oct 9, 2023 at 17:30 +0200, Vít Ondruch wrote:
> Dne 09. 10. 23 v 9:21 Petr Pisar napsal(a):
> > V Fri, Oct 06, 2023 at 06:06:14PM +0200, Vít Ondruch napsal(a):

<snip>
> > That's hilarious because it's completely out of specification for the genarators
> > https://rpm-software-management.github.io/rpm/manual/dependency_generators.html#generators:
> >  A generator is just an executable that reads file name(s) from stdin
> >
> > While your rpm-local-generator-test.spec redefines it to a function reading
> > an argument:
> >  %global __local_generator_provides() local-generator-provide(%%{basename:%{1}})
> >
> > I'd like to see comments from RPM maintainers.
>
> This is documented:
>
> https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...
>
> I have used the function just because of simplicity, nothing else. You 
> can see real life usage of the function generators here:
>
> https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/pyth...

Indeed, the parametric generators are quite convenient for simpler
usecases, as you don't need to execute a bunch of processes just to
print some text to stdout.
For packages with a lot of files (e.g. ansible which I maintain), this
really adds up.

[Vít Ondruch]

Just FTR, this is how the generators would be used in practice:

~~~

$ git diff
diff --git a/ruby.spec b/ruby.spec
index 1aea20b..51f3065 100644
--- a/ruby.spec
+++ b/ruby.spec
@@ -196,6 +196,11 @@ Source15: test_openssl_fips.rb
  %{load:%{SOURCE4}}
  %{load:%{SOURCE5}}

+%global __local_generator_requires make -C 
%{_builddir}/%{buildsubdir}/%{_vpath_builddir} -s runruby 
TESTRUN_SCRIPT="--enable-gems %{SOURCE9}"
+%global __local_generator_provides make -C 
%{_builddir}/%{buildsubdir}/%{_vpath_builddir} -s runruby 
TESTRUN_SCRIPT="--enable-gems %{SOURCE10}"
+%global __local_generator_conflicts make -C 
%{_builddir}/%{buildsubdir}/%{_vpath_builddir} -s runruby 
TESTRUN_SCRIPT="--enable-gems %{SOURCE11}"
+%global __local_generator_path ^%{gem_dir}/specifications/.*.gemspec$
+
  # Fix ruby_version abuse.
  # https://bugs.ruby-lang.org/issues/11002
  Patch0: ruby-2.3.0-ruby_version.patch
@@ -229,6 +234,7 @@ Requires: %{name}-libs%{?_isa} = %{version}-%{release}
  Recommends: ruby(rubygems) >= %{rubygems_version}
  Recommends: rubygem(bigdecimal) >= %{bigdecimal_version}

+BuildRequires: rpm-local-generator-support
  # Build dependencies
  BuildRequires: autoconf
  BuildRequires: gcc

~~~

For your convenience, full ruby.spec file is available here:

https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec

Please note that this is the default rubygems.attr file we are 
distributing with Ruby (actually via rubygems-devel package):

~~~

%__rubygems_requires    %{_rpmconfigdir}/rubygems.req
%__rubygems_provides    %{_rpmconfigdir}/rubygems.prov
%__rubygems_conflicts    %{_rpmconfigdir}/rubygems.con
# In non-gem packages, the %%{gem_name} macro is not available and the macro
# stays unexpanded which leads to "invalid regex" error (rhbz#1154067).
%__rubygems_path 
^%{?gem_name:%{gem_spec}}%{!?gem_name:this_should_never_match_anything}$

~~~

As you can see, we would not be able to use the generator even if it was 
somehow available because it need Ruby for its execution. But Ruby is 
not available for Ruby build, therefore we could not benefit from it 
without changing the macros.

Vít

Dne 06. 10. 23 v 18:06 Vít Ondruch napsal(a):
> I am still playing with idea of executing RPM dependency generators on 
> the .spec file which ships them. I have created RPM RFE ticket quite a 
> while ago here:
>
> https://github.com/rpm-software-management/rpm/issues/782
>
> So for no luck with having this implemented.
>
> However, I think there is reasonable way. It would be enough to have 
> some generic attr file around. I have create small test package 
> exploring this idea and you can find it in my Copr repo:
>
> https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> On the same place, you can also find package leveraging this concept:
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> Would somebody else benefit from this idea/package or can somebody 
> spot some issue with it? Thx for feedback.
>
> Vít

[Vít Ondruch]

Can somebody help me please with a package review? The package can't be 
simpler.

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2244428

Thx in advance

Vít

Dne 06. 10. 23 v 18:06 Vít Ondruch napsal(a):
> I am still playing with idea of executing RPM dependency generators on 
> the .spec file which ships them. I have created RPM RFE ticket quite a 
> while ago here:
>
> https://github.com/rpm-software-management/rpm/issues/782
>
> So for no luck with having this implemented.
>
> However, I think there is reasonable way. It would be enough to have 
> some generic attr file around. I have create small test package 
> exploring this idea and you can find it in my Copr repo:
>
> https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> On the same place, you can also find package leveraging this concept:
>
> https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generator...
>
> Would somebody else benefit from this idea/package or can somebody 
> spot some issue with it? Thx for feedback.
>
> Vít

[Miroslav Suchý]

Dne 16. 10. 23 v 16:16 Vít Ondruch napsal(a):
> Can somebody help me please with a package review? The package can't be simpler.
>
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2244428
>
> Thx in advance

Done.

You are welcome.

[Vít Ondruch]

Dne 16. 10. 23 v 16:35 Miroslav Suchý napsal(a):
> Dne 16. 10. 23 v 16:16 Vít Ondruch napsal(a):
> > Can somebody help me please with a package review? The package can't 
> > be simpler.
> >
> > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2244428
> >
> > Thx in advance
>
> Done.
>
> You are welcome.

Thx a lot 🤩

Vít

[Vít Ondruch]

The package is in Fedora now.

The practical demonstration are these two commits:

https://src.fedoraproject.org/fork/vondruch/rpms/ruby/c/6d8ecfca02947b5f1ce4...

https://src.fedoraproject.org/fork/vondruch/rpms/ruby/c/865f5b3a896ed1b423ad...

from this PR:

https://src.fedoraproject.org/rpms/ruby/pull-request/159

Vít

Dne 16. 10. 23 v 16:39 Vít Ondruch napsal(a):
> Dne 16. 10. 23 v 16:35 Miroslav Suchý napsal(a):
> > Dne 16. 10. 23 v 16:16 Vít Ondruch napsal(a):
> > > Can somebody help me please with a package review? The package can't 
> > > be simpler.
> > >
> > > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2244428
> > >
> > > Thx in advance
> >
> > Done.
> >
> > You are welcome.
>
> Thx a lot 🤩
>
> Vít

[Vít Ondruch]

And the package has been retired in Rawhide, because RPM 4.20+ provides 
its own implementation (and there should not be other users to my 
knowledge):

https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...

The difference in use is minimal:

https://src.fedoraproject.org/rpms/ruby/pull-request/187#request_diff

Vít

Dne 18. 10. 23 v 16:48 Vít Ondruch napsal(a):
> The package is in Fedora now.
>
> The practical demonstration are these two commits:
>
> https://src.fedoraproject.org/fork/vondruch/rpms/ruby/c/6d8ecfca02947b5f1ce4...
>
> https://src.fedoraproject.org/fork/vondruch/rpms/ruby/c/865f5b3a896ed1b423ad...
>
> from this PR:
>
> https://src.fedoraproject.org/rpms/ruby/pull-request/159
>
> Vít
>
> Dne 16. 10. 23 v 16:39 Vít Ondruch napsal(a):
> > Dne 16. 10. 23 v 16:35 Miroslav Suchý napsal(a):
> > > Dne 16. 10. 23 v 16:16 Vít Ondruch napsal(a):
> > > > Can somebody help me please with a package review? The package 
> > > > can't be simpler.
> > > >
> > > > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2244428
> > > >
> > > > Thx in advance
> > >
> > > Done.
> > >
> > > You are welcome.
> >
> > Thx a lot 🤩
> >
> > Vít

[Dridi Boukelmoune]

On Wed, Sep 4, 2024 at 12:39 PM Vít Ondruch  wrote:
> And the package has been retired in Rawhide, because RPM 4.20+ provides
> its own implementation (and there should not be other users to my
> knowledge):

I just started using it, it saves me the trouble of bootstrapping a
package, and I was about to sing you my praises for this neat little
hack.

If you don't wish to maintain it, I will unretire rpm-local-generator-support
and keep it around until RPM 4.20 becomes the baseline.
> https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...
>
> The difference in use is minimal:
>
> https://src.fedoraproject.org/rpms/ruby/pull-request/187#request_diff

Neat!

Dridi

[Vít Ondruch]

Dne 04. 09. 24 v 22:43 Dridi Boukelmoune napsal(a):
> On Wed, Sep 4, 2024 at 12:39 PM Vít Ondruch  wrote:
> > And the package has been retired in Rawhide, because RPM 4.20+ provides
> > its own implementation (and there should not be other users to my
> > knowledge):
>
> I just started using it, it saves me the trouble of bootstrapping a
> package, and I was about to sing you my praises for this neat little
> hack.

😊
> If you don't wish to maintain it, I will unretire rpm-local-generator-support
> and keep it around until RPM 4.20 becomes the baseline.

I have retired the package in Rawhide and orphaned elsewhere, because I 
don't think it really needs maintenance. So feel free to use it as it is.

If your concern is compatibility of .spec files over all supported 
Fedoras, then I'd suggest to have a bit different version for Rawhide. I 
have not tried to find a way to make single .spec file to work 
everywhere. It is likely doable with some conditionals.

Vít
> > https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...
> >
> > The difference in use is minimal:
> >
> > https://src.fedoraproject.org/rpms/ruby/pull-request/187#request_diff
>
> Neat!
>
> Dridi

[Vít Ondruch]

Dne 09. 09. 24 v 16:13 Vít Ondruch napsal(a):
> Dne 04. 09. 24 v 22:43 Dridi Boukelmoune napsal(a):
> > On Wed, Sep 4, 2024 at 12:39 PM Vít Ondruch  wrote:
> > > And the package has been retired in Rawhide, because RPM 4.20+ provides
> > > its own implementation (and there should not be other users to my
> > > knowledge):
> >
> > I just started using it, it saves me the trouble of bootstrapping a
> > package

Out of curiosity, where do you use it?

Thx

Vít
> > , and I was about to sing you my praises for this neat little
> > hack.
>
> 😊
> > If you don't wish to maintain it, I will unretire 
> > rpm-local-generator-support
> > and keep it around until RPM 4.20 becomes the baseline.
>
> I have retired the package in Rawhide and orphaned elsewhere, because 
> I don't think it really needs maintenance. So feel free to use it as 
> it is.
>
> If your concern is compatibility of .spec files over all supported 
> Fedoras, then I'd suggest to have a bit different version for Rawhide. 
> I have not tried to find a way to make single .spec file to work 
> everywhere. It is likely doable with some conditionals.
>
> Vít
> > > https://rpm-software-management.github.io/rpm/manual/dependency_generators.h...
> > >
> > > The difference in use is minimal:
> > >
> > > https://src.fedoraproject.org/rpms/ruby/pull-request/187#request_diff
> >
> > Neat!
> >
> > Dridi

[Dridi Boukelmoune]

> Out of curiosity, where do you use it?

I used it for the hare package I was working on:

https://copr.fedorainfracloud.org/coprs/dridi/harelang/packages/

I have locally removed the bootstrap step but haven't had enough spare
time since to realign the sample of packages I was working with.

Dridi