sysusers scriptlets: what to do if upstream includes the config files?

[Adam Williamson]

The guidelines for sysusers packaging:

https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...

say:

"Create a <package-name>.sysusers file with the user definition and add
it to the specfile as a source...use the %sysusers_create_compat macro
to consume it in the %pre section:

%pre
%sysusers_create_compat %{SOURCE3}
"

but...what are you supposed to do if *upstream* ships the config files?
openQA recently added these to its upstream distribution, so it seems
wrong to replace the config files upstream ships with ones separately
packaged downstream. But if I don't package them as source files, how
can I set up the %pre macro? I tried this:

%pre
%sysusers_create_compat usr/lib/sysusers.d/geekotest.conf

where usr/lib/sysusers.d/geekotest.conf is the path to one of the
sysusers config file within the upstream source, but it doesn't seem to
work. The built package has no %pre script. However, that does work if
I eval it locally from an openQA source tree:

[adamw@xps13k openQA (master %)]$ rpm --eval "%sysusers_create_compat
usr/lib/sysusers.d/geekotest.conf"

# generated from geekotest.conf
getent group 'geekotest' >/dev/null || groupadd -r 'geekotest'
...etc...

so...help? Is there any way to make this work right? Thanks!

[Frank Ch. Eigler]

Adam Williamson  writes:
> [...]
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...
>
> say:
>
> "Create a <package-name>.sysusers file with the user definition and add
> where usr/lib/sysusers.d/geekotest.conf is the path to one of the
> sysusers config file within the upstream source, but it doesn't seem to
> work. [...]

One problem with these sysusers rpm macros is that they expand to the
scriptlets very early: before even the main source tarball is extracted.
This is why the fedora packaging guideline more or less forces them to
be first-class spec sources.

In the case of systemtap, we worked around this by moving the sysusers
config files right into the spec file - out of the source tarball - and
feed them to %pre and %install scripts by hand.

https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...

IMO this is ugly and unfortunate.

- FChE

[Neal Gompa]

On Sat, Nov 27, 2021 at 9:02 AM Frank Ch. Eigler  wrote:
> Adam Williamson  writes:
> > [...]
> >
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...
> >
> > say:
> >
> > "Create a <package-name>.sysusers file with the user definition and add
> > where usr/lib/sysusers.d/geekotest.conf is the path to one of the
> > sysusers config file within the upstream source, but it doesn't seem to
> > work. [...]
>
> One problem with these sysusers rpm macros is that they expand to the
> scriptlets very early: before even the main source tarball is extracted.
> This is why the fedora packaging guideline more or less forces them to
> be first-class spec sources.
>
> In the case of systemtap, we worked around this by moving the sysusers
> config files right into the spec file - out of the source tarball - and
> feed them to %pre and %install scripts by hand.
>
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
>
> IMO this is ugly and unfortunate.

The design around sysusers expects a model where files are unpacked
and *then* scripts are run. RPM doesn't work that way, which makes all
of this fall apart. In the ideal case, we could generate preinstall
scriptlets for this stuff from detected sysusers files on the fly, but
there's currently no way to do that.

A more practical way to work around this is to always subpackage out
sysusers and use dependencies to guarantee that it's installed before
the package itself is. This would require the systemd file trigger to
make it so that sysusers is run per-package instead of
per-transaction, though. Alternatively, you could just manually run
systemd-sysusers in %post for those cases.

--
真実はいつも一つ！/ Always, there's only one truth!

[Matthew Miller]

On Sat, Nov 27, 2021 at 09:10:13AM -0500, Neal Gompa wrote:
> > IMO this is ugly and unfortunate.
>
> The design around sysusers expects a model where files are unpacked
> and *then* scripts are run. RPM doesn't work that way, which makes all
> of this fall apart. In the ideal case, we could generate preinstall
> scriptlets for this stuff from detected sysusers files on the fly, but
> there's currently no way to do that.

So.... not helping _currently_, but could we make rpm itself handle this in
a different way: stuff the systemd-sysusers files into metadata available
before anything is installed?
> A more practical way to work around this is to always subpackage out
> sysusers and use dependencies to guarantee that it's installed before
> the package itself is. This would require the systemd file trigger to

Ugggghhhhhh. That's practical but gross, unless there's some kind of
macro/automation for the subpackage.

[Igor Raits]

On Sat, Nov 27, 2021 at 6:42 PM Matthew Miller 
wrote:
> On Sat, Nov 27, 2021 at 09:10:13AM -0500, Neal Gompa wrote:
> > > IMO this is ugly and unfortunate.
> >
> > The design around sysusers expects a model where files are unpacked
> > and *then* scripts are run. RPM doesn't work that way, which makes all
> > of this fall apart. In the ideal case, we could generate preinstall
> > scriptlets for this stuff from detected sysusers files on the fly, but
> > there's currently no way to do that.
>
> So.... not helping _currently_, but could we make rpm itself handle this in
> a different way: stuff the systemd-sysusers files into metadata available
> before anything is installed?
> > A more practical way to work around this is to always subpackage out
> > sysusers and use dependencies to guarantee that it's installed before
> > the package itself is. This would require the systemd file trigger to
>
> Ugggghhhhhh. That's practical but gross, unless there's some kind of
> macro/automation for the subpackage.

https://github.com/rpm-software-management/rpm/pull/1485
> --
> Matthew Miller
>
> Fedora Project Leader
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure

[Neal Gompa]

On Sat, Nov 27, 2021 at 12:35 PM Matthew Miller
 wrote:
> On Sat, Nov 27, 2021 at 09:10:13AM -0500, Neal Gompa wrote:
> > > IMO this is ugly and unfortunate.
> >
> > The design around sysusers expects a model where files are unpacked
> > and *then* scripts are run. RPM doesn't work that way, which makes all
> > of this fall apart. In the ideal case, we could generate preinstall
> > scriptlets for this stuff from detected sysusers files on the fly, but
> > there's currently no way to do that.
>
> So.... not helping _currently_, but could we make rpm itself handle this in
> a different way: stuff the systemd-sysusers files into metadata available
> before anything is installed?

Sure, we could.
> > A more practical way to work around this is to always subpackage out
> > sysusers and use dependencies to guarantee that it's installed before
> > the package itself is. This would require the systemd file trigger to
>
> Ugggghhhhhh. That's practical but gross, unless there's some kind of
> macro/automation for the subpackage.

Well, this *was* one reason I wanted subpackage generators...
https://github.com/rpm-software-management/rpm/issues/329

[Panu Matilainen]

On 11/27/21 16:10, Neal Gompa wrote:
> On Sat, Nov 27, 2021 at 9:02 AM Frank Ch. Eigler  wrote:
> > Adam Williamson  writes:
> > > [...]
> > >
> > > https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...
> > >
> > > say:
> > >
> > > "Create a <package-name>.sysusers file with the user definition and add
> > > where usr/lib/sysusers.d/geekotest.conf is the path to one of the
> > > sysusers config file within the upstream source, but it doesn't seem to
> > > work. [...]
> >
> > One problem with these sysusers rpm macros is that they expand to the
> > scriptlets very early: before even the main source tarball is extracted.
> > This is why the fedora packaging guideline more or less forces them to
> > be first-class spec sources.

There's a way around this: change the %sysusers_create_package stuff to 
use "%pre -f <scriptfile>". Just like with %files -f, the script can be 
generated during build/install stages so you can use both separate 
sources and tarballed ones. And like with %files -f, the -f included 
script file gets appended to what else may be in that script.

So basically you'd want to turn %sysusers_create_package into a 
standalone script which can be called from %build/install, and included 
with %pre -f. Much like %find_lang is used.
> > In the case of systemtap, we worked around this by moving the sysusers
> > config files right into the spec file - out of the source tarball - and
> > feed them to %pre and %install scripts by hand.
> >
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> >
> > IMO this is ugly and unfortunate.
>
> The design around sysusers expects a model where files are unpacked
> and *then* scripts are run. RPM doesn't work that way, which makes all

Actually, in rpm >= 4.17 there technically is a window where users could 
be created based on content unpacked from the package itself. What's 
missing is a hook (aka script) to run after unpacking all files but 
prior to setting metadata on them all.
> of this fall apart. In the ideal case, we could generate preinstall
> scriptlets for this stuff from detected sysusers files on the fly, but
> there's currently no way to do that.
>
> A more practical way to work around this is to always subpackage out
> sysusers and use dependencies to guarantee that it's installed before
> the package itself is. This would require the systemd file trigger to
> make it so that sysusers is run per-package instead of
> per-transaction, though. Alternatively, you could just manually run
> systemd-sysusers in %post for those cases.

Yeah, conceptually user-only subpackages is a kinda neat solution, but 
not sure its tolerable in terms of package number explosion / metadata 
overhead.

- Panu -
> --
> 真実はいつも一つ！/ Always, there's only one truth!
> _______________________________________________
> devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Pavel Raiskup]

> There's a way around this: change the %sysusers_create_package stuff to

use "%pre -f <scriptfile>".

Sorry for opening an old thread, but has anyone tried to do what Panu suggested?
Seems like we could:

%build
/usr/lib/rpm/sysusers.generate-pre.sh config-file > pre.file
%pre -f pre.file

The side benefit is that this would fail the build in case of any problem.
https://github.com/rpm-software-management/mock/pull/1367
Pavel

[Adam Williamson]

On Sat, 2021-11-27 at 09:01 -0500, Frank Ch. Eigler wrote:
> Adam Williamson  writes:
> > [...]
> >
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...
> >
> > say:
> >
> > "Create a <package-name>.sysusers file with the user definition and add
> > where usr/lib/sysusers.d/geekotest.conf is the path to one of the
> > sysusers config file within the upstream source, but it doesn't seem to
> > work. [...]
>
> One problem with these sysusers rpm macros is that they expand to the
> scriptlets very early: before even the main source tarball is extracted.

Yeah, I figured that was probably the issue. I could not find any
documentation of exactly when macro expansion happens, but I probably
didn't look hard enough.
> This is why the fedora packaging guideline more or less forces them to
> be first-class spec sources.
>
> In the case of systemtap, we worked around this by moving the sysusers
> config files right into the spec file - out of the source tarball - and
> feed them to %pre and %install scripts by hand.
>
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
>
> IMO this is ugly and unfortunate.

I agree. I'd really like a less icky solution to this :(

[Zbigniew Jędrzejewski-Szmek]

On Sat, Nov 27, 2021 at 03:29:05PM -0800, Adam Williamson wrote:
> On Sat, 2021-11-27 at 09:01 -0500, Frank Ch. Eigler wrote:
> > Adam Williamson  writes:
> > > [...]
> > >
> > > https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_d...
> > >
> > > say:
> > >
> > > "Create a <package-name>.sysusers file with the user definition and add
> > > where usr/lib/sysusers.d/geekotest.conf is the path to one of the
> > > sysusers config file within the upstream source, but it doesn't seem to
> > > work. [...]
> >
> > One problem with these sysusers rpm macros is that they expand to the
> > scriptlets very early: before even the main source tarball is extracted.
>
> Yeah, I figured that was probably the issue. I could not find any
> documentation of exactly when macro expansion happens, but I probably
> didn't look hard enough.
> > This is why the fedora packaging guideline more or less forces them to
> > be first-class spec sources.
> >
> > In the case of systemtap, we worked around this by moving the sysusers
> > config files right into the spec file - out of the source tarball - and
> > feed them to %pre and %install scripts by hand.
> >
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> > https://src.fedoraproject.org/rpms/systemtap/blob/rawhide/f/systemtap.spec#_...
> >
> > IMO this is ugly and unfortunate.
>
> I agree. I'd really like a less icky solution to this :(

Yes, this is a problem. It would be nice if rpm allowed more flexibility here.
https://github.com/rpm-software-management/rpm/pull/1485 looks promising:
we could do away with *all* scriptlets in the spec file, and the sysusers
stuff would be handled by a generator.

A similar problem occurs with transfiletriggers in the systemd package:
they are defined in a long file that is created as part of the build. But
rpmbuild requires that their definition is always available. We copy
the file out [1] and after the build check that it hasn't changed [2],
but that's ugly and fragile.

[1] https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/triggers.systemd
[2] https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/systemd.spec#_533

Zbyszek