Fedora 32 System-Wide Change proposal: Adopting sysusers.d format

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format

== Summary ==
Files in sysusers.d format will be used to declare systems users so it
will be possible to introspect system users. Users will still be
created using old-style useradd calls.

== Owner ==
* Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
* Email: zbyszek at in waw pl

== Detailed Description ==

Many packages define their own user. Right now, those users are
created in %pre by calling getent, useradd, and groupadd
([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
guidelines]). This will be changed to define users in the
[https://www.freedesktop.org/software/systemd/man/sysusers.d.html
sysusers.d format]. A macro will be provided to generate a %pre
scriptlet that will call useradd and groupadd similarly to the
scriptlets that are currently required by the packaging guidelines.

In this proposal, systemd-sysusers will not be used to create the
user. Using the sysusers.d format makes it easy to switch to
systemd-sysusers as the implementation, and to experiment with
different way to actually create the users based on the declarative
syntax.

This approach is heavily based on OpenSUSE's
([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
guidelines]), but does not use separate rpm packages. I think using a
%pre macro is good enough.

== Benefit to Fedora ==

System users are declared by packages using a uniform syntax.

The scriptlet part is standardized. Current implementation of creating
users and groups is not changed, but may be switched easily in the
future. For example, for container images, the macro may be replaced
by a noop implementation, and the users created externally without
installing the user creation tools in the container.

Admins may easily introspect the system user list and which packages
require users.

Admins may easily override definitions of system users by providing
appropriate sysusers.d files with higher priority.

The difference between Fedora and other distros like OpenSUSE is reduced.

== Scope ==
* Proposal owners:
** provide the macro and any helper tools
** submit a proposal to FPC
** convert a subset of packages

* Other developers:
** FPC: review (and accept ;)) the guidelines changes
** other maintainers: convert other packages

* Release engineering: n/a

* Policies and guidelines: a pull request will be submitted

* Trademark approval: N/A (not needed for this Change)

== Upgrade/compatibility impact ==
None. This change should be backwards and forwards compatible, i.e.
unconverted packages can be still installed on new systems, and
converted packages can be installed on older systems.

== How To Test ==
This change should be mostly invisible to users. During installation,
users should be created as appropriate before packages are installed.
For packages that carry files owned by the user, check that the files
are created with appropriate ownership by rpm.

== User Experience ==
<code>systemd-analyze cat-config sysusers.d/</code>
shows the definitions of system users (incl. local overrides).

== Dependencies ==
N/A

== Contingency Plan ==

* Contingency mechanism: Revert to previous mechanism. This will
require a revert of changes to the spec file and a rebuild of the
package.

* Contingency deadline: beta freeze
* Blocks release? No
* Blocks product? No

== Documentation ==
TBD.

== Release Notes ==
Not needed.

-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis

[Neal Gompa]

On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
>
>  == Summary ==
>  Files in sysusers.d format will be used to declare systems users so it
>  will be possible to introspect system users. Users will still be
>  created using old-style useradd calls.
>
>  == Owner ==
>  * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
>  * Email: zbyszek at in waw pl
>
>  == Detailed Description ==
>
>  Many packages define their own user. Right now, those users are
>  created in %pre by calling getent, useradd, and groupadd
>
> ([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
>  guidelines]). This will be changed to define users in the
>  [https://www.freedesktop.org/software/systemd/man/sysusers.d.html
>  sysusers.d format]. A macro will be provided to generate a %pre
>  scriptlet that will call useradd and groupadd similarly to the
>  scriptlets that are currently required by the packaging guidelines.
>
>  In this proposal, systemd-sysusers will not be used to create the
>  user. Using the sysusers.d format makes it easy to switch to
>  systemd-sysusers as the implementation, and to experiment with
>  different way to actually create the users based on the declarative
>  syntax.
>
>  This approach is heavily based on OpenSUSE's
>  ([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
>  guidelines]), but does not use separate rpm packages. I think using a
>  %pre macro is good enough.
>
>  == Benefit to Fedora ==
>
>  System users are declared by packages using a uniform syntax.
>
>  The scriptlet part is standardized. Current implementation of creating
>  users and groups is not changed, but may be switched easily in the
>  future. For example, for container images, the macro may be replaced
>  by a noop implementation, and the users created externally without
>  installing the user creation tools in the container.
>
>  Admins may easily introspect the system user list and which packages
>  require users.
>
>  Admins may easily override definitions of system users by providing
>  appropriate sysusers.d files with higher priority.
>
>  The difference between Fedora and other distros like OpenSUSE is reduced.
>  
While this *is* incrementally better than before, there *is* a reason
for using separate RPM packages as openSUSE does. There are many cases
where a user+group is actually shared across multiple packages, and
having the user+group definition broken out into a subpackage means
that it's easy for multiple packages to depend on it without having to
depend on the program package.

For example, in openSUSE, there is a wwwrun user paired with the www
group (there's also a wwwrun group, but it doesn't matter quite that
much). This user+group combination is used by *all* webservers. It
also has a static ID, so that it's consistent across installs. Having
this be a package instead of being a scriptlet run by each of the
webservers means that it's easy for the webserver packages to depend
on them. Moreover, the *definition* of the user+group is centralized,
making it easier to audit and adjust over time. Also importantly,
there's a dependency generator that generates user() and group()
Provides for this, so that packages can just declare they need that
user+group combo to exist first before the package can be installed.

I would like us to be able to do that in Fedora too, as it would
drastically simplify some of the odd issues we have with managing
shared users and groups across multiple packages (web servers,
language runtime environments, git services, etc.).

For us, since we don't have the SUSE patches that make PreReq do
things, the way we'd declare this with upstream RPM features would be:

Requires: user(wwwrun)
OrderWithRequires: user(wwwrun)

That way, it's always installed before the package in question is.


--
真実はいつも一つ！/ Always, there's only one truth!

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2020 at 10:29:26AM -0500, Neal Gompa wrote:
>  On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
>  >
>  > https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
>  >
>  > == Summary ==
>  > Files in sysusers.d format will be used to declare systems users so it
>  > will be possible to introspect system users. Users will still be
>  > created using old-style useradd calls.
>  >
>  > == Owner ==
>  > * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
>  > * Email: zbyszek at in waw pl
>  >
>  > == Detailed Description ==
>  >
>  > Many packages define their own user. Right now, those users are
>  > created in %pre by calling getent, useradd, and groupadd
>  >
> ([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
>  > guidelines]). This will be changed to define users in the
>  > [https://www.freedesktop.org/software/systemd/man/sysusers.d.html
>  > sysusers.d format]. A macro will be provided to generate a %pre
>  > scriptlet that will call useradd and groupadd similarly to the
>  > scriptlets that are currently required by the packaging guidelines.
>  >
>  > In this proposal, systemd-sysusers will not be used to create the
>  > user. Using the sysusers.d format makes it easy to switch to
>  > systemd-sysusers as the implementation, and to experiment with
>  > different way to actually create the users based on the declarative
>  > syntax.
>  >
>  > This approach is heavily based on OpenSUSE's
>  > ([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
>  > guidelines]), but does not use separate rpm packages. I think using a
>  > %pre macro is good enough.
>  >
>  > == Benefit to Fedora ==
>  >
>  > System users are declared by packages using a uniform syntax.
>  >
>  > The scriptlet part is standardized. Current implementation of creating
>  > users and groups is not changed, but may be switched easily in the
>  > future. For example, for container images, the macro may be replaced
>  > by a noop implementation, and the users created externally without
>  > installing the user creation tools in the container.
>  >
>  > Admins may easily introspect the system user list and which packages
>  > require users.
>  >
>  > Admins may easily override definitions of system users by providing
>  > appropriate sysusers.d files with higher priority.
>  >
>  > The difference between Fedora and other distros like OpenSUSE is reduced.
>  >
>  
>  While this *is* incrementally better than before, there *is* a reason
>  for using separate RPM packages as openSUSE does. There are many cases
>  where a user+group is actually shared across multiple packages, and
>  having the user+group definition broken out into a subpackage means
>  that it's easy for multiple packages to depend on it without having to
>  depend on the program package. 
For those cases, a separate package can be split out, similarly to any
other case of resources shared between packages. I didn't want to make
the split mandatory because I assumed that sharing a user between
completely independent packages is something of a fringe case...
(The case when there's a single "main" package and a bunch of add-on
packages which use the same user doesn't count here, because in that
case it's easy enough to define the user in the "main" package.)
Apart from web servers, what are the use cases?

>  For example, in openSUSE, there is a wwwrun user paired with the www
>  group (there's also a wwwrun group, but it doesn't matter quite that
>  much). This user+group combination is used by *all* webservers. 
>  It also has a static ID, so that it's consistent across installs.
Fedora pretty much gave up on static UIDs a while back. We use "dynamic"
allocation pretty much all the time. Nevertheless, I don't think that
static vs. dynamic matters for this, since all the discussed mechanisms
(current useradd+groupadd, proposed mechanism, or opensuse mechanism
with separate packages) would give the same id allocations.

>  Moreover, the *definition* of the user+group is centralized,
>  making it easier to audit and adjust over time. 
I don't grok this sentence. In my proposal the definition of a user is
a single sysusers.d file. It is owned by some specific package, and in
this sense the definition is centralized (unique). IIUC, opensuse has
a similar setup, with the main difference being that the file is shunted
to a separate package, i.e. there is just as much "centralization".

>  Also importantly,
>  there's a dependency generator that generates user() and group()
>  Provides for this, so that packages can just declare they need that
>  user+group combo to exist first before the package can be installed. 
This can be added too.

>  I would like us to be able to do that in Fedora too, as it would
>  drastically simplify some of the odd issues we have with managing
>  shared users and groups across multiple packages (web servers,
>  language runtime environments, git services, etc.). 
I was not aware of those problems. Can you give me some pointers to
those issues?

>  For us, since we don't have the SUSE patches that make PreReq do
>  things, the way we'd declare this with upstream RPM features would be:
>  
>  Requires: user(wwwrun)
>  OrderWithRequires: user(wwwrun)
>  
>  That way, it's always installed before the package in question is. 
Sure, I don't see why not.

Zbyszek

[Neal Gompa]

On Thu, Jan 2, 2020 at 11:04 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Thu, Jan 02, 2020 at 10:29:26AM -0500, Neal Gompa wrote:
>  > On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  > >
>  > > https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
>  > >
>  > > == Summary ==
>  > > Files in sysusers.d format will be used to declare systems users so it
>  > > will be possible to introspect system users. Users will still be
>  > > created using old-style useradd calls.
>  > >
>  > > == Owner ==
>  > > * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
>  > > * Email: zbyszek at in waw pl
>  > >
>  > > == Detailed Description ==
>  > >
>  > > Many packages define their own user. Right now, those users are
>  > > created in %pre by calling getent, useradd, and groupadd
>  > >
> ([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
>  > > guidelines]). This will be changed to define users in the
>  > > [https://www.freedesktop.org/software/systemd/man/sysusers.d.html
>  > > sysusers.d format]. A macro will be provided to generate a %pre
>  > > scriptlet that will call useradd and groupadd similarly to the
>  > > scriptlets that are currently required by the packaging guidelines.
>  > >
>  > > In this proposal, systemd-sysusers will not be used to create the
>  > > user. Using the sysusers.d format makes it easy to switch to
>  > > systemd-sysusers as the implementation, and to experiment with
>  > > different way to actually create the users based on the declarative
>  > > syntax.
>  > >
>  > > This approach is heavily based on OpenSUSE's
>  > > ([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
>  > > guidelines]), but does not use separate rpm packages. I think using a
>  > > %pre macro is good enough.
>  > >
>  > > == Benefit to Fedora ==
>  > >
>  > > System users are declared by packages using a uniform syntax.
>  > >
>  > > The scriptlet part is standardized. Current implementation of creating
>  > > users and groups is not changed, but may be switched easily in the
>  > > future. For example, for container images, the macro may be replaced
>  > > by a noop implementation, and the users created externally without
>  > > installing the user creation tools in the container.
>  > >
>  > > Admins may easily introspect the system user list and which packages
>  > > require users.
>  > >
>  > > Admins may easily override definitions of system users by providing
>  > > appropriate sysusers.d files with higher priority.
>  > >
>  > > The difference between Fedora and other distros like OpenSUSE is reduced.
>  > >
>  >
>  > While this *is* incrementally better than before, there *is* a reason
>  > for using separate RPM packages as openSUSE does. There are many cases
>  > where a user+group is actually shared across multiple packages, and
>  > having the user+group definition broken out into a subpackage means
>  > that it's easy for multiple packages to depend on it without having to
>  > depend on the program package.
>
>  For those cases, a separate package can be split out, similarly to any
>  other case of resources shared between packages. I didn't want to make
>  the split mandatory because I assumed that sharing a user between
>  completely independent packages is something of a fringe case...
>  (The case when there's a single "main" package and a bunch of add-on
>  packages which use the same user doesn't count here, because in that
>  case it's easy enough to define the user in the "main" package.)
>  Apart from web servers, what are the use cases?
>
>  > For example, in openSUSE, there is a wwwrun user paired with the www
>  > group (there's also a wwwrun group, but it doesn't matter quite that
>  > much). This user+group combination is used by *all* webservers.
>
>  > It also has a static ID, so that it's consistent across installs.
>
>  Fedora pretty much gave up on static UIDs a while back. We use "dynamic"
>  allocation pretty much all the time. Nevertheless, I don't think that
>  static vs. dynamic matters for this, since all the discussed mechanisms
>  (current useradd+groupadd, proposed mechanism, or opensuse mechanism
>  with separate packages) would give the same id allocations.
>  
This is not true. Static IDs are hard when you potentially have
multiple packages defining the same user/group, but if they're shared
somehow, then static IDs are easier. We do not require static IDs by
default, and indeed most are non-static because they don't need to be
static. But some are, because if they aren't, things break (mock, for
example).

>  > Moreover, the *definition* of the user+group is centralized,
>  > making it easier to audit and adjust over time.
>
>  I don't grok this sentence. In my proposal the definition of a user is
>  a single sysusers.d file. It is owned by some specific package, and in
>  this sense the definition is centralized (unique). IIUC, opensuse has
>  a similar setup, with the main difference being that the file is shunted
>  to a separate package, i.e. there is just as much "centralization".
>  
The ownership is centralized in either case, true, but officially
supporting the case of splitting out as a package means that if
multiple consumers of a user/group are a thing, this can be handled a
lot more gracefully. It's pretty ugly as-is.

>  > Also importantly,
>  > there's a dependency generator that generates user() and group()
>  > Provides for this, so that packages can just declare they need that
>  > user+group combo to exist first before the package can be installed.
>
>  This can be added too.
>
>  > I would like us to be able to do that in Fedora too, as it would
>  > drastically simplify some of the odd issues we have with managing
>  > shared users and groups across multiple packages (web servers,
>  > language runtime environments, git services, etc.).
>
>  I was not aware of those problems. Can you give me some pointers to
>  those issues?
>  
Most of the problems I've had are related to me trying to make the
Pagure package be more flexible (not tied to apache, etc.) while still
having to deal with the complexities of web app packaging+deployment.
So far, I haven't been successful to a reasonable degree of
satisfaction, mostly because users and groups are messy. :(

For $DAYJOB and a couple of other projects, I've encountered issues
with having to support multiple providers of something, and they "own"
directories without sharing users and groups, so depending on what is
installed, I wind up in a situation where things don't work.

>  > For us, since we don't have the SUSE patches that make
> PreReq do
>  > things, the way we'd declare this with upstream RPM features would be:
>  >
>  > Requires: user(wwwrun)
>  > OrderWithRequires: user(wwwrun)
>  >
>  > That way, it's always installed before the package in question is.
>
>  Sure, I don't see why not.
>  
Cool.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2020 at 11:15:57AM -0500, Neal Gompa wrote:
>  On Thu, Jan 2, 2020 at 11:04 AM Zbigniew Jędrzejewski-Szmek
>  <zbyszek(a)in.waw.pl&gt; wrote:
>  >
>  > On Thu, Jan 02, 2020 at 10:29:26AM -0500, Neal Gompa wrote:
>  > > On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  > > >
>  > > > https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
>  > > >
>  > > > == Summary ==
>  > > > Files in sysusers.d format will be used to declare systems users so it
>  > > > will be possible to introspect system users. Users will still be
>  > > > created using old-style useradd calls.
>  > > >
>  > > > == Owner ==
>  > > > * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
>  > > > * Email: zbyszek at in waw pl
>  > > >
>  > > > == Detailed Description ==
>  > > >
>  > > > Many packages define their own user. Right now, those users are
>  > > > created in %pre by calling getent, useradd, and groupadd
>  > > >
> ([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
>  > > > guidelines]). This will be changed to define users in the
>  > > > [https://www.freedesktop.org/software/systemd/man/sysusers.d.html
>  > > > sysusers.d format]. A macro will be provided to generate a %pre
>  > > > scriptlet that will call useradd and groupadd similarly to the
>  > > > scriptlets that are currently required by the packaging guidelines.
>  > > >
>  > > > In this proposal, systemd-sysusers will not be used to create the
>  > > > user. Using the sysusers.d format makes it easy to switch to
>  > > > systemd-sysusers as the implementation, and to experiment with
>  > > > different way to actually create the users based on the declarative
>  > > > syntax.
>  > > >
>  > > > This approach is heavily based on OpenSUSE's
>  > > > ([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
>  > > > guidelines]), but does not use separate rpm packages. I think using a
>  > > > %pre macro is good enough.
>  > > >
>  > > > == Benefit to Fedora ==
>  > > >
>  > > > System users are declared by packages using a uniform syntax.
>  > > >
>  > > > The scriptlet part is standardized. Current implementation of creating
>  > > > users and groups is not changed, but may be switched easily in the
>  > > > future. For example, for container images, the macro may be replaced
>  > > > by a noop implementation, and the users created externally without
>  > > > installing the user creation tools in the container.
>  > > >
>  > > > Admins may easily introspect the system user list and which packages
>  > > > require users.
>  > > >
>  > > > Admins may easily override definitions of system users by providing
>  > > > appropriate sysusers.d files with higher priority.
>  > > >
>  > > > The difference between Fedora and other distros like OpenSUSE is reduced.
>  > > >
>  > >
>  > > While this *is* incrementally better than before, there *is* a reason
>  > > for using separate RPM packages as openSUSE does. There are many cases
>  > > where a user+group is actually shared across multiple packages, and
>  > > having the user+group definition broken out into a subpackage means
>  > > that it's easy for multiple packages to depend on it without having to
>  > > depend on the program package.
>  >
>  > For those cases, a separate package can be split out, similarly to any
>  > other case of resources shared between packages. I didn't want to make
>  > the split mandatory because I assumed that sharing a user between
>  > completely independent packages is something of a fringe case...
>  > (The case when there's a single "main" package and a bunch of add-on
>  > packages which use the same user doesn't count here, because in that
>  > case it's easy enough to define the user in the "main" package.)
>  > Apart from web servers, what are the use cases?
>  >
>  > > For example, in openSUSE, there is a wwwrun user paired with the www
>  > > group (there's also a wwwrun group, but it doesn't matter quite that
>  > > much). This user+group combination is used by *all* webservers.
>  >
>  > > It also has a static ID, so that it's consistent across installs.
>  >
>  > Fedora pretty much gave up on static UIDs a while back. We use "dynamic"
>  > allocation pretty much all the time. Nevertheless, I don't think that
>  > static vs. dynamic matters for this, since all the discussed mechanisms
>  > (current useradd+groupadd, proposed mechanism, or opensuse mechanism
>  > with separate packages) would give the same id allocations.
>  >
>  
>  This is not true.  
What exactly?

>  Static IDs are hard when you potentially have
>  multiple packages defining the same user/group, but if they're shared
>  somehow, then static IDs are easier. We do not require static IDs by
>  default, and indeed most are non-static because they don't need to be
>  static. But some are, because if they aren't, things break (mock, for
>  example). 
I agree that static IDs have their uses. By "gave up" I only meant
that they are rarely used and new ones are rarely added, not that they
are never used. I guess we are in total agreement here?

>  > > Moreover, the *definition* of the user+group is
> centralized,
>  > > making it easier to audit and adjust over time.
>  >
>  > I don't grok this sentence. In my proposal the definition of a user is
>  > a single sysusers.d file. It is owned by some specific package, and in
>  > this sense the definition is centralized (unique). IIUC, opensuse has
>  > a similar setup, with the main difference being that the file is shunted
>  > to a separate package, i.e. there is just as much "centralization".
>  >
>  
>  The ownership is centralized in either case, true, but officially
>  supporting the case of splitting out as a package means that if
>  multiple consumers of a user/group are a thing, this can be handled a
>  lot more gracefully. It's pretty ugly as-is. 
I think this is rather similar to the case of packages co-owning a
directory. For that case we declare that -filesystem package can be split
out. But we do this only when it's actually useful, i.e. when multiple
packages that are fully independent require the same directory and any
of them can be installed independently of others. For the common case
this isn't necessary.

Would it make you feel better if I added a para to the Change page saying
that the sysusers.d definition may be split out into a separate package
if desired?

>  > > Also importantly,
>  > > there's a dependency generator that generates user() and group()
>  > > Provides for this, so that packages can just declare they need that
>  > > user+group combo to exist first before the package can be installed.
>  >
>  > This can be added too.
>  >
>  > > I would like us to be able to do that in Fedora too, as it would
>  > > drastically simplify some of the odd issues we have with managing
>  > > shared users and groups across multiple packages (web servers,
>  > > language runtime environments, git services, etc.).
>  >
>  > I was not aware of those problems. Can you give me some pointers to
>  > those issues?
>  >
>  
>  Most of the problems I've had are related to me trying to make the
>  Pagure package be more flexible (not tied to apache, etc.) while still
>  having to deal with the complexities of web app packaging+deployment.
>  So far, I haven't been successful to a reasonable degree of
>  satisfaction, mostly because users and groups are messy. :(
>  
>  For $DAYJOB and a couple of other projects, I've encountered issues
>  with having to support multiple providers of something, and they "own"
>  directories without sharing users and groups, so depending on what is
>  installed, I wind up in a situation where things don't work. 
I fully agree that there should be a single definition of the user and
group (i.e. a single package providing the sysuser.d file).

We have guidelines how to share a directory between packages and they
work. I think we're veering off topic here into bugs in some package.

>  > > For us, since we don't have the SUSE patches that make
> PreReq do
>  > > things, the way we'd declare this with upstream RPM features would be:
>  > >
>  > > Requires: user(wwwrun)
>  > > OrderWithRequires: user(wwwrun)
>  > >
>  > > That way, it's always installed before the package in question is.
>  >
>  > Sure, I don't see why not.
>  >
>  
>  Cool. 
Zbyszek

[Neal Gompa]

On Thu, Jan 2, 2020 at 11:39 AM Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  On Thu, Jan 02, 2020 at 11:15:57AM -0500, Neal Gompa wrote:
>  > On Thu, Jan 2, 2020 at 11:04 AM Zbigniew Jędrzejewski-Szmek
>  > <zbyszek(a)in.waw.pl&gt; wrote:
>  > >
>  > > On Thu, Jan 02, 2020 at 10:29:26AM -0500, Neal Gompa wrote:
>  > > > On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
>  > > > >
>  > > > > https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
>  > > > >
>  > > > > == Summary ==
>  > > > > Files in sysusers.d format will be used to declare systems users so
> it
>  > > > > will be possible to introspect system users. Users will still be
>  > > > > created using old-style useradd calls.
>  > > > >
>  > > > > == Owner ==
>  > > > > * Name: [[User:zbyszek| Zbigniew Jędrzejewski-Szmek]]
>  > > > > * Email: zbyszek at in waw pl
>  > > > >
>  > > > > == Detailed Description ==
>  > > > >
>  > > > > Many packages define their own user. Right now, those users are
>  > > > > created in %pre by calling getent, useradd, and groupadd
>  > > > >
> ([https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/...
>  > > > > guidelines]). This will be changed to define users in the
>  > > > > [https://www.freedesktop.org/software/systemd/man/sysusers.d.html
>  > > > > sysusers.d format]. A macro will be provided to generate a %pre
>  > > > > scriptlet that will call useradd and groupadd similarly to the
>  > > > > scriptlets that are currently required by the packaging guidelines.
>  > > > >
>  > > > > In this proposal, systemd-sysusers will not be used to create the
>  > > > > user. Using the sysusers.d format makes it easy to switch to
>  > > > > systemd-sysusers as the implementation, and to experiment with
>  > > > > different way to actually create the users based on the declarative
>  > > > > syntax.
>  > > > >
>  > > > > This approach is heavily based on OpenSUSE's
>  > > > >
> ([https://en.opensuse.org/openSUSE:Packaging_guidelines#Users_and_Groups
>  > > > > guidelines]), but does not use separate rpm packages. I think using
> a
>  > > > > %pre macro is good enough.
>  > > > >
>  > > > > == Benefit to Fedora ==
>  > > > >
>  > > > > System users are declared by packages using a uniform syntax.
>  > > > >
>  > > > > The scriptlet part is standardized. Current implementation of
> creating
>  > > > > users and groups is not changed, but may be switched easily in the
>  > > > > future. For example, for container images, the macro may be replaced
>  > > > > by a noop implementation, and the users created externally without
>  > > > > installing the user creation tools in the container.
>  > > > >
>  > > > > Admins may easily introspect the system user list and which packages
>  > > > > require users.
>  > > > >
>  > > > > Admins may easily override definitions of system users by providing
>  > > > > appropriate sysusers.d files with higher priority.
>  > > > >
>  > > > > The difference between Fedora and other distros like OpenSUSE is
> reduced.
>  > > > >
>  > > >
>  > > > While this *is* incrementally better than before, there *is* a reason
>  > > > for using separate RPM packages as openSUSE does. There are many cases
>  > > > where a user+group is actually shared across multiple packages, and
>  > > > having the user+group definition broken out into a subpackage means
>  > > > that it's easy for multiple packages to depend on it without having
> to
>  > > > depend on the program package.
>  > >
>  > > For those cases, a separate package can be split out, similarly to any
>  > > other case of resources shared between packages. I didn't want to make
>  > > the split mandatory because I assumed that sharing a user between
>  > > completely independent packages is something of a fringe case...
>  > > (The case when there's a single "main" package and a bunch of
> add-on
>  > > packages which use the same user doesn't count here, because in that
>  > > case it's easy enough to define the user in the "main" package.)
>  > > Apart from web servers, what are the use cases?
>  > >
>  > > > For example, in openSUSE, there is a wwwrun user paired with the www
>  > > > group (there's also a wwwrun group, but it doesn't matter quite
> that
>  > > > much). This user+group combination is used by *all* webservers.
>  > >
>  > > > It also has a static ID, so that it's consistent across installs.
>  > >
>  > > Fedora pretty much gave up on static UIDs a while back. We use
> "dynamic"
>  > > allocation pretty much all the time. Nevertheless, I don't think that
>  > > static vs. dynamic matters for this, since all the discussed mechanisms
>  > > (current useradd+groupadd, proposed mechanism, or opensuse mechanism
>  > > with separate packages) would give the same id allocations.
>  > >
>  >
>  > This is not true.
>  What exactly?
>
>  > Static IDs are hard when you potentially have
>  > multiple packages defining the same user/group, but if they're shared
>  > somehow, then static IDs are easier. We do not require static IDs by
>  > default, and indeed most are non-static because they don't need to be
>  > static. But some are, because if they aren't, things break (mock, for
>  > example).
>
>  I agree that static IDs have their uses. By "gave up" I only meant
>  that they are rarely used and new ones are rarely added, not that they
>  are never used. I guess we are in total agreement here?
>  
I guess so?

>  > > > Moreover, the *definition* of the user+group is
> centralized,
>  > > > making it easier to audit and adjust over time.
>  > >
>  > > I don't grok this sentence. In my proposal the definition of a user is
>  > > a single sysusers.d file. It is owned by some specific package, and in
>  > > this sense the definition is centralized (unique). IIUC, opensuse has
>  > > a similar setup, with the main difference being that the file is shunted
>  > > to a separate package, i.e. there is just as much "centralization".
>  > >
>  >
>  > The ownership is centralized in either case, true, but officially
>  > supporting the case of splitting out as a package means that if
>  > multiple consumers of a user/group are a thing, this can be handled a
>  > lot more gracefully. It's pretty ugly as-is.
>
>  I think this is rather similar to the case of packages co-owning a
>  directory. For that case we declare that -filesystem package can be split
>  out. But we do this only when it's actually useful, i.e. when multiple
>  packages that are fully independent require the same directory and any
>  of them can be installed independently of others. For the common case
>  this isn't necessary.
>
>  Would it make you feel better if I added a para to the Change page saying
>  that the sysusers.d definition may be split out into a separate package
>  if desired?
>  
Yes, it would. Guidelines for these kinds of splits should also be
drafted for FPC review.

>  > > > Also importantly,
>  > > > there's a dependency generator that generates user() and group()
>  > > > Provides for this, so that packages can just declare they need that
>  > > > user+group combo to exist first before the package can be installed.
>  > >
>  > > This can be added too.
>  > >
>  > > > I would like us to be able to do that in Fedora too, as it would
>  > > > drastically simplify some of the odd issues we have with managing
>  > > > shared users and groups across multiple packages (web servers,
>  > > > language runtime environments, git services, etc.).
>  > >
>  > > I was not aware of those problems. Can you give me some pointers to
>  > > those issues?
>  > >
>  >
>  > Most of the problems I've had are related to me trying to make the
>  > Pagure package be more flexible (not tied to apache, etc.) while still
>  > having to deal with the complexities of web app packaging+deployment.
>  > So far, I haven't been successful to a reasonable degree of
>  > satisfaction, mostly because users and groups are messy. :(
>  >
>  > For $DAYJOB and a couple of other projects, I've encountered issues
>  > with having to support multiple providers of something, and they "own"
>  > directories without sharing users and groups, so depending on what is
>  > installed, I wind up in a situation where things don't work.
>
>  I fully agree that there should be a single definition of the user and
>  group (i.e. a single package providing the sysuser.d file).
>
>  We have guidelines how to share a directory between packages and they
>  work. I think we're veering off topic here into bugs in some package.
>  
Perhaps. I just want as part of this Change some clear guidance on how
to handle these scenarios, because right now we don't have any.

>  > > > For us, since we don't have the SUSE patches that
> make PreReq do
>  > > > things, the way we'd declare this with upstream RPM features would
> be:
>  > > >
>  > > > Requires: user(wwwrun)
>  > > > OrderWithRequires: user(wwwrun)
>  > > >
>  > > > That way, it's always installed before the package in question is.
>  > >
>  > > Sure, I don't see why not.
>  > >
>  >
>  > Cool.
>
>  Zbyszek 


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2020 at 12:27:46PM -0500, Neal Gompa wrote:
>  > Would it make you feel better if I added a para to the Change
> page saying
>  > that the sysusers.d definition may be split out into a separate package
>  > if desired?
>  >
>  
>  Yes, it would. Guidelines for these kinds of splits should also be
>  drafted for FPC review. 
https://fedoraproject.org/w/index.php?title=Changes%2FAdopting_sysusers.d...

Zbyszek

[Panu Matilainen]

On 1/2/20 5:29 PM, Neal Gompa wrote:
>  On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote: 
>  For us, since we don't have the SUSE patches that make PreReq do
>  things, the way we'd declare this with upstream RPM features would be: 
Please don't spread misinformation, openSUSE doesn't have patches to 
make PreReq anyhow special. They merely undo the deprecation warning 
that is issued on PreReq. With or without that, a PreReq will be 
translated to Requires(pre,preun) to simulate what the original PreReq did.

>  
>  Requires: user(wwwrun)
>  OrderWithRequires: user(wwwrun) 
I'm not sure how one is supposed to read the above, but certainly one 
does NOT need both Requires and OrderWithRequires on the same thing in a 
package. Requires(pre) might be in order because for user/group you 
really want them installed first in case of loops, but this is details 
without

	- Panu -

[Neal Gompa]

On Fri, Jan 3, 2020 at 4:01 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  On 1/2/20 5:29 PM, Neal Gompa wrote:
>  > On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>
>  > For us, since we don't have the SUSE patches that make PreReq do
>  > things, the way we'd declare this with upstream RPM features would be:
>
>  Please don't spread misinformation, openSUSE doesn't have patches to
>  make PreReq anyhow special. They merely undo the deprecation warning
>  that is issued on PreReq. With or without that, a PreReq will be
>  translated to Requires(pre,preun) to simulate what the original PreReq did.
>  
Huh, for some reason I thought it also did something else...

>  >
>  > Requires: user(wwwrun)
>  > OrderWithRequires: user(wwwrun)
>
>  I'm not sure how one is supposed to read the above, but certainly one
>  does NOT need both Requires and OrderWithRequires on the same thing in a
>  package. Requires(pre) might be in order because for user/group you
>  really want them installed first in case of loops, but this is details
>  without 
OtherWithRequires makes it so that if it's in the same transaction,
it'll get installed before this package. Otherwise it doesn't matter,
yes.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Panu Matilainen]

On 1/3/20 12:35 PM, Neal Gompa wrote:
>  On Fri, Jan 3, 2020 at 4:01 AM Panu Matilainen
> <pmatilai(a)redhat.com&gt; wrote:
> >
> > On 1/2/20 5:29 PM, Neal Gompa wrote:
> >> On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
> >
> >> For us, since we don't have the SUSE patches that make PreReq do
> >> things, the way we'd declare this with upstream RPM features would be:
> >
> > Please don't spread misinformation, openSUSE doesn't have patches to
> > make PreReq anyhow special. They merely undo the deprecation warning
> > that is issued on PreReq. With or without that, a PreReq will be
> > translated to Requires(pre,preun) to simulate what the original PreReq did.
> >
>  
>  Huh, for some reason I thought it also did something else...
>  
> >>
> >> Requires: user(wwwrun)
> >> OrderWithRequires: user(wwwrun)
> >
> > I'm not sure how one is supposed to read the above, but certainly one
> > does NOT need both Requires and OrderWithRequires on the same thing in a
> > package. Requires(pre) might be in order because for user/group you
> > really want them installed first in case of loops, but this is details
> > without
>  
>  OtherWithRequires makes it so that if it's in the same transaction,
>  it'll get installed before this package. Otherwise it doesn't matter,
>  yes. 
Um, what?

OrderWithRequires behaves exactly like Requires for *ordering*. There's 
no additional mystery magic there.

OtherWithRequires can be used to affect ordering in the case no hard 
dependency exists. With users, there's always a hard dependency, and 
OrderWithRequires achieves nothing at all but obfusctation.

	- Panu -

>

[Neal Gompa]

On Fri, Jan 3, 2020 at 5:42 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  On 1/3/20 12:35 PM, Neal Gompa wrote:
>  > On Fri, Jan 3, 2020 at 4:01 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  >>
>  >> On 1/2/20 5:29 PM, Neal Gompa wrote:
>  >>> On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
>  >>
>  >>> For us, since we don't have the SUSE patches that make PreReq do
>  >>> things, the way we'd declare this with upstream RPM features would be:
>  >>
>  >> Please don't spread misinformation, openSUSE doesn't have patches to
>  >> make PreReq anyhow special. They merely undo the deprecation warning
>  >> that is issued on PreReq. With or without that, a PreReq will be
>  >> translated to Requires(pre,preun) to simulate what the original PreReq did.
>  >>
>  >
>  > Huh, for some reason I thought it also did something else...
>  >
>  >>>
>  >>> Requires: user(wwwrun)
>  >>> OrderWithRequires: user(wwwrun)
>  >>
>  >> I'm not sure how one is supposed to read the above, but certainly one
>  >> does NOT need both Requires and OrderWithRequires on the same thing in a
>  >> package. Requires(pre) might be in order because for user/group you
>  >> really want them installed first in case of loops, but this is details
>  >> without
>  >
>  > OtherWithRequires makes it so that if it's in the same transaction,
>  > it'll get installed before this package. Otherwise it doesn't matter,
>  > yes.
>
>  Um, what?
>
>  OrderWithRequires behaves exactly like Requires for *ordering*. There's
>  no additional mystery magic there.
>
>  OtherWithRequires can be used to affect ordering in the case no hard
>  dependency exists. With users, there's always a hard dependency, and
>  OrderWithRequires achieves nothing at all but obfusctation.
>  
Well, then, today I learned I guess. Does Requires(pre) work even
without a %pre script?


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Panu Matilainen]

On 1/3/20 12:43 PM, Neal Gompa wrote:
>  On Fri, Jan 3, 2020 at 5:42 AM Panu Matilainen
> <pmatilai(a)redhat.com&gt; wrote:
> >
> > On 1/3/20 12:35 PM, Neal Gompa wrote:
> >> On Fri, Jan 3, 2020 at 4:01 AM Panu Matilainen <pmatilai(a)redhat.com&gt;
> wrote:
> >>>
> >>> On 1/2/20 5:29 PM, Neal Gompa wrote:
> >>>> On Thu, Jan 2, 2020 at 10:14 AM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
> >>>
> >>>> For us, since we don't have the SUSE patches that make PreReq do
> >>>> things, the way we'd declare this with upstream RPM features would
> be:
> >>>
> >>> Please don't spread misinformation, openSUSE doesn't have patches to
> >>> make PreReq anyhow special. They merely undo the deprecation warning
> >>> that is issued on PreReq. With or without that, a PreReq will be
> >>> translated to Requires(pre,preun) to simulate what the original PreReq did.
> >>>
> >>
> >> Huh, for some reason I thought it also did something else...
> >>
> >>>>
> >>>> Requires: user(wwwrun)
> >>>> OrderWithRequires: user(wwwrun)
> >>>
> >>> I'm not sure how one is supposed to read the above, but certainly one
> >>> does NOT need both Requires and OrderWithRequires on the same thing in a
> >>> package. Requires(pre) might be in order because for user/group you
> >>> really want them installed first in case of loops, but this is details
> >>> without
> >>
> >> OtherWithRequires makes it so that if it's in the same transaction,
> >> it'll get installed before this package. Otherwise it doesn't matter,
> >> yes.
> >
> > Um, what?
> >
> > OrderWithRequires behaves exactly like Requires for *ordering*. There's
> > no additional mystery magic there.
> >
> > OtherWithRequires can be used to affect ordering in the case no hard
> > dependency exists. With users, there's always a hard dependency, and
> > OrderWithRequires achieves nothing at all but obfusctation.
> >
>  
>  Well, then, today I learned I guess. Does Requires(pre) work even
>  without a %pre script? 
Yes, dependency processing is not connected to scriptlets in any way.

	- Panu -