Hey all,
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
We (as in rpm-team) are not pushing for doing all this in Fedora 39, this is more to start discussion and lay down the necessary steps. In the 4.19 builds so far, the sysusers integration has been entirely disabled because it needs more coordination than just drop it in. Much of it is between systemd and rpm, but any package with non-root ownerships will be affected in the end. At least the following, and not necessarily exactly in this order:
1. systemd has it's own user/group provides generator which directly conflicts (both on generated content and file level) with the new native generator in rpm, and the feature will not work with the provides generated by systemd.
2. systemd provides users and groups that are actually owned by the setup package. As rpm is now turning non-root file ownership into dependencies, systemd could end up pulled in where setup is needed (eg early install stage) which will not end up well. So systemd will need to stop providing users it does not actually own.
3. The various %sysuser_()* macros in systemd-rpm-macros need to be phased out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
4. The sysusers "hook" in rpm needs to be enabled (uncomment %__systemd_sysusers macro in rpm). It wont do anything at all before 1) and 3) are done though.
6. The user/group dependencies for non-root users need to be turned into hard requires (initially these are just recommends). I would be suprised if this doesn't cause some disruption somewhere, although the content that is not root:root owned is pretty scarce these days.
7. Packages creating or using non-root user/group need to be rebuilt.
7. One day a few years from now, replace https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ with "supply a sysusers file for your needs" :P In reality, it'll need adjusting long before that and for that, it'll need FPC recommendations and all.
8. Remove all user/group addition related macro and script fubar from specs for good. The first commit in rpm source tree is from 1995, it'd be a nice 30 year celebration... but I don't expect it to happen quite that soon. Maybe in 2035 new people will start look at old specs in horror, "What do you mean they had to deal with all this user/group stuff manually! For 30 years!"
I've begun from 1) now:
https://src.fedoraproject.org/rpms/systemd/pull-request/109 https://src.fedoraproject.org/rpms/rpm/pull-request/45
After those have been done, people can start experimenting with the feature. I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
Comments, thoughts etc?
- Panu -
On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:
Hey all,
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
Cool, let's do this!
We (as in rpm-team) are not pushing for doing all this in Fedora 39, this is more to start discussion and lay down the necessary steps. In the 4.19 builds so far, the sysusers integration has been entirely disabled because it needs more coordination than just drop it in. Much of it is between systemd and rpm, but any package with non-root ownerships will be affected in the end. At least the following, and not necessarily exactly in this order:
- systemd has it's own user/group provides generator which directly
conflicts (both on generated content and file level) with the new native generator in rpm, and the feature will not work with the provides generated by systemd.
I merged your PR to disable this, so this first step is done.
- systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies, systemd could end up pulled in where setup is needed (eg early install stage) which will not end up well. So systemd will need to stop providing users it does not actually own.
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb. setup is currently pulled in by dependencies, but e.g. in the initrd we should be fine without it. (And the same applies for e.g. minimal container images without login users or a shell.)
Maybe the non-essential stuff could be split out into a new subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow} with the base set of users and groups, and all other files moved to setup-clutter.
- The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
+1 to this plan.
- The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) and 3) are done though.
- The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised if this doesn't cause some disruption somewhere, although the content that is not root:root owned is pretty scarce these days.
Packages creating or using non-root user/group need to be rebuilt.
One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ with "supply a sysusers file for your needs" :P In reality, it'll need adjusting long before that and for that, it'll need FPC recommendations and all.
- Remove all user/group addition related macro and script fubar from specs
for good. The first commit in rpm source tree is from 1995, it'd be a nice 30 year celebration... but I don't expect it to happen quite that soon. Maybe in 2035 new people will start look at old specs in horror, "What do you mean they had to deal with all this user/group stuff manually! For 30 years!"
I've begun from 1) now:
This is merged now and the package is built. (I guess it's probably in gating now.)
https://src.fedoraproject.org/rpms/rpm/pull-request/45
After those have been done, people can start experimenting with the feature. I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
Zbyszek
Once upon a time, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl said:
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb.
/etc/services and /etc/protocols are AFAIK mandatory if you want to have network services (either client or server). I don't think there's anything else providing that information.
On Thu, Jun 22, 2023 at 10:25:10AM -0500, Chris Adams wrote:
Once upon a time, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl said:
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb.
/etc/services and /etc/protocols are AFAIK mandatory if you want to have network services (either client or server). I don't think there's anything else providing that information.
getservbyname would use /etc/services, but I'm not sure how widely it is used. A lot of code just hardcodes a specific number… Local configuration for port numbers is a concept that only works if somebody synchronizes the file across machines, which is unlikely to happen.
Similarly for /etc/protocols… Code would generally use a define, not resolve the protocol number from a string.
I removed both files on a machine and rebooted and there are no messages or any other indiciation that this makes a difference.
And in particular in the initrd, they are just not neeed.
Zbyszek
Once upon a time, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl said:
getservbyname would use /etc/services, but I'm not sure how widely it is used. A lot of code just hardcodes a specific number… Local configuration for port numbers is a concept that only works if somebody synchronizes the file across machines, which is unlikely to happen.
Similarly for /etc/protocols… Code would generally use a define, not resolve the protocol number from a string.
I removed both files on a machine and rebooted and there are no messages or any other indiciation that this makes a difference.
And in particular in the initrd, they are just not neeed.
Just checking one thing, NFS uses getservbyname, so they are needed, and needed in the initrd for NFS root.
* Zbigniew Jędrzejewski-Szmek:
On Thu, Jun 22, 2023 at 10:25:10AM -0500, Chris Adams wrote:
Once upon a time, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl said:
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb.
/etc/services and /etc/protocols are AFAIK mandatory if you want to have network services (either client or server). I don't think there's anything else providing that information.
getservbyname would use /etc/services, but I'm not sure how widely it is used.
The getaddrinfo function uses it as well, to fill in the port number.
Thanks, Florian
On Do, 22.06.23 10:25, Chris Adams (linux@cmadams.net) wrote:
Once upon a time, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl said:
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb.
/etc/services and /etc/protocols are AFAIK mandatory if you want to have network services (either client or server). I don't think there's anything else providing that information.
getservbyname() always and getaddrinfo() under certain circumstances (i.e. if 2nd argument actually uses a textual service name) use these files.
But I think in the year 2023 programs actively using this should probably be fixed not do so, and just use literal port numbers.
The service database in /etc/ is a really weird concept since it suggests this mapping was configurable. But it really isn't, because it's of course generally the *server* of an IP service that picks the port number, not the *client*. Hence if a client configures the mapping between service name and port at will this is generally useless since it's unlikely going to match what servers bound its sockets to.
hence, I think it would be wise to fix programs across the distribution to just use port numbers literally, and avoid the service database. it's probably what most admin folks assume anyway.
And for programs that cannot easily bit fixed to work like this, it would make sense to provide a split out package ("socket-databases.rpm"?), and then require those consuming packages to depend on that.
"tcpdump" can also uses these databases as well, but at least I personally always found that very much confusing and hindering, since the data in the database is sometimes a bit too historical to be useful. For example it thinks port 5355 is "hostmon", even though it almost certainly is llmnr if see in the wild... And what's even worse: local client sockets on Linux get auto-assigned port numbers from the range 32768-60999 by default (see /proc/sys/net/ipv4/ip_local_port_range), but /etc/services assigns a ton of names to various ports from that range. For all these cases the service info is just complete garbage.
Side note: because the service db is such a broken concept the socket activation feature in systemd explicitly never supported it btw. And while people complain about various of our "political" choices like that, noone so far has noticed or complained this omission... So I have serious doubts anyone cares about the concept at all...
(If such a split off "socket-databases.rpm" would be created I'd like to suggest moving the database to /usr/ somehwere at the same time, and then create a symlink from /etc/services to it, including via tmpfiles.d/, to at least push people away from assuming that this was actually configurable config file, and not just a dump of some outdated IANA data).
Lennart
-- Lennart Poettering, Berlin
Hello,
On Thursday, June 22, 2023 11:01:28 AM EDT Zbigniew Jędrzejewski-Szmek wrote:
- systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies, systemd could end up pulled in where setup is needed (eg early install stage) which will not end up well. So systemd will need to stop providing users it does not actually own.
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb. setup is currently pulled in by dependencies, but e.g. in the initrd we should be fine without it. (And the same applies for e.g. minimal container images without login users or a shell.)
There are several trusted databases there that things like getservbyport consult. I would think csh stuff could be installed by tcsh.
Maybe the non-essential stuff could be split out into a new subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow} with the base set of users and groups, and all other files moved to setup-clutter.
I think a few more files than that are still needed. But it could use some pruning.
- The various %sysuser_()* macros in systemd-rpm-macros need to be
phased out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
+1 to this plan.
- The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) and 3) are done though.
- The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised if this doesn't cause some disruption somewhere, although the content that is not root:root owned is pretty scarce these days.
Packages creating or using non-root user/group need to be rebuilt.
One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ with "supply a sysusers file for your needs" In reality, it'll need adjusting long before that and for that, it'll need FPC recommendations and all.
- Remove all user/group addition related macro and script fubar from
specs for good. The first commit in rpm source tree is from 1995, it'd be a nice 30 year celebration... but I don't expect it to happen quite that soon. Maybe in 2035 new people will start look at old specs in horror, "What do you mean they had to deal with all this user/group stuff manually! For 30 years!"
I've begun from 1) now:
This is merged now and the package is built. (I guess it's probably in gating now.)
https://src.fedoraproject.org/rpms/rpm/pull-request/45
After those have been done, people can start experimenting with the feature. I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Li...
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
The system accounts still need to be accessible by the getpw* family of functions or there could be a lot of breakage.
-Steve
On 6/22/23 19:55, Steve Grubb wrote:
Hello,
On Thursday, June 22, 2023 11:01:28 AM EDT Zbigniew Jędrzejewski-Szmek wrote:
- systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies, systemd could end up pulled in where setup is needed (eg early install stage) which will not end up well. So systemd will need to stop providing users it does not actually own.
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb. setup is currently pulled in by dependencies, but e.g. in the initrd we should be fine without it. (And the same applies for e.g. minimal container images without login users or a shell.)
There are several trusted databases there that things like getservbyport consult. I would think csh stuff could be installed by tcsh.
Maybe the non-essential stuff could be split out into a new subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow} with the base set of users and groups, and all other files moved to setup-clutter.
I think a few more files than that are still needed. But it could use some pruning.
- The various %sysuser_()* macros in systemd-rpm-macros need to be
phased out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
+1 to this plan.
- The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) and 3) are done though.
- The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised if this doesn't cause some disruption somewhere, although the content that is not root:root owned is pretty scarce these days.
Packages creating or using non-root user/group need to be rebuilt.
One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ with "supply a sysusers file for your needs" In reality, it'll need adjusting long before that and for that, it'll need FPC recommendations and all.
- Remove all user/group addition related macro and script fubar from
specs for good. The first commit in rpm source tree is from 1995, it'd be a nice 30 year celebration... but I don't expect it to happen quite that soon. Maybe in 2035 new people will start look at old specs in horror, "What do you mean they had to deal with all this user/group stuff manually! For 30 years!"
I've begun from 1) now:
This is merged now and the package is built. (I guess it's probably in gating now.)
https://src.fedoraproject.org/rpms/rpm/pull-request/45
After those have been done, people can start experimenting with the feature. I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Li...
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
Rpm's sysuser integration simply calls out to the configured helper to do the heavy lifting. By default that's systemd-sysusers, I have no idea what audit events if any it generates, but it's entirely possible to replace that with a script that calls out to useradd and groupadd instead.
The system accounts still need to be accessible by the getpw* family of functions or there could be a lot of breakage.
Uh, yes? Users and groups do need to be created. We're merely talking about letting rpm automate the task based on already packaged sysusers.d files instead of requiring packagers to fill out tedious boilerplate to handle it.
On a somewhat related note, we're considering to make rpm only ever look at local passwd/group files for user/group info as that's the only info that is reliably available to rpm: https://github.com/rpm-software-management/rpm/pull/2503, @keszbyz summarizes it nicely here: https://github.com/rpm-software-management/rpm/issues/882#issuecomment-15048...
- Panu -
-Steve
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 6/22/23 19:55, Steve Grubb wrote:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Li...
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
Looking at the systemd-sysusers source [1], it seems to do exactly zero audit logging. So there's a bit of work to do on that front...
- Panu -
[1] https://github.com/systemd/systemd/blob/main/src/sysusers/sysusers.c
On Di, 27.06.23 12:04, Panu Matilainen (pmatilai@redhat.com) wrote:
On 6/22/23 19:55, Steve Grubb wrote:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Li...
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
Looking at the systemd-sysusers source [1], it seems to do exactly zero audit logging. So there's a bit of work to do on that front...
last time I looked auditd is started later than systemd-sysusers. Hence not sure if sysusers would actually generate audit messages that auditd could pick them up.
In general though: people who care about audit need to send us patches for this, if this matters to them. I don't think anyone in systemd upstream wil work on this on their own.
Lennart
-- Lennart Poettering, Berlin
On Wednesday, June 28, 2023 10:15:48 AM EDT Lennart Poettering wrote:
On Di, 27.06.23 12:04, Panu Matilainen (pmatilai@redhat.com) wrote:
On 6/22/23 19:55, Steve Grubb wrote:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Accou nt-Lifecycle-Events
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
Looking at the systemd-sysusers source [1], it seems to do exactly zero audit logging. So there's a bit of work to do on that front...
last time I looked auditd is started later than systemd-sysusers. Hence not sure if sysusers would actually generate audit messages that auditd could pick them up.
When booted with audit=1, the kernel will look to audit_backlog_limit and hold that many event records until auditd can download them. The typical number people set is 8192.
-Steve
In general though: people who care about audit need to send us patches for this, if this matters to them. I don't think anyone in systemd upstream wil work on this on their own.
On 6/28/23 17:15, Lennart Poettering wrote:
On Di, 27.06.23 12:04, Panu Matilainen (pmatilai@redhat.com) wrote:
On 6/22/23 19:55, Steve Grubb wrote:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
I would caution against this whole proposal. Not that I'm against it, but just saying be careful doing it. People often forget about our security concerns. Currently, shadow-utils has about 400 places which generate audit events during the managing of system and user accounts. libuser (I saw the deprecation email) has 55 places where it sends audit events managing accounts.
There is a 10 year old (or more) standard published here: https://github.com/linux-audit/audit-documentation/wiki/SPEC-User-Account-Li...
If %pre getent, useradd, and groupadd are being replaced by something, that something needs to conform to the expected security safeguards that currently exist. It needs to match the kind of events and the format that currently exists.
Looking at the systemd-sysusers source [1], it seems to do exactly zero audit logging. So there's a bit of work to do on that front...
last time I looked auditd is started later than systemd-sysusers. Hence not sure if sysusers would actually generate audit messages that auditd could pick them up.
For the rpm integration, "started later" is irrelevant as the user/group creation takes place during rpm transactions.
In general though: people who care about audit need to send us patches for this, if this matters to them. I don't think anyone in systemd upstream wil work on this on their own.
Didn't imply any particular party, just that there's work to do.
Both rpm and systemd would like to see systemd-sysusers used for user/group creation but audit requirement prevents that. Who should do the work? Guess there's a bit of a Mexican stand-off here :D
The rpm integration doesn't technically require systemd-sysusers, we can write a script that calls useradd/groupadd instead. So for us it becomes a choice between writing that script or adding audit support to systemd-sysusers. Writing a script based on sysusers.generate-pre.sh may well be less work and would benefit the non-systemd audiences of rpm at the same time. We'll see.
- Panu -
On Thu, Jun 29, 2023, at 3:55 AM, Panu Matilainen wrote:
last time I looked auditd is started later than systemd-sysusers. Hence not sure if sysusers would actually generate audit messages that auditd could pick them up.
For the rpm integration, "started later" is irrelevant as the user/group creation takes place during rpm transactions.
Right, and that leads to an important point: for Anaconda based installations that run dnf/rpm in a special installation environment, unless one has gone to significant effort to inject some auditing setup into that, you aren't going to get events.
And the 90% case in cloud environments is to boot from pre-built images which will already have these users enabled (sure you could audit user events from your build environment but...why?).
So at best, the system user events only work for software injected *later*. And even then, how much sense does it really make to audit the useradd component of the install instead of the actual installation? And there's already rpm-plugin-audit for those who want that...
Anyways though, it's probably not a hard patch to add the auditing to sysusers.
On 6/29/23 09:55, Panu Matilainen wrote:
The rpm integration doesn't technically require systemd-sysusers, we can write a script that calls useradd/groupadd instead. So for us it becomes a choice between writing that script or adding audit support to systemd-sysusers. Writing a script based on sysusers.generate-pre.sh may well be less work and would benefit the non-systemd audiences of rpm at the same time. We'll see.
OK, I cobbled something together:
https://github.com/rpm-software-management/rpm/pull/2573
It still waits for the new test suite back end to pass the testsuite but that shouldn't prevent anyone from already having a look.
Does anyone have a suggestion for another way for checking for the existence of users groups and UIDs? While useradd and group add support the --root parameter things like id don't.
Florian
Dne 22. 06. 23 v 17:01 Zbigniew Jędrzejewski-Szmek napsal(a):
I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
Please create a Change. It will helps to track the progress and works great as single point to gather the knowledge how to do the conversion.
On 6/22/23 18:01, Zbigniew Jędrzejewski-Szmek wrote:
On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:
Hey all,
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
Cool, let's do this!
That's the spirit :D
We (as in rpm-team) are not pushing for doing all this in Fedora 39, this is more to start discussion and lay down the necessary steps. In the 4.19 builds so far, the sysusers integration has been entirely disabled because it needs more coordination than just drop it in. Much of it is between systemd and rpm, but any package with non-root ownerships will be affected in the end. At least the following, and not necessarily exactly in this order:
- systemd has it's own user/group provides generator which directly
conflicts (both on generated content and file level) with the new native generator in rpm, and the feature will not work with the provides generated by systemd.
I merged your PR to disable this, so this first step is done.
Thanks! I'll enable the generator in rpm then.
- systemd provides users and groups that are actually owned by the setup
package. As rpm is now turning non-root file ownership into dependencies, systemd could end up pulled in where setup is needed (eg early install stage) which will not end up well. So systemd will need to stop providing users it does not actually own.
I was hoping we would be make the dependency on setup optional. It is a fairly heavyweight package (700+ kb) and with lots of not-that-useful-on-a-typical-modern-installation stuff (mail alias support, csh profile, /etc/hosts, nfs exports, etc.). Most of this is tiny, but it clutters /etc, which ideally would be empty, and also /etc/services is 700 kb. setup is currently pulled in by dependencies, but e.g. in the initrd we should be fine without it. (And the same applies for e.g. minimal container images without login users or a shell.)
Maybe the non-essential stuff could be split out into a new subpackage, with setup only providing /etc/{passwd,group,shadow,gshadow} with the base set of users and groups, and all other files moved to setup-clutter.
Oh, "setup" is not special to rpm in any way, I only talk of it because that's where /etc/{passwd,group} traditionally reside. Rpm grabs the user/group provides from those files on encounter, so no rpm-side changes are needed if they get moved to some other package.
- The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
+1 to this plan.
- The sysusers "hook" in rpm needs to be enabled (uncomment
%__systemd_sysusers macro in rpm). It wont do anything at all before 1) and 3) are done though.
- The user/group dependencies for non-root users need to be turned into
hard requires (initially these are just recommends). I would be suprised if this doesn't cause some disruption somewhere, although the content that is not root:root owned is pretty scarce these days.
Packages creating or using non-root user/group need to be rebuilt.
One day a few years from now, replace
https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/ with "supply a sysusers file for your needs" :P In reality, it'll need adjusting long before that and for that, it'll need FPC recommendations and all.
- Remove all user/group addition related macro and script fubar from specs
for good. The first commit in rpm source tree is from 1995, it'd be a nice 30 year celebration... but I don't expect it to happen quite that soon. Maybe in 2035 new people will start look at old specs in horror, "What do you mean they had to deal with all this user/group stuff manually! For 30 years!"
I've begun from 1) now:
This is merged now and the package is built. (I guess it's probably in gating now.)
https://src.fedoraproject.org/rpms/rpm/pull-request/45
After those have been done, people can start experimenting with the feature. I don't remember seeing an actual Fedora Change for either file-trigger enablement or current %sysuser_* macros so I'm not sure it's needed here either?
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
Thanks, I've missed that.
Guess it's best to make a Change out of this too then.
- Panu -
On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
[...]
- The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
This proposal would effectively move all existing packages that create users or groups from useradd/groupadd (called by those %sysuser* macros underneath) to systemd-sysusers(8).
I wonder if we shouldn't first just move those macros over to systemd-sysusers to test-drive this utility at a larger scale and catch any potential bugs or issues before actually proceeding with the remaining steps as outlined in the email.
That's a lower-risk first step that should be fairly easy to implement right away, as mentioned in:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
On Fri, Jun 23, 2023 at 08:20:58PM +0200, Michal Domonkos wrote:
On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
[...]
- The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
This proposal would effectively move all existing packages that create users or groups from useradd/groupadd (called by those %sysuser* macros underneath) to systemd-sysusers(8).
I wonder if we shouldn't first just move those macros over to systemd-sysusers to test-drive this utility at a larger scale and catch any potential bugs or issues before actually proceeding with the remaining steps as outlined in the email.
I don't think so. Either way, the actual implementation is going to be a call to systemd-sysusers. But the rpm-internal approach is quite different in how the call is constructed from the macro-based approach, so the failure modes are likely to be different. If were to switch to the macro-based approach temporarily, we'd create quite a lot of churn and _different_ failure modes. So I think that if we're switching to sysusers as the implementation, we should go for the intended final approach immediately.
Zbyszek
On Sat, Jun 24, 2023 at 01:22:22AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
I don't think so. Either way, the actual implementation is going to be a call to systemd-sysusers. But the rpm-internal approach is quite different in how the call is constructed from the macro-based approach, so the failure modes are likely to be different. If were to switch to the macro-based approach temporarily, we'd create quite a lot of churn and _different_ failure modes. So I think that if we're switching to sysusers as the implementation, we should go for the intended final approach immediately.
My thinking was that the failure modes would then be limited to the switch to systemd-sysusers only so we wouldn't have to debug two new implementations (systemd-sysusers and RPM's integration) but just one (systemd-sysusers).
That said, having slept on it, I agree that such a two-staged approach would just make things needlessly more chaotic. Just switching the whole thing as proposed is going to be simpler, yup :)
On 6/23/23 21:20, Michal Domonkos wrote:
On Thu, Jun 22, 2023 at 01:18:27PM +0300, Panu Matilainen wrote:
Now that the initial hurdle of getting rpm 4.19 into rawhide is over, it's time to start looking towards enabling the sysusers integration: https://rpm-software-management.github.io/rpm/manual/users_and_groups.html
[...]
- The various %sysuser_()* macros in systemd-rpm-macros need to be phased
out. As it'll be a long time before the sysusers feature is in all Fedora versions, it needs a longer term plan. One simple possibility is do what was done with all those ldconfig from %post back then: change the %sysusers_() macros to no-ops in rawhide to let rpm handle it, and only actually bother updating packages once all relevant versions have the sysusers feature.
This proposal would effectively move all existing packages that create users or groups from useradd/groupadd (called by those %sysuser* macros underneath) to systemd-sysusers(8).
I wonder if we shouldn't first just move those macros over to systemd-sysusers to test-drive this utility at a larger scale and catch any potential bugs or issues before actually proceeding with the remaining steps as outlined in the email.
That's a lower-risk first step that should be fairly easy to implement right away, as mentioned in:
https://fedoraproject.org/wiki/Changes/Adopting_sysusers.d_format
The big problem with that approach is that it drags all of systemd into every user/group creation. Doesn't matter on an already running system but is disastrous in the early install stages where even shadow-utils easily creates ordering loops.
The rpm integration avoids that entirely by calling the user/group creation helper from outside of the chroot (ie systemd-sysusers --root=...) Which means one could pretty much drop all of the static /etc/{passwd,group} content (now shipped in setup) because the user/group creation facilities are available even to the first package being installed.
- Panu -
-
Chris Adams -
Colin Walters -
Florian Festi -
Florian Weimer -
Lennart Poettering -
Michal Domonkos -
Miroslav Suchý -
Panu Matilainen -
Steve Grubb -
Zbigniew Jędrzejewski-Szmek