List of our core services

[Peter Boy]

As of our IRC meeting 2023-02-15 we want to create a list of our core services in order to
check their proper shutdown value status.

I determined the following list:

• KVM/Libvirt VM's host
• KVM/libvirt VM instances
• Podman container host
• Podman container instances
• Systemd-nspawn container host
• Systemd-nspawn container instances
• Libvirt LXC container host
• Libvirt LXC container instances
• Domain Controller
• IPA service
• PostgreSQL database server
• Samba file server
• NFS file server 
• Apache Web server
• Web application service: Tomcat
• Web application service: Wildfly
• Postfix/Dovecot mail service

See issue #96
https://pagure.io/fedora-server/issue/96#comment-841351

Please comment here or comment / edit the issue.

[Emmanuel Seyman]

* Peter Boy [17/02/2023 12:29] :
>  Please comment here or comment / edit the issue. 
I will happily confirm everything on your list and add:

* DHCP/PXE server
* DNS resolving service

Emmanuel

[Dominik 'Rathann' Mierzejewski]

On Friday, 17 February 2023 at 23:07, Emmanuel Seyman wrote:
>  * Peter Boy [17/02/2023 12:29] :
>  >
>  > Please comment here or comment / edit the issue.
>  
>  I will happily confirm everything on your list and add:
>  
>  * DHCP/PXE server 
I assume that includes DHCPv6 and SLAAC+RDNSS. And TFTP.
Although for UEFI network boot you actually need a HTTP(S) server.

I'd also add:

* NTP/NTS time server

A scanner server (saned) could be added as well as it's fairly simple to
configure.

Regards,
Dominik
-- 
Fedora   https://getfedora.org  |  RPM Fusion  http://rpmfusion.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
        -- from "Collected Sayings of Muad'Dib" by the Princess Irulan

[Kevin Fenzi]

On Fri, Feb 17, 2023 at 12:29:45PM +0100, Peter Boy wrote:
>  As of our IRC meeting 2023-02-15 we want to create a list of our core
> services in order to check their proper shutdown value status.
>  
>  I determined the following list:
>   
libvirtd sets timeout to 'infinity' 
>  • KVM/Libvirt VM's host
>  • KVM/libvirt VM instances  
I'm not sure on containers... isn't that all in userspace? at least
podman I thought was specifically 'no big daemon'?

>  • Podman container host
>  • Podman container instances
>  • Systemd-nspawn container host
>  • Systemd-nspawn container instances
>  • Libvirt LXC container host
>  • Libvirt LXC container instances 
bind? Or nmbd?
>  • Domain Controller 
This one sets to 0 (old config for infinity)
>  • IPA service 
Infinity:
>  • PostgreSQL database server 
No special settings I can see off hand:
>  • Samba file server
>  • NFS file server 
>  • Apache Web server 
Not sure on these:
>  • Web application service: Tomcat
>  • Web application service: Wildfly 
No special handling: 
>  • Postfix/Dovecot mail service 
kevin

[Matthew Miller]

On Fri, Feb 17, 2023 at 12:29:45PM +0100, Peter Boy wrote:
>  • Systemd-nspawn container instances
>  • Libvirt LXC container host
>  • Libvirt LXC container instances
>  • Domain Controller
>  • IPA service
>  • PostgreSQL database server
>  • Samba file server
>  • NFS file server 
>  • Apache Web server
>  • Web application service: Tomcat
>  • Web application service: Wildfly
>  • Postfix/Dovecot mail service 
What about nginx and mariadb?


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Kevin Fenzi]

On Mon, Feb 20, 2023 at 01:59:51PM -0500, Matthew Miller wrote:
>  On Fri, Feb 17, 2023 at 12:29:45PM +0100, Peter Boy wrote:
>  > • Systemd-nspawn container instances
>  > • Libvirt LXC container host
>  > • Libvirt LXC container instances
>  > • Domain Controller
>  > • IPA service
>  > • PostgreSQL database server
>  > • Samba file server
>  > • NFS file server 
>  > • Apache Web server
>  > • Web application service: Tomcat
>  > • Web application service: Wildfly
>  > • Postfix/Dovecot mail service
>  
>  What about nginx and mariadb? 
From what I recall about the initial PRD, we tried to pick just one
application from overlapping ones that also did the same thing. 
I think it was an attempt to avoid:

"hey, I want a database, whats the Fedora Server db?"
"well, it depends, you could use mariadb or postgresql or sqlite or
tiedot or rocksdb or..."

I'm not sure if we want to keep doing that, or try and say "hey, all
these things are important and we should try and support them".

Additionally, if all these things are 'core services'... we should
likely test them all and add release critera and such for them so we
don't ship with any of them broken?

kevin

[Daniel Schier]

Don’t want to argue here, but are systemd-nspawn and LXC really more „core“ than Podman?
Podman is shipped pre-installed in all Fedora variants, afaik.

What about Cockpit, which is also shipped in the default setup?

>  Am 20.02.2023 um 21:17 schrieb Kevin Fenzi <kevin(a)scrye.com&gt;:
>  
>  On Mon, Feb 20, 2023 at 01:59:51PM -0500, Matthew Miller wrote:
> >> On Fri, Feb 17, 2023 at 12:29:45PM +0100, Peter Boy wrote:
> >> • Systemd-nspawn container instances
> >> • Libvirt LXC container host
> >> • Libvirt LXC container instances
> >> • Domain Controller
> >> • IPA service
> >> • PostgreSQL database server
> >> • Samba file server
> >> • NFS file server 
> >> • Apache Web server
> >> • Web application service: Tomcat
> >> • Web application service: Wildfly
> >> • Postfix/Dovecot mail service
> > 
> > What about nginx and mariadb?
>  
>  From what I recall about the initial PRD, we tried to pick just one
>  application from overlapping ones that also did the same thing. 
>  I think it was an attempt to avoid:
>  
>  "hey, I want a database, whats the Fedora Server db?"
>  "well, it depends, you could use mariadb or postgresql or sqlite or
>  tiedot or rocksdb or..."
>  
>  I'm not sure if we want to keep doing that, or try and say "hey, all
>  these things are important and we should try and support them".
>  
>  Additionally, if all these things are 'core services'... we should
>  likely test them all and add release critera and such for them so we
>  don't ship with any of them broken?
>  
>  kevin
>  _______________________________________________
>  server mailing list -- server(a)lists.fedoraproject.org
>  To unsubscribe send an email to server-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Adam Williamson]

On Mon, 2023-02-20 at 22:41 +0100, Daniel Schier wrote:
>  Don’t want to argue here, but are systemd-nspawn and LXC really more
> „core“ than Podman? Podman is shipped pre-installed in all Fedora variants, afaik. 
Podman is in Peter's list. Matthew edited it out in his reply for some
reason (space?)
>  
>  What about Cockpit, which is also shipped in the default setup? 
I was going to say that, but then I thought, this is about shutdown
policy; is it really a big problem if Cockpit doesn't shut down
cleanly? AIUI the question is "which services do we need to make sure
we wait for to close down cleanly, rather than just killing them if
they haven't shut down in some rather short period of time".
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net

[Daniel Schier]

Thanks for the clarification about Podman. i appreciate it.

Honestly, I am not sure about Cockpit here.

It has an addon for packagekit and rpmostree, which might be a problem to be killed during
an update. But, I haven’t inspected how this works in the background.

>  Am 21.02.2023 um 18:03 schrieb Adam Williamson
> <adamwill(a)fedoraproject.org&gt;:
>  
>  On Mon, 2023-02-20 at 22:41 +0100, Daniel Schier wrote:
> > Don’t want to argue here, but are systemd-nspawn and LXC really more „core“ than
> Podman? Podman is shipped pre-installed in all Fedora variants, afaik.
>  
>  Podman is in Peter's list. Matthew edited it out in his reply for some
>  reason (space?)
> > 
> > What about Cockpit, which is also shipped in the default setup?
>  
>  I was going to say that, but then I thought, this is about shutdown
>  policy; is it really a big problem if Cockpit doesn't shut down
>  cleanly? AIUI the question is "which services do we need to make sure
>  we wait for to close down cleanly, rather than just killing them if
>  they haven't shut down in some rather short period of time".
>  -- 
>  Adam Williamson (he/him/his)
>  Fedora QA
>  Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
>  https://www.happyassassin.net
>  
>  
>  
>  _______________________________________________
>  server mailing list -- server(a)lists.fedoraproject.org
>  To unsubscribe send an email to server-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Martin Pitt]

Hello Adam, all,

Adam Williamson [2023-02-21  9:02 -0800]:
>  > What about Cockpit, which is also shipped in the default setup?
>
>  I was going to say that, but then I thought, this is about shutdown
>  policy; is it really a big problem if Cockpit doesn't shut down
>  cleanly? 
No, it isn't. Brittle as network connections are, we design cockpit with a "no
long-running operations tied to the cockpit session" rule in mind. Any such
operations, like creating a file system, applying updates, or starting a VM or
container are done only as control messages to their respective system services
(udisks2, packagekit, libvirt, and podman). You can disrupt the network,
disconnect the cockpit session, reconnect later, and the UI picks up the
current state.

IOW, there should be no reason why shutting down cockpit.service takes longer
than a few milliseconds.

>  AIUI the question is "which services do we need to make sure
>  we wait for to close down cleanly, rather than just killing them if
>  they haven't shut down in some rather short period of time". 
Full ack. There should be very few of these.

Thanks,

Pitti

[Matthew Miller]

On Tue, Feb 21, 2023 at 09:02:55AM -0800, Adam Williamson wrote:
>  On Mon, 2023-02-20 at 22:41 +0100, Daniel Schier wrote:
>  > Don’t want to argue here, but are systemd-nspawn and LXC really more „core“ than
> Podman? Podman is shipped pre-installed in all Fedora variants, afaik.
>  
>  Podman is in Peter's list. Matthew edited it out in his reply for some
>  reason (space?) 
"Wasn't being careful", most likely. Wasn't intentional.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Adam Williamson]

On Mon, 2023-02-20 at 12:17 -0800, Kevin Fenzi wrote:
>  On Mon, Feb 20, 2023 at 01:59:51PM -0500, Matthew Miller wrote:
>  > On Fri, Feb 17, 2023 at 12:29:45PM +0100, Peter Boy wrote:
>  > > • Systemd-nspawn container instances
>  > > • Libvirt LXC container host
>  > > • Libvirt LXC container instances
>  > > • Domain Controller
>  > > • IPA service
>  > > • PostgreSQL database server
>  > > • Samba file server
>  > > • NFS file server 
>  > > • Apache Web server
>  > > • Web application service: Tomcat
>  > > • Web application service: Wildfly
>  > > • Postfix/Dovecot mail service
>  > 
>  > What about nginx and mariadb?
>  
>  From what I recall about the initial PRD, we tried to pick just one
>  application from overlapping ones that also did the same thing. 
>  I think it was an attempt to avoid:
>  
>  "hey, I want a database, whats the Fedora Server db?"
>  "well, it depends, you could use mariadb or postgresql or sqlite or
>  tiedot or rocksdb or..."
>  
>  I'm not sure if we want to keep doing that, or try and say "hey, all
>  these things are important and we should try and support them". 
As the person who gets to write and maintain the tests and deal with
all the failures, I very much support "pick one damn thing".

Of course, this depends on what the issue is. If we're just doing a
one-time check that a specific config value is set appropriately for
all the things someone might run on Fedora Server, it doesn't hurt to
cast a wide net if someone is willing to do the work.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net

[Peter Boy]

I have now transferred all findings to the list. See
https://pagure.io/fedora-server/issue/96#comment-841351. Many thanks to Kevin, who
contributed all the specific configurations of our "big" core services. 


As I see it, the result is mixed. 

1. As far as we know, either a service has set the timeout value to infinity, or has set
no timeout at all. All the latter rely on the system default value.

2. At least most of our "huge core services" as virtualization, PostgreSQL, IPA,
etc. which are "suspected" of taking longer for a clean shutdown, have the
timeout set to infinity.

3. But some of our "huge core services" did not specify a timeout value, e.g.
all the container services (for information: CoreOS, the "pod man container OS",
decided to keep the old timeout value).

4. There are a number of "alternative" services as e.g. MariaDB database, which
may need longer as well. We have no information yet about their timeout value. These are
not "core" (in the sense we tested everything, and it is a potential release
blocker), but nevertheless everyone would expect that these work flawlessly and safely on
Fedora Server, nevertheless. So we should take these into consideration as well.

5. There are some services that must rely on a system default, as WildFly or various
collaboration software, because there is (currently) no rpm


Maybe I'm being overly cautious. But I prefer "better safe than sorry" and
like the decision of CoreOS - at least as long as we don't have better data and
measurements.