1:37 p.m.
systemd and autofs/nfs are at war and have been ever since systemd
appeared.
Specifically, if machine A has an open connection to machine B
and B goes down or become inaccessible, then A cannot shutdown.
A's shutdown sequence hangs, waiting for B to respond to an unmount
command, which will not/cannot happen.
WAITING WILL NOT HELP.
Only a dirty disconnect will work; there's no way to avoid it.
Here's what I've tried:
- Precede 'shutdown' or 'reboot' or 'systemctl reboot' with
umount -fl -t nfs <each nfs-mounted-filesystem>
(You might expect a forced lazy umount to do the job.
It doesn't work; umount just hangs)
- Change nfs mount option to 'soft' instead of 'hard'.
Edit /etc/nfsmount.conf to say:
Hard=False
Soft=True
(This did change the autofs/nfs mount option, but didn't improve
the tolerance for a dirty disconnect. Shutdown still hangs.)
- Reduce systemd's timeout values from 90 to 10 sec;
edit /etc/systemd/system.conf to say:
DefaultTimeoutStartSec=10s
DefaultTimeoutStopSec=10s
(During shutdown, systemd displays two times; the time it has waited
for an unmount event, and the time limit to wait. Incomprehensibly,
when the time limit is reached, it bumps it up and continues to wait.
Changing the default timeout values seems to accelerate this recycling,
but doesn't do anything for the actual time limit.)
- Change the nfs timeout from 600 (default) to 20 decisec to reduce
the time nfs waits for any response, including umount, before
declaring a timeout. Edit /etc/nfsmount.conf to say:
# Timeo=600
Timeo=20
(The option displayed by the mount command did change, but it did not
help the shutdown hangup.)
- That leaves ONLY TWO WAYS to shutdown machine A, both ugly:
SysReq R E I S U B - which forces a reboot NOW
Hold the Power button - which just stops everything.
This is NOT NEW. Systemd has had this design defect from day one.
SysV never hung on shutdown. It Just Knew when a dirty disconnect
was the only viable way and handled it.
How many more years must we wait?
Have I overlooked something obvious? Is there a way to make systemd
perform the simple function 'shutdown' smoothly, reliably and quickly?
If anyone knows how, I would love to hear it.
--
David A. De Graaf DATIX, Inc. Hendersonville, NC
dad(a)datix.us www.datix.us
"Tolerance becomes a crime when applied to evil."
-- Thomas Mann
1:53 p.m.
On Thu, 28 Jul 2016 14:37:41 -0400
David A. De Graaf wrote:
Have I overlooked something obvious? Is there a way to make systemd
perform the simple function 'shutdown' smoothly, reliably and quickly?
If anyone knows how, I would love to hear it.
I can't make systemd itself work, but I've been using an
outside systemd solution for a while now: I setup
an alias for the "reboot" command that arranges to
kill off all the things systemd unreasonably waits
for, then does a real reboot.
Since systemd now has nothing to stop it, it reboots
rather fast (until something new shows up which I have
to track down and add to my list :-).
My current set of things to do before shutdown includes:
umount -l -t nfs -a
apachectl -k stop
kill all the "user deamon" process trees.
The user daemon stuff is handled by a program
described here:
http://tomhorsley.com/game/punch.html
Of course, since (according to the systemd biggest
myths page) systemd is easy to script and not at all
confusing, then it should be trivial for me to script
things so shutdown works this way automatically
without me having to remember to run my alias
in a terminal, but, alas, they don't appear to be
myths at all.
2:40 p.m.
On 07/28/2016 11:53 AM, Tom Horsley wrote:
On Thu, 28 Jul 2016 14:37:41 -0400
David A. De Graaf wrote:
> Have I overlooked something obvious? Is there a way to make systemd
> perform the simple function 'shutdown' smoothly, reliably and quickly?
> If anyone knows how, I would love to hear it.
I can't make systemd itself work, but I've been using an
outside systemd solution for a while now: I setup
an alias for the "reboot" command that arranges to
kill off all the things systemd unreasonably waits
for, then does a real reboot.
Since systemd now has nothing to stop it, it reboots
rather fast (until something new shows up which I have
to track down and add to my list :-).
My current set of things to do before shutdown includes:
umount -l -t nfs -a
apachectl -k stop
kill all the "user deamon" process trees.
The user daemon stuff is handled by a program
described here:
http://tomhorsley.com/game/punch.html
Of course, since (according to the systemd biggest
myths page) systemd is easy to script and not at all
confusing, then it should be trivial for me to script
things so shutdown works this way automatically
without me having to remember to run my alias
in a terminal, but, alas, they don't appear to be
myths at all.
I've said it before, systemd is a huge, cumbersome, useless solution
looking for a problem. The decision to use this crud was one of the most
boneheaded things any distribution (not just RHEL/Fedora) ever chose
to adopt. Unfortunately, we're stuck with it now.
As far as the NFS unmount issue, this was an interesting dialog
that happened in September last year:
https://github.com/systemd/systemd/issues/1257
So, "but I doubt that lazy unmounting is really a comprehensive
solution for anything..." comes from one of the developers. Fine, then
when it times out, do a forced umount. This is pretty obvious. The
server isn't listening and we (the client) are shutting down. Pull the
g*ddamned plug!
I'm with David...but this is only one of a litany of stupid decisions
systemd makes. It seems that developers nowadays are never taught how
(or are simply too lazy) to test for error conditions.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com -
- AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 -
- -
- Political Correctness: The insane doctrine that postulates that it -
- is entirely possible to pick up a turd by the clean end. -
----------------------------------------------------------------------
5:43 p.m.
Hey guys...
Not to start a flame war...
But I've looked at different linux flavors.. ubuntu/mint/centos/etc.. and
I'm thinking of taking the step to centos 7/fed24(or whatever it is...)
But, the whole systemd/dnf stuff... is that really useful, as opposed to
the philosophy of some of the other flavors?
On Thu, Jul 28, 2016 at 3:40 PM, Rick Stevens <ricks(a)alldigital.com> wrote:
On 07/28/2016 11:53 AM, Tom Horsley wrote:
> On Thu, 28 Jul 2016 14:37:41 -0400
> David A. De Graaf wrote:
>
>> Have I overlooked something obvious? Is there a way to make systemd
>> perform the simple function 'shutdown' smoothly, reliably and quickly?
>> If anyone knows how, I would love to hear it.
>
> I can't make systemd itself work, but I've been using an
> outside systemd solution for a while now: I setup
> an alias for the "reboot" command that arranges to
> kill off all the things systemd unreasonably waits
> for, then does a real reboot.
>
> Since systemd now has nothing to stop it, it reboots
> rather fast (until something new shows up which I have
> to track down and add to my list :-).
>
> My current set of things to do before shutdown includes:
>
> umount -l -t nfs -a
> apachectl -k stop
> kill all the "user deamon" process trees.
>
> The user daemon stuff is handled by a program
> described here:
>
> http://tomhorsley.com/game/punch.html
>
> Of course, since (according to the systemd biggest
> myths page) systemd is easy to script and not at all
> confusing, then it should be trivial for me to script
> things so shutdown works this way automatically
> without me having to remember to run my alias
> in a terminal, but, alas, they don't appear to be
> myths at all.
I've said it before, systemd is a huge, cumbersome, useless solution
looking for a problem. The decision to use this crud was one of the most
boneheaded things any distribution (not just RHEL/Fedora) ever chose
to adopt. Unfortunately, we're stuck with it now.
As far as the NFS unmount issue, this was an interesting dialog
that happened in September last year:
https://github.com/systemd/systemd/issues/1257
So, "but I doubt that lazy unmounting is really a comprehensive
solution for anything..." comes from one of the developers. Fine, then
when it times out, do a forced umount. This is pretty obvious. The
server isn't listening and we (the client) are shutting down. Pull the
g*ddamned plug!
I'm with David...but this is only one of a litany of stupid decisions
systemd makes. It seems that developers nowadays are never taught how
(or are simply too lazy) to test for error conditions.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com -
- AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 -
- -
- Political Correctness: The insane doctrine that postulates that it -
- is entirely possible to pick up a turd by the clean end. -
----------------------------------------------------------------------
--
users mailing list
users(a)lists.fedoraproject.org
To unsubscribe or change subscription options:
https://lists.fedoraproject.org/admin/lists/users@lists.fedoraproject.org
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
5:52 p.m.
On Thu, 28 Jul 2016 18:43:09 -0400
bruce wrote:
But, the whole systemd/dnf stuff... is that really useful, as opposed
to
the philosophy of some of the other flavors?
There is no other flavor. Systemd will assimilate you. Every
distro uses it now (as near as I can tell). DNF is totally separate
(so far, though you never know what new piece of linux systemd
will engulf).
The debian/ubuntu flavors use .deb files rather than .rpm files
and have tools that do much the same jobs, but are slightly
different than the .rpm/yum/dnf world.
6:03 p.m.
Hey Tom!
So.. If I undertand the overall point. Most linux flavors are more or less
moving in the same direction.. Just some a bit faster than others..
Heaven or Hell!!
Thanks
On Thu, Jul 28, 2016 at 6:52 PM, Tom Horsley <horsley1953(a)gmail.com> wrote:
On Thu, 28 Jul 2016 18:43:09 -0400
bruce wrote:
> But, the whole systemd/dnf stuff... is that really useful, as opposed to
> the philosophy of some of the other flavors?
There is no other flavor. Systemd will assimilate you. Every
distro uses it now (as near as I can tell). DNF is totally separate
(so far, though you never know what new piece of linux systemd
will engulf).
The debian/ubuntu flavors use .deb files rather than .rpm files
and have tools that do much the same jobs, but are slightly
different than the .rpm/yum/dnf world.
--
users mailing list
users(a)lists.fedoraproject.org
To unsubscribe or change subscription options:
https://lists.fedoraproject.org/admin/lists/users@lists.fedoraproject.org
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
11:26 p.m.
On 16/07/28 18:52, Tom Horsley wrote:
On Thu, 28 Jul 2016 18:43:09 -0400
bruce wrote:
> But, the whole systemd/dnf stuff... is that really useful, as opposed to
> the philosophy of some of the other flavors?
There is no other flavor. Systemd will assimilate you. Every
distro uses it now (as near as I can tell). DNF is totally separate
(so far, though you never know what new piece of linux systemd
will engulf).
Iirc Gentoo doesn't use it, Devuan (a Debian-fork that got forked to not
use systemd) doesn't use it and will never use it, there is Manjaro
OpenRC and Arch OpenRC, also Obarun and Slackware and some others. Have
a look here: http://systemd-free.org/
Also: if the init-system is really important to you, have a look at *BSD
(PC-BSD is quite desktop-friendly). Depending on your needs it might
work well for you.
Niels
4:26 p.m.
On Thu, Jul 28, 2016 at 02:53:34PM -0400, Tom Horsley wrote:
On Thu, 28 Jul 2016 14:37:41 -0400
David A. De Graaf wrote:
> Have I overlooked something obvious? Is there a way to make systemd
> perform the simple function 'shutdown' smoothly, reliably and quickly?
> If anyone knows how, I would love to hear it.
I can't make systemd itself work, but I've been using an
outside systemd solution for a while now: I setup
an alias for the "reboot" command that arranges to
kill off all the things systemd unreasonably waits
for, then does a real reboot.
Since systemd now has nothing to stop it, it reboots
rather fast (until something new shows up which I have
to track down and add to my list :-).
My current set of things to do before shutdown includes:
umount -l -t nfs -a
apachectl -k stop
kill all the "user deamon" process trees.
The user daemon stuff is handled by a program
described here:
http://tomhorsley.com/game/punch.html
Thank you Tom Horsley for all your brilliant insights,
and especially this one.
I have taken your 'locate-user-daemons.c' program, lock, stock
and barrel, stirred with some other condiments and come up with
this alternate /usr/local/bin/reboot script that works for me:
$ cat /usr/local/bin/reboot
# Precede normal reboot command with unmounting of nfs mounts
# and stop other impediments to progress
/usr/local/bin/nfsumount
/usr/bin/pkill -9 gkrellmd
/usr/local/bin/locate-user-daemons | /bin/bash -i
systemctl reboot
where /usr/local/bin/nfsumount contains:
# Forcibly umount all autofs-mounted filesystems
mount | grep /net/ |
sed -e 's/.*on //' | sed -e 's/ .*//' |
sort -u | tac |
while read fn; do
umount -fl -t nfs $fn &
done
Your simpler 'umount -l -t nfs -a' works for mounts listed
in /etc/fstab, but not for autofs-mounted filesystems, I believe.
I (re)discovered that gkrellmd takes a full 90 sec to be stopped
by systemd, but no time at all by pkill; I added that.
As you say, more may turn up.
Now the 'reboot' command makes my systems shut down smoothly, reliably
and quickly, as proper Linux systems should.
--
David A. De Graaf DATIX, Inc. Hendersonville, NC
dad(a)datix.us www.datix.us
"Those who hear not the music, think the dancers mad."
5:45 p.m.
Hey David..
Thanks for the thoughts on this...
Would you mind posting/pasting what your code/shell scripts are/is.. I'm
sure someone will need something similar in life!
-Peace!
On Thu, Jul 28, 2016 at 5:26 PM, David A. De Graaf <dad(a)datix.us> wrote:
On Thu, Jul 28, 2016 at 02:53:34PM -0400, Tom Horsley wrote:
> On Thu, 28 Jul 2016 14:37:41 -0400
> David A. De Graaf wrote:
>
> > Have I overlooked something obvious? Is there a way to make systemd
> > perform the simple function 'shutdown' smoothly, reliably and quickly?
> > If anyone knows how, I would love to hear it.
>
> I can't make systemd itself work, but I've been using an
> outside systemd solution for a while now: I setup
> an alias for the "reboot" command that arranges to
> kill off all the things systemd unreasonably waits
> for, then does a real reboot.
>
> Since systemd now has nothing to stop it, it reboots
> rather fast (until something new shows up which I have
> to track down and add to my list :-).
>
> My current set of things to do before shutdown includes:
>
> umount -l -t nfs -a
> apachectl -k stop
> kill all the "user deamon" process trees.
>
> The user daemon stuff is handled by a program
> described here:
>
> http://tomhorsley.com/game/punch.html
>
Thank you Tom Horsley for all your brilliant insights,
and especially this one.
I have taken your 'locate-user-daemons.c' program, lock, stock
and barrel, stirred with some other condiments and come up with
this alternate /usr/local/bin/reboot script that works for me:
$ cat /usr/local/bin/reboot
# Precede normal reboot command with unmounting of nfs mounts
# and stop other impediments to progress
/usr/local/bin/nfsumount
/usr/bin/pkill -9 gkrellmd
/usr/local/bin/locate-user-daemons | /bin/bash -i
systemctl reboot
where /usr/local/bin/nfsumount contains:
# Forcibly umount all autofs-mounted filesystems
mount | grep /net/ |
sed -e 's/.*on //' | sed -e 's/ .*//' |
sort -u | tac |
while read fn; do
umount -fl -t nfs $fn &
done
Your simpler 'umount -l -t nfs -a' works for mounts listed
in /etc/fstab, but not for autofs-mounted filesystems, I believe.
I (re)discovered that gkrellmd takes a full 90 sec to be stopped
by systemd, but no time at all by pkill; I added that.
As you say, more may turn up.
Now the 'reboot' command makes my systems shut down smoothly, reliably
and quickly, as proper Linux systems should.
--
David A. De Graaf DATIX, Inc. Hendersonville, NC
dad(a)datix.us www.datix.us
"Those who hear not the music, think the dancers mad."
--
users mailing list
users(a)lists.fedoraproject.org
To unsubscribe or change subscription options:
https://lists.fedoraproject.org/admin/lists/users@lists.fedoraproject.org
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
12:20 p.m.
On Thu, Jul 28, 2016 at 06:45:17PM -0400, bruce wrote:
Hey David..
Thanks for the thoughts on this...
Would you mind posting/pasting what your code/shell scripts are/is.. I'm sure
someone will need something similar in life!
Well, I have. And you have regurgitated them below (in your somewhat
inconsiderate top-posting).
The locate-user-daemons program is kindly provided by Tom Horsley
at his website, also listed below.
It's all here, for a careful reader.
-Peace!
On Thu, Jul 28, 2016 at 5:26 PM, David A. De Graaf <dad(a)datix.us> wrote:
On Thu, Jul 28, 2016 at 02:53:34PM -0400, Tom Horsley wrote:
> On Thu, 28 Jul 2016 14:37:41 -0400
> David A. De Graaf wrote:
>
> > Have I overlooked something obvious? Is there a way to make systemd
> > perform the simple function 'shutdown' smoothly, reliably and
quickly?
> > If anyone knows how, I would love to hear it.
>
> I can't make systemd itself work, but I've been using an
> outside systemd solution for a while now: I setup
> an alias for the "reboot" command that arranges to
> kill off all the things systemd unreasonably waits
> for, then does a real reboot.
>
> Since systemd now has nothing to stop it, it reboots
> rather fast (until something new shows up which I have
> to track down and add to my list :-).
>
> My current set of things to do before shutdown includes:
>
> umount -l -t nfs -a
> apachectl -k stop
> kill all the "user deamon" process trees.
>
> The user daemon stuff is handled by a program
> described here:
>
> http://tomhorsley.com/game/punch.html
>
Thank you Tom Horsley for all your brilliant insights,
and especially this one.
I have taken your 'locate-user-daemons.c' program, lock, stock
and barrel, stirred with some other condiments and come up with
this alternate /usr/local/bin/reboot script that works for me:
$ cat /usr/local/bin/reboot
# Precede normal reboot command with unmounting of nfs mounts
# and stop other impediments to progress
/usr/local/bin/nfsumount
/usr/bin/pkill -9 gkrellmd
/usr/local/bin/locate-user-daemons | /bin/bash -i
systemctl reboot
where /usr/local/bin/nfsumount contains:
# Forcibly umount all autofs-mounted filesystems
mount | grep /net/ |
sed -e 's/.*on //' | sed -e 's/ .*//' |
sort -u | tac |
while read fn; do
umount -fl -t nfs $fn &
done
Your simpler 'umount -l -t nfs -a' works for mounts listed
in /etc/fstab, but not for autofs-mounted filesystems, I believe.
I (re)discovered that gkrellmd takes a full 90 sec to be stopped
by systemd, but no time at all by pkill; I added that.
As you say, more may turn up.
Now the 'reboot' command makes my systems shut down smoothly, reliably
and quickly, as proper Linux systems should.
--
David A. De Graaf DATIX, Inc. Hendersonville, NC
dad(a)datix.us www.datix.us
If lawyers are disbarred and clergymen defrocked, then doesn't it follow
that electricians can be delighted, musicians denoted, cowboys deranged,
models deposed, tree surgeons debarked, and dry cleaners depressed?
-- R. C. Bagley III
6:34 p.m.
David A. De Graaf wrote:
systemd and autofs/nfs are at war and have been ever since systemd
appeared.
Specifically, if machine A has an open connection to machine B
and B goes down or become inaccessible, then A cannot shutdown.
A's shutdown sequence hangs, waiting for B to respond to an unmount
command, which will not/cannot happen.
Like this bug?
https://bugzilla.redhat.com/show_bug.cgi?id=1359423
-- Rex
12:04 p.m.
On Thu, Jul 28, 2016 at 06:34:55PM -0500, Rex Dieter wrote:
David A. De Graaf wrote:
> systemd and autofs/nfs are at war and have been ever since systemd
> appeared.
>
> Specifically, if machine A has an open connection to machine B
> and B goes down or become inaccessible, then A cannot shutdown.
> A's shutdown sequence hangs, waiting for B to respond to an unmount
> command, which will not/cannot happen.
Like this bug?
https://bugzilla.redhat.com/show_bug.cgi?id=1359423
-- Rex
No, not really. That BZ claims that merely having an nfs mount
active will cause a slow shutdown.
In my experience, if the nfs connection is working properly, ie,
the network is still connected, the server is still working and
responsive, then shutdown is not delayed.
Only when the remote machine fails to respond is the nfs umount
command blocked. It then waits for a response that will not and
cannot come. That's what's dumb.
--
David A. De Graaf DATIX, Inc. Hendersonville, NC
dad(a)datix.us www.datix.us
"Flint must be an extremely wealthy town: I see that each of you
bought two or three seats."
-Victor Borge, playing to a half-filled house in Flint, Mich.
12:16 p.m.
"David A. De Graaf" <dad(a)datix.us> writes:
Only when the remote machine fails to respond is the nfs umount
command blocked. It then waits for a response that will not and
cannot come. That's what's dumb.
I've had this happen to me. I even waited about 20 minutes to see if it
would *ever* time out. It didn't. Had to hard-reset the machine to
reboot it, with the usual risks to filesystem data.
2824
days inactive
2829
days old
12 comments
7 participants
participants (7)
-
bruce -
David A. De Graaf -
DJ Delorie -
Niels Kobschaetzki -
Rex Dieter -
Rick Stevens -
Tom Horsley