Re: [PATCH v2] Append both nofail and x-systemd.before to kdump mount target

[Kairui Song]

Thanks for the test! That warning is expected and we can't avoid that
unless the underlying bug is fixed, but we really need to revert the
previous workaround which is failing NFSv3 (maybe multipath/FCoE or
any other target that require extra services), I'll backport them to
RHEL then.

On Fri, May 15, 2020 at 2:42 AM HAGIO KAZUHITO(萩尾　一仁)
<k-hagio-ab(a)nec.com&gt; wrote:
>  > -----Original Message-----
>  > Hi, Kazu,
>  >
>  > You commented on my previous patch 'Partially Revert "Don't mount the
>  > dump target unless needed"'
>  > that patch is failing in RHEL for the vfs issue, could you help have a
>  > try of this new workaround?
>  >
>  > There might be still a warning but it should not fail kdump.
>  >
>  > I got following log in my test:
>  >
>  > [    8.723761] systemd[1]: kdumproot-mnt-nfs.mount: Mount process
>  > finished, but there is no mount.
>  > [    8.725684] systemd[1]: kdumproot-mnt-nfs.mount: Failed with result
>  > 'protocol'.
>  > [    8.727500] systemd[1]: Failed to mount /kdumproot/mnt/nfs.
>  > [FAILED] Failed to mount /kdumproot/mnt/nfs.
>  > See 'systemctl status kdumproot-mnt-nfs.mount' for details.
>  > [    8.752847] systemd[1]: Started Reload Configuration from the Real Root.
>  > [  OK  ] Started Reload Configuration from the Real Root.
>  > [    8.756231] systemd[1]: Reached target Initrd File Systems.
>  > [  OK  ] Reached target Initrd File Systems.
>  > [    8.760391] systemd[1]: Reached target Initrd Default Target.
>  > [  OK  ] Reached target Initrd Default Target.
>  > [    8.764978] systemd[1]: Starting dracut pre-pivot and cleanup hook...
>  >          Starting dracut pre-pivot and cleanup hook...
>  > [  OK  ] Started dracut pre-pivot and cleanup hook.
>  >          Starting Kdump Vmcore Save Service...
>  > [    8.950571] systemd[1]: Started dracut pre-pivot and cleanup hook.
>  > [    8.952191] systemd[1]: Starting Kdump Vmcore Save Service...
>  > kdump: dump target is 192.168.122.1:/srv/local/nfs
>  > kdump: saving to
>  > /kdumproot/mnt/nfs//var/crash/192.168.122.97-2020-05-13-04:53:56/
>  > kdump: saving vmcore-dmesg.txt
>  > kdump: saving vmcore-dmesg.txt complete
>  > kdump: saving vmcore
>  > Copying data                                      : [100.0 %] \
>  >    eta: 0s
>  > kdump: saving vmcore complete
>  >
>  > kdump works, despite there is an warning caused by the kernel vfs bug,
>  > the warning will be gone after that bug is fixed.
>
>  Hi Kairui,
>
>  I've tried this patch replaced with
> "nofail,x-systemd.before=initrd-fs.target",
>  and sometimes see the same "Failed to mount" messages above, but kdump works.
>
>  Thanks!
>  Kazu
>
>  >
>  >
>  > On Wed, May 13, 2020 at 5:32 PM Kairui Song <kasong(a)redhat.com&gt; wrote:
>  > >
>  > > By this point, there is still an unresolved vfs kernel issue that blocks
>  > > systemd from mounting the dump target properly from time to time. To
>  > > prevent systemd from failing by mounting the dump target, we can add
>  > > nofail option to the kdump mount point.
>  > >
>  > > But adding nofail will wipe out default dependency of the mount point,
>  > > see commit 94a7b43, so systemd randomize the order of calling kdump.sh
>  > > and mounting the dump target and lead to unexpected behavior.
>  > > However we can use x-systemd.before to ensure the mount is done
>  > > in right order.
>  > >
>  > > In dracut-kdump-capture.service, we have "After=initrd.target",
>  > > and look at dracut.bootup.7, systems start processing of fstab by
>  > > initrd-fs.target, so set "After=initrd.target" could ensure the
> mount
>  > > is ready before pre-pivot and kdump-capture service.
>  > >
>  > > Now with both nofail and x-systemd.before=initrd.target, systemd
>  > > will try to mount the dump target before calling kdump, and even if the
>  > > mount failed, kdump.sh will still be called and try to mount again. See
>  > > dump_fs function, which will try to mount if the target is not mounted.
>  > > Kdump will only fail if both mount attemp fails.
>  > >
>  > > Else if the kdump target mount failed or unstable, systemd will directly
>  > > jump to kdump failure action, and kdump fails.
>  > >
>  > > This should improve the robustness in general with no other risk.
>  > > ---
>  > >  mkdumprd | 3 +++
>  > >  1 file changed, 3 insertions(+)
>  > >
>  > > diff --git a/mkdumprd b/mkdumprd
>  > > index cedf536..7ab06fa 100644
>  > > --- a/mkdumprd
>  > > +++ b/mkdumprd
>  > > @@ -83,6 +83,9 @@ to_mount() {
>  > >      # drop nofail or nobootwait
>  > >      _options=$(echo $_options | sed 's/\(^\|,\)nofail\($\|,\)/\1/g')
>  > >      _options=$(echo $_options | sed
> 's/\(^\|,\)nobootwait\($\|,\)/\1/g')
>  > > +    # use both nofail and x-systemd.before to ensure systemd will try best to
>  > > +    # mount it before kdump starts, this is an attempt to improve robustness
>  > > +    _options="$_options,nofail,x-systemd.before=initrd.target"
>  > >
>  > >      _mntopts="$_target $_fstype $_options"
>  > >      #for non-nfs _dev converting to use udev persistent name
>  > > --
>  > > 2.26.2
>  > >
>  >
>  >
>  > --
>  > Best Regards,
>  > Kairui Song
>  > _______________________________________________
>  > kexec mailing list -- kexec(a)lists.fedoraproject.org
>  > To unsubscribe send an email to kexec-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 


-- 
Best Regards,
Kairui Song