kexec September 2019

kexec@lists.fedoraproject.org

4 participants
9 discussions

[PATCH] mkdumprd: allow spaces after 'path' config phrase when network dump
by Kazuhito Hagio 27 May '22

27 May '22

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

4 3

[PATCH v2] Don't mount the dump target unless needed
by Kairui Song 29 Sep '19

29 Sep '19

For fadump, this helps to reduce the risk of boot failure, and may also help speed up the boot by a bit. For normal kdump, this will delay the dump target mounting, and no longer depend on systemd to do the mounting job. And currently there is a failure that caused by some mount handling bug with kernel and systemd that is failing the system booting: [FAILED] Failed to mount /kdumproot/home. See 'systemctl status kdumproot-home.mount' for details. [DEPEND] Dependency failed for Local File Systems. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting udev Coldplug all Devices... Starting Create Volatile Files and Directories... Starting Kdump Emergency... This patch can bypass it. The fix of root cause is still WIP, but this patch itself is a nice to have optimization so it's reasonable to do so. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- Update from V1: Always mount dump target on required, for both fadump and kdump. Tested with NFS/local dump and fail into Emergency shell case. kdump-lib-initramfs.sh | 30 +++++++++++++++++++++++++----- mkdumprd | 5 +++-- 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 608dc6e..c409dce 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -96,16 +96,31 @@ get_kdump_confs() # dump_fs <mount point| device> dump_fs() { - + local _do_umount="" local _dev=$(findmnt -k -f -n -r -o SOURCE $1) local _mp=$(findmnt -k -f -n -r -o TARGET $1) local _op=$(findmnt -k -f -n -r -o OPTIONS $1) - echo "kdump: dump target is $_dev" - if [ -z "$_mp" ]; then - echo "kdump: error: Dump target $_dev is not mounted." - return 1 + _dev=$(findmnt -s -f -n -r -o SOURCE $1) + _mp=$(findmnt -s -f -n -r -o TARGET $1) + _op=$(findmnt -s -f -n -r -o OPTIONS $1) + + if [ -n "$_dev" ] && [ -n "$_mp" ]; then + echo "kdump: dump target $_dev is not mounted, trying to mount..." + mkdir -p $_mp + mount -o $_op $_dev $_mp + + if [ $? -ne 0 ]; then + echo "kdump: mounting failed (mount point: $_mp, option: $_op)" + return 1 + fi + _do_umount=1 + else + echo "kdump: error: Dump target $_dev is not usable" + fi + else + echo "kdump: dump target is $_dev" fi # Remove -F in makedumpfile case. We don't want a flat format dump here. @@ -129,6 +144,11 @@ dump_fs() sync echo "kdump: saving vmcore complete" + + if [ $_do_umount ]; then + umount $_mp || echo "kdump: warn: failed to umount target" + fi + # improper kernel cmdline can cause the failure of echo, we can ignore this kind of failure return 0 } diff --git a/mkdumprd b/mkdumprd index cf3533f..50472a9 100644 --- a/mkdumprd +++ b/mkdumprd @@ -108,12 +108,13 @@ to_mount() { fi # mount fs target as rw in 2nd kernel _options=$(echo $_options | sed 's/$^\|,$ro$$\|,$/\1rw\2/g') - # with 'noauto' in fstab nfs and non-root disk mount will fail in 2nd - # kernel, filter it out here. + # filter out 'noauto' here, it will be force appended later, avoid duplication _options=$(echo $_options | sed 's/$^\|,$noauto$$\|,$/\1/g') # drop nofail or nobootwait _options=$(echo $_options | sed 's/$^\|,$nofail$$\|,$/\1/g') _options=$(echo $_options | sed 's/$^\|,$nobootwait$$\|,$/\1/g') + # only mount the dump target when needed. + _options="$_options,noauto" _mntopts="$_target $_fstype $_options" #for non-nfs _dev converting to use udev persistent name -- 2.21.0

2 1

[PATCH] fadump: don't mount the dump target unless needed
by Kairui Song 29 Sep '19

29 Sep '19

This helps to reduce the risk of boot failure, and may also help speed up the boot by a bit. And currently there is a failure that caused by some mount handling bug with kernel and systemd that is failing the system booting: [FAILED] Failed to mount /kdumproot/home. See 'systemctl status kdumproot-home.mount' for details. [DEPEND] Dependency failed for Local File Systems. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting udev Coldplug all Devices... Starting Create Volatile Files and Directories... Starting Kdump Emergency... This patch can bypass it. The fix of root cause is still WIP, but this patch itself is a nice to have optimization so it's reasonable to do so. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- kdump-lib-initramfs.sh | 30 +++++++++++++++++++++++++----- mkdumprd | 5 +++++ 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 608dc6e..0a2429b 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -96,16 +96,31 @@ get_kdump_confs() # dump_fs <mount point| device> dump_fs() { - + local _do_umount local _dev=$(findmnt -k -f -n -r -o SOURCE $1) local _mp=$(findmnt -k -f -n -r -o TARGET $1) local _op=$(findmnt -k -f -n -r -o OPTIONS $1) - echo "kdump: dump target is $_dev" - if [ -z "$_mp" ]; then - echo "kdump: error: Dump target $_dev is not mounted." - return 1 + _dev=$(findmnt -s -f -n -r -o SOURCE $1) + _mp=$(findmnt -s -f -n -r -o TARGET $1) + _op=$(findmnt -s -f -n -r -o OPTIONS $1) + + if [ -n "$_dev" ] && [ -n "$_mp" ]; then + echo "kdump: dump target $_dev is not mounted, trying to mount..." + mkdir -p $_mp + mount -o $_op $_dev $_mp + + if [ $? -ne 0 ]; then + echo "kdump: mounting failed (mount point: $_mp, option: $_op)" + return 1 + fi + _do_umount=1 + else + echo "kdump: error: Dump target $_dev is not usable" + fi + else + echo "kdump: dump target is $_dev" fi # Remove -F in makedumpfile case. We don't want a flat format dump here. @@ -129,6 +144,11 @@ dump_fs() sync echo "kdump: saving vmcore complete" + + if [ $_do_umount ]; then + umount $_mp || echo "kdump: warn: failed to umount target" + fi + # improper kernel cmdline can cause the failure of echo, we can ignore this kind of failure return 0 } diff --git a/mkdumprd b/mkdumprd index cf3533f..2526351 100644 --- a/mkdumprd +++ b/mkdumprd @@ -115,6 +115,11 @@ to_mount() { _options=$(echo $_options | sed 's/$^\|,$nofail$$\|,$/\1/g') _options=$(echo $_options | sed 's/$^\|,$nobootwait$$\|,$/\1/g') + # Only for fadump, don't mount the dump target unless needed. + if is_fadump_capable; then + _options="$_options,noauto" + fi + _mntopts="$_target $_fstype $_options" #for non-nfs _dev converting to use udev persistent name if [ -b "$_source" ]; then -- 2.21.0

2 2

[PATCHv2] kdumpctl: echo msg when waiting for connection
by Pingfan Liu 24 Sep '19

24 Sep '19

Print some message during the long wait period to reflect the process. The message will look like: Network dump target is not usable, waiting for it to be ready ... Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdumpctl | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kdumpctl b/kdumpctl index f75b820..dc0c64b 100755 --- a/kdumpctl +++ b/kdumpctl @@ -736,6 +736,7 @@ check_ssh_config() check_and_wait_network_ready() { local start_time=$(date +%s) + local warn_once=1 local cur local diff local retval @@ -760,6 +761,12 @@ check_and_wait_network_ready() return 1 fi + if [ $warn_once -eq 1 ]; then + echo "Network dump target is not usable, waiting for it to be ready" + warn_once=0 + fi + echo -n . + cur=$(date +%s) let "diff = $cur - $start_time" # 60s time out -- 2.7.5

2 1

[PATCH] kdumpctl: echo msg when waiting for connection
by Pingfan Liu 23 Sep '19

23 Sep '19

Print some message during the long wait period to reflect the process. The message will look like: try harder to connect the server, maybe another 3mins ... Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdumpctl | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kdumpctl b/kdumpctl index f75b820..3d9373f 100755 --- a/kdumpctl +++ b/kdumpctl @@ -736,6 +736,7 @@ check_ssh_config() check_and_wait_network_ready() { local start_time=$(date +%s) + local warn_once=1 local cur local diff local retval @@ -760,6 +761,12 @@ check_and_wait_network_ready() return 1 fi + if [ $warn_once -eq 1 ]; then + echo "try harder to connect the server, maybe another 3mins" + warn_once=0 + fi + echo -n . + cur=$(date +%s) let "diff = $cur - $start_time" # 60s time out -- 2.7.5

3 2

[PATCH 0/3] makedumpfile: Fixes for -e option
by Kazuhito Hagio 20 Sep '19

20 Sep '19

The -e option excludes the page structures (vmemmap) which represent excluded pages and greatly shortens the dump of a very large memory system. This option is only for x86_64. But currently it does not support KASLR-enabled vmcore, as a result, it excludes incorrect pages and can create a broken dumpfile without errors on KASLR-enabled system. Patch 1/3 fixes this issue. Patch 2/3 fixes an issue that some boundary vmemmap pages that should be excluded are not excluded correctly. Patch 3/3 fixes an issue that the -e option does some unnecessary processing with no effect and marks the dump DUMP_DH_EXCLUDED_VMEMMAP unexpectedly on architectures other than x86_64. Kazuhito Hagio (3): makedumpfile: x86_64: Fix incorrect exclusion by -e option with KASLR makedumpfile: Fix exclusion range in find_vmemmap_pages() makedumpfile: Fix inconsistent return value from find_vmemmap() ...xclusion-range-in-find_vmemmap_pages.patch | 37 ++ ...stent-return-value-from-find_vmemmap.patch | 327 ++++++++++++++++++ ...-Fix-incorrect-exclusion-by-e-option.patch | 41 +++ kexec-tools.spec | 6 + 4 files changed, 411 insertions(+) create mode 100644 kexec-tools-2.0.20-makedumpfile-Fix-exclusion-range-in-find_vmemmap_pages.patch create mode 100644 kexec-tools-2.0.20-makedumpfile-Fix-inconsistent-return-value-from-find_vmemmap.patch create mode 100644 kexec-tools-2.0.20-makedumpfile-x86_64-Fix-incorrect-exclusion-by-e-option.patch -- 2.23.0

2 4

[RESEND PATCH v4 0/2] Disable device dump by default and update doc
by Kairui Song 02 Sep '19

02 Sep '19

Update from V3: Rebase and resend patch as previousely the kernel parameter is not ready in Fedora, no change Update from V2: Adjust the doc. Update from V1: Split into two patches and update docs as well. Kairui Song (2): Disable device dump by default kexec-kdump-howto.txt: Add notes about device dump kdump.sysconfig | 2 +- kdump.sysconfig.aarch64 | 2 +- kdump.sysconfig.i386 | 2 +- kdump.sysconfig.ppc64 | 2 +- kdump.sysconfig.ppc64le | 2 +- kdump.sysconfig.s390x | 2 +- kdump.sysconfig.x86_64 | 2 +- kexec-kdump-howto.txt | 17 +++++++++++++++++ 8 files changed, 24 insertions(+), 7 deletions(-) -- 2.21.0

2 3

[PATCH v2] dracut-module-setup: fix bond ifcfg processing
by Kairui Song 02 Sep '19

02 Sep '19

Bond options in ifcfg is space separated, dracut expected it to be comma separated, so it have to be parsed and converted during initramfs building. The currently parsing and convert pattern is flawed, for example: " downdelay=0 miimon=100 mode=802.3ad updelay=0 " is converted to : ":,downdelay=0 miimon=100 mode=802.3ad updelay=0 " should be: ":downdelay=0,miimon=100,mode=802.3ad,updelay=0" So fix this issue by using more simple but robust method for processing the options. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- Update from V1: - Add a xargs call to deduplicate the spaces. Although bash should convert the unquated string into a list of parameters and deduplicate the spaces automatically, but if someone add quotes it will break and it's confusing. --- dracut-module-setup.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 3fa696d..b395160 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -264,7 +264,7 @@ kdump_setup_bond() { source_ifcfg_file $_netdev - bondoptions="$(echo :$BONDING_OPTS | sed 's/\s\+/,/')" + bondoptions=":$(echo $BONDING_OPTS | xargs echo | tr " " ",")" echo "$bondoptions" >> ${initdir}/etc/cmdline.d/42bond.conf } -- 2.21.0

2 1

[PATCH] kdumpctl: distinguish the failed reason of ssh
by Pingfan Liu 02 Sep '19

02 Sep '19

On a host with ipaddr not ready before kdump service, ssh return errno 255. While if no ssh-key, ssh also return errno 255. For both of cases, the current kdump code promote user to run 'kdumpctl propagate'. This confuses user who already installs ssh-key. In order to tell these two cases from each other, the ssh warning message should be involved, and parsed. For the no ssh-key case , warning message is "Permission denied" or "No such file or directory". For the other, warning message is "Network Unreachable" This patch also does a slight change to enlarge the timeout from 60s to 180s. This value can meet test at the time being Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdumpctl | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/kdumpctl b/kdumpctl index 2f2d819..f75b820 100755 --- a/kdumpctl +++ b/kdumpctl @@ -738,24 +738,38 @@ check_and_wait_network_ready() local start_time=$(date +%s) local cur local diff + local retval + local errmsg while true; do - ssh -q -i $SSH_KEY_LOCATION -o BatchMode=yes $DUMP_TARGET mkdir -p $SAVE_PATH + errmsg=$(ssh -i $SSH_KEY_LOCATION -o BatchMode=yes $DUMP_TARGET mkdir -p $SAVE_PATH 2>&1) + retval=$? + # ssh exits with the exit status of the remote command or with 255 if an error occurred - if [ $? -eq 0 ]; then + if [ $retval -eq 0 ]; then return 0 - elif [ $? -ne 255 ]; then + elif [ $retval -ne 255 ]; then + echo "Could not create $DUMP_TARGET:$SAVE_PATH, you should check the privilege on server side" >&2 return 1 fi + + # if server removes the authorized_keys or, no /root/.ssh/kdump_id_rsa + echo $errmsg | grep -q "Permission denied\|No such file or directory" + if [ $? -eq 0 ]; then + echo "Could not create $DUMP_TARGET:$SAVE_PATH, you probably need to run \"kdumpctl propagate\"" >&2 + return 1 + fi + cur=$(date +%s) - diff=$( $cur - $start_time ) + let "diff = $cur - $start_time" # 60s time out - if [ $diff -gt 60 ]; then + if [ $diff -gt 180 ]; then break; fi sleep 1 done + echo "Could not create $DUMP_TARGET:$SAVE_PATH, ipaddr is not ready yet. You should check network connection" >&2 return 1 } @@ -763,7 +777,6 @@ check_ssh_target() { check_and_wait_network_ready if [ $? -ne 0 ]; then - echo "Could not create $DUMP_TARGET:$SAVE_PATH, you probably need to run \"kdumpctl propagate\"" >&2 return 1 fi return 0 -- 2.20.1

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec September 2019