kexec April 2021

kexec@lists.fedoraproject.org

10 participants
34 discussions

[PATCH] kdumpctl: fix check_config error when kdump.conf is empty
by Kairui Song 28 Apr '21

28 Apr '21

Kdump scirpt already have default values for core_collector, path in many other place. Empty kdump.conf still works. Fix this corner case and fix the error message. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- kdumpctl | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/kdumpctl b/kdumpctl index 81448be..00703f2 100755 --- a/kdumpctl +++ b/kdumpctl @@ -238,12 +238,7 @@ restore_default_initrd() check_config() { local -A _opt_rec - while read config_opt config_val; do - if [ -z "$config_val" ]; then - derror "Invalid kdump config value for option $config_opt" - return 1 - fi - + while read -r config_opt config_val; do case "$config_opt" in dracut_args) if [[ $config_val == *--mount* ]]; then @@ -269,12 +264,20 @@ check_config() derror "Deprecated kdump config option: $config_opt. Refer to kdump.conf manpage for alternatives." return 1 ;; + '') + continue + ;; *) derror "Invalid kdump config option $config_opt" return 1 ;; esac + if [[ -z "$config_val" ]]; then + derror "Invalid kdump config value for option '$config_opt'" + return 1 + fi + if [ -n "${_opt_rec[$config_opt]}" ]; then if [ $config_opt == _target ]; then derror "More than one dump targets specified" -- 2.30.2

2 1

[PATCH] Revert "Always set vm.zone_reclaim_mode = 3 in kdump kernel"
by Kairui Song 28 Apr '21

28 Apr '21

This reverts commit 5633e8331866098c97e72e99f233a254fa479a4d. vm.zone_reclaim_mode may cause trashing on some machines. And after second thought, vm.zone_reclaim_mode is barely helpful for machines with high mem stress, so just revert it. --- dracut-module-setup.sh | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 066d0ce..b32b39b 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -645,18 +645,16 @@ kdump_install_conf() { rm -f ${initdir}/tmp/$$-kdump.conf } -# Remove user custom configurations sysctl.conf & sysctl.d/* -# and apply some optimization for kdump -overwrite_sysctl_conf() { +# Default sysctl parameters should suffice for kdump kernel. +# Remove custom configurations sysctl.conf & sysctl.d/* +remove_sysctl_conf() { + # As custom configurations like vm.min_free_kbytes can lead # to OOM issues in kdump kernel, avoid them rm -f "${initdir}/etc/sysctl.conf" rm -rf "${initdir}/etc/sysctl.d" rm -rf "${initdir}/run/sysctl.d" rm -rf "${initdir}/usr/lib/sysctl.d" - - mkdir -p "${initdir}/etc/sysctl.d" - echo "vm.zone_reclaim_mode = 3" > "${initdir}/etc/sysctl.d/99-zone-reclaim.conf" } kdump_iscsi_get_rec_val() { @@ -947,7 +945,7 @@ kdump_install_systemd_conf() { install() { kdump_module_init kdump_install_conf - overwrite_sysctl_conf + remove_sysctl_conf if is_ssh_dump_target; then kdump_install_random_seed -- 2.30.2

2 1

[PATCH] fadump: fix dump capture failure to root disk
by Hari Bathini 28 Apr '21

28 Apr '21

If the dump target is the root disk, kdump scripts add an entry in /etc/fstab for root disk with /sysroot as the mount point. The root disk, passed through root=<> kernel commandline parameter, is mounted at /sysroot in read-only mode before switching from initial ramdisk. So, in fadump mode, a remount of /sysroot to read-write mode is needed to capture dump successfully, because /sysroot is already mounted as read-only based on root=<> boot parameter. Commit e8ef4db8ff91 ("Fix dump_fs mount point detection and fallback mount") removed initialization of $_op variable, the variable holding the options the dump target was mounted with, leading to the below error as remount was skipped: kdump[586]: saving to /sysroot/var/crash/127.0.0.1-2021-04-22-07:22:08/ kdump.sh[587]: mkdir: cannot create directory '/sysroot/var/crash/127.0.0.1-2021-04-22-07:22:08/': Read-only file system kdump[589]: saving vmcore failed Restore $_op variable initialization in dump_fs() function to fix this. Fixes: e8ef4db8ff91 ("Fix dump_fs mount point detection and fallback mount") Signed-off-by: Hari Bathini <hbathini(a)linux.ibm.com> --- kdump-lib-initramfs.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 5cb0223..15bbd85 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -119,7 +119,8 @@ dump_fs() { local _exitcode local _mp=$1 - ddebug "dump_fs _mp=$_mp" + local _op=$(get_mount_info OPTIONS target $_mp -f) + ddebug "dump_fs _mp=$_mp _opts=$_op" if ! is_mounted "$_mp"; then dinfo "dump path \"$_mp\" is not mounted, trying to mount..." @@ -139,8 +140,8 @@ dump_fs() # Only remount to read-write mode if the dump target is mounted read-only. if [[ "$_op" = "ro"* ]]; then - dinfo "Mounting Dump target $_dev in rw mode." - mount -o remount,rw $_dev $_mp || return 1 + dinfo "Remounting the dump target in rw mode." + mount -o remount,rw $_mp || return 1 fi mkdir -p $_dump_path || return 1

2 1

[PATCH 0/3] Fixing the bug kexec-tools generates rd.route for eth0 instead of kdump-eth0
by Coiby Xu 26 Apr '21

26 Apr '21

The main goal of this patch set is to fix bz1854037 which happens because rd.route doesn't use the name from kdump_setup_ifname which in turn causes NetworkManger to get an IP for the slave NIC and then time out. Meanwhile, I think it's better to avoid repeatedly calling kdump_setup_ifname. Coiby Xu (3): get kdump ifname once in kdump_install_net pass kdumpnic to kdump_setup_/bond/bridge/vlan directly rd.route should use the name from kdump_setup_ifname dracut-module-setup.sh | 58 ++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 30 deletions(-) -- 2.31.1

2 7

[PATCH 0/2] Drop drm module by default unless failure shell enabled
by Kairui Song 22 Apr '21

22 Apr '21

drm modules have been causing many troubles and they often doesn't work and only consume extra memory in second kernel. And if failure shell is not enabled, they are not very useful either even if they works. So drop it by default and add some comment to allow user add it back if really needed. Kairui Song (2): Drop drm modules if failure_action is not shell kdump.conf: Add some inline comment about drm module dracut-module-setup.sh | 6 ++++-- kdump.conf | 6 +++++- 2 files changed, 9 insertions(+), 3 deletions(-) -- 2.30.2

1 2

[PATCH v2] kdumpctl: Add kdumpctl estimate
by Kairui Song 22 Apr '21

22 Apr '21

Add a rough esitimation support, currently, following memory usage are checked by this sub command: - System RAM - Kdump Initramfs size - Kdump Kernel image size - Kdump Kernel module size - Kdump userspace user and other runtime allocated memory (currently simply using a fixed value: 64M) This will provide user three values as reference, The output of kdumpctl estimate looks like this: # kdumpctl estimate Reserved crashkernel: 128M Estimated crashkernel: 143M Recommanded crashkernel: 160M Kernel image size: 47M Kernel modules size: 9M Initramfs size: 22M Runtime reservation: 64M Large modules: nouveau: 2281472 xfs: 1515520 WARNING: Current crashkernel size is lower than recommanded size 160M. "Reserved crashkernel" is currently reserved crashkernel value. "Baseline crashkernel" is the crashkernel size when `crashkernel=auto` is used, based on crashkernel auto rules from RHEL. "Estimated crashkernel" is the estimate value based on the memory usage items listed above. "Recommanded crashkernel" will use the largest value of estimated crashkernel and the baseline crashkernel. All values are rounded up and converted to MB. All modules with static size larger than 1M will be listed as large module. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- Update from v1: - Fix typos - Remove "Baseline crashkernel:" from output --- kdump-lib.sh | 67 +++++++++++++++++++++++++++++++++++++++ kdumpctl | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 153 insertions(+), 2 deletions(-) diff --git a/kdump-lib.sh b/kdump-lib.sh index 21271cf..d56622a 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -900,3 +900,70 @@ kdump_get_arch_recommend_size() echo $result return 0 } + +check_vmlinux() +{ + # Use readelf to check if it's a valid ELF + readelf -h $1 &>/dev/null || return 1 +} + +get_vmlinux_size() +{ + local size=0 + + while read _type _offset _virtaddr _physaddr _fsize _msize _flg _aln; do + size=$(( $size + $_msize )) + done <<< $(readelf -l -W $1 | grep "^ LOAD" 2>/dev/stderr) + + echo $size +} + +try_decompress() +{ + # The obscure use of the "tr" filter is to work around older versions of + # "grep" that report the byte offset of the line instead of the pattern. + + # Try to find the header ($1) and decompress from here + for pos in `tr "$1\n$2" "\n$2=" < "$4" | grep -abo "^$2"` + do + if ! type -P $3 > /dev/null; then + ddebug "Signiature detected but '$3' is missing, skip this decompressor" + break + fi + + pos=${pos%%:*} + tail -c+$pos "$img" | $3 > $5 2> /dev/null + if check_vmlinux $5; then + ddebug "Kernel is extracted with '$3'" + return 0 + fi + done + + return 1 +} + +# Borrowed from linux/scripts/extract-vmlinux +get_kernel_size() +{ + # Prepare temp files: + local img=$1 tmp=$(mktemp /tmp/vmlinux-XXX) + trap "rm -f $tmp" 0 + + # Try to check if it's a vmlinux already + check_vmlinux $img && get_vmlinux_size $img && return 0 + + # That didn't work, so retry after decompression. + try_decompress '\037\213\010' xy gunzip $img $tmp || \ + try_decompress '\3757zXZ\000' abcde unxz $img $tmp || \ + try_decompress 'BZh' xy bunzip2 $img $tmp || \ + try_decompress '\135\0\0\0' xxx unlzma $img $tmp || \ + try_decompress '\211\114\132' xy 'lzop -d' $img $tmp || \ + try_decompress '\002!L\030' xxx 'lz4 -d' $img $tmp || \ + try_decompress '(\265/\375' xxx unzstd $img $tmp + + # Finally check for uncompressed images or objects: + get_vmlinux_size $tmp && return 0 + + derror "Failed to get kernel image size" + return 1 +} diff --git a/kdumpctl b/kdumpctl index c3311ad..4109539 100755 --- a/kdumpctl +++ b/kdumpctl @@ -1246,6 +1246,87 @@ rebuild() { return $? } +do_estimate() { + local kdump_mods + local -A large_mods + local baseline + local kernel_size mod_size initrd_size baseline_size runtime_size reserved_size estimated_size recommanded_size + local size_mb=$(( 1024 * 1024 )) + + setup_initrd + + if [ ! -f $TARGET_INITRD ]; then + derror "kdumpctl estimate: kdump initramfs is not built yet." + exit 1 + fi + + kdump_mods="$(lsinitrd $TARGET_INITRD -f /usr/lib/dracut/hostonly-kernel-modules.txt | tr '\n' ' ')" + + baseline=$(kdump_get_arch_recommend_size) + if [[ "${baseline: -1}" == "M" ]]; then + baseline=${baseline%M} + elif [[ "${baseline: -1}" == "G" ]]; then + baseline=$(( ${baseline%G} * 1024 )) + elif [[ "${baseline: -1}" == "T" ]]; then + baseline=$(( ${baseline%Y} * 1048576 )) + fi + + # The default value when using crashkernel=auto + baseline_size=$(( $baseline * $size_mb )) + # Current reserved crashkernel size + reserved_size=$(cat /sys/kernel/kexec_crash_size) + # A pre-estimated value for userspace usage and kernel + # runtime allocation, 64M should good for most cases + runtime_size=$(( 64 * $size_mb )) + # Kernel image size + kernel_size=$(get_kernel_size $KDUMP_KERNEL) + # Kdump initramfs size + initrd_size=$(du -b $TARGET_INITRD | awk '{print $1}') + # Kernel modules size after loaded + mod_size=0 + while read -r _name _size _; do + if [[ ! " $kdump_mods " == *" $_name "* ]]; then + continue + fi + mod_size=$(( $mod_size + $_size )) + + # Mark module with static size larger than 2M as large module + if [[ $(( $_size / $size_mb )) -ge 1 ]]; then + large_mods[$_name]=$_size + fi + done <<< "$(< /proc/modules)" + + estimated_size=$(( $kernel_size + $mod_size + $initrd_size + $runtime_size )) + + if [[ $baseline_size -gt $estimated_size ]]; then + recommanded_size=$baseline_size + else + recommanded_size=$estimated_size + fi + + echo "Reserved crashkernel: $(( ( $reserved_size + $size_mb - 1 ) / $size_mb ))M" + echo "Estimated crashkernel: $(( ( $estimated_size + $size_mb - 1 ) / $size_mb ))M" + echo "Recommanded crashkernel: $(( ( $recommanded_size + $size_mb - 1 ) / $size_mb ))M" + echo + echo "Kernel image size: $(( ( $kernel_size + $size_mb - 1 ) / $size_mb ))M" + echo "Kernel modules size: $(( ( $mod_size + $size_mb - 1 ) / $size_mb ))M" + echo "Initramfs size: $(( ( $initrd_size + $size_mb - 1 ) / $size_mb ))M" + echo "Runtime reservation: $(( ( $runtime_size + $size_mb - 1 ) / $size_mb ))M" + echo -n "Large modules:" + if [[ "${#large_mods[@]}" -eq 0 ]]; then + echo " <none>" + else + echo "" + for _mod in "${!large_mods[@]}"; do + echo " $_mod: ${large_mods[$_mod]}" + done + fi + + if [[ $reserved_size -le $recommanded_size ]]; then + echo "WARNING: Current crashkernel size is lower than recommanded size $(( ( $recommanded_size + 1023 ) / $size_mb ))M." + fi +} + if [ ! -f "$KDUMP_CONFIG_FILE" ]; then derror "Error: No kdump config file found!" exit 1 @@ -1301,8 +1382,11 @@ main () showmem) show_reserved_mem ;; + estimate) + do_estimate $2 + ;; *) - dinfo $"Usage: $0 {start|stop|status|restart|reload|rebuild|propagate|showmem}" + dinfo $"Usage: $0 {start|stop|status|restart|reload|rebuild|propagate|showmem|estimate}" exit 1 esac } @@ -1312,6 +1396,6 @@ single_instance_lock # To avoid fd 9 leaking, we invoke a subshell, close fd 9 and call main. # So that fd isn't leaking when main is invoking a subshell. -(exec 9<&-; main $1) +(exec 9<&-; main $@) exit $? -- 2.29.2

1 1

[PATCH v3] kdumpctl: Add kdumpctl estimate
by Kairui Song 22 Apr '21

22 Apr '21

This patch is based on previous patch series: [PATCH 0/8] Better support for LUKS targets Add a rough esitimation support, currently, following memory usage are checked by this sub command: - System RAM - Kdump Initramfs size - Kdump Kernel image size - Kdump Kernel module size - Kdump userspace user and other runtime allocated memory (currently simply using a fixed value: 64M) - LUKS encryption memory usage This will provide user three values as reference, The output of kdumpctl estimate looks like this: # kdumpctl estimate This encrypted kdump target requires extra memory, assuming using the keyslow with minimun memory requirement Reserved crashkernel: 256M Estimated crashkernel: 655M Recommanded crashkernel: 655M Kernel image size: 48M Kernel modules size: 13M Initramfs size: 19M Runtime reservation: 64M LUKS required size: 512M Large modules: xfs: 1892352 nouveau: 2318336 WARNING: Current crashkernel size is lower than recommanded size 654M. "Reserved crashkernel" is currently reserved crashkernel value. "Baseline crashkernel" is the crashkernel size when `crashkernel=auto` is used, based on crashkernel auto rules from RHEL. "Estimated crashkernel" is the estimate value based on the memory usage items listed above. "LUKS required size" is printed only if LUKS2 encryption is used for dump target and key decryption requires extra memory. "Recommanded crashkernel" will use the largest value of estimated crashkernel and the baseline crashkernel. All values are rounded up and converted to MB. All modules with static size larger than 1M will be listed as large module. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- Update from v2: - Update get_kernel_size, if binutils or kernel decompression tools are not installed, will get kernel size from iomem. - Add support for detecting LUKS memory requirement. Update from v1: - Fix typos - Remove "Baseline crashkernel:" from output kdump-lib.sh | 70 +++++++++++++++++++++++++++++++++++ kdumpctl | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 169 insertions(+), 2 deletions(-) diff --git a/kdump-lib.sh b/kdump-lib.sh index 0b547a4..ad9fd90 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -1024,5 +1024,75 @@ get_luks_dev_master_key() fi dwarn "Can't get key of LUKS device $(readlink -f /dev/block/$_dev)" +} + +check_vmlinux() +{ + # Use readelf to check if it's a valid ELF + readelf -h $1 &>/dev/null || return 1 +} + +get_vmlinux_size() +{ + local size=0 + + while read _type _offset _virtaddr _physaddr _fsize _msize _flg _aln; do + size=$(( $size + $_msize )) + done <<< $(readelf -l -W $1 | grep "^ LOAD" 2>/dev/stderr) + + echo $size +} + +try_decompress() +{ + # The obscure use of the "tr" filter is to work around older versions of + # "grep" that report the byte offset of the line instead of the pattern. + + # Try to find the header ($1) and decompress from here + for pos in `tr "$1\n$2" "\n$2=" < "$4" | grep -abo "^$2"` + do + if ! type -P $3 > /dev/null; then + ddebug "Signiature detected but '$3' is missing, skip this decompressor" + break + fi + + pos=${pos%%:*} + tail -c+$pos "$img" | $3 > $5 2> /dev/null + if check_vmlinux $5; then + ddebug "Kernel is extracted with '$3'" + return 0 + fi + done + return 1 } + +# Borrowed from linux/scripts/extract-vmlinux +get_kernel_size() +{ + # Prepare temp files: + local img=$1 tmp=$(mktemp /tmp/vmlinux-XXX) + trap "rm -f $tmp" 0 + + # Try to check if it's a vmlinux already + check_vmlinux $img && get_vmlinux_size $img && return 0 + + # That didn't work, so retry after decompression. + try_decompress '\037\213\010' xy gunzip $img $tmp || \ + try_decompress '\3757zXZ\000' abcde unxz $img $tmp || \ + try_decompress 'BZh' xy bunzip2 $img $tmp || \ + try_decompress '\135\0\0\0' xxx unlzma $img $tmp || \ + try_decompress '\211\114\132' xy 'lzop -d' $img $tmp || \ + try_decompress '\002!L\030' xxx 'lz4 -d' $img $tmp || \ + try_decompress '(\265/\375' xxx unzstd $img $tmp + + # Finally check for uncompressed images or objects: + [[ $? -eq 0 ]] && get_vmlinux_size $tmp && return 0 + + # Fallback to use iomem + local _size=0 + for _seg in $(cat /proc/iomem | grep -E "Kernel (code|rodata|data|bss)" | cut -d ":" -f 1); do + _size=$(( $_size + 0x${_seg#*-} - 0x${_seg%-*} )) + done + echo $_size +} diff --git a/kdumpctl b/kdumpctl index 18c3971..fdedfc6 100755 --- a/kdumpctl +++ b/kdumpctl @@ -1314,6 +1314,100 @@ rebuild() { return $? } +do_estimate() { + local kdump_mods + local -A large_mods + local baseline + local kernel_size mod_size initrd_size baseline_size runtime_size reserved_size estimated_size recommanded_size + local size_mb=$(( 1024 * 1024 )) + + setup_initrd + + if [ ! -f $TARGET_INITRD ]; then + derror "kdumpctl estimate: kdump initramfs is not built yet." + exit 1 + fi + + kdump_mods="$(lsinitrd $TARGET_INITRD -f /usr/lib/dracut/hostonly-kernel-modules.txt | tr '\n' ' ')" + + baseline=$(kdump_get_arch_recommend_size) + if [[ "${baseline: -1}" == "M" ]]; then + baseline=${baseline%M} + elif [[ "${baseline: -1}" == "G" ]]; then + baseline=$(( ${baseline%G} * 1024 )) + elif [[ "${baseline: -1}" == "T" ]]; then + baseline=$(( ${baseline%Y} * 1048576 )) + fi + + # The default value when using crashkernel=auto + baseline_size=$(( $baseline * $size_mb )) + # Current reserved crashkernel size + reserved_size=$(cat /sys/kernel/kexec_crash_size) + # A pre-estimated value for userspace usage and kernel + # runtime allocation, 64M should good for most cases + runtime_size=$(( 64 * $size_mb )) + # Kernel image size + kernel_size=$(get_kernel_size $KDUMP_KERNEL) + # Kdump initramfs size + initrd_size=$(du -b $TARGET_INITRD | awk '{print $1}') + # Kernel modules size after loaded + mod_size=0 + while read -r _name _size _; do + if [[ ! " $kdump_mods " == *" $_name "* ]]; then + continue + fi + mod_size=$(( $mod_size + $_size )) + + # Mark module with static size larger than 2M as large module + if [[ $(( $_size / $size_mb )) -ge 1 ]]; then + large_mods[$_name]=$_size + fi + done <<< "$(< /proc/modules)" + + crypt_size=0 + for _dev in $(get_all_kdump_crypt_dev); do + _crypt_info=$(cryptsetup luksDump /dev/block/$_dev) + [[ $(echo "$_crypt_info" | sed -n "s/^Version:\s*$.*$/\1/p" ) == "2" ]] || continue + for _mem in $(echo "$_crypt_info" | sed -n "s/\sMemory:\s*$.*$/\1/p" | sort -n ); do + crypt_size=$(( $crypt_size + $_mem * 1024 )) + break + done + done + [[ $crypt_size -ne 0 ]] && echo -e "Encrypted kdump target requires extra memory, assuming using the keyslot with minimun memory requirement\n" + + estimated_size=$(( $kernel_size + $mod_size + $initrd_size + $runtime_size + $crypt_size )) + + if [[ $baseline_size -gt $estimated_size ]]; then + recommanded_size=$baseline_size + else + recommanded_size=$estimated_size + fi + + echo "Reserved crashkernel: $(( ( $reserved_size + $size_mb - 1 ) / $size_mb ))M" + echo "Estimated crashkernel: $(( ( $estimated_size + $size_mb - 1 ) / $size_mb ))M" + echo "Recommanded crashkernel: $(( ( $recommanded_size + $size_mb - 1 ) / $size_mb ))M" + echo + echo "Kernel image size: $(( ( $kernel_size + $size_mb - 1 ) / $size_mb ))M" + echo "Kernel modules size: $(( ( $mod_size + $size_mb - 1 ) / $size_mb ))M" + echo "Initramfs size: $(( ( $initrd_size + $size_mb - 1 ) / $size_mb ))M" + echo "Runtime reservation: $(( ( $runtime_size + $size_mb - 1 ) / $size_mb ))M" + [[ $crypt_size -ne 0 ]] && \ + echo "LUKS required size: $(( ( $crypt_size + $size_mb - 1 ) / $size_mb ))M" + echo -n "Large modules:" + if [[ "${#large_mods[@]}" -eq 0 ]]; then + echo " <none>" + else + echo "" + for _mod in "${!large_mods[@]}"; do + echo " $_mod: ${large_mods[$_mod]}" + done + fi + + if [[ $reserved_size -le $recommanded_size ]]; then + echo "WARNING: Current crashkernel size is lower than recommanded size $(( ( $recommanded_size + 1023 ) / $size_mb ))M." + fi +} + if [ ! -f "$KDUMP_CONFIG_FILE" ]; then derror "Error: No kdump config file found!" exit 1 @@ -1321,7 +1415,7 @@ fi usage() { - dinfo $"Usage: $0 {start|stop|status|restart|reload|rebuild|propagate|showmem} [--askpass]" + dinfo $"Usage: $0 {start|stop|status|restart|reload|rebuild|propagate|showmem|estimate} [--askpass]" exit 1 } @@ -1333,7 +1427,7 @@ main() local cmd case "$1" in - condrestart|start|stop|status|restart|reload|rebuild|propagate|showmem) + condrestart|start|stop|status|restart|reload|rebuild|propagate|showmem|estimate) cmd=$1 ;; *) @@ -1403,6 +1497,9 @@ main() showmem) show_reserved_mem ;; + estimate) + do_estimate + ;; *) usage exit 1 -- 2.30.2

1 0

Kdump with full-disk LUKS encryption
by Kairui Song 19 Apr '21

19 Apr '21

Hi all, I'm currently trying to add kdump support for systemd with full-disk LUKS encryption. vmcores contain sensitive data so they should also be protected, and network dumps sometimes are not available. So kdump has to open the LUKS encrypted device in the kdump environment. I'm using systemd/dracut, my work machine is running Fedora 34, and there are several problems I'm trying to solve: 1. Users have to input the password in the kdump kernel environment. But users often don't have shell access to the kdump environment. (headless server, graphic card not working after kexec, both are very common) 2. LUKS2 prefers Argon2 as the key derivation function, designed to use a lot of memory. kdump is expected to use a minimal amount of memory. Users will have to reserve a huge amount of memory for kdump to work (eg. 1G reserve for kdump with 4G total memory which is not reasonable). To fix these problems, I tried to pass the master key to the second kernel directly via initramfs. Kdump will modify the initramfs in ramfs to include the key, kexec_load it, and never write to any actual back storage. This worked with old LUKS configurations. But LUKS2/cryptsetup now stores the key in the kernel keyring by default. The key is accessible from userspace. Users can enter the password to start kdump manually and then it will work, but usually people expect kdump service to start automatically. (WIP patch series: https://lists.fedoraproject.org/archives/list/kexec@lists.fedoraproject.org…) I've several ideas about how to improve it but not sure which one is better, and if this is a good idea after all: 1. Simply introduce a config to let systemd-cryptsetup disable kernel keyring on setup, there is currently no such option. 2. If we can let the key stay in userspace for a little longer, eg. for systems booted with dracut/systemd, when systemd-cryptsetup(a)%s.service opens the crypt device, keep the key in dm-crypt. And later when services like kdump have finished loading, cryptsetup can refresh the device and store the key in the kernel keyring again. 3. Let kdump use some custom helper/service to load all needed resources in the early initrd boot stage, prior to systemd-cryptsetup(a)%s.service. It will ask the password / parse the keyfile and load kdump, then provide info for systemd-cryptsetup or just do the setup. Or maybe let systemd-cryptsetup support some kind of "plugins" so other tools can use it. 4. A better and safer solution seems to keep a consistent key ring between kexec boots but also more complex and difficult as different arch implements kexec differently. Any suggestions are welcomed! -- Best Regards, Kairui Song

1 0

[PATCH v3] Write to `/var/lib/kdump` if $KDUMP_BOOTDIR not writable
by Kelvin Fan 19 Apr '21

19 Apr '21

The `/boot` directory on some operating systems might be read-only. If we cannot write to `$KDUMP_BOOTDIR` when generating the kdump initrd, attempt to place the generated initrd at `/var/lib/kdump` instead. Signed-off by: Kelvin Fan <kelvinfan001(a)gmail.com> --- 60-kdump.install | 14 ++++++++++---- kdump-lib.sh | 24 +++++++++++++++++------- kdumpctl | 4 ++-- kexec-tools.spec | 2 ++ 4 files changed, 31 insertions(+), 13 deletions(-) diff --git a/60-kdump.install b/60-kdump.install index 0a3b40e..5b0e021 100755 --- a/60-kdump.install +++ b/60-kdump.install @@ -2,17 +2,23 @@ COMMAND="$1" KERNEL_VERSION="$2" -BOOT_DIR_ABS="$3" +KDUMP_INITRD_DIR_ABS="$3" KERNEL_IMAGE="$4" if ! [[ ${KERNEL_INSTALL_MACHINE_ID-x} ]]; then exit 0 fi -if [[ -d "$BOOT_DIR_ABS" ]]; then +if [[ -d "$KDUMP_INITRD_DIR_ABS" ]]; then KDUMP_INITRD="initrdkdump" else - BOOT_DIR_ABS="/boot" + # If `KDUMP_BOOTDIR` is not writable, then the kdump + # initrd must have been placed at `/var/lib/kdump` + if [[ ! -w "/boot" ]]; then + KDUMP_INITRD_DIR_ABS="/var/lib/kdump" + else + KDUMP_INITRD_DIR_ABS="/boot" + fi KDUMP_INITRD="initramfs-${KERNEL_VERSION}kdump.img" fi @@ -23,7 +29,7 @@ case "$COMMAND" in # and managed by kdump service ;; remove) - rm -f -- "$BOOT_DIR_ABS/$KDUMP_INITRD" + rm -f -- "$KDUMP_INITRD_DIR_ABS/$KDUMP_INITRD" ret=$? ;; esac diff --git a/kdump-lib.sh b/kdump-lib.sh index 21271cf..5f53a8a 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -733,20 +733,30 @@ prepare_kdump_bootinfo() boot_initrdlist="initramfs-$KDUMP_KERNELVER.img initrd" for initrd in $boot_initrdlist; do if [ -f "$KDUMP_BOOTDIR/$initrd" ]; then - DEFAULT_INITRD="$KDUMP_BOOTDIR/$initrd" + defaut_initrd_base="$initrd" + DEFAULT_INITRD="$KDUMP_BOOTDIR/$defaut_initrd_base" break fi done - # Get kdump initrd from default initrd filename + # Create kdump initrd basename from default initrd basename # initramfs-5.7.9-200.fc32.x86_64.img => initramfs-5.7.9-200.fc32.x86_64kdump.img # initrd => initrdkdump - if [[ -z "$DEFAULT_INITRD" ]]; then - KDUMP_INITRD=${KDUMP_BOOTDIR}/initramfs-${KDUMP_KERNELVER}kdump.img - elif [[ $(basename $DEFAULT_INITRD) == *.* ]]; then - KDUMP_INITRD=${DEFAULT_INITRD%.*}kdump.${DEFAULT_INITRD##*.} + if [[ -z "$defaut_initrd_base" ]]; then + kdump_initrd_base=initramfs-${KDUMP_KERNELVER}kdump.img + elif [[ $defaut_initrd_base == *.* ]]; then + kdump_initrd_base=${defaut_initrd_base%.*}kdump.${DEFAULT_INITRD##*.} else - KDUMP_INITRD=${DEFAULT_INITRD}kdump + kdump_initrd_base=${defaut_initrd_base}kdump + fi + + # Place kdump initrd in `/var/lib/kdump` if `KDUMP_BOOTDIR` not writable + if [[ ! -w "$KDUMP_BOOTDIR" ]];then + var_target_initrd_dir="/var/lib/kdump" + mkdir -p "$var_target_initrd_dir" + KDUMP_INITRD="$var_target_initrd_dir/$kdump_initrd_base" + else + KDUMP_INITRD="$KDUMP_BOOTDIR/$kdump_initrd_base" fi } diff --git a/kdumpctl b/kdumpctl index c3311ad..b0846fa 100755 --- a/kdumpctl +++ b/kdumpctl @@ -151,8 +151,8 @@ rebuild_kdump_initrd() rebuild_initrd() { - if [[ ! -w "$KDUMP_BOOTDIR" ]];then - derror "$KDUMP_BOOTDIR does not have write permission. Can not rebuild $TARGET_INITRD" + if [[ ! -w $(dirname $TARGET_INITRD) ]];then + derror "$(dirname $TARGET_INITRD) does not have write permission. Cannot rebuild $TARGET_INITRD" return 1 fi diff --git a/kexec-tools.spec b/kexec-tools.spec index 24c6b36..74b7877 100644 --- a/kexec-tools.spec +++ b/kexec-tools.spec @@ -170,6 +170,7 @@ mkdir -p $RPM_BUILD_ROOT%{_unitdir} mkdir -p -m755 $RPM_BUILD_ROOT%{_bindir} mkdir -p -m755 $RPM_BUILD_ROOT%{_libdir} mkdir -p -m755 $RPM_BUILD_ROOT%{_prefix}/lib/kdump +mkdir -p -m755 $RPM_BUILD_ROOT%{_sharedstatedir}/kdump install -m 755 %{SOURCE1} $RPM_BUILD_ROOT%{_bindir}/kdumpctl install -m 755 build/sbin/kexec $RPM_BUILD_ROOT/usr/sbin/kexec @@ -335,6 +336,7 @@ done %dir %{_sysconfdir}/kdump %dir %{_sysconfdir}/kdump/pre.d %dir %{_sysconfdir}/kdump/post.d +%dir %{_sharedstatedir}/kdump %{_mandir}/man8/kdumpctl.8.gz %{_mandir}/man8/kexec.8.gz %ifarch %{ix86} x86_64 ppc64 s390x ppc64le aarch64 -- 2.29.2

2 1

Re: [PATCH] Add /etc/hosts to kdump initramfs for nfs/ssh dump
by Kairui Song 19 Apr '21

19 Apr '21

On Fri, Apr 2, 2021 at 10:01 AM HAGIO KAZUHITO(萩尾　一仁) <k-hagio-ab(a)nec.com> wrote: > > Hi, > > I hit the following issue and wrote this patch, but I found later that > it might be better to fix the dracut nfs/ssh-client modules in this case. > but I'm not familiar with dracut.. could I have any comments? Hi Kazu, I agree with you that we better frix it in dracut, I made a simple patch below, can you have a look or try it? https://github.com/ryncsn/dracut/commit/c69c223f649e1392e09b6c89bf2df00e900… And I think after the dracut fix, we still need to detect for host/nss file change in kdumpctl and rebuild the initramfs on change. Should be a simple change. > > Thanks, > Kazu > > From f39a89d5da01cb5f27ff4b1c8a81d6b58601e80c Mon Sep 17 00:00:00 2001 > From: Kazuhito Hagio <k-hagio-ab(a)nec.com> > Subject: [PATCH] Add /etc/hosts to kdump initramfs for nfs/ssh dump > > Currently the /etc/hosts file is not contained in kdump initramfs > by default. Hosts that depend only on /etc/hosts to resolve IP > address (i.e. do not have DNS servers) can start kdump service with > hostname nfs/ssh configuration, but fail in resolving the target > server's IP address on the 2nd kernel: > > mount[355]: mount.nfs: Failed to resolve server dumpsrv: Name or service not known > ... > systemd[1]: kdumproot-dump.mount: Mount process exited, code=exited, status=32/n/a > systemd[1]: kdumproot-dump.mount: Failed with result 'exit-code'. > systemd[1]: Failed to mount /kdumproot/dump. > > Add the /etc/hosts file to kdump initramfs when nfs/ssh dump is > configured. The /etc/nsswitch.conf file is also required for ssh > dump according to a test. > > Signed-off-by: Kazuhito Hagio <k-hagio-ab(a)nec.com> > --- > dracut-module-setup.sh | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh > index 8316589b3ac8..c59750810cf0 100755 > --- a/dracut-module-setup.sh > +++ b/dracut-module-setup.sh > @@ -874,6 +874,10 @@ install() { > # nfs/ssh dump will need to get host ip in second kernel and need to call 'ip' tool, see get_host_ip for more detail > if is_nfs_dump_target || is_ssh_dump_target; then > inst "ip" > + # For hosts that depend only on /etc/hosts to resolve IP address. > + # /etc/nsswitch.conf is also required for ssh dump. > + inst "/etc/hosts" > + inst "/etc/nsswitch.conf" > fi > > # For the lvm type target under kdump, in /etc/lvm/lvm.conf we can > -- > 2.18.4 > -- Best Regards, Kairui Song

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec April 2021