[PATCH v2] kdump-lib.sh: add extra 64M to default crashkernel if sme/sev is active

[Baoquan He]

It's reported that kdump kernel failed to boot and can't dump vmcore
when crashkernel=192M and SME/SEV is active.

This is because swiotlb will be enabled and reserves 64M memory by
default on system with SME/SEV enabled. Then kdump kernel will be out of
memory after taking 64M away for swiotlb init.

So here add extra 64M memory to default crashkernel value so that kdump
kernel can function well as before. When doing that, search journalctl
for the "Memory Encryption Features active: AMD" to check if SME or SEV
is active. This line of log is printed out in kernel function as below
and the type SME is mutual exclusive with type SEV.
  ***:
  arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()

Note:
1) The conditional check is relying on journalctl log because I didn't
   find available system interface to check if SEV is active. Even
   though we can check if SME is active via /proc/cpuinfo. For
   consistency, I take the same check for both SME and SEV by searching
   journalctl.

2) The conditional check is relying on journalctl log, means it won't
   work for crashkernel setting in anoconda because the installation
   kernel doesn't have the SME/SEV setting. So customer need manually
   run 'kdumpctl reset-crashkernel' to reset crashkernel to add the
   extra 64M after OS installation.

3) We need watch the line of log printing in
   print_mem_encrypt_feature_info() in kernel just in case people may
   change it in the future.

Signed-off-by: Baoquan He <bhe(a)redhat.com&gt;
---
 kdump-lib.sh | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kdump-lib.sh b/kdump-lib.sh
index 16238c508f65..eadb57a1828b 100755
--- a/kdump-lib.sh
+++ b/kdump-lib.sh
@@ -84,6 +84,11 @@ is_generic_fence_kdump()
 	[[ $(kdump_get_conf_val fence_kdump_nodes) ]]
 }
 
+is_sme_or_sev_active()
+{
+	journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD
(SME|SEV)$"
+}
+
 to_dev_name()
 {
 	local dev="${1//\"/}"
@@ -930,7 +935,13 @@ kdump_get_arch_recommend_crashkernel()
 
 	_arch=$(uname -m)
 
-	if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then
+	if [[ $_arch == "x86_64" ]] ; then
+		if is_sme_or_sev_active; then
+			_ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
+		else
+			_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
+		fi
+	elif [[ $_arch == "s390x" ]]; then
 		_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
 	elif [[ $_arch == "aarch64" ]]; then
 		local _running_kernel
-- 
2.41.0

[Philipp Rudo]

Hi Baoquan,

On Mon, 18 Sep 2023 12:17:56 +0800
Baoquan He <bhe(a)redhat.com&gt; wrote:

>  It's reported that kdump kernel failed to boot and can't dump
> vmcore
>  when crashkernel=192M and SME/SEV is active.
>  
>  This is because swiotlb will be enabled and reserves 64M memory by
>  default on system with SME/SEV enabled. Then kdump kernel will be out of
>  memory after taking 64M away for swiotlb init.
>  
>  So here add extra 64M memory to default crashkernel value so that kdump
>  kernel can function well as before. When doing that, search journalctl
>  for the "Memory Encryption Features active: AMD" to check if SME or SEV
>  is active. This line of log is printed out in kernel function as below
>  and the type SME is mutual exclusive with type SEV.
>    ***:
>    arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
>  
>  Note:
>  1) The conditional check is relying on journalctl log because I didn't
>     find available system interface to check if SEV is active. Even
>     though we can check if SME is active via /proc/cpuinfo. For
>     consistency, I take the same check for both SME and SEV by searching
>     journalctl.
>  
>  2) The conditional check is relying on journalctl log, means it won't
>     work for crashkernel setting in anoconda because the installation
>     kernel doesn't have the SME/SEV setting. So customer need manually
>     run 'kdumpctl reset-crashkernel' to reset crashkernel to add the
>     extra 64M after OS installation.
>  
>  3) We need watch the line of log printing in
>     print_mem_encrypt_feature_info() in kernel just in case people may
>     change it in the future.
>  
>  Signed-off-by: Baoquan He <bhe(a)redhat.com&gt;
>  ---
>   kdump-lib.sh | 13 ++++++++++++-
>   1 file changed, 12 insertions(+), 1 deletion(-)
>  
>  diff --git a/kdump-lib.sh b/kdump-lib.sh
>  index 16238c508f65..eadb57a1828b 100755
>  --- a/kdump-lib.sh
>  +++ b/kdump-lib.sh
>  @@ -84,6 +84,11 @@ is_generic_fence_kdump()
>   	[[ $(kdump_get_conf_val fence_kdump_nodes) ]]
>   }
>   
>  +is_sme_or_sev_active()
>  +{
>  +	journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD
> (SME|SEV)$"
>  +}
>  +
>   to_dev_name()
>   {
>   	local dev="${1//\"/}"
>  @@ -930,7 +935,13 @@ kdump_get_arch_recommend_crashkernel()
>   
>   	_arch=$(uname -m)
>   
>  -	if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then
>  +	if [[ $_arch == "x86_64" ]] ; then
>  +		if is_sme_or_sev_active; then
>  +			_ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
>  +		else
>  +			_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
>  +		fi
>  +	elif [[ $_arch == "s390x" ]]; then
>   		_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
>   	elif [[ $_arch == "aarch64" ]]; then
>   		local _running_kernel 
Like I've said in v1. Rather than hard coding the crashkernel value
with an additional 64M I think it would be much better if you copy what
Pingfan introduced for arm. IMHO this makes the code less error prone
and lets us increase the crashkernel value for any weird combination of
reasons. In particular the last hunk should look like below

Thanks
Philipp

diff --git a/kdump-lib.sh b/kdump-lib.sh
index 042ac87..4dd800f 100755
--- a/kdump-lib.sh
+++ b/kdump-lib.sh
@@ -980,6 +980,7 @@ _crashkernel_add()
 kdump_get_arch_recommend_crashkernel()
 {
        local _arch _ck_cmdline _dump_mode
+       local _delta=0
 
        if [[ -z "$1" ]]; then
                if is_fadump_capable; then
@@ -995,9 +996,9 @@ kdump_get_arch_recommend_crashkernel()
 
        if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then
                _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
+               is_sme_or_sev_active && ((_delta += 64))
        elif [[ $_arch == "aarch64" ]]; then
                local _running_kernel
-               local _delta=0
 
                # Base line for 4K variant kernel. The formula is based on x86 plus extra
= 64M
                _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
@@ -1021,7 +1022,6 @@ kdump_get_arch_recommend_crashkernel()
                        #4k kernel, mlx5 consumes extra 124M memory, and choose 150M
                        has_mlx5 && ((_delta += 150))
                fi
-               _ck_cmdline=$(_crashkernel_add "$_ck_cmdline"
"${_delta}M")
        elif [[ $_arch == "ppc64le" ]]; then
                if [[ $_dump_mode == "fadump" ]]; then
                       
_ck_cmdline="4G-16G:768M,16G-64G:1G,64G-128G:2G,128G-1T:4G,1T-2T:6G,2T-4T:12G,4T-8T:20G,8T-16T:36G,16T-32T:64G,32T-64T:128G,64T-:180G"
@@ -1030,7 +1030,7 @@ kdump_get_arch_recommend_crashkernel()
                fi
        fi
 
-       echo -n "$_ck_cmdline"
+       echo -n "$(_crashkernel_add "$_ck_cmdline"
"${_delta}M")"
 }
 
 # return recommended size based on current system RAM size

[Baoquan He]

On 09/19/23 at 08:29pm, Philipp Rudo wrote:
>  Hi Baoquan,
>  
>  On Mon, 18 Sep 2023 12:17:56 +0800
>  Baoquan He <bhe(a)redhat.com&gt; wrote:
>  
>  > It's reported that kdump kernel failed to boot and can't dump vmcore
>  > when crashkernel=192M and SME/SEV is active.
>  > 
>  > This is because swiotlb will be enabled and reserves 64M memory by
>  > default on system with SME/SEV enabled. Then kdump kernel will be out of
>  > memory after taking 64M away for swiotlb init.
>  > 
>  > So here add extra 64M memory to default crashkernel value so that kdump
>  > kernel can function well as before. When doing that, search journalctl
>  > for the "Memory Encryption Features active: AMD" to check if SME or SEV
>  > is active. This line of log is printed out in kernel function as below
>  > and the type SME is mutual exclusive with type SEV.
>  >   ***:
>  >   arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
>  > 
>  > Note:
>  > 1) The conditional check is relying on journalctl log because I didn't
>  >    find available system interface to check if SEV is active. Even
>  >    though we can check if SME is active via /proc/cpuinfo. For
>  >    consistency, I take the same check for both SME and SEV by searching
>  >    journalctl.
>  > 
>  > 2) The conditional check is relying on journalctl log, means it won't
>  >    work for crashkernel setting in anoconda because the installation
>  >    kernel doesn't have the SME/SEV setting. So customer need manually
>  >    run 'kdumpctl reset-crashkernel' to reset crashkernel to add the
>  >    extra 64M after OS installation.
>  > 
>  > 3) We need watch the line of log printing in
>  >    print_mem_encrypt_feature_info() in kernel just in case people may
>  >    change it in the future.
>  > 
>  > Signed-off-by: Baoquan He <bhe(a)redhat.com&gt;
>  > ---
>  >  kdump-lib.sh | 13 ++++++++++++-
>  >  1 file changed, 12 insertions(+), 1 deletion(-)
>  > 
>  > diff --git a/kdump-lib.sh b/kdump-lib.sh
>  > index 16238c508f65..eadb57a1828b 100755
>  > --- a/kdump-lib.sh
>  > +++ b/kdump-lib.sh
>  > @@ -84,6 +84,11 @@ is_generic_fence_kdump()
>  >  	[[ $(kdump_get_conf_val fence_kdump_nodes) ]]
>  >  }
>  >  
>  > +is_sme_or_sev_active()
>  > +{
>  > +	journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD
> (SME|SEV)$"
>  > +}
>  > +
>  >  to_dev_name()
>  >  {
>  >  	local dev="${1//\"/}"
>  > @@ -930,7 +935,13 @@ kdump_get_arch_recommend_crashkernel()
>  >  
>  >  	_arch=$(uname -m)
>  >  
>  > -	if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then
>  > +	if [[ $_arch == "x86_64" ]] ; then
>  > +		if is_sme_or_sev_active; then
>  > +			_ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
>  > +		else
>  > +			_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
>  > +		fi
>  > +	elif [[ $_arch == "s390x" ]]; then
>  >  		_ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
>  >  	elif [[ $_arch == "aarch64" ]]; then
>  >  		local _running_kernel
>  
>  Like I've said in v1. Rather than hard coding the crashkernel value
>  with an additional 64M I think it would be much better if you copy what
>  Pingfan introduced for arm. IMHO this makes the code less error prone
>  and lets us increase the crashkernel value for any weird combination of
>  reasons. In particular the last hunk should look like below 
Ah, I didn't scroll down your earlier reply. It's fine to me. Will
post v3 to include it. Thanks.

>  
>  Thanks
>  Philipp
>  
>  diff --git a/kdump-lib.sh b/kdump-lib.sh
>  index 042ac87..4dd800f 100755
>  --- a/kdump-lib.sh
>  +++ b/kdump-lib.sh
>  @@ -980,6 +980,7 @@ _crashkernel_add()
>   kdump_get_arch_recommend_crashkernel()
>   {
>          local _arch _ck_cmdline _dump_mode
>  +       local _delta=0
>   
>          if [[ -z "$1" ]]; then
>                  if is_fadump_capable; then
>  @@ -995,9 +996,9 @@ kdump_get_arch_recommend_crashkernel()
>   
>          if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then
>                  _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
>  +               is_sme_or_sev_active && ((_delta += 64))
>          elif [[ $_arch == "aarch64" ]]; then
>                  local _running_kernel
>  -               local _delta=0
>   
>                  # Base line for 4K variant kernel. The formula is based on x86 plus extra
> = 64M
>                  _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
>  @@ -1021,7 +1022,6 @@ kdump_get_arch_recommend_crashkernel()
>                          #4k kernel, mlx5 consumes extra 124M memory, and choose 150M
>                          has_mlx5 && ((_delta += 150))
>                  fi
>  -               _ck_cmdline=$(_crashkernel_add "$_ck_cmdline"
> "${_delta}M")
>          elif [[ $_arch == "ppc64le" ]]; then
>                  if [[ $_dump_mode == "fadump" ]]; then
>                         
> _ck_cmdline="4G-16G:768M,16G-64G:1G,64G-128G:2G,128G-1T:4G,1T-2T:6G,2T-4T:12G,4T-8T:20G,8T-16T:36G,16T-32T:64G,32T-64T:128G,64T-:180G"
>  @@ -1030,7 +1030,7 @@ kdump_get_arch_recommend_crashkernel()
>                  fi
>          fi
>   
>  -       echo -n "$_ck_cmdline"
>  +       echo -n "$(_crashkernel_add "$_ck_cmdline"
> "${_delta}M")"
>   }
>   
>   # return recommended size based on current system RAM size
>