Re: Detecting kdump-lib-initramfs.sh errors

[Dave Young]

Hi, Joe

On 02/26/16 at 04:09pm, Joe Lawrence wrote:
> Hi Dave,
>
> Stratus has a question about monitoring various error scenarios out of
> kdump-lib-initramfs.sh.
>
> This file looks to be Red Hat specific (at least it's not in my upstream
> kexec-tools git tree), so I wasn't sure where to direct the question.
> Feel free to point us in the direction of an appropriate mailing list,
> bugzilla, etc.

We are using Fedora kexec mailing list as the upstream of kdump scripts.
see below page:
https://lists.fedoraproject.org/admin/lists/kexec.lists.fedoraproject.org/
> Basically we're interested in reporting kdump progress over IPMI,
> including the mounting / creation of kdump directories, vmcore-dmesg and
> vmcore files.
>
> We could obviously modify kdump-lib-initramfs.sh directly to accomplish
> this [1], but we were wondering if there was a cleaner approach -- for
> example, functions in kdump-lib-initramfs.sh could return status or log
> errors somewhere that we could read later.
>
> We would gladly prototype and test examples if this is something that
> could be reasonably supported.
>
> [1] https://paste.fedoraproject.org/330114/

The better way is to send patches to [e-mail]
so that we can review and discuss it.

The latest code stays in Fedora kexec git tree, you can clone the tree by:
fedpkg clone -a kexec-tools

Thanks
Dave

Detecting kdump-lib-initramfs.sh errors

[Joe Lawrence]

On 02/28/2016 08:37 PM, Dave Young wrote:
> Hi, Joe
>
> On 02/26/16 at 04:09pm, Joe Lawrence wrote:
> > Hi Dave,
> >
> > Stratus has a question about monitoring various error scenarios out of
> > kdump-lib-initramfs.sh.
> >
> > This file looks to be Red Hat specific (at least it's not in my upstream
> > kexec-tools git tree), so I wasn't sure where to direct the question.
> > Feel free to point us in the direction of an appropriate mailing list,
> > bugzilla, etc.
>
> We are using Fedora kexec mailing list as the upstream of kdump scripts.
> see below page:
> https://lists.fedoraproject.org/admin/lists/kexec.lists.fedoraproject.org/

Thanks, Dave, and hello fedora kexec list.
> > Basically we're interested in reporting kdump progress over IPMI,
> > including the mounting / creation of kdump directories, vmcore-dmesg and
> > vmcore files.
> >
> > We could obviously modify kdump-lib-initramfs.sh directly to accomplish
> > this [1], but we were wondering if there was a cleaner approach -- for
> > example, functions in kdump-lib-initramfs.sh could return status or log
> > errors somewhere that we could read later.
> >
> > We would gladly prototype and test examples if this is something that
> > could be reasonably supported.
> >
> > [1] https://paste.fedoraproject.org/330114/
>
> The better way is to send patches to [e-mail]
> so that we can review and discuss it.
>
> The latest code stays in Fedora kexec git tree, you can clone the tree by:
> fedpkg clone -a kexec-tools

Got it.  I can see the kdump-lib-initramfs.sh file now in the fedora
variant of the kexec-tools tree.

As I mentioned above, we were interested in having a few kdump process
mile markers logged to help us debug no-dump-no-clue situations.  The
pastebin link[1] is an example of directly modifying
kdump-lib-initramfs.sh to accomplish this for the progress and specific
dump action we support.

Looking more into the code, there are a lot of potential points of
failure along the way, so perhaps we are asking for something too
specific to be of general use here.  We can edit kdump-lib-initramfs.sh
post kexec-tools install to accomplish what we want, but that would open
potential package upgrade/verification/etc. can of worms.

Is there a cleaner way of extracting such status from the kdump
(library) scripts?  Ideally the kdump_post script could be empowered
with this information and then the user could do whatever they want with it.

Thanks,

-- Joe

Detecting kdump-lib-initramfs.sh errors

[Dave Young]

On 02/29/16 at 02:10pm, Joe Lawrence wrote:
> On 02/28/2016 08:37 PM, Dave Young wrote:
> > Hi, Joe
> >
> > On 02/26/16 at 04:09pm, Joe Lawrence wrote:
> > > Hi Dave,
> > >
> > > Stratus has a question about monitoring various error scenarios out of
> > > kdump-lib-initramfs.sh.
> > >
> > > This file looks to be Red Hat specific (at least it's not in my upstream
> > > kexec-tools git tree), so I wasn't sure where to direct the question.
> > > Feel free to point us in the direction of an appropriate mailing list,
> > > bugzilla, etc.
> >
> > We are using Fedora kexec mailing list as the upstream of kdump scripts.
> > see below page:
> > https://lists.fedoraproject.org/admin/lists/kexec.lists.fedoraproject.org/
>
> Thanks, Dave, and hello fedora kexec list.
> > > Basically we're interested in reporting kdump progress over IPMI,
> > > including the mounting / creation of kdump directories, vmcore-dmesg and
> > > vmcore files.
> > >
> > > We could obviously modify kdump-lib-initramfs.sh directly to accomplish
> > > this [1], but we were wondering if there was a cleaner approach -- for
> > > example, functions in kdump-lib-initramfs.sh could return status or log
> > > errors somewhere that we could read later.
> > >
> > > We would gladly prototype and test examples if this is something that
> > > could be reasonably supported.
> > >
> > > [1] https://paste.fedoraproject.org/330114/
> >
> > The better way is to send patches to [e-mail]
> > so that we can review and discuss it.
> >
> > The latest code stays in Fedora kexec git tree, you can clone the tree by:
> > fedpkg clone -a kexec-tools
>
> Got it.  I can see the kdump-lib-initramfs.sh file now in the fedora
> variant of the kexec-tools tree.
>
> As I mentioned above, we were interested in having a few kdump process
> mile markers logged to help us debug no-dump-no-clue situations.  The
> pastebin link[1] is an example of directly modifying
> kdump-lib-initramfs.sh to accomplish this for the progress and specific
> dump action we support.
>
> Looking more into the code, there are a lot of potential points of
> failure along the way, so perhaps we are asking for something too
> specific to be of general use here.  We can edit kdump-lib-initramfs.sh
> post kexec-tools install to accomplish what we want, but that would open
> potential package upgrade/verification/etc. can of worms.
>
> Is there a cleaner way of extracting such status from the kdump
> (library) scripts?  Ideally the kdump_post script could be empowered
> with this information and then the user could do whatever they want with it.

In kdump scripts we can do limited trace for example add a functionality like
dump_status() it will print the status to console by default.

$1: message to print
dump_status() {
  print the status to console;
  if a status facility is registered; then
  call the registerd script to print the status to somewhere else.
  fi 
}

save_vmcore_dmesg_fs() {
 dump_status("save_vmcore_dmesg_fs: begin")
 [snip]
 dump_status("save_vmcore_dmesg_fs: finished")
}

Ditto for other save_vmcore functions...

A problems with the design is:
a) if the status dumping hangs for some reason, kdump will hang as well
b) should we check the status dumping return value?
c) how to register the facility to kdump.

For c) I think add it as a dracut module will be better, reason is kdump script
only take care of dumping. Mounting filesystem, bringing up network etc are
handled in dracut. So to add more hooks to dracut and provide a dracut module
will be better. Kdump can just use the facility directly.

The dracut upstream mail list is [e-mail]

Thanks
Dave