[fedora-java] [Crash-catcher] java reports by crash catcher

Wed Oct 13 14:47:11 UTC 2010

On 10/12/2010 02:30 PM, Jiri Moskovcak wrote:
> On 10/11/2010 04:03 PM, Mark Wielaard wrote:
>> Looks like the java-devel list disgarded any email from non-subscribers
>> (so that included my initial email and your reply). Hope that is fixed
>> now. I left all text in the email, even though my reply consists of only
>> on tiny paragraph so others can catch up on the discussion.
>>
>> On Mon, 2010-10-11 at 15:32 +0200, Jiri Moskovcak wrote:
>>> On 10/11/2010 03:19 PM, Mark Wielaard wrote:
>>>> Hi Jiri,
>>>>
>>>> On Wed, 2010-10-06 at 10:11 +0200, Jiri Moskovcak wrote:
>>>>> what about resurrecting this feature for F15? We made some changes to
>>>>> abrt so it uses socket instead of the helper app, so we can send you the
>>>>> info on how to use it if you'd be interested in implementing it to javavm.
>>>>
>>>> I currently don't have the time to work on this, but do think they are
>>>> good ideas that would improve abrt a lot for java/jvm based packages.
>>>>
>>>> I added Fedora Java Developers list to the CC in the hope someone would
>>>> be interested. So if someone is looking for a fun (python!) hack, this
>>>> might be interesting.
>>>>
>>> Actually it would be probably more C fun.
>>>
>>>> One idea is that abrt/crash-catcher creates a lot of bugzilla reports
>>>> against the jvm package. Those do include a native backtrace, but don't
>>>> include the (often far more useful) hs_err_pid.log file. It would be
>>>> nice if abrt would find it and offer to automagically attach it to the
>>>> bug report.
>>>
>>> Attaching the file is not a problem, but does it live in some
>>> predictable location?
>>
>> It lives in the current working directory of the JVM. In theory one
>> could hack the hotspot sources to place it somewhere else. But what
>> would be a good place?
>>
>
> ABRT knows the cwd and the pid of the crashing process so it should be
> able to read hss_err_<pid>.log file I will play with it.
>

The problem with <CWD> is that it could be different every time and even 
if it's writable for JVM it could be not-readable for ABRT (like $HOME) 
and makes SELinux to complain. So what about /var/log/openjdk/ or (not 
sure how nice it is..) jvm could open stderr write the log there and not 
close it, the abrt handler could open /proc/<pid>/2 and read the log 
from there (not sure how secure this is and how selinux will like it...)

>>>> The other idea discussed was when a java program exits through an
>>>> uncaught exception in the main thread. In that case you might want to
>>>> catch that and create a bug report against the package that contains the
>>>> main class file (instead of against the jvm package).
>>>>
>>>
>>> This one is actually what I'd like to be done in F15.
>>>
>>>> The first idea is probably the least work and has the most benefit in
>>>> the short term (at least for the java-1.6.0-openjdk bug maintainers).
>>>>
>>>>> On 11/16/2009 09:43 AM, Jiri Moskovcak wrote:
>>>>>> On 11/13/2009 11:00 AM, Mark Wielaard wrote:
>>>>>>> On Wed, 2009-11-11 at 11:05 +0100, Jiri Moskovcak wrote:
>>>>>>>> On 11/05/2009 05:49 PM, Mark Wielaard wrote:
>>>>>>>>> The code is already setup to save if in a different place if necessary
>>>>>>>>> (in fact if the current directory isn't writable for the user it will
>>>>>>>>> try saving in /tmp). If /var/log/java is made writable for all users
>>>>>>>>> that could be a place to dump the log also.
>>>>>>>>>
>>>>>>>>> The code can also be modified to actually call abrt (or an helper
>>>>>>>>> executable/script) if necessary either with the path of the log file or
>>>>>>>>> even with an open file descriptor to the log.
>>>>>>>>>
>>>>>>>> We use a helper to handle the python logs, so I think we could use the
>>>>>>>> same helper for saving the java logs.
>>>>>>>
>>>>>>> Do you have a pointer to the code for this helper?
>>>>>>
>>>>>> http://git.fedorahosted.org/git/abrt.git?p=abrt.git;a=blob;f=src/Hooks/abrt-pyhook-helper.cpp;h=348fbc72bd12b3b0f6757bac69a44e725706cf5f;hb=HEAD
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>>> The other thing I have in mind is how to catch an unhandled
>>>>>>>>>> exception in
>>>>>>>>>> java programs, because in this case the VM exits "normally" and abrt
>>>>>>>>>> can't detect it. We managed to catch these exception in python by
>>>>>>>>>> overriding the default exception handler by script that is
>>>>>>>>>> automatically
>>>>>>>>>> loaded everytime when python VM is started. If there would be some way
>>>>>>>>>> to this for java we could wire this to ABRT.
>>>>>>>>>
>>>>>>>>> In principal we could install some uncaught exception handler, but
>>>>>>>>> uncaught exceptions might not be fatal (although they are admittedly
>>>>>>>>> sloppy). The program may even happily run even if one thread has an
>>>>>>>>> uncaught exception (as long as there are other non-daemon threads
>>>>>>>>> running).
>>>>>>>>>
>>>>>>>> ABRT doesn't care if it is or isn't fatal, this is up to your exception
>>>>>>>> handler - you can just log the exception using the abrt helper (because
>>>>>>>> even if the exception is not fatal, it's usually a bug..) and contiune
>>>>>>>> running the program.
>>>>>>>
>>>>>>> This part is trickier than the above. In case of a JVM crash there is a
>>>>>>> clear point where we catch that crash and produce the necessary logs for
>>>>>>> a bug report. In case of an application specific uncaught exception
>>>>>>> there is an uncaught exception handler mechanism, but the application
>>>>>>> could already be using it (either for a specific Thread, the ThreadGroup
>>>>>>> or system wide). This might require some surgery to get right (and
>>>>>>> unobtrusive for the application running on the JVM).
>>>>>>>
>>>>>>
>>>>>> This is really up to you as I don't know much about about insides of JVM.
>>>>>>
>>>>>>>>> How would you determine which package the exception belongs to? For
>>>>>>>>> a VM
>>>>>>>>> crash it is almost always the java VM package that should get the bug
>>>>>>>>> report (since the VM just shouldn't crash ever).
>>>>>>>>
>>>>>>>> If I'm right the Java VM is compiled, so it creates a coredump and would
>>>>>>>> be handled by a different hook then scripts, but that applies only if
>>>>>>>> you don't catch the sigsegv, sigabrt (whichI think you do, to create the
>>>>>>>> logs..) and let it die.
>>>>>>>
>>>>>>> Yes, the VM catches fatal signals and creates an hs_err log file based
>>>>>>> on the information it can still retrieve at that point before dying.
>>>>>>>
>>>>>>
>>>>>> Ok, so we need to find the way how to pass this logs to abrt hook.
>>>>>>
>>>>>>>> I think, as Java VM is a non-trivial programme,
>>>>>>>> we should write a special handler for it, or we can try to improve the
>>>>>>>> general hook for compiled programs to be able to handle some additional
>>>>>>>> data as the log file if that would be enough.
>>>>>>>>
>>>>>>>>> But for uncaught
>>>>>>>>> exceptions reporting it against the java VM package is definitely the
>>>>>>>>> wrong thing to do. How do you solve that in the python case?
>>>>>>>>>
>>>>>>>>
>>>>>>>> The python exception hook is run in the context of the running script,
>>>>>>>> so it knows the script name and the path to the script and then we can
>>>>>>>> simply run $ rpm -qf /path/to/script to determine the package, the code
>>>>>>>> to do this is:
>>>>>>>>
>>>>>>>> executable = os.path.abspath(sys.argv[0])
>>>>>>>
>>>>>>> Aha. I think we could determine the main class that is being run and the
>>>>>>> classpath with .jar/.zip files that this class comes from. With that we
>>>>>>> could probably achieve similar heuristics about the package that
>>>>>>> provided the classes.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Mark
>>
>
> _______________________________________________
> Crash-catcher mailing list
> Crash-catcher at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/crash-catcher