Hi, we have made (together with Karel) some todo list for retrace server. If you find any part interesting, let us know, but you are basically free to go :).
At the moment, I'm working on: Task cleanup Reporting statistics Incomplete backtraces - a little HTTPS certificates In the future, I'd like to work on remote GDB sessions.
Michal
Here's the list:
--- What NEEDS to be done for F15 ---
* CLI - Command line tool providing retrace client functionality: create new task, ask about status, log, backtrace. * GUI - Integrate retrace server with ABRT GUI. * Task cleanup - A tool cleaning /var/spool/abrt-retrace directory from tasks older than some threshold (5 days?). Periodically called from cron. * Trusted HTTPS certificates - Get trusted HTTPS certificates from Fedora infrastructure. * Incomplete backtraces - Find out why some packages do not generate full backtraces and deal with the differences between backtraces generated on F14 and RHEL6 hosts. * Reporting statistics - Retrace worker should save some statistics. What information exactly do we want to save? (package name, duration, version, parallel tasks?) * Manual - Convert design document to manual. * Security audit - Look the code for possible security holes - running file contents in shell etc. :).
--- What SHOULD be done in the future ---
* Only use coredump - Do not require additional files (architecture, package etc.), coredump should contain all necessary information. * Support different archive formats - Accept gz, bz2 (...?) comressed archives and also archives with no compression - fastest on LAN. * Calculate estimated time - When enough statistics are gathered, try to estimate retrace duration and send it to the client. Client does not need to ask about status before. * Support keep-alive HTTP connection - Status script should not close HTTP connection, but send task status to the client every ~10 seconds. The connection is closed once retrace job is complete. * Handle specific crashes (gdb, libc etc.) - Handle crashes from binaries and libraries required by the retrace server itself. * Coredump stripping - Only send some smaller part of (possibly large) coredump, the one really required by GDB. * Write SELinux policy - Do we really need it? * Support rawhide - Deal with the problem of enormous number of packages. * User authentication - Mostly for corporate use, every retrace job has an owner, who is able to re-run, delete, modify... it. Impossible in Fedora. * Multiarch support - Support multiple architectures (i386, x86_64, ppc, ppc64 etc.) on the same machine. * Providing GDB debugging sessions - Allow users to debug their crashes remotely. * Statistics cleanup - Delete old statistics from stats database. Do we really need it? * 3rd party packages support - Allow users to add their own packages and debuginfos to the retrace request. Allow also to process non-official repositories like rpmfusion? * WebUI - Provide the statistics and even the possibility of uploading coredump from web browser. Do we really need it? * Speed optimizations: ** RPM package stripping - Discard all unnecessary information from RPMs. Store only binaries, libraries and debuginfos. Would be very useful. ** Jobs planning - Change the priority of retrace jobs to gain better efficiency. ** Updates-testing cleanup - Do not store packages in updates and updates-testing repositories twice. ** Integrate with DebuginfoFS - Once DebuginfoFS is complete.
On 20.1.2011 17:08, Michal Toman wrote:
--- What NEEDS to be done for F15 ---
- CLI - Command line tool providing retrace client functionality: create
new task, ask about status, log, backtrace.
I am currently working on this. New version will reach git soon, it actually starts to work correctly.
- GUI - Integrate retrace server with ABRT GUI.
- Task cleanup - A tool cleaning /var/spool/abrt-retrace directory from
tasks older than some threshold (5 days?). Periodically called from cron.
- Trusted HTTPS certificates - Get trusted HTTPS certificates from
Fedora infrastructure.
- Incomplete backtraces - Find out why some packages do not generate
full backtraces and deal with the differences between backtraces generated on F14 and RHEL6 hosts.
I'll look at this too, as it's important to get it fixed soon. We want to convince people to use the server to get better backtraces, but currently the server is not good in that.
- Reporting statistics - Retrace worker should save some statistics.
What information exactly do we want to save? (package name, duration, version, parallel tasks?)
- Manual - Convert design document to manual.
And we should add Deployment section. This is my obligation, but I'll be able to work on it only sometimes in February.
- Security audit - Look the code for possible security holes - running
file contents in shell etc. :).
We should fix obvious issues as soon as possible. However, let's discuss them privately :)
--- What SHOULD be done in the future ---
- Only use coredump - Do not require additional files (architecture,
package etc.), coredump should contain all necessary information.
We need this, but not for Fedora 15. I'll add the support for this to the client now, and the server can be extended later.
- Support different archive formats - Accept gz, bz2 (...?) comressed
archives and also archives with no compression - fastest on LAN.
We need the "no compression" option, but not for Fedora 15. I'll add the support for this to the client now, and the server can be extended later.
- Calculate estimated time - When enough statistics are gathered, try to
estimate retrace duration and send it to the client. Client does not need to ask about status before.
- Support keep-alive HTTP connection - Status script should not close
HTTP connection, but send task status to the client every ~10 seconds. The connection is closed once retrace job is complete.
- Handle specific crashes (gdb, libc etc.) - Handle crashes from
binaries and libraries required by the retrace server itself.
- Coredump stripping - Only send some smaller part of (possibly large)
coredump, the one really required by GDB.
- Write SELinux policy - Do we really need it?
- Support rawhide - Deal with the problem of enormous number of packages.
- User authentication - Mostly for corporate use, every retrace job has
an owner, who is able to re-run, delete, modify... it. Impossible in Fedora.
- Multiarch support - Support multiple architectures (i386, x86_64, ppc,
ppc64 etc.) on the same machine.
- Providing GDB debugging sessions - Allow users to debug their crashes
remotely.
- Statistics cleanup - Delete old statistics from stats database. Do we
really need it?
- 3rd party packages support - Allow users to add their own packages and
debuginfos to the retrace request. Allow also to process non-official repositories like rpmfusion?
- WebUI - Provide the statistics and even the possibility of uploading
coredump from web browser. Do we really need it?
- Speed optimizations:
** RPM package stripping - Discard all unnecessary information from RPMs. Store only binaries, libraries and debuginfos. Would be very useful. ** Jobs planning - Change the priority of retrace jobs to gain better efficiency. ** Updates-testing cleanup - Do not store packages in updates and updates-testing repositories twice. ** Integrate with DebuginfoFS - Once DebuginfoFS is complete.
I think nobody actively works on DebuginfoFS. When done right, it would make "RPM package stripping", "updates-testing cleanup", createrepo, our reposync script unnecessary. It would also help local (no server) ABRT retracing a lot (but it would probably need changes in GDB).
* Add support of Jan Kratochvil's gdbserver
coredump2packages would need to be executed locally, before creating the retrace task, because coredump would not be available on the server.
Karel
On Thu, 20 Jan 2011 17:08:13 +0100, Michal Toman mtoman@redhat.com wrote:
- WebUI - Provide the statistics and even the possibility of uploading
coredump from web browser. Do we really need it?
This part is quite interesting. I can imagine that retrace server can be promote full abrt webui app. If we add webui with accounts(roles) to independently retrace bugs, xmlrpc to communicate to bugzilla (rhtsupport) than we have powerful management tool.
-- Nikola
crash-catcher@lists.fedorahosted.org
-
Karel Klic -
Michal Toman -
Nikola Pajkovsky