Re: wiki/code updates
by Adam Stokes
im curious if adding this sort of check would be to much overhead and cause cas to stall for an unspecified amount of time.. unless we just put an option in the config file to point to a 32bit binary?
any suggestions?
thanks
----- Original Message -----
From: sdodson(a)sdodson.com
To: "core analysis system" <cas(a)lists.fedorahosted.org>
Sent: Monday, December 15, 2008 11:14:15 AM GMT -05:00 Columbia
Subject: Re: wiki/code updates
On Mon, Dec 15, 2008 at 10:56:19AM -0500, Jon Stanley wrote:
> On Mon, Dec 15, 2008 at 8:21 AM, Adam Stokes <astokes(a)redhat.com> wrote:
> > Thanks to some feedback I've went ahead and pushed code in to allow for local processing of core files through cas. If the machine architecture doesn't match up with the corefile it will attempt to use func if available, otherwise complete the job without processing. Because of this func is no longer required, but optional :)
>
> Stupid question - an x86_64 machine could process an i686 core,
> correct? I wouldn't see any reason why not.
You'd just need to install the i686 version of crash somewhere and
use that. I've done this by installing to /usr/local/i386 with the
--relocate option for rpm.
We'd have to add the ability for CAS to use alternate paths.
--
Scott
_______________________________________________
cas mailing list
cas(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/cas
--
__ __
.-----.| |_.-----.| |--.-----.-----.
|__ --|| _| _ || <| -__|__ --|
|_____||____|_____||__|__|_____|_____|
.----------------------------------.
( ajs(a)redhat.com || 919.754.4187 )
`----------------------------------'
15 years, 4 months
wiki/code updates
by Adam Stokes
Thanks to some feedback I've went ahead and pushed code in to allow for local processing of core files through cas. If the machine architecture doesn't match up with the corefile it will attempt to use func if available, otherwise complete the job without processing. Because of this func is no longer required, but optional :)
Some wiki updates, mainly to the how to use cas, https://fedorahosted.org/cas/wiki/HowToUseCas and I've also got a page on building from source which is what I recommend since releases aren't pushed out quite as often.
https://fedorahosted.org/cas/wiki/BuildingFromSource
Please post any questions, comments you have here
--
__ __
.-----.| |_.-----.| |--.-----.-----.
|__ --|| _| _ || <| -__|__ --|
|_____||____|_____||__|__|_____|_____|
.----------------------------------.
( ajs(a)redhat.com || 919.754.4187 )
`----------------------------------'
15 years, 4 months
cas cas.spec lib/cas version
by Adam Stokes
cas | 14 ++++++++++++--
cas.spec | 2 +-
lib/cas/util.py | 2 +-
version | 2 +-
4 files changed, 15 insertions(+), 5 deletions(-)
New commits:
commit aa7943b13ae2c90742cfaacd3581a45eb93c5a3a
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Mon Dec 15 08:14:14 2008 -0500
- catch smtplib socket error since no exception is provided
(that i could find at least)
- still working out how to handle logging properly
diff --git a/cas b/cas
index 96510b3..307bc04 100755
--- a/cas
+++ b/cas
@@ -78,7 +78,9 @@ class TimestampHandler(object):
self.tool = CoreBase()
def run(self):
- cprint("Running timestamp")
+ # dig through the buildstamp database and attempt to match it with the
+ # one found in the core
+ cprint("Running timestamp on %s" % (self.corefile,))
rpmDB = self.util.load(RPMS)
try:
coreTimestamp = self.tool.timestamp(self.corefile)
@@ -101,6 +103,8 @@ class CasApplication(object):
self.rpmTool = RPMBase()
def parse_options(self, args):
+ # build option - arguement list in the form of
+ # cas -i <id> -f <filename> -m user(a)example.com
parser = optparse.OptionParser(usage="cas [opts] args")
parser.add_option("-i","--identifier", dest="identifier",
help="Unique ID for core")
@@ -213,7 +217,13 @@ class CasApplication(object):
crashOutFH = open(crashOutFile,'r')
msg += crashOutFH.read()
crashOutFH.close()
- mailServer = smtplib.SMTP(SMTPHOST)
+ try:
+ # for some reason smtplib doesn't have proper exception handling
+ # for name or service not known :(
+ mailServer = smtplib.SMTP(SMTPHOST)
+ except:
+ dprint("Unable to connect to mail server: %s, no email " \
+ "results sent." % (SMTPHOST,))
mailServer.set_debuglevel(0)
mailServer.sendmail(self.email,self.email,msg)
mailServer.quit()
diff --git a/cas.spec b/cas.spec
index b7ee24a..a516164 100644
--- a/cas.spec
+++ b/cas.spec
@@ -40,7 +40,7 @@ rm -rf ${RPM_BUILD_ROOT}
%{python_sitelib}/*
%{_mandir}/man1/cas.1.gz
%{_mandir}/man1/cas-admin.1.gz
-%doc LICENSE README
+%doc LICENSE README PKG-INFO
%changelog
* Mon Dec 15 2008 Adam Stokes <ajs at redhat dot com> - 0.13-94
diff --git a/lib/cas/util.py b/lib/cas/util.py
index 97a02b8..8086993 100755
--- a/lib/cas/util.py
+++ b/lib/cas/util.py
@@ -36,7 +36,7 @@ fh.setFormatter(fh_fmt)
def dprint(msg, debug=True):
if debug:
log.setLevel(logging.DEBUG)
- log.debug(msg)
+ log.debug("[.cas.][debug] :: " +msg)
def sprint(msg):
""" function to print status messages
diff --git a/version b/version
index 281a13a..88e3999 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 95
+0.13 97
15 years, 4 months
cas.spec version
by Adam Stokes
cas.spec | 25 ++++++++++++++++++++-----
version | 2 +-
2 files changed, 21 insertions(+), 6 deletions(-)
New commits:
commit 2fff9acb1c912e7a3bb5e2517776b5da4f69588e
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Mon Dec 15 07:55:33 2008 -0500
- spec file update
diff --git a/cas.spec b/cas.spec
index 007d03f..b7ee24a 100644
--- a/cas.spec
+++ b/cas.spec
@@ -1,7 +1,7 @@
-%define name cas
+%{!?python_sitelib: %define python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")}
+Name: cas
Summary: Tool to analyze and configure core file environment.
-Name: %{name}
Source1: version
Version: %(echo `awk '{print $1 }' %{SOURCE1}`)
Release: %(echo `awk '{print $2 }' %{SOURCE1}`)%{?dist}
@@ -23,35 +23,50 @@ boxes, finding machines suitable for a particular core is now taken care of.
%setup -q
%build
-python setup.py build
+%{__python} setup.py build
%install
rm -rf ${RPM_BUILD_ROOT}
-python setup.py install --optimize 1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
+%{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
%clean
rm -rf ${RPM_BUILD_ROOT}
-%files -f INSTALLED_FILES
+%files
%defattr(-,root,root,-)
+%config(noreplace) %{_sysconfdir}/cas.conf
+%{_bindir}/cas
+%{_bindir}/cas-admin
+%{python_sitelib}/*
+%{_mandir}/man1/cas.1.gz
+%{_mandir}/man1/cas-admin.1.gz
%doc LICENSE README
%changelog
+* Mon Dec 15 2008 Adam Stokes <ajs at redhat dot com> - 0.13-94
+- no replace on config file
+- cas now processes locally and remotely via func
+
* Wed Aug 20 2008 Adam Stokes <ajs at redhat dot com> - 0.13
- Updated build and spec
+
* Mon Feb 10 2008 Scott Dodson <sdodson at redhat dot com> - 0.11
- Minor changes to permissions
+
* Mon Dec 10 2007 Adam Stokes <astokes at redhat dot com> - 0.9.1
- splitting off grabcore to be a download/extract only service
- core of the work to be done specifically by their intended
modules
+
* Fri Dec 7 2007 Adam Stokes <astokes at redhat dot com> - 0.9
- release bump
- decompression module added
+
* Tue Nov 13 2007 Adam Stokes <astokes at redhat dot com> - 0.8
- threading added
- better exception handling
- bug fixes
- added initscripts, service capabilities
+
* Mon Oct 22 2007 Adam Stokes <astokes at redhat dot com> - 0.1
- initial build
diff --git a/version b/version
index 0610003..281a13a 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 94
+0.13 95
15 years, 4 months
2 commits - cas cas.conf lib/cas version
by Adam Stokes
cas | 21 ++++++++++++++-------
cas.conf | 2 +-
lib/cas/util.py | 21 ++++++++++++++-------
version | 2 +-
4 files changed, 30 insertions(+), 16 deletions(-)
New commits:
commit 365648c84335331f6615b700edce90ad17bacfda
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Mon Dec 15 07:39:06 2008 -0500
- working on better logging facilities
diff --git a/cas b/cas
index 8b8dfbd..96510b3 100755
--- a/cas
+++ b/cas
@@ -175,9 +175,9 @@ class CasApplication(object):
# Only necessary for debugging why running of crash failed.
# (sts, out, err) = client_dict[client_dict.keys()[0]]
else:
- cprint("No servers available for arch, please run cas-admin -h " \
- "for more information")
- cprint("Attempting a local query.")
+ cprint("No servers available for arch and current system not "\
+ "suitable for processing, please run cas-admin -h " \
+ "for more information")
else:
cprint("No servers database found, please run cas-admin -h for " \
"more information")
diff --git a/cas.conf b/cas.conf
index 3efe358..950b791 100644
--- a/cas.conf
+++ b/cas.conf
@@ -1,5 +1,5 @@
[settings]
-# Where kernels are stored
+# Where kernel-debuginfo packages are stored
# NOTE: this _can_ include symlinked directories, just be careful they are
# indefinately recursive
kernels=/mnt/kernels
diff --git a/version b/version
index 03dca87..0610003 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 93
+0.13 94
commit 24f2bbf7f21cdffd4f098b2cc32480ba4403bba7
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Mon Dec 15 07:16:49 2008 -0500
- testing runs completed for running cas locally
code cleanup and rework as a result of testing
diff --git a/cas b/cas
index 32cc964..8b8dfbd 100755
--- a/cas
+++ b/cas
@@ -32,7 +32,6 @@ if sys.version_info[:2] < (2,4):
raise SystemExit("Python >= 2.4 required")
# Configuration parsing of /etc/cas.conf
-# TODO: verify configuration is setup properly
config = ConfigParser.ConfigParser()
config.read("/etc/cas.conf")
WORKDIRECTORY = config.get("settings","workDirectory")
@@ -156,7 +155,7 @@ class CasApplication(object):
# with processing the core on the current machine
currentMachineArch = Popen(["uname","-m"], stdout=PIPE, stderr=PIPE)
currentMachineArch = currentMachineArch.stdout.read().strip()
- if not debugKernelArch == currentMachineArch:
+ if debugKernelArch != currentMachineArch:
# The machine running cas isn't capable of processing this core, lets
# attempt with Func. Assuming Func is installed and a server database
# is configured we attempt to process the core at another machine.
@@ -190,10 +189,18 @@ class CasApplication(object):
cprint("Finishing job without processing the core, please find a suitable %s "\
"machine in order to view this core in crash." % (debugKernelArch,))
else:
+ import platform
+ # Define current machine hostname, mainly used for email results.
+ casProcessMachine = platform.uname()[1]
# The machine is suitable for processing the core through crash.
+ cprint("Current machine suitable for processing core, running crash.")
cmd = os.path.join(self.storagePath,"crash")
- # TODO: capture any errors returned from crash when processing core.
- cmdPipe = Popen([cmd],stdout=PIPE, stderr=PIPE)
+ # DONE: capture any errors returned from crash when processing core.
+ cmdPipe = Popen([cmd], stdout=PIPE, stderr=PIPE)
+ cmdData = cmdPipe.communicate()
+ sts, out, err = (cmdPipe.returncode, cmdData[0], cmdData[1])
+ if err:
+ dprint(err, DPRINT)
crashOutFile = os.path.join(self.storagePath,"crash.out")
if os.path.isfile(crashOutFile):
cprint("Crash output processed, sending email to %s" % (self.email,))
diff --git a/lib/cas/util.py b/lib/cas/util.py
index b7bfab5..97a02b8 100755
--- a/lib/cas/util.py
+++ b/lib/cas/util.py
@@ -22,15 +22,21 @@ import logging
from subprocess import Popen, PIPE, call
# setup logging
-logging.basicConfig(level=logging.DEBUG,
- format='%(asctime)s %(levelname)-8s %(message)s',
- filename='/var/log/cas.log',
- filemode='a')
+logFile = '/var/log/cas.log'
+log = logging.getLogger()
+ch = logging.StreamHandler()
+fh = logging.FileHandler(logFile)
+log.addHandler(ch)
+log.addHandler(fh)
+ch_fmt = logging.Formatter("%(message)s")
+fh_fmt = logging.Formatter("%(asctime)s %(process)d (%(levelname)s)\t: %(message)s")
+ch.setFormatter(ch_fmt)
+fh.setFormatter(fh_fmt)
def dprint(msg, debug=True):
if debug:
- print("[.cas.](debug) :: %s" % (msg,))
- logging.debug(msg)
+ log.setLevel(logging.DEBUG)
+ log.debug(msg)
def sprint(msg):
""" function to print status messages
@@ -41,7 +47,8 @@ def sprint(msg):
def cprint(msg):
""" function to print procedure
"""
- print("[.cas.] :: " +msg)
+ log.setLevel(logging.INFO)
+ log.info("[.cas.] :: " +msg)
class UtilException(Exception): pass
diff --git a/version b/version
index 45e95d3..03dca87 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 91
+0.13 93
15 years, 4 months
cas cas.spec
by Adam Stokes
cas | 79 ++++++++++++++++++++++++++++++++++++---------------------------
cas.spec | 1
2 files changed, 46 insertions(+), 34 deletions(-)
New commits:
commit b20a2757b24840d03b5f200b993dcbd3cff22082
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Sun Dec 14 17:48:54 2008 -0500
- removed func requirement
- added code to handle processing the core on the machine cas
is ran from if architecture's match. otherwise we attempt to
use func and in worse case finish job without processing core
through crash
diff --git a/cas b/cas
index a4b15bb..32cc964 100755
--- a/cas
+++ b/cas
@@ -21,6 +21,8 @@ import ConfigParser
import shutil
import smtplib
+from subprocess import Popen, PIPE
+
from cas.network import Download
from cas.core import CoreBase, CoreException
from cas.util import UtilBase, dprint, cprint
@@ -147,40 +149,51 @@ class CasApplication(object):
debugKernel = os.path.abspath(debugKernel)
# setup crash file to finalize the processing of the core file
self.util.buildCrashFile(self.storagePath, corefile, debugKernel)
- # If func is installed we will pre-process the output from crash on
- # a random machine tied to the architecture of the debug file.
- # If no servers exists in your setup then this will return and
- # finish with just a location of the data.
- try:
- import func.overlord.client as fc
- # Pull the architecture from the elf file to match up with a
- # server providing this architecture
- debugKernelArch = self.util.debugKernelArch(debugKernel)
- cprint("Crash file built, locating suitable %s system for " \
- "processing" % (debugKernelArch,))
- if os.path.isfile(SERVERS):
- serverList = self.util.load(SERVERS)
- if serverList.has_key(debugKernelArch):
- # TODO: Randomize server selection
- casProcessMachine = serverList[debugKernelArch][0]
- cprint("Machine %s found, processing crash output" % (casProcessMachine,))
- cmd = os.path.join(self.storagePath,"crash")
- client = fc.Overlord(casProcessMachine)
- client_dict = client.command.run(cmd)
- # Only necessary for debugging why running of crash failed.
- # (sts, out, err) = client_dict[client_dict.keys()[0]]
+ # Pull the architecture from the elf file to match up with a
+ # server providing this architecture
+ debugKernelArch = self.util.debugKernelArch(debugKernel)
+ # Read current machine arch to see if we can bypass func and proceed
+ # with processing the core on the current machine
+ currentMachineArch = Popen(["uname","-m"], stdout=PIPE, stderr=PIPE)
+ currentMachineArch = currentMachineArch.stdout.read().strip()
+ if not debugKernelArch == currentMachineArch:
+ # The machine running cas isn't capable of processing this core, lets
+ # attempt with Func. Assuming Func is installed and a server database
+ # is configured we attempt to process the core at another machine.
+ try:
+ import func.overlord.client as fc
+ cprint("Crash file built, locating suitable %s system for " \
+ "processing" % (debugKernelArch,))
+ if os.path.isfile(SERVERS):
+ serverList = self.util.load(SERVERS)
+ if serverList.has_key(debugKernelArch):
+ # TODO: Randomize server selection
+ casProcessMachine = serverList[debugKernelArch][0]
+ cprint("Machine %s found, processing crash output" % (casProcessMachine,))
+ cmd = os.path.join(self.storagePath,"crash")
+ client = fc.Overlord(casProcessMachine)
+ client_dict = client.command.run(cmd)
+ # Only necessary for debugging why running of crash failed.
+ # (sts, out, err) = client_dict[client_dict.keys()[0]]
+ else:
+ cprint("No servers available for arch, please run cas-admin -h " \
+ "for more information")
+ cprint("Attempting a local query.")
else:
- cprint("No servers available for arch, please run cas-admin -h " \
- "for more information")
- cprint("Continueing without processing core.")
- else:
- cprint("No servers database found, please run cas-admin -h for " \
- "more information")
- sys.exit(1)
- # TODO: Possibly handle this exception more gracefully?
- except ImportError:
- cprint("You need to install func (http://fedorahosted.org/func if you "\
- "wish to have the data processed automatically.")
+ cprint("No servers database found, please run cas-admin -h for " \
+ "more information")
+ sys.exit(1)
+ # DONE: Possibly handle this exception more gracefully?
+ except ImportError:
+ cprint("Current running machine is not suitable for processing this core " \
+ "and http://fedorahosted.org/func is not installed/configured properly.")
+ cprint("Finishing job without processing the core, please find a suitable %s "\
+ "machine in order to view this core in crash." % (debugKernelArch,))
+ else:
+ # The machine is suitable for processing the core through crash.
+ cmd = os.path.join(self.storagePath,"crash")
+ # TODO: capture any errors returned from crash when processing core.
+ cmdPipe = Popen([cmd],stdout=PIPE, stderr=PIPE)
crashOutFile = os.path.join(self.storagePath,"crash.out")
if os.path.isfile(crashOutFile):
cprint("Crash output processed, sending email to %s" % (self.email,))
diff --git a/cas.spec b/cas.spec
index 8d7e14d..007d03f 100644
--- a/cas.spec
+++ b/cas.spec
@@ -13,7 +13,6 @@ BuildArch: noarch
Url: http://fedorahosted.org/cas
BuildRequires: python-devel
Requires: python >= 2.4
-Requires: func
%description
CAS provides a support engineer the ability to configure an environment for
15 years, 4 months
Probably stupid question
by Jon Stanley
So my TAM just told me about cas and that testers are needed. I'm the
token Fedora QA guy. So there's a perfect fit there :)
So what I've got in a CAS rpm installed on a RHEL5.2 xen domU (can I
run it on Fedora?). I noticed that doing a 'yum localinstall' pulled
in func and certmaster, and the wiki mentions that is useful.
Question I have is "what for?" Can cas be used to simply be a local
analysis tool (i.e. I copy a core there, run the tool, and perform any
further analysis on the same box? I think that this would useful for
environments where we pull back cores to some other spot for analysis
(that is network isolated from the systems that actually generated
them).
Another question is what to put in /mnt/kernels per the default
configuration - is it vmlinuz+initrd, vmlinux, or RPM's, or some other
thing I haven't thought of? :) I can see there's some logic to
extract a vmlinux from a kernel-debuginfo, but I'm not exactly clear
where that gets called and where it expects to have the
kernel-debuginfo.
I think that's all the questions that I have for the moment, I'm sure
I'll think of more :)
15 years, 4 months
cas-0.13-91 set free
by Adam Stokes
Starting to get pretty close to needed testers for some initial runs of cas.
Please download the latest release:
https://fedorahosted.org/releases/c/a/cas/
Post configuration/running questions to the mailing list for now. I'll have some step by step documentation up in the next few days.
CAS still needs several testing iterations before I can consider it somewhat stable for use so please test!
--
__ __
.-----.| |_.-----.| |--.-----.-----.
|__ --|| _| _ || <| -__|__ --|
|_____||____|_____||__|__|_____|_____|
.----------------------------------.
( ajs(a)redhat.com || 919.754.4187 )
`----------------------------------'
15 years, 4 months
cas cas.1 cas-admin cas-admin.1 cas.conf lib/cas version
by Adam Stokes
cas | 92 ++++++++++++++++++++++++++++++++++++++++++--------------
cas-admin | 11 +++---
cas-admin.1 | 10 ++++--
cas.1 | 14 +++++++-
cas.conf | 4 ++
lib/cas/core.py | 3 -
lib/cas/util.py | 16 ++++++---
version | 2 -
8 files changed, 113 insertions(+), 39 deletions(-)
New commits:
commit 806b6924d71b1a30aa0f6402d5f16c9d0a8ff31a
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Thu Dec 11 13:03:46 2008 -0500
- update man pages
- fixes to cas+func processing, crash should now run
remotely
- added email support
- other minor cleanups/fixes
diff --git a/cas b/cas
index 586e1e3..a4b15bb 100755
--- a/cas
+++ b/cas
@@ -19,46 +19,55 @@ import optparse
import os
import ConfigParser
import shutil
+import smtplib
from cas.network import Download
-from cas.core import CoreBase
-from cas.util import UtilBase, dprint
+from cas.core import CoreBase, CoreException
+from cas.util import UtilBase, dprint, cprint
from cas.rpmutils import RPMBase
if sys.version_info[:2] < (2,4):
raise SystemExit("Python >= 2.4 required")
+# Configuration parsing of /etc/cas.conf
+# TODO: verify configuration is setup properly
config = ConfigParser.ConfigParser()
config.read("/etc/cas.conf")
WORKDIRECTORY = config.get("settings","workDirectory")
RPMS = config.get("settings","rpms")
DPRINT = config.get("settings","dprint")
SERVERS = config.get("settings", "servers")
+SMTPHOST = config.get("settings", "mailServer")
class CoreHandler(object):
- def __init__(self, filename, dst):
- self.filename = filename
+ def __init__(self, filename, dst, is_url):
+ self.filename = filename # abspath of file or url
+ self.is_url = is_url
self.dst = dst
self.tool = CoreBase()
def run(self):
- if (self.filename.startswith("http") or self.filename.startswith("ftp")):
+ if self.is_url:
+ # return abspath of location of downloaded corefile
self.filename = Download(self.filename, self.dst).get()
- self.filename = os.path.basename(self.filename)
if not os.path.isfile(self.filename):
- print("Unable to find file %s" % (self.filename,))
+ dprint("Unable to find file %s" % (self.filename,), DPRINT)
sys.exit(1)
if self.tool.isCorefile(self.filename):
# No need to proceed to extracting corefile since we assume
# this is already at the proper stage.
shutil.move(self.filename,
os.path.join(self.dst, self.filename))
- return os.path.realpath(self.filename)
+ return os.path.join(self.dst, self.filename)
try:
+ cprint("Detected compressed archive, extracting.")
corepath = self.tool.extractCore(self.filename, self.dst)
- return os.path.realpath(corepath)
- except:
- dprint("Unable to extract corefile", DPRINT)
+ # corefile extracted now move it to work directory, pull basename
+ # from corepath since we auto-detect the core file from extraction
+ shutil.move(corepath,os.path.join(self.dst, os.path.basename(corepath)))
+ return os.path.join(self.dst, os.path.basename(corepath))
+ except CoreException, err:
+ dprint(err, DPRINT)
sys.exit(1)
class TimestampHandler(object):
@@ -68,6 +77,7 @@ class TimestampHandler(object):
self.tool = CoreBase()
def run(self):
+ cprint("Running timestamp")
rpmDB = self.util.load(RPMS)
try:
coreTimestamp = self.tool.timestamp(self.corefile)
@@ -78,8 +88,9 @@ class TimestampHandler(object):
for coreObj in rpmDB[k]:
debugKernel, timestamp = coreObj
if timestamp and coreTimestamp in timestamp:
+ cprint("Timestamp found %s" % (coreTimestamp,))
return (k, debugKernel)
- dprint("Unable to match (%s,%s) with debugKernel\n" % (self.corefile, coreTimestamp), DPRINT)
+ dprint("Unable to match (%s,%s) with debugKernel" % (self.corefile, coreTimestamp), DPRINT)
sys.exit(1)
class CasApplication(object):
@@ -89,11 +100,14 @@ class CasApplication(object):
self.rpmTool = RPMBase()
def parse_options(self, args):
- parser = optparse.OptionParser(usage="cas -i ID -f FILE")
+ parser = optparse.OptionParser(usage="cas [opts] args")
parser.add_option("-i","--identifier", dest="identifier",
help="Unique ID for core")
parser.add_option("-f","--file", dest="filename",
help="Filename")
+ parser.add_option("-m","--email", dest="email",
+ help="Define email for results (must be valid!)",
+ action="store", default="root(a)localhost.localdomain")
self.opts, args = parser.parse_args()
if not self.opts.identifier:
@@ -102,17 +116,29 @@ class CasApplication(object):
parser.error("A file object is missing.")
self.filename = self.opts.filename
+ self.is_url = True
+ # not a url, so we assume its a local file, pre-pend absolute path
+ if (not self.filename.startswith("http") or not self.filename.startswith("ftp")):
+ self.filename = os.path.abspath(self.filename)
+ self.is_url = False
self.identifier = self.opts.identifier
+ self.email = self.opts.email
+
self.storagePath = os.path.join(WORKDIRECTORY, self.identifier)
def run(self):
# setup directory structure
if not os.path.isdir(self.storagePath):
os.makedirs(self.storagePath)
- corefile = CoreHandler(self.filename, self.storagePath).run()
- dprint(corefile + "\n", DPRINT)
+ cprint("Starting job at %s on file %s" % (self.storagePath, self.filename))
+ # change into processed directory for completion
+ # os.chdir(self.storagePath)
+ # begin core extraction analysis
+ corefile = CoreHandler(self.filename, self.storagePath, self.is_url).run()
+ cprint("Corefile prepared, processing %s" % (corefile,))
debuginfo, debugKernel = TimestampHandler(corefile).run()
filterString = "*/%s" % (debugKernel,)
+ cprint("Extracting debug kernel with filter %s" % (filterString,))
self.rpmTool.extract(debuginfo, self.storagePath,
filter=filterString,
return_results=False)
@@ -130,32 +156,54 @@ class CasApplication(object):
# Pull the architecture from the elf file to match up with a
# server providing this architecture
debugKernelArch = self.util.debugKernelArch(debugKernel)
+ cprint("Crash file built, locating suitable %s system for " \
+ "processing" % (debugKernelArch,))
if os.path.isfile(SERVERS):
serverList = self.util.load(SERVERS)
if serverList.has_key(debugKernelArch):
# TODO: Randomize server selection
casProcessMachine = serverList[debugKernelArch][0]
+ cprint("Machine %s found, processing crash output" % (casProcessMachine,))
cmd = os.path.join(self.storagePath,"crash")
client = fc.Overlord(casProcessMachine)
client_dict = client.command.run(cmd)
# Only necessary for debugging why running of crash failed.
# (sts, out, err) = client_dict[client_dict.keys()[0]]
else:
- print("No servers available for arch, please run cas-admin -h " \
+ cprint("No servers available for arch, please run cas-admin -h " \
"for more information")
- print("Continueing without processing core.")
+ cprint("Continueing without processing core.")
else:
- print("No servers database found, please run cas-admin -h for " \
+ cprint("No servers database found, please run cas-admin -h for " \
"more information")
sys.exit(1)
+ # TODO: Possibly handle this exception more gracefully?
except ImportError:
- print("You need to install func (http://fedorahosted.org/func if you "\
+ cprint("You need to install func (http://fedorahosted.org/func if you "\
"wish to have the data processed automatically.")
- # TODO: Possibly handle this exception more gracefully?
- print("Job on %s complete and located in %s." % (self.filename,self.storagePath))
+ crashOutFile = os.path.join(self.storagePath,"crash.out")
+ if os.path.isfile(crashOutFile):
+ cprint("Crash output processed, sending email to %s" % (self.email,))
+ try:
+ # Compose email msg of results
+ msg = "Subject: CAS results for %s\r\n\n" % (self.identifier,)
+ msg += "Location: %s\n" % (self.storagePath,)
+ msg += "Server: %s\n" % (casProcessMachine,)
+ msg += "Output data:\n"
+ crashOutFH = open(crashOutFile,'r')
+ msg += crashOutFH.read()
+ crashOutFH.close()
+ mailServer = smtplib.SMTP(SMTPHOST)
+ mailServer.set_debuglevel(0)
+ mailServer.sendmail(self.email,self.email,msg)
+ mailServer.quit()
+ except smtplib.SMTPException, e:
+ dprint(e)
+ cprint("Job on %s complete and located in %s." % (self.filename,
+ self.storagePath))
return
if __name__=="__main__":
- "Start of application"
+ # Begin CAS
app = CasApplication(sys.argv[1:])
sys.exit(app.run())
diff --git a/cas-admin b/cas-admin
index 073e877..d441f2c 100755
--- a/cas-admin
+++ b/cas-admin
@@ -21,7 +21,7 @@ import sys
import urlparse
from cas.core import CoreBase
-from cas.util import UtilBase, sprint, dprint
+from cas.util import UtilBase, sprint, dprint, cprint
from cas.rpmutils import RPMBase
from subprocess import Popen, PIPE
from shutil import rmtree
@@ -103,6 +103,7 @@ class CasServerHandler(object):
import func.overlord.client as fc
parent_func = fc.Overlord("*")
minions = parent_func.minions
+ # TODO: add only servers that respond, purge the rest
for i in minions:
scheme, netloc, path, query, frag = urlparse.urlsplit(i)
hostname, port = netloc.split(":")
@@ -134,7 +135,7 @@ class CasAdminApplication(object):
self.parse_options(args)
def parse_options(self, args):
- parser = optparse.OptionParser(usage="casprint {-b, -s}")
+ parser = optparse.OptionParser(usage="casprint [opts] args")
parser.add_option("-b","--build", dest="buildDB",
help="Build CAS DB", action="store_true", default=False)
parser.add_option("-s","--server", dest="server_init",
@@ -155,13 +156,13 @@ class CasAdminApplication(object):
os.makedirs(DEBUGS)
if self.buildDB:
- print("::: Starting CAS DB instance. :::")
+ cprint("Starting CAS DB instance.")
dbHandler = CasDatabaseHandler().run()
elif self.server_init:
- print("::: Building CAS Server DB instance. :::")
+ cprint("Building CAS Server DB instance.")
serverHandler = CasServerHandler().run()
else:
- print("Missing options, please run with --help.")
+ cprint("Missing options, please run with --help.")
sys.exit(1)
if __name__=="__main__":
diff --git a/cas-admin.1 b/cas-admin.1
index b13a053..7163f03 100644
--- a/cas-admin.1
+++ b/cas-admin.1
@@ -6,7 +6,13 @@ cas-admin \- Administrator tools for building rpm/server database.
.SH DESCRIPTION
.TP
\fIcas-admin\fP is a tool used to build necessary data for use with cas.
-
+.SH OPTIONS
+.TP
+.BI \-s
+Populate server database for automatic processing of core files.
+.TP
+.B \-b
+Populate a database with debug kernel information.
.SH RESOURCES
A configuration file is maintained in /etc/cas.conf.
.SH AUTHOR
@@ -20,4 +26,4 @@ This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.
.SH "SEE ALSO"
-Full documentation should be found in /usr/share/cas-<version>.
+Full documentation should be found in /usr/share/doc/cas-<version>.
diff --git a/cas.1 b/cas.1
index 1146ec2..81c1ae6 100644
--- a/cas.1
+++ b/cas.1
@@ -7,7 +7,17 @@ cas \- Tool to configure core analysis environment.
.TP
\fIcas\fP is a tool used to automatically configure an environment for
viewing coredumps.
-
+.SH OPTIONS
+.TP
+.BI \-i " identifier"
+Specify an id to associate with current cas job, normally a support ticket of
+some sort.
+.TP
+.BI \-f " file"
+Define a core file to be processed.
+.TP
+.BI \-m " email"
+Provide an email address for result notification
.SH RESOURCES
A configuration file is maintained in /etc/cas.conf.
.SH AUTHOR
@@ -21,4 +31,4 @@ This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.
.SH "SEE ALSO"
-Full documentation should be found in /usr/share/cas-<version>.
+Full documentation should be found in /usr/share/doc/cas-<version>.
diff --git a/cas.conf b/cas.conf
index 09e6a6a..3efe358 100644
--- a/cas.conf
+++ b/cas.conf
@@ -31,3 +31,7 @@ servers=/var/db/cas/servers.db
# define processing jobs db
# TODO: determine format type, (e.g raw, json, xml)
jobs=/var/db/cas/jobs.db
+
+# Mail server, e.g. mail.example.com
+# Provides job results via email
+mailServer=mail.example.com
diff --git a/lib/cas/core.py b/lib/cas/core.py
index c1d9bc2..937df39 100644
--- a/lib/cas/core.py
+++ b/lib/cas/core.py
@@ -62,13 +62,12 @@ class CoreBase(object):
if not format:
raise CoreException("Can not determine compression format.")
else:
- os.chdir(self.dst)
format.append(self.filepath)
p = Popen(format, stdout=PIPE, stderr=PIPE)
err = p.stderr.read()
out = p.stdout.read()
if err:
- raise CoreException("Unable to extract file based on compression format.: %s" % (err,))
+ raise CoreException("Unable to extract file: %s" % (err,))
for root, dirs, files in self.util.directoryList(self.dst):
for file in files:
if self.isCorefile(file):
diff --git a/lib/cas/util.py b/lib/cas/util.py
index bcaf21d..b7bfab5 100755
--- a/lib/cas/util.py
+++ b/lib/cas/util.py
@@ -29,14 +29,19 @@ logging.basicConfig(level=logging.DEBUG,
def dprint(msg, debug=True):
if debug:
- sys.stderr.write("(debug) %s" % (msg,))
+ print("[.cas.](debug) :: %s" % (msg,))
logging.debug(msg)
def sprint(msg):
""" function to print status messages
"""
- sys.stdout.write(msg + "\r")
+ sys.stdout.write("[.cas.] :: " + msg + "\r")
sys.stdout.flush()
+
+def cprint(msg):
+ """ function to print procedure
+ """
+ print("[.cas.] :: " +msg)
class UtilException(Exception): pass
@@ -125,10 +130,11 @@ class UtilBase(object):
commands
"""
# Optional commands can be placed here, for example,
- # kmem -f > kmem-f.out\n
+ # kmem -f, sys -c
+ # please note some commands like the ones above take a looong time to
+ # run and can possibly timeout the xmlrpc server provided with func
crashInputCmds = ['bt -a >>', 'sys >>',
- 'sys -c >>', 'log >>',
- 'mod >>']
+ 'log >>', 'mod >>']
crashOutputPath = os.path.join(dst, "crash.out")
# test for output file existance
if os.path.isfile(crashOutputPath):
diff --git a/version b/version
index 7382259..45e95d3 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 81
+0.13 91
15 years, 4 months
cas cas-admin lib/cas version
by Adam Stokes
cas | 7 +++++--
cas-admin | 2 +-
lib/cas/util.py | 23 ++++++++++++++++++-----
version | 2 +-
4 files changed, 25 insertions(+), 9 deletions(-)
New commits:
commit 909e594b88fae6da92d305035570d790148faa22
Author: Adam Stokes <adam(a)conans.battleaxe>
Date: Wed Dec 10 11:30:30 2008 -0500
finishing up func integration
crash buildout cleanup
diff --git a/cas b/cas
index 4035b9e..586e1e3 100755
--- a/cas
+++ b/cas
@@ -44,6 +44,7 @@ class CoreHandler(object):
def run(self):
if (self.filename.startswith("http") or self.filename.startswith("ftp")):
self.filename = Download(self.filename, self.dst).get()
+ self.filename = os.path.basename(self.filename)
if not os.path.isfile(self.filename):
print("Unable to find file %s" % (self.filename,))
sys.exit(1)
@@ -116,6 +117,9 @@ class CasApplication(object):
filter=filterString,
return_results=False)
+ # define absolute path to debugkernel
+ debugKernel = os.path.abspath(debugKernel)
+ # setup crash file to finalize the processing of the core file
self.util.buildCrashFile(self.storagePath, corefile, debugKernel)
# If func is installed we will pre-process the output from crash on
# a random machine tied to the architecture of the debug file.
@@ -131,8 +135,7 @@ class CasApplication(object):
if serverList.has_key(debugKernelArch):
# TODO: Randomize server selection
casProcessMachine = serverList[debugKernelArch][0]
- cmd = "cd %s; %s" % (self.storagePath,
- os.path.join(self.storagePath,"crash"))
+ cmd = os.path.join(self.storagePath,"crash")
client = fc.Overlord(casProcessMachine)
client_dict = client.command.run(cmd)
# Only necessary for debugging why running of crash failed.
diff --git a/cas-admin b/cas-admin
index 34c0c74..073e877 100755
--- a/cas-admin
+++ b/cas-admin
@@ -134,7 +134,7 @@ class CasAdminApplication(object):
self.parse_options(args)
def parse_options(self, args):
- parser = optparse.OptionParser(usage="casprint -b")
+ parser = optparse.OptionParser(usage="casprint {-b, -s}")
parser.add_option("-b","--build", dest="buildDB",
help="Build CAS DB", action="store_true", default=False)
parser.add_option("-s","--server", dest="server_init",
diff --git a/lib/cas/util.py b/lib/cas/util.py
index c261b16..bcaf21d 100755
--- a/lib/cas/util.py
+++ b/lib/cas/util.py
@@ -95,7 +95,7 @@ class UtilBase(object):
out=cPickle.load(FILE)
FILE.close()
else:
- raise UtilitiesException("%s : Unable to locate/load file." % (fname,))
+ raise UtilException("%s : Unable to locate/load file." % (fname,))
return out
def debugKernelArch(self, debug):
@@ -107,6 +107,9 @@ class UtilBase(object):
"Intel IA-64": "ia64",
"PowerPC64": "ppc64"}
+ # readelf pulls in various information about elf object files
+ # we only care about data in the elf header at the start of file
+ # more specifically the machine type
cmd = ["readelf", "-h", debug]
cmd2 = ["grep", "Machine"]
pipe = Popen(cmd, stdout=PIPE, stderr=PIPE)
@@ -123,9 +126,19 @@ class UtilBase(object):
"""
# Optional commands can be placed here, for example,
# kmem -f > kmem-f.out\n
- crashInputCmds = ['bt -a > bt.out\n', 'sys > sys.out\n',
- 'sys -c > sys-c.out\n', 'log > log.out\n',
- 'mod > mod.out\n', 'exit']
+ crashInputCmds = ['bt -a >>', 'sys >>',
+ 'sys -c >>', 'log >>',
+ 'mod >>']
+ crashOutputPath = os.path.join(dst, "crash.out")
+ # test for output file existance
+ if os.path.isfile(crashOutputPath):
+ os.remove(crashOutputPath)
+ # alter list to append the output path of the crash.out file
+ # e.g. bt -a >> /cores/processed/123/crash.out\n
+ crashInputCmds = [item+crashOutputPath+"\n" for item in crashInputCmds]
+ # append the crash exit command since this doesn't give any additional
+ # output and doesn't need alteration
+ crashInputCmds.append("exit\n")
# Build crash input file
crashInputPath = os.path.join(dst, file_in)
@@ -134,7 +147,7 @@ class UtilBase(object):
crashInputFH.close()
vmcorePath = os.path.join(dst, vmcore)
- crashCmd = "#!/bin/sh\ncrash %s %s -i %s -s\n" % (vmcorePath,debug,
+ crashCmd = "#!/bin/sh\ncrash %s %s -i %s -s\n" % (vmcorePath, debug,
crashInputPath)
crashExe = os.path.join(dst,"crash")
fh = open(crashExe,"w")
diff --git a/version b/version
index df52f92..7382259 100644
--- a/version
+++ b/version
@@ -1 +1 @@
-0.13 78
+0.13 81
15 years, 4 months