Hi list
Referring to ticket #34 at https://hosted.fedoraproject.org/projects/pungi/ticket/34 stating that the logger needed a little work so that it didn't depend on the gather.py (or at least thats what I understood :) I propose either a new file (pungiLog.py) located in the pypungi directory or a new function in the "pungi" file that contains the logging stuff. The log services would be started somewhere before the line containing "# Actually do work." of the "pungi" file. The logging root would be called "pungi" and would be called in each file that logging is needed with the logging.getlogger("pungi") command. If "quiet" is specified in the config file the logging will be turned off.
*Diff for the pungi file:* 1. Initializes the logger by calling to the new file. 2. specify quiet value. 3. logging function. * Diff for pungi.py file: *1. use the correct logger. * Diff for gather.py file:* change all the if statements for each logging call.
Files attached... Comments appreciated. Regards
--- gather.py-original 2007-05-03 17:01:18.000000000 +0200 +++ gather.py-JG 2007-05-03 17:01:18.000000000 +0200 @@ -16,6 +16,9 @@ import os import shutil import sys +import logging + +log = logging.getLogger("pungi")
class Gather(yum.YumBase): def __init__(self, config, pkglist): @@ -46,7 +49,6 @@ #self.doSackSetup(arches) self.doSackSetup(archlist=arches) # work around temp break in yum api self.doSackFilelistPopulate() - self.logger = yum.logging.getLogger("yum.verbose.pungi") self.pkglist = pkglist self.polist = [] self.srpmlist = [] @@ -56,15 +58,8 @@ self.compsobj.add(self.config.get('default', 'comps'))
def doLoggingSetup(self, debuglevel, errorlevel): - """Setup the logging facility.""" - - - logdir = os.path.join(self.config.get('default', 'destdir'), 'logs') - if not os.path.exists(logdir): - os.makedirs(logdir) - logfile = os.path.join(logdir, '%s.%s.log' % (self.config.get('default', 'flavor'), - self.config.get('default', 'arch'))) - yum.logging.basicConfig(level=yum.logging.DEBUG, filename=logfile) + # We want to use our own logger. + pass
def doFileLogSetup(self, uid, logfile): # This function overrides a yum function, allowing pungi to control @@ -76,8 +71,7 @@ transaction info"""
- if not self.config.has_option('default', 'quiet'): - self.logger.info('Checking deps of %s.%s' % (po.name, po.arch)) + log.info('Checking deps of %s.%s' % (po.name, po.arch))
reqs = po.requires provs = po.provides @@ -93,13 +87,12 @@
deps = self.whatProvides(r, f, v).returnPackages() if deps is None: - self.logger.warning("Unresolvable dependency %s in %s.%s" % (r, po.name, po.arch)) + log.warning("Unresolvable dependency %s in %s.%s" % (r, po.name, po.arch)) continue
for dep in deps: self.tsInfo.addInstall(dep) - if not self.config.has_option('default', 'quiet'): - self.logger.info('Added %s.%s for %s.%s' % (dep.name, dep.arch, po.name, po.arch)) + log.info('Added %s.%s for %s.%s' % (dep.name, dep.arch, po.name, po.arch))
self.resolved_deps[req] = None
@@ -124,7 +117,7 @@
# Check if we have the group if not self.compsobj.has_group(group): - self.logger.error("Group %s not found in comps!" % group) + log.error("Group %s not found in comps!" % group) return packages
# Get the group object to work with @@ -175,22 +168,18 @@ for line in self.pkglist: line = line.strip() if line.startswith('#'): - if not self.config.has_option('default', 'quiet'): - self.logger.info('Skipping comment: %s' % line) + log.info('Skipping comment: %s' % line) continue if line.startswith('@'): - if not self.config.has_option('default', 'quiet'): - self.logger.info('Adding group: %s' % line) + log.info('Adding group: %s' % line) grouplist.append(line.strip('@')) continue if line.startswith('-'): - if not self.config.has_option('default', 'quiet'): - self.logger.info('Adding exclude: %s' % line) + log.info('Adding exclude: %s' % line) excludelist.append(line.strip('-')) continue else: - if not self.config.has_option('default', 'quiet'): - self.logger.info('Adding package: %s' % line) + log.info('Adding package: %s' % line) addlist.append(line)
# First remove the excludes @@ -219,12 +208,10 @@ mysack = yum.packageSack.ListPackageSack(matches) for match in mysack.returnNewestByNameArch(): self.tsInfo.addInstall(match) - if not self.config.has_option('default', 'quiet'): - self.logger.info('Found %s.%s' % (match.name, match.arch)) + log.info('Found %s.%s' % (match.name, match.arch))
for pkg in unmatched: - if not pkg in matchdict.keys(): - self.logger.warn('Could not find a match for %s' % pkg) + log.warn('Could not find a match for %s' % pkg)
if len(self.tsInfo) == 0: raise yum.Errors.MiscError, 'No packages found to download.' @@ -262,7 +249,7 @@ for pkg in self.polist: downloads.append('%s.%s' % (pkg.name, pkg.arch)) downloads.sort() - self.logger.info("Download list: %s" % downloads) + log.info("Download list: %s" % downloads)
# Package location within destdir, name subject to change/config pkgdir = os.path.join(self.config.get('default', 'destdir'), self.config.get('default', 'version'), @@ -282,8 +269,7 @@ if (os.path.exists(local) and str(os.path.getsize(local)) == pkg.packagesize):
- if not self.config.has_option('default', 'quiet'): - self.logger.info("%s already exists and appears to be complete" % local) + log.info("%s already exists and appears to be complete" % local) target = os.path.join(pkgdir, os.path.basename(remote)) if os.path.exists(target): os.remove(target) # avoid traceback after interrupted download @@ -292,8 +278,7 @@
# Disable cache otherwise things won't download repo.cache = 0 - if not self.config.has_option('default', 'quiet'): - self.logger.info('Downloading %s' % os.path.basename(remote)) + log.info('Downloading %s' % os.path.basename(remote)) pkg.localpath = local # Hack: to set the localpath to what we want.
# do a little dance for file:// repos... @@ -348,19 +333,16 @@ local = os.path.join(self.config.get('default', 'cachedir'), local) if os.path.exists(local) and str(os.path.getsize(local)) == pkg.packagesize:
- if not self.config.has_option('default', 'quiet'): - self.logger.info("%s already exists and appears to be complete" % local) + log.info("%s already exists and appears to be complete" % local) if os.path.exists(os.path.join(pkgdir, os.path.basename(remote))) and str(os.path.getsize(os.path.join(pkgdir, os.path.basename(remote)))) == pkg.packagesize: - if not self.config.has_option('default', 'quiet'): - self.logger.info("%s already exists in tree and appears to be complete" % local) + log.info("%s already exists in tree and appears to be complete" % local) else: os.link(local, os.path.join(pkgdir, os.path.basename(remote))) continue
# Disable cache otherwise things won't download repo.cache = 0 - if not self.config.has_option('default', 'quiet'): - self.logger.info('Downloading %s' % os.path.basename(remote)) + log.info('Downloading %s' % os.path.basename(remote)) pkg.localpath = local # Hack: to set the localpath to what we want.
# do a little dance for file:// repos...
--- pungi.py-original 2007-05-03 11:01:18.000000000 -0400 +++ pungi.py-JG 2007-05-03 11:17:00.000000000 -0400 @@ -21,7 +21,7 @@ import shutil import re
-log = logging.getLogger("pypungi.pungi") +log = logging.getLogger("pungi")
class Pungi: def __init__(self, config):
--- pungi-original 2007-05-03 11:01:18.000000000 -0400 +++ pungi-JG 2007-05-03 11:47:12.000000000 -0400 @@ -16,6 +16,7 @@ import pypungi.gather import pypungi.pungi import yum +import logging
from ConfigParser import SafeConfigParser
@@ -79,6 +80,16 @@
destdir = config.get('default', 'destdir')
+ if not config.has_option('default', 'quiet'):quiet=False + else:quiet=True + config.set('default','quiet',str(quiet)) + + # initialize logging. + initLogger(config.get('default', 'destdir'), + '%s.%s.log' % (config.get('default', 'flavor'), config.get('default', 'arch')), + quiet=quiet) + log = logging.getLogger("pungi") + if not os.path.exists(destdir): try: os.makedirs(destdir) @@ -180,4 +190,25 @@ manifestfile.close() return pkglist
+ def initLogger(path, filename, quiet=True): + """Initialize the log stuff so other files can use it. + + path - Is the directory where the log will be located + filename - Is the log files name + useLogFile - Whether to log to the file or not + """ + logger = logging.getLogger("pungi") + if not quiet: + logdir = os.path.join(path , 'logs') + if not os.path.exists(logdir): + os.makedirs(logdir) + logfile = os.path.join(logdir, filename) + + fileHandler = logging.FileHandler(logfile) + fileHandler.setLevel(logging.DEBUG) + logger.addHandler(fileHandler) + logger.setLevel(logging.DEBUG) + else: + logger.disable = 1 + main()
On Friday 04 May 2007 10:13:34 Joel Andres Granados wrote:
Referring to ticket #34 at https://hosted.fedoraproject.org/projects/pungi/ticket/34 stating that the logger needed a little work so that it didn't depend on the gather.py (or at least thats what I understood :) I propose either a new file (pungiLog.py) located in the pypungi directory or a new function in the "pungi" file that contains the logging stuff. The log services would be started somewhere before the line containing "# Actually do work." of the "pungi" file. The logging root would be called "pungi" and would be called in each file that logging is needed with the logging.getlogger("pungi") command. If "quiet" is specified in the config file the logging will be turned off.
*Diff for the pungi file:*
- Initializes the logger by calling to the new file.
- specify quiet value.
- logging function.
Diff for pungi.py file: *1. use the correct logger.
Diff for gather.py file:* change all the if statements for each logging call.
Files attached... Comments appreciated.
Thanks for this. I think this is the right direction. However I'm reluctant to make such a change this late in Fedora 7 development. I'll want to look at this once I start doing Fedora 8 changes.
On Tuesday 15 May 2007 14:20:29 Jesse Keating wrote:
On Friday 04 May 2007 10:13:34 Joel Andres Granados wrote:
Referring to ticket #34 at https://hosted.fedoraproject.org/projects/pungi/ticket/34 stating that the logger needed a little work so that it didn't depend on the gather.py (or at least thats what I understood :) I propose either a new file (pungiLog.py) located in the pypungi directory or a new function in the "pungi" file that contains the logging stuff. The log services would be started somewhere before the line containing "# Actually do work." of the "pungi" file. The logging root would be called "pungi" and would be called in each file that logging is needed with the logging.getlogger("pungi") command. If "quiet" is specified in the config file the logging will be turned off.
*Diff for the pungi file:*
- Initializes the logger by calling to the new file.
- specify quiet value.
- logging function.
Diff for pungi.py file: *1. use the correct logger.
Diff for gather.py file:* change all the if statements for each logging call.
Files attached... Comments appreciated.
Thanks for this. I think this is the right direction. However I'm reluctant to make such a change this late in Fedora 7 development. I'll want to look at this once I start doing Fedora 8 changes.
I just looked at this again and I don't think it'll quite work. The thing is I want logging to work if somebody just imported pypungi and started using commands. Maybe that's not possible, maybe it is. Something to look into. I think I can have a pypungi.logger module/class that has some reasonable defaults that can be overridden if the config file is used, or if say /usr/bin/pungi is envoked, but would work otherwise.
Jesse Keating wrote:
On Tuesday 15 May 2007 14:20:29 Jesse Keating wrote:
On Friday 04 May 2007 10:13:34 Joel Andres Granados wrote:
Referring to ticket #34 at https://hosted.fedoraproject.org/projects/pungi/ticket/34 stating that the logger needed a little work so that it didn't depend on the gather.py (or at least thats what I understood :) I propose either a new file (pungiLog.py) located in the pypungi directory or a new function in the "pungi" file that contains the logging stuff. The log services would be started somewhere before the line containing "# Actually do work." of the "pungi" file. The logging root would be called "pungi" and would be called in each file that logging is needed with the logging.getlogger("pungi") command. If "quiet" is specified in the config file the logging will be turned off.
*Diff for the pungi file:*
- Initializes the logger by calling to the new file.
- specify quiet value.
- logging function.
Diff for pungi.py file: *1. use the correct logger.
Diff for gather.py file:* change all the if statements for each logging call.
Files attached... Comments appreciated.
Thanks for this. I think this is the right direction. However I'm reluctant to make such a change this late in Fedora 7 development. I'll want to look at this once I start doing Fedora 8 changes.
I just looked at this again and I don't think it'll quite work. The thing is I want logging to work if somebody just imported pypungi and started using commands. Maybe that's not possible, maybe it is. Something to look into. I think I can have a pypungi.logger module/class that has some reasonable defaults that can be overridden if the config file is used, or if say /usr/bin/pungi is envoked, but would work otherwise.
-- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Something like this??? The logger must still be initialized, but after that you just have to import the Plog variable from pypungi to begin. I put the logging code in the __init__.py because I wanted it to execute when "import pypungi" was called. I couldn't find a way to pass arguments to it though (within python). It would be ideal to set global variables before the import and for the function to somehow notice those globals. I would suppose it is possible but have no idea how. Look for more info on that tomorrow.
For now here is my proposal :) Comments greatly appreciated :)
Regards
I just looked at this again and I don't think it'll quite work. The thing is I want logging to work if somebody just imported pypungi and started using commands. Maybe that's not possible, maybe it is. Something to look into. I think I can have a pypungi.logger module/class that has some reasonable defaults that can be overridden if the config file is used, or if say /usr/bin/pungi is envoked, but would work otherwise.
-- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
This one can also be considered... This solution activates logging as soon as the pypungi module is imported and puts it in the __builtin__ namespace as "Plog" (I called it "Plog" instead of "log" to give it a 'pungi' feel :). This allows the user to use "Plog" from that poing on without having to do anything else. The behavior of the "Plog" can be changed by calling on the pypungi.pungiLogO object and specifying the values that need to be changed. I did a small test run and everything seems OK. Jesse: I think this is close to what you mentioned on the previous mail and can be considered for inclusion. The diff is attached.
buildsys@lists.fedoraproject.org
-
Jesse Keating -
Joel Andres Granados