Re: [hyperkitty-devel] Archive logger

Hi,

Thanks to both of you. this was quite helpful

Aurelien, i've created a branch with patches (except the postgres 
related one)

https://code.launchpad.net/~nkarageuzian/mailman/hyperkitty_compliance

and now I have the following stack when i'm trying to release messages 
:
Dec 01 14:46:57 2013 (22078) HOLD: [e-mail] 
post from nico@vostrodeb held, 
message-id=: n/a
Dec 01 14:46:58 2013 (22078) OWNER: 

Dec 01 14:47:01 2013 (22085) lists.menestrel.local - - "GET 
/3.0/lists/[e-mail] HTTP/1.1" 200 369
Traceback (most recent call last):
   File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
     self.result = application(self.environ, self.start_response)
   File "/home/nico/wspace/mailman/src/mailman/database/transaction.py", 
line 60, in wrapper
     rtn = function(*args, **kws)
   File "/home/nico/wspace/mailman/src/mailman/rest/wsgiapp.py", line 
59, in __call__
     environ, start_response)
   File 
"/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/app.py", 
line 18, in __call__
     response = self.get_response(request, resource_or_response)
   File 
"/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/app.py", 
line 65, in get_response
     resource_or_response = resource_or_response(request)
   File 
"/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", 
line 212, in __call__
     return _dispatch(request, match, lambda r: callable(self, r))
   File 
"/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", 
line 243, in _dispatch
     response = func(request)
   File 
"/home/nico/wspace/venv_hk/local/lib/python2.7/site-packages/restish-0.12.1-py2.7.egg/restish/resource.py", 
line 212, in <lambda>
     return _dispatch(request, match, lambda r: callable(self, r))
   File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 
162, in requests
     resource = self._make_collection(request)
   File "/home/nico/wspace/mailman/src/mailman/rest/helpers.py", line 
180, in _make_collection
     for resource in collection]
   File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 
151, in _resource_as_dict
     return self._make_resource(request.id)
   File "/home/nico/wspace/mailman/src/mailman/rest/moderation.py", line 
86, in _make_resource
     msg = getUtility(IMessageStore).get_message_by_id(key)
   File "/home/nico/wspace/mailman/src/mailman/database/transaction.py", 
line 78, in wrapper
     return function(args[0], config.db.store, *args[1:], **kws)
   File "/home/nico/wspace/mailman/src/mailman/model/messagestore.py", 
line 113, in get_message_by_id
     return self._get_message(row)
   File "/home/nico/wspace/mailman/src/mailman/model/messagestore.py", 
line 105, in _get_message
     with open(path) as fp:
IOError: [Errno 2] No such file or directory: 
'/home/nico/wspace/mailman/var/messages/WR/TF/WRTFIZG5HYLEDRWDDT4PZDVBQZW26SOV'
Dec 01 14:47:01 2013 (22085) lists.menestrel.local - - "GET 
/3.0/lists/[e-mail]/held HTTP/1.1" 500 59
and when i look in directory, there are no files.

That's some concern, but i have now some more hyperkitty specific 
(related to report about postgres i have in common with mayrin)
Given the following script :

#-*- coding: utf-8 -*-
from __future__ import absolute_import, print_function, 
unicode_literals

from mailman.interfaces.mailinglist import IListArchiverSet
from mailman.testing.helpers import (
     specialized_message_from_string as mfs)
def showArchiver(mlist):
     ""
     archiver_set = IListArchiverSet(mlist) # new feature of 
http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/revision/7228
     archiver = archiver_set.get(u'hyperkitty')
     print("%s - %s" % (mlist.list_name,archiver.name) )
     msg = mfs("""\
To: %s
 From: [e-mail]
Subject: Testing the test list
Message-ID: <ant>
X-Message-ID-Hash: MS6QLWERIJLGCRF44J7USBFDELMNT2BW

Tests are better than no tests
but the water deserves to be swum.
""" % mlist.list_name)
     print(archiver)
     archiver.system_archiver.archive_message(msg,mlist)

got the following output :

(venv_hk)nico@vostrodeb:~/wspace/mailman$ mailman withlist -r 
mlist.showArchiver [e-mail]
newsletter - hyperkitty
<mailman.model.mailinglist.ListArchiver object at 0x38fdad0>
[09:12:37.936526] EXECUTE: 'SELECT * FROM patch WHERE version=0', ()
[09:12:37.938982] DONE
[09:12:37.939733] EXECUTE: 'SELECT patch.version FROM patch', ()
[09:12:37.940496] DONE
[09:12:37.943501] EXECUTE: 'SELECT patch.version FROM patch', ()
[09:12:37.944198] DONE

some long time ~ 3mn
        while we can observe :
23624 nico      20   0  307m 202m 5952 R  99,9  5,4   2:31.86 mailman
1 full CPU for mailman
then shows up the stack :
Traceback (most recent call last):
   File "/home/nico/wspace/venv_hk/bin/mailman", line 9, in <module>
     load_entry_point('mailman==3.0.0b3-', 'console_scripts', 
'mailman')()
   File "/home/nico/wspace/mailman/src/mailman/bin/mailman.py", line 98, 
in main
     args.func(args)
   File 
"/home/nico/wspace/mailman/src/mailman/commands/cli_withlist.py", line 
134, in process
     r = call_name(dotted_name, m)
   File "/home/nico/wspace/mailman/src/mailman/utilities/modules.py", 
line 65, in call_name
     return named_callable(*args, **kws)
   File "/home/nico/wspace/mlist.py", line 22, in showArchiver
     archiver.system_archiver.archive_message(msg,mlist)
   File "/home/nico/wspace/hyperkitty/hyperkitty/archiver.py", line 113, 
in archive_message
     msg.message_id_hash = self.store.add_to_list(mlist, msg)
   File "/home/nico/wspace/kittystore/kittystore/storm/store.py", line 
92, in add_to_list
     list_name = unicode(mlist.fqdn_listname)
AttributeError: Message instance has no attribute 'fqdn_listname'

another run give exact similar comportement.

Feel free to ask any further information

Cheers

Le 2013-12-01 11:13, Aurelien Bompard a écrit :
> Oh, I forgot: I'm using patches on Mailman from all the bugs I've
> reported and are not yet merged:
> https://bugs.launchpad.net/mailman/+bug/1130957
> https://bugs.launchpad.net/mailman/+bug/1158721
> https://bugs.launchpad.net/mailman/+bug/1130696
> https://bugs.launchpad.net/mailman/+bug/1170347
> https://bugs.launchpad.net/mailman/+bug/1236297
>
> All those have patches attached. There's also my work on the import21
> command, but I don't think it would impact you.
>
> This may be linked to why the archiver does not work for you. Cant 
> you
> try with the paches?
>
> Thanks. I should have thought of that earlier, sorry... :-/
>
> Aurélien
>
> P.S.: Hi Barry, cool to see you around :-)
>
> 2013/11/30 Barry Warsaw :
> > On Nov 30, 2013, at 08:37 PM, nicolas wrote:
> > > I think we have to qualify a better report before sending to mailman 
> > > team,
> > > i'm using r7229 of lp:mailman, and i wrote this simple mlist.py 
> > > module :
> > > #-*- coding: utf-8 -*-
> >
> > ^^^ This isn't enough to make it work.  MM3 core is written in a 
> > dialect that
> > tries to be as close to Python 3 as possible[*].  You should put the 
> > following
> > at the top of your module:
> >
> > from __future__ import absolute_import, print_function, 
> > unicode_literals
> >
> > It's possible there are some corner cases when using this as a 
> > withlist
> > script.  If so, please submit a bug report!
> >
> > Cheers,
> > -Barry
> >
> > hyperkitty-devel mailing list
> > [e-mail]
> > https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel
>
> hyperkitty-devel mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/hyperkitty-devel