email notifications

[Toshio Kuratomi]

On Wed, Aug 04, 2010 at 09:12:16AM +0300, Ville Skyttä wrote:
> On Wednesday 21 July 2010, Ville Skyttä wrote:
> > On Wednesday 21 July 2010, Dan Horák wrote:
> > > Florian La Roche píše v Čt 15. 07. 2010 v 06:00 +0200:
> > > > Hallo,
> > > > 
> > > > With koji-1.4 running on RHEL6-beta2, some email notifications
> > > > are not sent out correctly:
> > > > 
> > > > Traceback (most recent call last):
> > > >   File "/usr/sbin/kojid", line 1437, in runTask
> > > >   
> > > >     response = (handler.run(),)
> > > >   
> > > >   File "/usr/sbin/kojid", line 1513, in run
> > > >   
> > > >     return self.handler(*self.params,**self.opts)
> > > >   
> > > >   File "/usr/sbin/kojid", line 3649, in handler
> > > >   
> > > >     message = message.encode('utf-8')
> > > > 
> > > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
> > > > 546: ordinal not in range(128)
> > > > 
> > > > Thie seems to happen e.g. on dejavu-fonts, ctags, crontabs,
> > > > cronie, amanda, akonadi.
> > > 
> > > And that's the reason why I didn't received any notification from Fedora
> > > Koji about successful builds ...
> > 
> > Neither have I, and not getting those mails has already started to
> > interfere with my workflow which I'd rather not change... when can we
> > expect a fix for this in the Fedora koji instance?
> 
> Ping?  Still no notification mails for me, for example the modplugtools builds 
> done yesterday, and I haven't seen any comments when this might get fixed.
>
I know practically nothing about the koji code but that code seems very
wrong.  This is earlier in the class::

    message_templ = \
    """From: %(from_addr)s\r
    Subject: %(nvr)s %(result)s %(operation)s by %(user_name)s\r
    To: %(to_addrs)s\r
    X-Koji-Package: %(pkg_name)s\r
    X-Koji-NVR: %(nvr)s\r
    X-Koji-User: %(user_name)s\r
    X-Koji-Status: %(status)s\r
    %(tag_headers)s\r
    \r
    Package: %(pkg_name)s\r
    NVR: %(nvr)s\r
    User: %(user_name)s\r
    Status: %(status)s\r
    %(operation_details)s\r
    %(nvr)s %(result)s %(operation)s by %(user_name)s\r
    %(failure_info)s\r
    """

Then in this method::
        message = self.message_templ % locals()
        # ensure message is in UTF-8
        message = message.encode('utf-8')

Notice that message_templ is a byte string?  If all of the variables in
locals() is also a byte string then message will be a byte string
(if locals() contains unicode strings it will either become a unicode string
or throw a traceback if it can't decode some other byte string to unicode).

The encode method of byte strings is pretty much useless.  It allows you to
take a byte string that's all ascii and transform it into another byte
string.  You can use it if you want to rot13 it but not much else.  If the
byte string contains any bytes that aren't in the ascii subset, it'll throw
the error given above.

How to solve this depends on what you're actually trying to achieve.  If
it's really to make sure that the byte string only contains valid utf8
data and you don't care what the non-utf8 data becomes I'd do it like this:

   # Note this will be unsafe if locals contains strings of type <unicode>
   message = self.message_templ % locals()
   message = unicode(message, encoding='utf8', errors='replace').encode('utf8')

That takes the byte string that is the message, turns it into a unicode
string by treating the bytes as 'utf8', replacing any invalid bytes with �,
and then reencoding it to utf8.

If you can't guarantee that you start off with only byte strings, you could
use kitchen.text.converters.to_unicode to do the work or copy the essentials
of what it does::

   message = self.message_templ % locals()
   if isinstance(message, unicode):
       message = message.encode('utf8')
   else:
       message = unicode(message, encoding='utf8', errors='replace').encode('utf8')

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/buildsys/attachments/20100812/ac587322/attachment.bin