Error: "<type 'exceptions.TypeError'>: object of type 'NoneType' has no len()"> in koji

[Viacheslav Dubrovskyi]

Hi.

After update koji from 17 -> 1.21.0-2.el7 all build failed with error: 
<Fault 1: "<type 'exceptions.TypeError'>: object of type
'NoneType' has 
no len()">


[slava@dubrwork ~]$ koji build el7 
/home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
Uploading srpm: 
/home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
[====================================] 100% 00:00:00  17.38 KiB 64.31 
KiB/sec
Created task: 5873
Task info: http://koji.oro.cloud/koji/taskinfo?taskID=5873
Watching tasks (this may be safely interrupted)...
5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free
5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free -> open 
(koji.oro.cloud)
   5874 rebuildSRPM (noarch): open (koji.oro.cloud)
   5874 rebuildSRPM (noarch): open (koji.oro.cloud) -> closed
   0 free  1 open  1 done  0 failed
   5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
   5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free -> 
open (koji.oro.cloud)
5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): open 
(koji.oro.cloud) -> FAILED: Fault: <Fault 1: "<type 
'exceptions.TypeError'>: object of type 'NoneType' has no
len()">
   0 free  1 open  1 done  1 failed
   5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): open 
(koji.oro.cloud) -> closed
   0 free  0 open  2 done  1 failed

5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm) failed

In /var/log/kojid.log:

2020-06-10 16:37:14,329 [INFO] koji.TaskManager: Task 5875 (pid 3358) 
exited with status 0
2020-06-10 16:37:14,380 [INFO] koji.TaskManager: Expiring subsession 
6833 (task 5875)
2020-06-10 16:37:14,529 [WARNING] koji.TaskManager: FAULT:
Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1339, in 
runTask
     response = (handler.run(),)
   File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 329, in run
     return koji.util.call_with_argcheck(self.handler, self.params, 
self.opts)
   File "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in 
call_with_argcheck
     return func(*args, **kwargs)
   File "/usr/sbin/kojid", line 1062, in handler
     self.session.host.completeBuild(self.id, build_id, srpm, rpms, 
brmap, logs=logs)
   File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2263, 
in __call__
     return self.__func(self.__name, args, opts)
   File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2804, 
in _callMethod
     raise err
Fault: <Fault 1: "<type 'exceptions.TypeError'>: object of type 
'NoneType' has no len()">

$ rpm -qa | grep koji | sort
koji-1.21.0-2.el7.noarch
koji-builder-1.21.0-2.el7.noarch
koji-hub-1.21.0-2.el7.noarch
koji-utils-1.21.0-2.el7.noarch
koji-web-1.21.0-2.el7.noarch
python2-koji-1.21.0-2.el7.noarch
python2-koji-cli-plugins-1.21.0-2.el7.noarch
python2-koji-hub-1.21.0-2.el7.noarch
python2-koji-web-1.21.0-2.el7.noarch

Help please to fix it.

Thank you!

-- 
WBR,
Viacheslav Dubrovskyi

[Ken Dreyer]

This looks like kojid is getting an error from koji-hub when it calls
the "host.completeBuild" RPC.

The hub is raising a TypeError when calling the len() method on a
value that unexpectedly happens to be None, but I don't know much more
beyond that.

I don't see any len() calls in the completeBuild() or import_build()
methods in hub/kojihub.py. I don't see any additional len() calls
added between git log -p koji-1.17.0..koji-1.21.0.

Would you please paste the entire traceback if it's available in your
koji-hub's Apache error logs?  You should see it in
/var/log/httpd/error_log or ssl_error_log or thereabouts.

- Ken

On Wed, Jun 10, 2020 at 10:58 AM Viacheslav Dubrovskyi <dubrsl(a)gmail.com&gt; wrote:
>
> Hi.
>
> After update koji from 17 -> 1.21.0-2.el7 all build failed with error:
> <Fault 1: "<type 'exceptions.TypeError'>: object of type
'NoneType' has
> no len()">
>
>
> [slava@dubrwork ~]$ koji build el7
> /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> Uploading srpm:
> /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> [====================================] 100% 00:00:00  17.38 KiB 64.31
> KiB/sec
> Created task: 5873
> Task info: http://koji.oro.cloud/koji/taskinfo?taskID=5873
> Watching tasks (this may be safely interrupted)...
> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free
> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free -> open
> (koji.oro.cloud)
>    5874 rebuildSRPM (noarch): open (koji.oro.cloud)
>    5874 rebuildSRPM (noarch): open (koji.oro.cloud) -> closed
>    0 free  1 open  1 done  0 failed
>    5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
>    5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free ->
> open (koji.oro.cloud)
> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): open
> (koji.oro.cloud) -> FAILED: Fault: <Fault 1: "<type
> 'exceptions.TypeError'>: object of type 'NoneType' has no
len()">
>    0 free  1 open  1 done  1 failed
>    5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): open
> (koji.oro.cloud) -> closed
>    0 free  0 open  2 done  1 failed
>
> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm) failed
>
> In /var/log/kojid.log:
>
> 2020-06-10 16:37:14,329 [INFO] koji.TaskManager: Task 5875 (pid 3358)
> exited with status 0
> 2020-06-10 16:37:14,380 [INFO] koji.TaskManager: Expiring subsession
> 6833 (task 5875)
> 2020-06-10 16:37:14,529 [WARNING] koji.TaskManager: FAULT:
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1339, in
> runTask
>      response = (handler.run(),)
>    File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 329, in run
>      return koji.util.call_with_argcheck(self.handler, self.params,
> self.opts)
>    File "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in
> call_with_argcheck
>      return func(*args, **kwargs)
>    File "/usr/sbin/kojid", line 1062, in handler
>      self.session.host.completeBuild(self.id, build_id, srpm, rpms,
> brmap, logs=logs)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2263,
> in __call__
>      return self.__func(self.__name, args, opts)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2804,
> in _callMethod
>      raise err
> Fault: <Fault 1: "<type 'exceptions.TypeError'>: object of
type
> 'NoneType' has no len()">
>
> $ rpm -qa | grep koji | sort
> koji-1.21.0-2.el7.noarch
> koji-builder-1.21.0-2.el7.noarch
> koji-hub-1.21.0-2.el7.noarch
> koji-utils-1.21.0-2.el7.noarch
> koji-web-1.21.0-2.el7.noarch
> python2-koji-1.21.0-2.el7.noarch
> python2-koji-cli-plugins-1.21.0-2.el7.noarch
> python2-koji-hub-1.21.0-2.el7.noarch
> python2-koji-web-1.21.0-2.el7.noarch
>
> Help please to fix it.
>
> Thank you!
>
> --
> WBR,
> Viacheslav Dubrovskyi
>
>
> _______________________________________________
> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedorahosted.org/archives/list/[e-mail]...

[Viacheslav Dubrovskyi]

Hi Ken,

Here is entire traceback from /var/log/httpd/error_log

[Wed Jun 10 17:27:09.418133 2020] [:error] [pid 7760] 2020-06-10 
17:27:09,417 [WARNING] m=host.completeBuild u=koji.oro.cloud p=7760 
r=35.192.190.50:50042 koji.xmlrpc: Traceback (most recent call last):
[Wed Jun 10 17:27:09.418166 2020] [:error] [pid 7760]   File 
"/usr/share/koji-hub/kojixmlrpc.py", line 238, in _wrap_handler
[Wed Jun 10 17:27:09.418169 2020] [:error] [pid 7760]     response = 
handler(environ)
[Wed Jun 10 17:27:09.418171 2020] [:error] [pid 7760]   File 
"/usr/share/koji-hub/kojixmlrpc.py", line 281, in handle_rpc
[Wed Jun 10 17:27:09.418174 2020] [:error] [pid 7760]     return 
self._dispatch(method, params)
[Wed Jun 10 17:27:09.418176 2020] [:error] [pid 7760]   File 
"/usr/share/koji-hub/kojixmlrpc.py", line 318, in _dispatch
[Wed Jun 10 17:27:09.418178 2020] [:error] [pid 7760]     ret = 
koji.util.call_with_argcheck(func, params, opts)
[Wed Jun 10 17:27:09.418181 2020] [:error] [pid 7760]   File 
"/usr/lib/python2.7/site-packages/koji/util.py", line 258, in 
call_with_argcheck
[Wed Jun 10 17:27:09.418183 2020] [:error] [pid 7760]     return 
func(*args, **kwargs)
[Wed Jun 10 17:27:09.418185 2020] [:error] [pid 7760]   File 
"/usr/share/koji-hub/kojihub.py", line 13602, in completeBuild
[Wed Jun 10 17:27:09.418187 2020] [:error] [pid 7760] 
build_notification(task_id, build_id)
[Wed Jun 10 17:27:09.418189 2020] [:error] [pid 7760]   File 
"/usr/share/koji-hub/kojihub.py", line 8401, in build_notification
[Wed Jun 10 17:27:09.418192 2020] [:error] [pid 7760]     if 
len(recipients) > 0:
[Wed Jun 10 17:27:09.418194 2020] [:error] [pid 7760] TypeError: object 
of type 'NoneType' has no len()
[Wed Jun 10 17:27:09.418196 2020] [:error] [pid 7760]
[Wed Jun 10 19:00:08.153189 2020] [:error] [pid 16444] 2020-06-10 
19:00:08,152 [ERROR] m=None u=None p=16444 r=?:? koji.web: Traceback 
(most recent call last):
[Wed Jun 10 19:00:08.153234 2020] [:error] [pid 16444]   File 
"/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in handle_request
[Wed Jun 10 19:00:08.153238 2020] [:error] [pid 16444]     result = 
func(environ, **data)
[Wed Jun 10 19:00:08.153240 2020] [:error] [pid 16444] TypeError: 
taskinfo() takes exactly 2 arguments (1 given)
[Wed Jun 10 19:00:08.153243 2020] [:error] [pid 16444]
[Wed Jun 10 19:00:08.153285 2020] [:error] [pid 16444] 152 [ERROR] 
m=taskinfo u=None p=16444 r=203.133.180.188:45870 koji.web: Traceback 
(most recent call last):
[Wed Jun 10 19:00:08.153289 2020] [:error] [pid 16444]   File 
"/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in handle_request
[Wed Jun 10 19:00:08.153291 2020] [:error] [pid 16444]     result = 
func(environ, **data)
[Wed Jun 10 19:00:08.153304 2020] [:error] [pid 16444] TypeError: 
taskinfo() takes exactly 2 arguments (1 given)
[Wed Jun 10 19:00:08.153307 2020] [:error] [pid 16444]
[Wed Jun 10 19:00:10.020415 2020] [:error] [pid 18062] 2020-06-10 
19:00:10,020 [ERROR] m=None u=None p=18062 r=?:? koji.web: Traceback 
(most recent call last):
[Wed Jun 10 19:00:10.020446 2020] [:error] [pid 18062]   File 
"/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in handle_request
[Wed Jun 10 19:00:10.020449 2020] [:error] [pid 18062]     result = 
func(environ, **data)
[Wed Jun 10 19:00:10.020452 2020] [:error] [pid 18062] TypeError: 
taskinfo() takes exactly 2 arguments (1 given)
[Wed Jun 10 19:00:10.020454 2020] [:error] [pid 18062]
[Wed Jun 10 19:00:10.020502 2020] [:error] [pid 18062] 20 [ERROR] 
m=taskinfo u=None p=18062 r=203.133.180.188:45905 koji.web: Traceback 
(most recent call last):
[Wed Jun 10 19:00:10.020506 2020] [:error] [pid 18062]   File 
"/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in handle_request
[Wed Jun 10 19:00:10.020508 2020] [:error] [pid 18062]     result = 
func(environ, **data)
[Wed Jun 10 19:00:10.020510 2020] [:error] [pid 18062] TypeError: 
taskinfo() takes exactly 2 arguments (1 given)
[Wed Jun 10 19:00:10.020512 2020] [:error] [pid 18062]

No notifications in system.



10.06.2020 21:59, Ken Dreyer пишет:
>  This looks like kojid is getting an error from koji-hub when it
> calls
>  the "host.completeBuild" RPC.
>
>  The hub is raising a TypeError when calling the len() method on a
>  value that unexpectedly happens to be None, but I don't know much more
>  beyond that.
>
>  I don't see any len() calls in the completeBuild() or import_build()
>  methods in hub/kojihub.py. I don't see any additional len() calls
>  added between git log -p koji-1.17.0..koji-1.21.0.
>
>  Would you please paste the entire traceback if it's available in your
>  koji-hub's Apache error logs?  You should see it in
>  /var/log/httpd/error_log or ssl_error_log or thereabouts.
>
>  - Ken
>
>  On Wed, Jun 10, 2020 at 10:58 AM Viacheslav Dubrovskyi <dubrsl(a)gmail.com&gt; wrote:
> > Hi.
> >
> > After update koji from 17 -> 1.21.0-2.el7 all build failed with error:
> > <Fault 1: "<type 'exceptions.TypeError'>: object of type
> 'NoneType' has
> > no len()">
> >
> >
> > [slava@dubrwork ~]$ koji build el7
> > /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> > Uploading srpm:
> > /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> > [====================================] 100% 00:00:00  17.38 KiB 64.31
> > KiB/sec
> > Created task: 5873
> > Task info: http://koji.oro.cloud/koji/taskinfo?taskID=5873
> > Watching tasks (this may be safely interrupted)...
> > 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free
> > 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free -> open
> > (koji.oro.cloud)
> >     5874 rebuildSRPM (noarch): open (koji.oro.cloud)
> >     5874 rebuildSRPM (noarch): open (koji.oro.cloud) -> closed
> >     0 free  1 open  1 done  0 failed
> >     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
> >     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free ->
> > open (koji.oro.cloud)
> > 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): open
> > (koji.oro.cloud) -> FAILED: Fault: <Fault 1: "<type
> > 'exceptions.TypeError'>: object of type 'NoneType' has no
> len()">
> >     0 free  1 open  1 done  1 failed
> >     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): open
> > (koji.oro.cloud) -> closed
> >     0 free  0 open  2 done  1 failed
> >
> > 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm) failed
> >
> > In /var/log/kojid.log:
> >
> > 2020-06-10 16:37:14,329 [INFO] koji.TaskManager: Task 5875 (pid 3358)
> > exited with status 0
> > 2020-06-10 16:37:14,380 [INFO] koji.TaskManager: Expiring subsession
> > 6833 (task 5875)
> > 2020-06-10 16:37:14,529 [WARNING] koji.TaskManager: FAULT:
> > Traceback (most recent call last):
> >     File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 1339, in
> > runTask
> >       response = (handler.run(),)
> >     File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 329, in
> run
> >       return koji.util.call_with_argcheck(self.handler, self.params,
> > self.opts)
> >     File "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in
> > call_with_argcheck
> >       return func(*args, **kwargs)
> >     File "/usr/sbin/kojid", line 1062, in handler
> >       self.session.host.completeBuild(self.id, build_id, srpm, rpms,
> > brmap, logs=logs)
> >     File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2263,
> > in __call__
> >       return self.__func(self.__name, args, opts)
> >     File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2804,
> > in _callMethod
> >       raise err
> > Fault: <Fault 1: "<type 'exceptions.TypeError'>: object of
> type
> > 'NoneType' has no len()">
> >
> > $ rpm -qa | grep koji | sort
> > koji-1.21.0-2.el7.noarch
> > koji-builder-1.21.0-2.el7.noarch
> > koji-hub-1.21.0-2.el7.noarch
> > koji-utils-1.21.0-2.el7.noarch
> > koji-web-1.21.0-2.el7.noarch
> > python2-koji-1.21.0-2.el7.noarch
> > python2-koji-cli-plugins-1.21.0-2.el7.noarch
> > python2-koji-hub-1.21.0-2.el7.noarch
> > python2-koji-web-1.21.0-2.el7.noarch
> >
> > Help please to fix it.
> >
> > Thank you!
> >
> > --
> > WBR,
> > Viacheslav Dubrovskyi
> >
> >
> > _______________________________________________
> > koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> > To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
>  _______________________________________________
>  koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
>  To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]... 
-- 
WBR,
Viacheslav Dubrovskyi

[Viacheslav Dubrovskyi]

Hi Ken,

After set in /etc/koji-hub/hub.conf

DisableNotifications = True

EmailDomain = mydomain.com

Builds passed.

I don't know if it's bug, but anyway I found workaround. Thank you for 
point about error from koji-hub.


11.06.2020 08:49, Viacheslav Dubrovskyi пишет:
>  Hi Ken,
>
>  Here is entire traceback from /var/log/httpd/error_log
>
>  [Wed Jun 10 17:27:09.418133 2020] [:error] [pid 7760] 2020-06-10 
>  17:27:09,417 [WARNING] m=host.completeBuild u=koji.oro.cloud p=7760 
>  r=35.192.190.50:50042 koji.xmlrpc: Traceback (most recent call last):
>  [Wed Jun 10 17:27:09.418166 2020] [:error] [pid 7760]   File 
>  "/usr/share/koji-hub/kojixmlrpc.py", line 238, in _wrap_handler
>  [Wed Jun 10 17:27:09.418169 2020] [:error] [pid 7760]     response = 
>  handler(environ)
>  [Wed Jun 10 17:27:09.418171 2020] [:error] [pid 7760]   File 
>  "/usr/share/koji-hub/kojixmlrpc.py", line 281, in handle_rpc
>  [Wed Jun 10 17:27:09.418174 2020] [:error] [pid 7760]     return 
>  self._dispatch(method, params)
>  [Wed Jun 10 17:27:09.418176 2020] [:error] [pid 7760]   File 
>  "/usr/share/koji-hub/kojixmlrpc.py", line 318, in _dispatch
>  [Wed Jun 10 17:27:09.418178 2020] [:error] [pid 7760]     ret = 
>  koji.util.call_with_argcheck(func, params, opts)
>  [Wed Jun 10 17:27:09.418181 2020] [:error] [pid 7760]   File 
>  "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in 
>  call_with_argcheck
>  [Wed Jun 10 17:27:09.418183 2020] [:error] [pid 7760]     return 
>  func(*args, **kwargs)
>  [Wed Jun 10 17:27:09.418185 2020] [:error] [pid 7760]   File 
>  "/usr/share/koji-hub/kojihub.py", line 13602, in completeBuild
>  [Wed Jun 10 17:27:09.418187 2020] [:error] [pid 7760] 
>  build_notification(task_id, build_id)
>  [Wed Jun 10 17:27:09.418189 2020] [:error] [pid 7760]   File 
>  "/usr/share/koji-hub/kojihub.py", line 8401, in build_notification
>  [Wed Jun 10 17:27:09.418192 2020] [:error] [pid 7760]     if 
>  len(recipients) > 0:
>  [Wed Jun 10 17:27:09.418194 2020] [:error] [pid 7760] TypeError: 
>  object of type 'NoneType' has no len()
>  [Wed Jun 10 17:27:09.418196 2020] [:error] [pid 7760]
>  [Wed Jun 10 19:00:08.153189 2020] [:error] [pid 16444] 2020-06-10 
>  19:00:08,152 [ERROR] m=None u=None p=16444 r=?:? koji.web: Traceback 
>  (most recent call last):
>  [Wed Jun 10 19:00:08.153234 2020] [:error] [pid 16444]   File 
>  "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in 
>  handle_request
>  [Wed Jun 10 19:00:08.153238 2020] [:error] [pid 16444]     result = 
>  func(environ, **data)
>  [Wed Jun 10 19:00:08.153240 2020] [:error] [pid 16444] TypeError: 
>  taskinfo() takes exactly 2 arguments (1 given)
>  [Wed Jun 10 19:00:08.153243 2020] [:error] [pid 16444]
>  [Wed Jun 10 19:00:08.153285 2020] [:error] [pid 16444] 152 [ERROR] 
>  m=taskinfo u=None p=16444 r=203.133.180.188:45870 koji.web: Traceback 
>  (most recent call last):
>  [Wed Jun 10 19:00:08.153289 2020] [:error] [pid 16444]   File 
>  "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in 
>  handle_request
>  [Wed Jun 10 19:00:08.153291 2020] [:error] [pid 16444]     result = 
>  func(environ, **data)
>  [Wed Jun 10 19:00:08.153304 2020] [:error] [pid 16444] TypeError: 
>  taskinfo() takes exactly 2 arguments (1 given)
>  [Wed Jun 10 19:00:08.153307 2020] [:error] [pid 16444]
>  [Wed Jun 10 19:00:10.020415 2020] [:error] [pid 18062] 2020-06-10 
>  19:00:10,020 [ERROR] m=None u=None p=18062 r=?:? koji.web: Traceback 
>  (most recent call last):
>  [Wed Jun 10 19:00:10.020446 2020] [:error] [pid 18062]   File 
>  "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in 
>  handle_request
>  [Wed Jun 10 19:00:10.020449 2020] [:error] [pid 18062]     result = 
>  func(environ, **data)
>  [Wed Jun 10 19:00:10.020452 2020] [:error] [pid 18062] TypeError: 
>  taskinfo() takes exactly 2 arguments (1 given)
>  [Wed Jun 10 19:00:10.020454 2020] [:error] [pid 18062]
>  [Wed Jun 10 19:00:10.020502 2020] [:error] [pid 18062] 20 [ERROR] 
>  m=taskinfo u=None p=18062 r=203.133.180.188:45905 koji.web: Traceback 
>  (most recent call last):
>  [Wed Jun 10 19:00:10.020506 2020] [:error] [pid 18062]   File 
>  "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in 
>  handle_request
>  [Wed Jun 10 19:00:10.020508 2020] [:error] [pid 18062]     result = 
>  func(environ, **data)
>  [Wed Jun 10 19:00:10.020510 2020] [:error] [pid 18062] TypeError: 
>  taskinfo() takes exactly 2 arguments (1 given)
>  [Wed Jun 10 19:00:10.020512 2020] [:error] [pid 18062]
>
>  No notifications in system.
>
>
>
>  10.06.2020 21:59, Ken Dreyer пишет:
> > This looks like kojid is getting an error from koji-hub when it calls
> > the "host.completeBuild" RPC.
> >
> > The hub is raising a TypeError when calling the len() method on a
> > value that unexpectedly happens to be None, but I don't know much more
> > beyond that.
> >
> > I don't see any len() calls in the completeBuild() or import_build()
> > methods in hub/kojihub.py. I don't see any additional len() calls
> > added between git log -p koji-1.17.0..koji-1.21.0.
> >
> > Would you please paste the entire traceback if it's available in your
> > koji-hub's Apache error logs?  You should see it in
> > /var/log/httpd/error_log or ssl_error_log or thereabouts.
> >
> > - Ken
> >
> > On Wed, Jun 10, 2020 at 10:58 AM Viacheslav Dubrovskyi 
> > <dubrsl(a)gmail.com&gt; wrote:
> >> Hi.
> >>
> >> After update koji from 17 -> 1.21.0-2.el7 all build failed with error:
> >> <Fault 1: "<type 'exceptions.TypeError'>: object of type
> 'NoneType' has
> >> no len()">
> >>
> >>
> >> [slava@dubrwork ~]$ koji build el7
> >> /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
>
> >>
> >> Uploading srpm:
> >> /home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
>
> >>
> >> [====================================] 100% 00:00:00  17.38 KiB 64.31
> >> KiB/sec
> >> Created task: 5873
> >> Task info: http://koji.oro.cloud/koji/taskinfo?taskID=5873
> >> Watching tasks (this may be safely interrupted)...
> >> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free
> >> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free -> open
> >> (koji.oro.cloud)
> >>     5874 rebuildSRPM (noarch): open (koji.oro.cloud)
> >>     5874 rebuildSRPM (noarch): open (koji.oro.cloud) -> closed
> >>     0 free  1 open  1 done  0 failed
> >>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
> >>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free ->
> >> open (koji.oro.cloud)
> >> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): open
> >> (koji.oro.cloud) -> FAILED: Fault: <Fault 1: "<type
> >> 'exceptions.TypeError'>: object of type 'NoneType' has no
> len()">
> >>     0 free  1 open  1 done  1 failed
> >>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): open
> >> (koji.oro.cloud) -> closed
> >>     0 free  0 open  2 done  1 failed
> >>
> >> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm) failed
> >>
> >> In /var/log/kojid.log:
> >>
> >> 2020-06-10 16:37:14,329 [INFO] koji.TaskManager: Task 5875 (pid 3358)
> >> exited with status 0
> >> 2020-06-10 16:37:14,380 [INFO] koji.TaskManager: Expiring subsession
> >> 6833 (task 5875)
> >> 2020-06-10 16:37:14,529 [WARNING] koji.TaskManager: FAULT:
> >> Traceback (most recent call last):
> >>     File "/usr/lib/python2.7/site-packages/koji/daemon.py", line 
> >> 1339, in
> >> runTask
> >>       response = (handler.run(),)
> >>     File "/usr/lib/python2.7/site-packages/koji/tasks.py", line 329, 
> >> in run
> >>       return koji.util.call_with_argcheck(self.handler, self.params,
> >> self.opts)
> >>     File "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in
> >> call_with_argcheck
> >>       return func(*args, **kwargs)
> >>     File "/usr/sbin/kojid", line 1062, in handler
> >>       self.session.host.completeBuild(self.id, build_id, srpm, rpms,
> >> brmap, logs=logs)
> >>     File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 
> >> 2263,
> >> in __call__
> >>       return self.__func(self.__name, args, opts)
> >>     File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 
> >> 2804,
> >> in _callMethod
> >>       raise err
> >> Fault: <Fault 1: "<type 'exceptions.TypeError'>: object of
> type
> >> 'NoneType' has no len()">
> >>
> >> $ rpm -qa | grep koji | sort
> >> koji-1.21.0-2.el7.noarch
> >> koji-builder-1.21.0-2.el7.noarch
> >> koji-hub-1.21.0-2.el7.noarch
> >> koji-utils-1.21.0-2.el7.noarch
> >> koji-web-1.21.0-2.el7.noarch
> >> python2-koji-1.21.0-2.el7.noarch
> >> python2-koji-cli-plugins-1.21.0-2.el7.noarch
> >> python2-koji-hub-1.21.0-2.el7.noarch
> >> python2-koji-web-1.21.0-2.el7.noarch
> >>
> >> Help please to fix it.
> >>
> >> Thank you!
> >>
> >> -- 
> >> WBR,
> >> Viacheslav Dubrovskyi
> >>
> >>
> >> _______________________________________________
> >> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> >> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> >> Fedora Code of Conduct: 
> >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives: 
> >> https://lists.fedorahosted.org/archives/list/[e-mail]...
> > _______________________________________________
> > koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> > To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct: 
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: 
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
>  
-- 
WBR,
Viacheslav Dubrovskyi

[Ken Dreyer]

Thanks! Yes, it's a bug, apparently introduced in 1.18.0. Want to try
the patch at https://pagure.io/koji/pull-request/2308 ?

- Ken

On Thu, Jun 11, 2020 at 10:06 AM Viacheslav Dubrovskyi <dubrsl(a)gmail.com&gt; wrote:
>
> Hi Ken,
>
> After set in /etc/koji-hub/hub.conf
>
> DisableNotifications = True
>
> EmailDomain = mydomain.com
>
> Builds passed.
>
> I don't know if it's bug, but anyway I found workaround. Thank you for
> point about error from koji-hub.
>
>
> 11.06.2020 08:49, Viacheslav Dubrovskyi пишет:
> > Hi Ken,
> >
> > Here is entire traceback from /var/log/httpd/error_log
> >
> > [Wed Jun 10 17:27:09.418133 2020] [:error] [pid 7760] 2020-06-10
> > 17:27:09,417 [WARNING] m=host.completeBuild u=koji.oro.cloud p=7760
> > r=35.192.190.50:50042 koji.xmlrpc: Traceback (most recent call last):
> > [Wed Jun 10 17:27:09.418166 2020] [:error] [pid 7760]   File
> > "/usr/share/koji-hub/kojixmlrpc.py", line 238, in _wrap_handler
> > [Wed Jun 10 17:27:09.418169 2020] [:error] [pid 7760]     response =
> > handler(environ)
> > [Wed Jun 10 17:27:09.418171 2020] [:error] [pid 7760]   File
> > "/usr/share/koji-hub/kojixmlrpc.py", line 281, in handle_rpc
> > [Wed Jun 10 17:27:09.418174 2020] [:error] [pid 7760]     return
> > self._dispatch(method, params)
> > [Wed Jun 10 17:27:09.418176 2020] [:error] [pid 7760]   File
> > "/usr/share/koji-hub/kojixmlrpc.py", line 318, in _dispatch
> > [Wed Jun 10 17:27:09.418178 2020] [:error] [pid 7760]     ret =
> > koji.util.call_with_argcheck(func, params, opts)
> > [Wed Jun 10 17:27:09.418181 2020] [:error] [pid 7760]   File
> > "/usr/lib/python2.7/site-packages/koji/util.py", line 258, in
> > call_with_argcheck
> > [Wed Jun 10 17:27:09.418183 2020] [:error] [pid 7760]     return
> > func(*args, **kwargs)
> > [Wed Jun 10 17:27:09.418185 2020] [:error] [pid 7760]   File
> > "/usr/share/koji-hub/kojihub.py", line 13602, in completeBuild
> > [Wed Jun 10 17:27:09.418187 2020] [:error] [pid 7760]
> > build_notification(task_id, build_id)
> > [Wed Jun 10 17:27:09.418189 2020] [:error] [pid 7760]   File
> > "/usr/share/koji-hub/kojihub.py", line 8401, in build_notification
> > [Wed Jun 10 17:27:09.418192 2020] [:error] [pid 7760]     if
> > len(recipients) > 0:
> > [Wed Jun 10 17:27:09.418194 2020] [:error] [pid 7760] TypeError:
> > object of type 'NoneType' has no len()
> > [Wed Jun 10 17:27:09.418196 2020] [:error] [pid 7760]
> > [Wed Jun 10 19:00:08.153189 2020] [:error] [pid 16444] 2020-06-10
> > 19:00:08,152 [ERROR] m=None u=None p=16444 r=?:? koji.web: Traceback
> > (most recent call last):
> > [Wed Jun 10 19:00:08.153234 2020] [:error] [pid 16444]   File
> > "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in
> > handle_request
> > [Wed Jun 10 19:00:08.153238 2020] [:error] [pid 16444]     result =
> > func(environ, **data)
> > [Wed Jun 10 19:00:08.153240 2020] [:error] [pid 16444] TypeError:
> > taskinfo() takes exactly 2 arguments (1 given)
> > [Wed Jun 10 19:00:08.153243 2020] [:error] [pid 16444]
> > [Wed Jun 10 19:00:08.153285 2020] [:error] [pid 16444] 152 [ERROR]
> > m=taskinfo u=None p=16444 r=203.133.180.188:45870 koji.web: Traceback
> > (most recent call last):
> > [Wed Jun 10 19:00:08.153289 2020] [:error] [pid 16444]   File
> > "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in
> > handle_request
> > [Wed Jun 10 19:00:08.153291 2020] [:error] [pid 16444]     result =
> > func(environ, **data)
> > [Wed Jun 10 19:00:08.153304 2020] [:error] [pid 16444] TypeError:
> > taskinfo() takes exactly 2 arguments (1 given)
> > [Wed Jun 10 19:00:08.153307 2020] [:error] [pid 16444]
> > [Wed Jun 10 19:00:10.020415 2020] [:error] [pid 18062] 2020-06-10
> > 19:00:10,020 [ERROR] m=None u=None p=18062 r=?:? koji.web: Traceback
> > (most recent call last):
> > [Wed Jun 10 19:00:10.020446 2020] [:error] [pid 18062]   File
> > "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in
> > handle_request
> > [Wed Jun 10 19:00:10.020449 2020] [:error] [pid 18062]     result =
> > func(environ, **data)
> > [Wed Jun 10 19:00:10.020452 2020] [:error] [pid 18062] TypeError:
> > taskinfo() takes exactly 2 arguments (1 given)
> > [Wed Jun 10 19:00:10.020454 2020] [:error] [pid 18062]
> > [Wed Jun 10 19:00:10.020502 2020] [:error] [pid 18062] 20 [ERROR]
> > m=taskinfo u=None p=18062 r=203.133.180.188:45905 koji.web: Traceback
> > (most recent call last):
> > [Wed Jun 10 19:00:10.020506 2020] [:error] [pid 18062]   File
> > "/usr/share/koji-web/scripts/wsgi_publisher.py", line 377, in
> > handle_request
> > [Wed Jun 10 19:00:10.020508 2020] [:error] [pid 18062]     result =
> > func(environ, **data)
> > [Wed Jun 10 19:00:10.020510 2020] [:error] [pid 18062] TypeError:
> > taskinfo() takes exactly 2 arguments (1 given)
> > [Wed Jun 10 19:00:10.020512 2020] [:error] [pid 18062]
> >
> > No notifications in system.
> >
> >
> >
> > 10.06.2020 21:59, Ken Dreyer пишет:
> >> This looks like kojid is getting an error from koji-hub when it calls
> >> the "host.completeBuild" RPC.
> >>
> >> The hub is raising a TypeError when calling the len() method on a
> >> value that unexpectedly happens to be None, but I don't know much more
> >> beyond that.
> >>
> >> I don't see any len() calls in the completeBuild() or import_build()
> >> methods in hub/kojihub.py. I don't see any additional len() calls
> >> added between git log -p koji-1.17.0..koji-1.21.0.
> >>
> >> Would you please paste the entire traceback if it's available in your
> >> koji-hub's Apache error logs?  You should see it in
> >> /var/log/httpd/error_log or ssl_error_log or thereabouts.
> >>
> >> - Ken
> >>
> >> On Wed, Jun 10, 2020 at 10:58 AM Viacheslav Dubrovskyi
> >> <dubrsl(a)gmail.com&gt; wrote:
> >>> Hi.
> >>>
> >>> After update koji from 17 -> 1.21.0-2.el7 all build failed with
error:
> >>> <Fault 1: "<type 'exceptions.TypeError'>: object
of type 'NoneType' has
> >>> no len()">
> >>>
> >>>
> >>> [slava@dubrwork ~]$ koji build el7
> >>>
/home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> >>>
> >>> Uploading srpm:
> >>>
/home/slava/Documents/ORO/SCL/rh-postgresql12/rh-postgresql12-3.4-2.fc32.src.rpm
> >>>
> >>> [====================================] 100% 00:00:00  17.38 KiB 64.31
> >>> KiB/sec
> >>> Created task: 5873
> >>> Task info: http://koji.oro.cloud/koji/taskinfo?taskID=5873
> >>> Watching tasks (this may be safely interrupted)...
> >>> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free
> >>> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): free -> open
> >>> (koji.oro.cloud)
> >>>     5874 rebuildSRPM (noarch): open (koji.oro.cloud)
> >>>     5874 rebuildSRPM (noarch): open (koji.oro.cloud) -> closed
> >>>     0 free  1 open  1 done  0 failed
> >>>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
> >>>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): free
->
> >>> open (koji.oro.cloud)
> >>> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm): open
> >>> (koji.oro.cloud) -> FAILED: Fault: <Fault 1: "<type
> >>> 'exceptions.TypeError'>: object of type 'NoneType'
has no len()">
> >>>     0 free  1 open  1 done  1 failed
> >>>     5875 buildArch (rh-postgresql12-3.4-2.el7.src.rpm, x86_64): open
> >>> (koji.oro.cloud) -> closed
> >>>     0 free  0 open  2 done  1 failed
> >>>
> >>> 5873 build (el7, rh-postgresql12-3.4-2.fc32.src.rpm) failed
> >>>
> >>> In /var/log/kojid.log:
> >>>
> >>> 2020-06-10 16:37:14,329 [INFO] koji.TaskManager: Task 5875 (pid 3358)
> >>> exited with status 0
> >>> 2020-06-10 16:37:14,380 [INFO] koji.TaskManager: Expiring subsession
> >>> 6833 (task 5875)
> >>> 2020-06-10 16:37:14,529 [WARNING] koji.TaskManager: FAULT:
> >>> Traceback (most recent call last):
> >>>     File "/usr/lib/python2.7/site-packages/koji/daemon.py",
line
> >>> 1339, in
> >>> runTask
> >>>       response = (handler.run(),)
> >>>     File "/usr/lib/python2.7/site-packages/koji/tasks.py",
line 329,
> >>> in run
> >>>       return koji.util.call_with_argcheck(self.handler, self.params,
> >>> self.opts)
> >>>     File "/usr/lib/python2.7/site-packages/koji/util.py", line
258, in
> >>> call_with_argcheck
> >>>       return func(*args, **kwargs)
> >>>     File "/usr/sbin/kojid", line 1062, in handler
> >>>       self.session.host.completeBuild(self.id, build_id, srpm, rpms,
> >>> brmap, logs=logs)
> >>>     File "/usr/lib/python2.7/site-packages/koji/__init__.py",
line
> >>> 2263,
> >>> in __call__
> >>>       return self.__func(self.__name, args, opts)
> >>>     File "/usr/lib/python2.7/site-packages/koji/__init__.py",
line
> >>> 2804,
> >>> in _callMethod
> >>>       raise err
> >>> Fault: <Fault 1: "<type 'exceptions.TypeError'>:
object of type
> >>> 'NoneType' has no len()">
> >>>
> >>> $ rpm -qa | grep koji | sort
> >>> koji-1.21.0-2.el7.noarch
> >>> koji-builder-1.21.0-2.el7.noarch
> >>> koji-hub-1.21.0-2.el7.noarch
> >>> koji-utils-1.21.0-2.el7.noarch
> >>> koji-web-1.21.0-2.el7.noarch
> >>> python2-koji-1.21.0-2.el7.noarch
> >>> python2-koji-cli-plugins-1.21.0-2.el7.noarch
> >>> python2-koji-hub-1.21.0-2.el7.noarch
> >>> python2-koji-web-1.21.0-2.el7.noarch
> >>>
> >>> Help please to fix it.
> >>>
> >>> Thank you!
> >>>
> >>> --
> >>> WBR,
> >>> Viacheslav Dubrovskyi
> >>>
> >>>
> >>> _______________________________________________
> >>> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> >>> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> >>> Fedora Code of Conduct:
> >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >>> List Archives:
> >>>
https://lists.fedorahosted.org/archives/list/[e-mail]...
> >> _______________________________________________
> >> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> >> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> >> Fedora Code of Conduct:
> >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> >> List Archives:
> >>
https://lists.fedorahosted.org/archives/list/[e-mail]...
> >
> --
> WBR,
> Viacheslav Dubrovskyi
>
>
> _______________________________________________
> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedorahosted.org/archives/list/[e-mail]...