12:24 p.m.
I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
tasks never complete, and I see this message in the logs:
For buildSRPMFromSCM:
Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
For createrepo:
Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
buildArch tasks are fine.
Can you tell me what this means?
Thanks,
-Mat
--
Mátyás (Mat) Selmeci
Open Science Grid Software Team / Center for High-Throughput Computing
University of Wisconsin-Madison Department of Computer Sciences
6:14 a.m.
Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
I'm having problems with kojid 1.11: buildSRPMFromSCM and
createrepo
tasks never complete, and I see this message in the logs:
For buildSRPMFromSCM:
Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
For createrepo:
Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
buildArch tasks are fine.
Can you tell me what this means?
Thanks,
-Mat
Which version are you using? I'm hitting similar problems with very
latest master branch code (related to psycopg2 change), but not with
released code.
Technically it means, that there was probably some server-side traceback
during xmlrpc call which resulted in returning error instead of
correctly formed xml.
--
Tomas Kopecek <tkopecek(a)redhat.com>
Release Engineering Development, RedHat
11:09 a.m.
On 01/19/17 06:14, Tomas Kopecek wrote:
Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
> I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
> tasks never complete, and I see this message in the logs:
>
> For buildSRPMFromSCM:
>
> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
>
> For createrepo:
>
> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
>
> buildArch tasks are fine.
>
> Can you tell me what this means?
>
> Thanks,
> -Mat
Which version are you using? I'm hitting similar problems with very
latest master branch code (related to psycopg2 change), but not with
released code.
Technically it means, that there was probably some server-side traceback
during xmlrpc call which resulted in returning error instead of
correctly formed xml.
We're using 1.11.0-1 from EPEL, but we have some patches on top of it.
I'm not able to reproduce the problem on a test instance, however. Some
of the python dependencies on our production server are older (most
notably python-requests, which was 1.1.0 instead of 2.6.0), so maybe
that's it.
-Mat
1:51 p.m.
On 01/19/2017 07:14 AM, Tomas Kopecek wrote:
Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
> I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
> tasks never complete, and I see this message in the logs:
>
> For buildSRPMFromSCM:
>
> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
>
> For createrepo:
>
> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
>
> buildArch tasks are fine.
>
> Can you tell me what this means?
>
> Thanks,
> -Mat
Which version are you using? I'm hitting similar problems with very
latest master branch code (related to psycopg2 change), but not with
released code.
Technically it means, that there was probably some server-side traceback
during xmlrpc call which resulted in returning error instead of
correctly formed xml.
Well this is possible. It's fairly unusual for a server side traceback
to do this. It would have to happen in a very odd way to not turn into a
Fault (would are not retried by the clients).
Please check the following things.
1) Look at the hub logs and see if there are some related tracebacks there.
2) Try running kojid in the foreground with debugging on (koji --fg
--debug). That should at least show the full traceback for those retried
calls.
2:14 p.m.
On 01/19/17 13:51, Mike McLean wrote:
On 01/19/2017 07:14 AM, Tomas Kopecek wrote:
> Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
>> I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
>> tasks never complete, and I see this message in the logs:
>>
>> For buildSRPMFromSCM:
>>
>> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
>>
>> For createrepo:
>>
>> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
>>
>> buildArch tasks are fine.
>>
>> Can you tell me what this means?
>>
>> Thanks,
>> -Mat
>
> Which version are you using? I'm hitting similar problems with very
> latest master branch code (related to psycopg2 change), but not with
> released code.
>
> Technically it means, that there was probably some server-side traceback
> during xmlrpc call which resulted in returning error instead of
> correctly formed xml.
Well this is possible. It's fairly unusual for a server side traceback
to do this. It would have to happen in a very odd way to not turn into a
Fault (would are not retried by the clients).
Please check the following things.
1) Look at the hub logs and see if there are some related tracebacks there.
2) Try running kojid in the foreground with debugging on (koji --fg
--debug). That should at least show the full traceback for those retried
calls.
I didn't see any tracebacks in the hub logs, though ssl_error_log shows
"Invalid Content-Length" messages that appear to be correlated. I'll try
your suggestion when I can reproduce the problem on a non-production host.
Thanks,
-Mat
4:06 p.m.
On 01/19/17 14:14, Mátyás Selmeci wrote:
On 01/19/17 13:51, Mike McLean wrote:
> On 01/19/2017 07:14 AM, Tomas Kopecek wrote:
>> Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
>>> I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
>>> tasks never complete, and I see this message in the logs:
>>>
>>> For buildSRPMFromSCM:
>>>
>>> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
>>>
>>> For createrepo:
>>>
>>> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
>>>
>>> buildArch tasks are fine.
>>>
>>> Can you tell me what this means?
>>>
>>> Thanks,
>>> -Mat
>>
>> Which version are you using? I'm hitting similar problems with very
>> latest master branch code (related to psycopg2 change), but not with
>> released code.
>>
>> Technically it means, that there was probably some server-side traceback
>> during xmlrpc call which resulted in returning error instead of
>> correctly formed xml.
>
> Well this is possible. It's fairly unusual for a server side traceback
> to do this. It would have to happen in a very odd way to not turn into a
> Fault (would are not retried by the clients).
>
> Please check the following things.
>
> 1) Look at the hub logs and see if there are some related tracebacks
> there.
>
> 2) Try running kojid in the foreground with debugging on (koji --fg
> --debug). That should at least show the full traceback for those retried
> calls.
I didn't see any tracebacks in the hub logs, though ssl_error_log shows
"Invalid Content-Length" messages that appear to be correlated. I'll try
your suggestion when I can reproduce the problem on a non-production host.
The full traceback from kojid was:
koji:2000 Opening new requests session
koji:2404 Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line
2360, in _callMethod
return self._sendCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line
2274, in _sendCall
return self._sendOneCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line
2321, in _sendOneCall
ret = self._read_xmlrpc_response(r)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line
2334, in _read_xmlrpc_response
p.feed(chunk)
File "/usr/lib64/python2.6/xmlrpclib.py", line 601, in feed
self._parser.Parse(data, 0)
ExpatError: syntax error: line 1, column 49
I needed to downgrade python-requests to 1.1.0-4 (which required
downgrading python-urllib3 to 1.5-7) in order to reproduce the error, so
I guess it was just a case of outdated dependencies!
My problem is likely solved. Thanks for the help!
-Mat
11:07 a.m.
On 01/19/2017 05:06 PM, Mátyás Selmeci wrote:
On 01/19/17 14:14, Mátyás Selmeci wrote:
> On 01/19/17 13:51, Mike McLean wrote:
>> On 01/19/2017 07:14 AM, Tomas Kopecek wrote:
>>> Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
>>>> I'm having problems with kojid 1.11: buildSRPMFromSCM and createrepo
>>>> tasks never complete, and I see this message in the logs:
>>>>
>>>> For buildSRPMFromSCM:
>>>>
>>>> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column 49
>>>>
>>>> For createrepo:
>>>>
>>>> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column 49
>>>>
>>>> buildArch tasks are fine.
>>>>
>>>> Can you tell me what this means?
>>>>
>>>> Thanks,
>>>> -Mat
>>>
>>> Which version are you using? I'm hitting similar problems with very
>>> latest master branch code (related to psycopg2 change), but not with
>>> released code.
>>>
>>> Technically it means, that there was probably some server-side
>>> traceback
>>> during xmlrpc call which resulted in returning error instead of
>>> correctly formed xml.
>>
>> Well this is possible. It's fairly unusual for a server side traceback
>> to do this. It would have to happen in a very odd way to not turn into a
>> Fault (would are not retried by the clients).
>>
>> Please check the following things.
>>
>> 1) Look at the hub logs and see if there are some related tracebacks
>> there.
>>
>> 2) Try running kojid in the foreground with debugging on (koji --fg
>> --debug). That should at least show the full traceback for those retried
>> calls.
>
> I didn't see any tracebacks in the hub logs, though ssl_error_log shows
> "Invalid Content-Length" messages that appear to be correlated. I'll
try
> your suggestion when I can reproduce the problem on a non-production
> host.
The full traceback from kojid was:
koji:2000 Opening new requests session
koji:2404 Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2360,
in _callMethod
return self._sendCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2274,
in _sendCall
return self._sendOneCall(handler, headers, request)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2321,
in _sendOneCall
ret = self._read_xmlrpc_response(r)
File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2334,
in _read_xmlrpc_response
p.feed(chunk)
File "/usr/lib64/python2.6/xmlrpclib.py", line 601, in feed
self._parser.Parse(data, 0)
ExpatError: syntax error: line 1, column 49
I needed to downgrade python-requests to 1.1.0-4 (which required
downgrading python-urllib3 to 1.5-7) in order to reproduce the error, so
I guess it was just a case of outdated dependencies!
I'm curious what OS this is on?
My problem is likely solved. Thanks for the help!
-Mat
_______________________________________________
koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
11:48 a.m.
On 01/23/17 11:07, Mike McLean wrote:
On 01/19/2017 05:06 PM, Mátyás Selmeci wrote:
> On 01/19/17 14:14, Mátyás Selmeci wrote:
>> On 01/19/17 13:51, Mike McLean wrote:
>>> On 01/19/2017 07:14 AM, Tomas Kopecek wrote:
>>>> Dne 18.1.2017 v 19:24 Mátyás Selmeci napsal(a):
>>>>> I'm having problems with kojid 1.11: buildSRPMFromSCM and
createrepo
>>>>> tasks never complete, and I see this message in the logs:
>>>>>
>>>>> For buildSRPMFromSCM:
>>>>>
>>>>> Try #1 for call 8 (rawUpload) failed: syntax error: line 1, column
49
>>>>>
>>>>> For createrepo:
>>>>>
>>>>> Try #1 for call 2 (rawUpload) failed: syntax error: line 1, column
49
>>>>>
>>>>> buildArch tasks are fine.
>>>>>
>>>>> Can you tell me what this means?
>>>>>
>>>>> Thanks,
>>>>> -Mat
>>>>
>>>> Which version are you using? I'm hitting similar problems with very
>>>> latest master branch code (related to psycopg2 change), but not with
>>>> released code.
>>>>
>>>> Technically it means, that there was probably some server-side
>>>> traceback
>>>> during xmlrpc call which resulted in returning error instead of
>>>> correctly formed xml.
>>>
>>> Well this is possible. It's fairly unusual for a server side traceback
>>> to do this. It would have to happen in a very odd way to not turn
>>> into a
>>> Fault (would are not retried by the clients).
>>>
>>> Please check the following things.
>>>
>>> 1) Look at the hub logs and see if there are some related tracebacks
>>> there.
>>>
>>> 2) Try running kojid in the foreground with debugging on (koji --fg
>>> --debug). That should at least show the full traceback for those
>>> retried
>>> calls.
>>
>> I didn't see any tracebacks in the hub logs, though ssl_error_log shows
>> "Invalid Content-Length" messages that appear to be correlated.
I'll try
>> your suggestion when I can reproduce the problem on a non-production
>> host.
>
> The full traceback from kojid was:
> koji:2000 Opening new requests session
> koji:2404 Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2360,
> in _callMethod
> return self._sendCall(handler, headers, request)
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2274,
> in _sendCall
> return self._sendOneCall(handler, headers, request)
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2321,
> in _sendOneCall
> ret = self._read_xmlrpc_response(r)
> File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 2334,
> in _read_xmlrpc_response
> p.feed(chunk)
> File "/usr/lib64/python2.6/xmlrpclib.py", line 601, in feed
> self._parser.Parse(data, 0)
> ExpatError: syntax error: line 1, column 49
>
> I needed to downgrade python-requests to 1.1.0-4 (which required
> downgrading python-urllib3 to 1.5-7) in order to reproduce the error, so
> I guess it was just a case of outdated dependencies!
I'm curious what OS this is on?
This was Scientific Linux 6.6. Looking at its repos, it did not include
python-requests at all; I'm guessing we got python-requests from EPEL,
and it was later dropped from EPEL once it got included in 6.7.
-Mat
2650
days inactive
2655
days old
koji-devel@lists.fedorahosted.org
7 comments
3 participants
participants (3)
-
Mike McLean -
Mátyás Selmeci -
Tomas Kopecek