Koji running on RHEL / CentOS 7

[Allen Hewes]

Hi,

I am trying to move from Koji 1.9.0 on RHEL 5.11 to Koji 1.9.0 (gitcd45e886) on CentOS 7.
But it didn't help my issue:

Testing out the new Koji instance with a task submission:
$ koji --debug regen-repo el5-decisiv
successfully connected to hub
Warning: el5-decisiv is not a build tag
Warning: tag el5-decisiv has an empty arch list Regenerating repo for tag el5-decisiv
Watching tasks (this may be safely interrupted)...
30 newRepo (el5-decisiv): open (koji.decisiv.net) Traceback (most recent call last):
  File "/usr/bin/koji", line 6566, in <module>
    rv = locals()[command].__call__(options, session, args)
  File "/usr/bin/koji", line 6410, in handle_regen_repo
    return watch_tasks(session, [task_id], quiet=options.quiet)
  File "/usr/bin/koji", line 472, in watch_tasks
    changed = task.update()
  File "/usr/bin/koji", line 377, in update
    self.info = self.session.getTaskInfo(self.id, request=True)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1556, in
__call__
    return self.__func(self.__name,args,opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1899, in
_callMethod
    return self._sendCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1810, in
_sendCall
    return self._sendOneCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1830, in
_sendOneCall
    response = cnx.getresponse()
  File "/usr/lib64/python2.7/httplib.py", line 1045, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 409, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib64/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line 140,
in recv
    return con.recv(bufsize, flags)
OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF')

I added a rescue of SSL.SysCallError in the recv() function in
/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py:

diff -uab /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py
/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py.b
--- /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py  2015-01-29
04:00:08.000000000 +0000
+++ /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py.b        2015-01-29
04:00:56.224059850 +0000
@@ -138,6 +138,11 @@

             try:
                 return con.recv(bufsize, flags)
+            except SSL.SysCallError as e:
+                if e.args == (-1, 'Unexpected EOF'):
+                    break
+                else:
+                    raise
             except SSL.ZeroReturnError:
                 return None
             except SSL.WantReadError:

Which then the task submission succeeds:
$ koji --debug regen-repo el5-decisiv
successfully connected to hub
Warning: el5-decisiv is not a build tag
Warning: tag el5-decisiv has an empty arch list Regenerating repo for tag el5-decisiv
Watching tasks (this may be safely interrupted)...
32 newRepo (el5-decisiv): free
32 newRepo (el5-decisiv): free -> closed
  0 free  0 open  1 done  0 failed

32 newRepo (el5-decisiv) completed successfully

I couldn't find a Koji ticket for this error so I don't know if I am making it
worse or what I added is Good Enough to use? Is this a known issue?

I also had to install a newer createrepo. I was getting the mergerepo error here:
https://bugzilla.redhat.com/show_bug.cgi?id=1058975

Maybe createrepo should be updated for EPEL 7?

/allen

________________________________

Disclaimer Confidentiality Notice: This e-mail, and any attachments and/or documents
linked to this email, are intended for the addressee and may contain information that is
privileged, confidential, proprietary, or otherwise protected by law. Any dissemination,
distribution, or copying is prohibited. This notice serves as a confidentiality marking
for the purpose of any confidentiality or nondisclosure agreement. If you have received
this communication in error, please contact the original sender.

[Didier FABERT]

On Thursday 29 January 2015 18:11:53 Allen Hewes wrote:

Hi,

el7 have md5 disable and if you have your ssl certificates with 
'default_md=md5' parameter, you must recreate your pki with this parameter to 
sha1 or better sha256 in your ssl.cnf 
(http://fedoraproject.org/wiki/Koji/ServerHowTo).

to be sure that's the problem:
OPENSSL_ENABLE_MD5_VERIFY=1 koji regen-repo el5-decisiv

if this command run successfully, you know what to do ...

>  Hi,
>  
>  I am trying to move from Koji 1.9.0 on RHEL 5.11 to Koji 1.9.0 (gitcd45e886)
>  on CentOS 7. But it didn't help my issue:
>  
>  Testing out the new Koji instance with a task submission:
>  $ koji --debug regen-repo el5-decisiv
>  successfully connected to hub
>  Warning: el5-decisiv is not a build tag
>  Warning: tag el5-decisiv has an empty arch list Regenerating repo for tag
>  el5-decisiv Watching tasks (this may be safely interrupted)... 30 newRepo
>  (el5-decisiv): open (koji.decisiv.net) Traceback (most recent call last):
>  File "/usr/bin/koji", line 6566, in <module>
>      rv = locals()[command].__call__(options, session, args)
>    File "/usr/bin/koji", line 6410, in handle_regen_repo
>      return watch_tasks(session, [task_id], quiet=options.quiet)
>    File "/usr/bin/koji", line 472, in watch_tasks
>      changed = task.update()
>    File "/usr/bin/koji", line 377, in update
>      self.info = self.session.getTaskInfo(self.id, request=True)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1556, in
>  __call__ return self.__func(self.__name,args,opts)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1899, in
>  _callMethod return self._sendCall(handler, headers, request)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1810, in
>  _sendCall return self._sendOneCall(handler, headers, request)
>    File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1830, in
>  _sendOneCall response = cnx.getresponse()
>    File "/usr/lib64/python2.7/httplib.py", line 1045, in getresponse
>      response.begin()
>    File "/usr/lib64/python2.7/httplib.py", line 409, in begin
>      version, status, reason = self._read_status()
>    File "/usr/lib64/python2.7/httplib.py", line 365, in _read_status
>      line = self.fp.readline(_MAXLINE + 1)
>    File "/usr/lib64/python2.7/socket.py", line 476, in readline
>      data = self._sock.recv(self._rbufsize)
>    File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line
>  140, in recv return con.recv(bufsize, flags)
>  OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF')
>  
>  I added a rescue of SSL.SysCallError in the recv() function in
>  /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py:
>  
>  diff -uab /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py
>  /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py.b ---
>  /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py  2015-01-29
>  04:00:08.000000000 +0000 +++
>  /usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py.b       
>  2015-01-29 04:00:56.224059850 +0000 @@ -138,6 +138,11 @@
>  
>               try:
>                   return con.recv(bufsize, flags)
>  +            except SSL.SysCallError as e:
>  +                if e.args == (-1, 'Unexpected EOF'):
>  +                    break
>  +                else:
>  +                    raise
>               except SSL.ZeroReturnError:
>                   return None
>               except SSL.WantReadError:
>  
>  Which then the task submission succeeds:
>  $ koji --debug regen-repo el5-decisiv
>  successfully connected to hub
>  Warning: el5-decisiv is not a build tag
>  Warning: tag el5-decisiv has an empty arch list Regenerating repo for tag
>  el5-decisiv Watching tasks (this may be safely interrupted)... 32 newRepo
>  (el5-decisiv): free
>  32 newRepo (el5-decisiv): free -> closed
>    0 free  0 open  1 done  0 failed
>  
>  32 newRepo (el5-decisiv) completed successfully
>  
>  I couldn't find a Koji ticket for this error so I don't know if I am making
>  it worse or what I added is Good Enough to use? Is this a known issue?
>  
>  I also had to install a newer createrepo. I was getting the mergerepo error
>  here: https://bugzilla.redhat.com/show_bug.cgi?id=1058975
>  
>  Maybe createrepo should be updated for EPEL 7?
>  
>  /allen
>  
>  ________________________________
>  
>  Disclaimer Confidentiality Notice: This e-mail, and any attachments and/or
>  documents linked to this email, are intended for the addressee and may
>  contain information that is privileged, confidential, proprietary, or
>  otherwise protected by law. Any dissemination, distribution, or copying is
>  prohibited. This notice serves as a confidentiality marking for the purpose
>  of any confidentiality or nondisclosure agreement. If you have received
>  this communication in error, please contact the original sender. --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys 
-- 
Didier

[Allen Hewes]

>  el7 have md5 disable and if you have your ssl certificates with
>  'default_md=md5' parameter, you must recreate your pki with this
>  parameter to
>  sha1 or better sha256 in your ssl.cnf
>  (http://fedoraproject.org/wiki/Koji/ServerHowTo).
>
>  to be sure that's the problem:
>  OPENSSL_ENABLE_MD5_VERIFY=1 koji regen-repo el5-decisiv
>
>  if this command run successfully, you know what to do ...
>  
Hi Didier,

Yep, I knew this. I remembered the e-mail on the list. Also, I didn't move/use any of
my current Koji configuration files from my running instance. I made a new Koji instance
from scratch. I made sure to use the SHA256 crypto. It's also the crypto specified the
example ssl.cnf on the Fedora documentation link you sent.

Also, koji commands work. It's just the polling watching function doesn't unless I
rescue the OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF'). From what I can find
out, this is a NO-OP situation that isn't currently handled in Koji's code. The
koji client *is* authenticating via SSL but then the polling (watching the task/request)
doesn't work.

I see the same "Unexpected EOF" (unless I rescue it) in /var/log/kojid.log:
2015-01-29 03:12:50,257 [INFO] koji: Try #1 for call 362 (listBuildroots) failed: (-1,
'Unexpected EOF')

I will double check the SSL certs but I am confident that I would get a different error
message.

Thanks,

/allen

________________________________

Disclaimer Confidentiality Notice: This e-mail, and any attachments and/or documents
linked to this email, are intended for the addressee and may contain information that is
privileged, confidential, proprietary, or otherwise protected by law. Any dissemination,
distribution, or copying is prohibited. This notice serves as a confidentiality marking
for the purpose of any confidentiality or nondisclosure agreement. If you have received
this communication in error, please contact the original sender.

[Sérgio Basto]

On Sex, 2015-01-30 at 18:29 +0000, Allen Hewes wrote:
>  >
>  > el7 have md5 disable and if you have your ssl certificates with
>  > 'default_md=md5' parameter, you must recreate your pki with this
>  > parameter to
>  > sha1 or better sha256 in your ssl.cnf
>  > (http://fedoraproject.org/wiki/Koji/ServerHowTo).
>  >
>  > to be sure that's the problem:
>  > OPENSSL_ENABLE_MD5_VERIFY=1 koji regen-repo el5-decisiv
>  >
>  > if this command run successfully, you know what to do ...
>  
>  Hi Didier,
>  
>  Yep, I knew this. I remembered the e-mail on the list. Also, I didn't move/use any of
> my current Koji configuration files from my running instance. I made a new Koji instance
> from scratch. I made sure to use the SHA256 crypto. It's also the crypto specified the
> example ssl.cnf on the Fedora documentation link you sent.
>  
>  Also, koji commands work. It's just the polling watching function doesn't unless
> I rescue the OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF'). From what I can
> find out, this is a NO-OP situation that isn't currently handled in Koji's code.
> The koji client *is* authenticating via SSL but then the polling (watching the
> task/request) doesn't work.
>  
>  I see the same "Unexpected EOF" (unless I rescue it) in /var/log/kojid.log:
>  2015-01-29 03:12:50,257 [INFO] koji: Try #1 for call 362 (listBuildroots) failed: (-1,
> 'Unexpected EOF')
>  
>  I will double check the SSL certs but I am confident that I would get a different error
> message. 
yeap koji server now needs to be build in a sha256 certs and I'm getting
same problem on Fedora 21 with all updates-testing available for this
area 

your patch mention in first message of this thread works great and I
could bootstrap one koji server ( with self signed certificates ) 

it also looks like this bug
https://bugzilla.redhat.com/show_bug.cgi?id=1186994 


Thanks,
-- 
Sérgio M. B.