Freeze break: link on pkgs lookaside for fedpkg namespace changes

[Kevin Fenzi]

Greetings. 

In recent fedpkg/rpkg versions (at least the ones in rawhide now) it's
incorrectly looking for a namespaced (rpms vs anything else) lookaside
instead of the non namespaced one we have. 

So, this gives: 

% fedpkg -v srpm
Creating repo object from /home/kevin/git/pkgs/midori/master
Downloading midori_0.5.11_all_.tar.bz2
Full url:
http://pkgs.fedoraproject.org/repo/pkgs/rpms/midori/midori_0.5.11_all_.ta...
######################################################################## 100.0%
Could not execute srpm: Server returned status code 404
Traceback (most recent call last):
  File "/usr/bin/fedpkg", line 16, in <module>
    main()
  File "/usr/lib/python2.7/site-packages/fedpkg/__main__.py", line 69, in main
    sys.exit(client.args.command())
  File "/usr/lib/python2.7/site-packages/pyrpkg/cli.py", line 1234, in srpm
    self.cmd.sources()
  File "/usr/lib/python2.7/site-packages/pyrpkg/__init__.py", line 1696, in
sources
    hashtype=entry.hashtype, branch=self.branch_merge)
  File "/usr/lib/python2.7/site-packages/pyrpkg/lookaside.py", line 174, in
download
    raise DownloadError('Server returned status code %d' % status)
pyrpkg.errors.DownloadError: Server returned status code 404

And on upload it puts it in a rpms dir thats not correct. 

We can see 3 packages already have this: 

-bash-4.2$ pwd
/srv/cache/lookaside/pkgs/rpms
-bash-4.2$ ls -l
total 12
drwxr-sr-x. 3 apache apache 4096 Mar 15 12:21 compose-utils
drwxr-sr-x. 3 apache apache 4096 Mar 16 11:06 dbus
drwxr-sr-x. 3 apache apache 4096 Mar 16 11:00 xdg-app

So, I would like to do the following: 

* Move all the /srv/cache/lookaside/pkgs/rpms/ to
  their /src/cache/lookaside/pkgs/ versions. 

* create a link from /srv/cache/lookaside/pkgs/rpms
  to /srv/cache/lookaside/pkgs

This should fix this issue until we push out a fedpkg/rpkg fix or want
to namespace lookaside. 

+1s?

kevin

[Peter Robinson]

On Wed, Mar 16, 2016 at 3:33 PM, Kevin Fenzi <kevin(a)scrye.com&gt; wrote:
>  Greetings.
>
>  In recent fedpkg/rpkg versions (at least the ones in rawhide now) it's
>  incorrectly looking for a namespaced (rpms vs anything else) lookaside
>  instead of the non namespaced one we have.
>
>  So, this gives:
>
>  % fedpkg -v srpm
>  Creating repo object from /home/kevin/git/pkgs/midori/master
>  Downloading midori_0.5.11_all_.tar.bz2
>  Full url:
> http://pkgs.fedoraproject.org/repo/pkgs/rpms/midori/midori_0.5.11_all_.ta...
>  ######################################################################## 100.0%
>  Could not execute srpm: Server returned status code 404
>  Traceback (most recent call last):
>    File "/usr/bin/fedpkg", line 16, in <module>
>      main()
>    File "/usr/lib/python2.7/site-packages/fedpkg/__main__.py", line 69, in main
>      sys.exit(client.args.command())
>    File "/usr/lib/python2.7/site-packages/pyrpkg/cli.py", line 1234, in srpm
>      self.cmd.sources()
>    File "/usr/lib/python2.7/site-packages/pyrpkg/__init__.py", line 1696, in
> sources
>      hashtype=entry.hashtype, branch=self.branch_merge)
>    File "/usr/lib/python2.7/site-packages/pyrpkg/lookaside.py", line 174, in
> download
>      raise DownloadError('Server returned status code %d' % status)
>  pyrpkg.errors.DownloadError: Server returned status code 404
>
>  And on upload it puts it in a rpms dir thats not correct.
>
>  We can see 3 packages already have this:
>
>  -bash-4.2$ pwd
>  /srv/cache/lookaside/pkgs/rpms
>  -bash-4.2$ ls -l
>  total 12
>  drwxr-sr-x. 3 apache apache 4096 Mar 15 12:21 compose-utils
>  drwxr-sr-x. 3 apache apache 4096 Mar 16 11:06 dbus
>  drwxr-sr-x. 3 apache apache 4096 Mar 16 11:00 xdg-app
>
>  So, I would like to do the following:
>
>  * Move all the /srv/cache/lookaside/pkgs/rpms/ to
>    their /src/cache/lookaside/pkgs/ versions.
>
>  * create a link from /srv/cache/lookaside/pkgs/rpms
>    to /srv/cache/lookaside/pkgs
>
>  This should fix this issue until we push out a fedpkg/rpkg fix or want
>  to namespace lookaside.
>
>  +1s? 

+1 from me

[Pierre-Yves Chibon]

On Wed, Mar 16, 2016 at 09:33:20AM -0600, Kevin Fenzi wrote:
>  Greetings. 
>  
>  In recent fedpkg/rpkg versions (at least the ones in rawhide now) it's
>  incorrectly looking for a namespaced (rpms vs anything else) lookaside
>  instead of the non namespaced one we have.  
AFAIK we haven't pushed changes to fedpkg/rpkg related to the namespacing, so it
might be something that changed in the repo making the old code behaving
inconsistently.
 
>  So, this gives: 
>  
>  % fedpkg -v srpm
>  Creating repo object from /home/kevin/git/pkgs/midori/master
>  Downloading midori_0.5.11_all_.tar.bz2
>  Full url:
> http://pkgs.fedoraproject.org/repo/pkgs/rpms/midori/midori_0.5.11_all_.ta...
>  ######################################################################## 100.0%
>  Could not execute srpm: Server returned status code 404
>  Traceback (most recent call last):
>    File "/usr/bin/fedpkg", line 16, in <module>
>      main()
>    File "/usr/lib/python2.7/site-packages/fedpkg/__main__.py", line 69, in main
>      sys.exit(client.args.command())
>    File "/usr/lib/python2.7/site-packages/pyrpkg/cli.py", line 1234, in srpm
>      self.cmd.sources()
>    File "/usr/lib/python2.7/site-packages/pyrpkg/__init__.py", line 1696, in
> sources
>      hashtype=entry.hashtype, branch=self.branch_merge)
>    File "/usr/lib/python2.7/site-packages/pyrpkg/lookaside.py", line 174, in
> download
>      raise DownloadError('Server returned status code %d' % status)
>  pyrpkg.errors.DownloadError: Server returned status code 404
>  
>  And on upload it puts it in a rpms dir thats not correct.  
What is the git remote URL for this repo?
I suspect that's what fedpkg/rpkg are using and therefore the source of the
problem.

>  So, I would like to do the following: 
>  
>  * Move all the /srv/cache/lookaside/pkgs/rpms/ to
>    their /src/cache/lookaside/pkgs/ versions. 
>  
>  * create a link from /srv/cache/lookaside/pkgs/rpms
>    to /srv/cache/lookaside/pkgs
>  
>  This should fix this issue until we push out a fedpkg/rpkg fix or want
>  to namespace lookaside. 
>  
>  +1s? 
+1 for me, we should get the situation fixed anyway (+ if my intuition is
correct we'll have a chicken-egg problem between this and the package with ++ in
their names).

Pierre

[Kevin Fenzi]

On Wed, 16 Mar 2016 16:50:02 +0100
Pierre-Yves Chibon <pingou(a)pingoured.fr&gt; wrote:

>  On Wed, Mar 16, 2016 at 09:33:20AM -0600, Kevin Fenzi wrote:
>  > Greetings. 
>  > 
>  > In recent fedpkg/rpkg versions (at least the ones in rawhide now)
>  > it's incorrectly looking for a namespaced (rpms vs anything else)
>  > lookaside instead of the non namespaced one we have.   
>  
>  AFAIK we haven't pushed changes to fedpkg/rpkg related to the
>  namespacing, so it might be something that changed in the repo making
>  the old code behaving inconsistently. 
well, pyrpkg is updated in rawhide. ;) 

* Mon Mar 14 2016 Lubomír Sedlář <lubomir.sedlar(a)gmail.com&gt; - 1.41-2
- Depend on python-osbs directly to avoid python3

* Mon Mar 14 2016 Lubomír Sedlář <lubomir.sedlar(a)gmail.com&gt; - 1.41-1
- Update upstream URL (lsedlar)
- Fixes based on lsedlar's feedback (maxamillion)
- add distgit namespacing for non-rpm content (docker, xdg-app, etc)
  (maxamillion)
- Container-build: dont't allow to build with unpushed changes (araszka)
- Suggest --dist option when can't get OS ver from branch (araszka)
- fix: print all tags without filter (araszka)
- Fix lookaside upload when --path is specified (araszka)
- Lookaside: encoding repo name to UTF-8 (araszka)
- Fix errors on Python 2.6 (lsedlar)
- Add test and docstring to byte offset convertor (araszka)
- Decode .spec file with UTF-8 (araszka)
- 1271741 - add copr command (Recommends: copr-cli) (msuchy)
- Suggest --target option when unknown target (araszka)

I think this is the proposed fix: 
https://pagure.io/rpkg/pull-request/38

>  What is the git remote URL for this repo? 
        url = ssh://[e-mail]/rpms/midori

I did it from a fresh 'fedpkg clone'

>  I suspect that's what fedpkg/rpkg are using and therefore the
> source
>  of the problem. 
Perhaps. 

>  +1 for me, we should get the situation fixed anyway (+ if my
>  intuition is correct we'll have a chicken-egg problem between this
>  and the package with ++ in their names). 
Yeah, we will need to fix the packages that have mistakenly uploaded to
rpms/ in any case. ;) 

kevin