Bugzilla components of retired packages

[Pierre-Yves Chibon]

On Thu, Apr 02, 2015 at 04:22:27PM +0200, Pierre-Yves Chibon wrote:
> On Thu, Apr 02, 2015 at 09:04:38AM -0500, Michael Cronenworth wrote:
> > On 04/02/2015 03:37 AM, Pierre-Yves Chibon wrote:
> > >Retrieving the list of retired packages from pkgdb is the easy step:
> > >https://admin.fedoraproject.org/pkgdb/api/#list_packages
> > >
> > >So this should do it:
> > >https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired&eol=True
> > 
> > Thanks. I got a package list in just a few seconds using:
> > 
> > https://admin.fedoraproject.org/pkgdb/api/packages?branches=master&branches=f22&branches=f21&branches=f20&status=Retired&limit=500&page=[1-8]
> > 
> > List: http://fpaste.org/206455/79833981/
> 
> hm, looking through the pkgdb code it iterates through the branch instead of
> asking for the specified status on all the specified branches.
> So maybe do one request per branch and find the list of packages present in all
> requests.
>
> I can probably script something quickly if you want.

Attached is my attempt and below is its output:

https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired&branches=master
13 pages to retrieve for master
3093 packages retrieved in master
https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired&branches=f22
1 pages to retrieve for f22
36 packages retrieved in f22
https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired&branches=f21
1 pages to retrieve for f21
134 packages retrieved in f21
https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired&branches=f20
1 pages to retrieve for f20
203 packages retrieved in f20
203 packages found to be retired

But that's also likely un-complete as lots of retired package will not have the
f22 branch, so I guess the solution is really to get the list of retired
packages and then go through them one by one to check all their branch (which
might be as slow as the pkgdb-cli approach).


Pierre
-------------- next part --------------
#!/usr/bin/env python2

import requests

url = 'https://admin.fedoraproject.org/pkgdb/api/packages?status=Retired'

def get_retired_pkgs_in_branch(branch):
    branch_url = '%s&branches=%s' % (url, branch)
    print branch_url
    req = requests.get(branch_url)

    data = req.json()
    packages = data['packages']
    page = int(data['page'])
    total = data['page_total']

    print '%s pages to retrieve for %s' % (total, branch)
    while page <= total:
        nurl = '%s&page=%s' % (branch_url, page + 1)
        data = requests.get(nurl).json()
        packages.extend(data['packages'])
        page += 1

    print '%s packages retrieved in %s' % (len(packages), branch)
    pkgs_name = [pkg['name'] for pkg in packages]
    return pkgs_name


if __name__ == '__main__':
    pkgs = set()
    for branch in ['master', 'f22', 'f21', 'f20']:
        if len(pkgs) == 0:
            pkgs = set(get_retired_pkgs_in_branch(branch))
        else:
            pkgs = pkgs.intersection(
                set(get_retired_pkgs_in_branch(branch)))

    print '%s packages found to be retired' % len(pkgs)

    # un-comment below to get the list of packages
    #for pkg in pkgs:
        #print pkg