[releng] PR #6970: modulepkg.py - Look up modules for packages

[Filip Valder]

fivaldi opened a new pull-request against the project: `releng` that you are following:
``
modulepkg.py - Look up modules for packages
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
@fivaldi - this repo has a policy that all commits have to be "signed off"

Can you rebase with `git commit --amend -s`, and then force push to fix that?
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Filip Valder]

fivaldi commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
@ralph , I altered the code a bit, added these flags:
```
  --all-releases      By default, only latest module releases are listed. This
                      flag causes all module releases to be listed
  --include-inactive  By default, only active modules are taken in mind. This
                      flag causes both active/inactive modules to be looked up
```

The commit should be "signed off" now. Thank you for notifying.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Filip Valder]

fivaldi commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
This is the diff to the previous "version":
https://paste.fedoraproject.org/paste/yiU1FJ5GJ40Roi095pF-Nw/
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Filip Valder]

fivaldi commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
Added enhancements:
```
- defaults to Python3
- requests-cache for improved performance
- arg `--branch` renamed to `--filter-branch`
- new args:
  - `--filter-api`
  - `--filter-profile`
- code refactored
```

If needed, I can squash it to a single commit after review is done.

Thanks!
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
Note, @nphilipp said he'd have a look at this in `#fedora-modularity`.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Nils Philippsen]

nphilipp commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
This is surely a useful tool, thanks! Perhaps we could move it somewhere where module
creators/maintainers can get at it someday.

* Is it normal that it takes pretty long to complete? (Probably yes.)
* I'm unsure about what the `--filter-branch` option does exactly—does it match
components whose `ref` is reachable in the specified branch? For instance, I get
`flatpak-runtime-f26-20170701152209` as one match for `python modulepkg.py -v
--filter-branch ... wayland`, regardless of if I filter for `f26` or `master`.
* The output isn't labeled correctly and could be formatted better if we want module
creators/maintainers to use it:
    * You map `ref` to `branch_or_ref`, but the former is used in `modulemd` and git for
commit hashes, branches or tags.
    * Similarly, `module_nvr` uses the RPM terms ("version",
"release"), but modules use "stream" (to signify that there is no
lower or higher) and "version" (this both was changed after the relevant PDC API
was defined, sorry!). You should also use colons as separators between name, stream,
version (see https://pagure.io/modularity/pull-request/43), the PDC API returns the parts
in `variant_id`, `variant_version` and `variant_release`, respectively, e.g. like this:

            # PDC API doesn't match the current terminology. Modules use colons as
separators.
            m_dict['module_nsv'] =
"{variant_id}:{variant_version}:{variant_release}".format(m)

        …and when displaying it, map `module_nsv` to a string like `module
name:stream:version`.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
>  Perhaps we could move it somewhere where module creators/maintainers
> can get at it someday.
>  
Agreed.  The `fedora-packager` package is a good eventual landing place for this.

In the meantime, thanks for all the feedback.  @fivaldi, can you work with @nphilipp to
refine it?
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Filip Valder]

fivaldi commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
>  This is surely a useful tool, thanks! Perhaps we could move it
> somewhere where module creators/maintainers can get at it someday.
>  
>  Is it normal that it takes pretty long to complete? (Probably yes.)
>  
Yes, it really takes. The very same (as below) can be seen via web browser in the
network tab. The majority of the time is spent by waiting for a response from PDC. I think
it's worth analyzing/optimizing/profiling whether it's the infrastructure, DB or
PDC itself. 
```
time wget -qO /dev/null
https://pdc.fedoraproject.org/rest_api/v1/unreleasedvariants/\?component_...
wget -qO /dev/null   0,04s user 0,03s system 0% cpu 2:21,62 total
```

>  I'm unsure about what the --filter-branch option does
> exactly—does it match components whose ref is reachable in the specified branch? For
> instance, I get flatpak-runtime-f26-20170701152209 as one match for python modulepkg.py -v
> --filter-branch ... wayland, regardless of if I filter for f26 or master.
>  
I rely on what's returned by PDC. I also noticed that filtering `master`
usually returns the same as a what is then returned for the specific branch. Thanks for
pointing it out, I will double-check that behaviour. 

>  The output isn't labeled correctly and could be formatted better
> if we want module creators/maintainers to use it:
>  You map ref to branch_or_ref, but the former is used in modulemd and git for commit
> hashes, branches or tags.
>  
>  Similarly, module_nvr uses the RPM terms ("version", "release"), but
> modules use "stream" (to signify that there is no lower or higher) and
> "version" (this both was changed after the relevant PDC API was defined,
> sorry!). You should also use colons as separators between name, stream, version (see
> https://pagure.io/modularity/pull-request/43), the PDC API returns the parts in
> variant_id, variant_version and variant_release, respectively, e.g. like this:
>  # PDC API doesn't match the current terminology. Modules use colons as separators.
>  m_dict['module_nsv'] =
> "{variant_id}:{variant_version}:{variant_release}".format(m)
>  
>  
>  
>  …and when displaying it, map module_nsv to a string like module name:stream:version.
>  
>  
Thanks for recommendation on naming conventions. I will fix it.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Nils Philippsen]

nphilipp commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
Thanks for these additions! Looks good to me now, but I'd need to know if you want
this change (one commit) to be merged, or just optionally rebased and pushed­, I'm not
familiar with the repo policies.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
I think either is fine.  I usually see other releng admins doing merge commits.
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph commented on the pull-request: `modulepkg.py - Look up modules for packages` that
you are following:
``
Thanks guys!
``

To reply, visit the link below or just reply to this email
https://pagure.io/releng/pull-request/6970

[Ralph Bean]

ralph merged a pull-request against the project: `releng` that you are following.

Merged pull-request:

``
modulepkg.py - Look up modules for packages
``

https://pagure.io/releng/pull-request/6970