https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Bug ID: 1795077 Summary: Review Request: python-shodan - Python library and command-line utility for Shodan.io Product: Fedora Version: rawhide Hardware: All OS: Linux Status: NEW Component: Package Review Severity: medium Priority: medium Assignee: nobody@fedoraproject.org Reporter: mail@fabian-affolter.ch QA Contact: extras-qa@fedoraproject.org CC: package-review@lists.fedoraproject.org Target Milestone: --- Classification: Fedora
Spec URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan.spec SRPM URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan-1.21.3-1.fc31.src....
Project URL: http://github.com/achillean/shodan-python
Description: The official Python library and CLI for Shodan Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools.
Koji scratch build: https://koji.fedoraproject.org/koji/taskinfo?taskID=41055240
rpmlint output: $ rpmlint python-shodan-1.21.3-1.fc31.src.rpm 1 packages and 0 specfiles checked; 0 errors, 0 warnings.
$ rpmlint python3-shodan-1.21.3-1.fc31.noarch.rpm 1 packages and 0 specfiles checked; 0 errors, 0 warnings.
Fedora Account System Username: fab
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fabian Affolter mail@fabian-affolter.ch changed:
What |Removed |Added ---------------------------------------------------------------------------- Blocks| |563471 (FE-SECLAB) Alias| |python-shodan Doc Type|--- |If docs needed, set a value
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=563471 [Bug 563471] Tracker: Review Requests for Fedora Security Lab related packages
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fabio Valentini decathorpe@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED CC| |decathorpe@gmail.com Assignee|nobody@fedoraproject.org |decathorpe@gmail.com Flags| |fedora-review?
--- Comment #1 from Fabio Valentini decathorpe@gmail.com --- Hi, I'll review your package.
Some recommendations:
1) I assume you're hiding tests behind the api_key bcond because they require an actual API key and internet access? So do you run tests locally with "rpmbuild -bb --with api_key"? Maybe add a comment for the bcond or in %check.
2) I recommend you use HTTPS for the URL as well. Then you can also replace the URL prefix of the Source0 with %{url}.
Also, why are you not using %{pypi_source} directly? Do the pypi sources miss some files? If so, adding a comment why you're using the GitHub tarball instead would be helpful for anybody who's looking at the package.
3) You could deduplicate the %description, with something like:
%global _description %{expand: The official Python library and CLI for Shodan Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools.}
Then you can use:
%description %{_description}
and
%description python3-%{pypi_name} %{_description}
4) Since you're shipping both python3-shodan and shodan binary packages, you could just name the source package shodan directly. But that's a matter of style, I guess. If you'll keep the source package named python-shodan, then "%files -n python-%{pypi_name}-doc" is superfluous and could be just "%files doc".
5) I guess you're removing a stray shebang line from worldmap.py in %prep? Adding a descriptive comment would be great.
6) For listing python3 modules, please use trailing slashes:
%{python3_sitelib}/%{pypi_name}/ %{python3_sitelib}/%{pypi_name}-*.egg-info/
This will prevent upgrade issues if these ever change from directories to files.
You could also use something like this instead of the second line:
%{python3_sitelib}/%{pypi_name}-%{version}-py%{python3_version}.egg-info/
Then you'd not even need the glob.
Let me know if these points are helpful, and when the package is ready for final review.
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #2 from Fabian Affolter mail@fabian-affolter.ch --- (In reply to Fabio Valentini from comment #1)
Hi, I'll review your package.
Thanks
Some recommendations:
- I assume you're hiding tests behind the api_key bcond because they
require an actual API key and internet access? So do you run tests locally with "rpmbuild -bb --with api_key"? Maybe add a comment for the bcond or in %check.
Comment added. Also, create the file for the API key during the build process.
- I recommend you use HTTPS for the URL as well. Then you can also replace
the URL prefix of the Source0 with %{url}.
GitHub is making sure with a HTTP 301 message that one uses HTTPS. Changed anyway. Seems to be coming from pyp2rpm as this is the URL that is used in the setup.py file (https://github.com/achillean/shodan-python/pull/119).
%{url} could be an option. If the reviewer wants that then I will add it. Otherwise I prefer to keep things "copy-and-pastable" for humans in the spec file.
Also, why are you not using %{pypi_source} directly? Do the pypi sources miss some files? If so, adding a comment why you're using the GitHub tarball instead would be helpful for anybody who's looking at the package.
GitHub is the upstream location for the source and not PyPI. As a long as the project is providing proper tarballs I think that we should stick to the original upstream location and not a third-party delivery mechanism.
- You could deduplicate the %description, with something like:
%global _description %{expand: The official Python library and CLI for Shodan Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools.}
Then you can use:
%description %{_description}
and
%description python3-%{pypi_name} %{_description}
Again, this removes the possibility to re-use the description outside of the spec file with copy-and-paste. It makes sense if you have a dozen of subpackages and want to use the same text but for regular packages it doesn't add much value from my point of view.
- Since you're shipping both python3-shodan and shodan binary packages, you
could just name the source package shodan directly. But that's a matter of style, I guess. If you'll keep the source package named python-shodan, then "%files -n python-%{pypi_name}-doc" is superfluous and could be just "%files doc".
From my point of view, it's at the packager's discretion. This comes down to the decision if it is more a tool or a lib. In most cases the CLI part is an add-on to the lib.
- I guess you're removing a stray shebang line from worldmap.py in %prep?
Adding a descriptive comment would be great.
I really think that we should not add comments to every standard/common building block like remove shebang, fix permissions, remove the egg, remove the left-overs from the docs generation, etc.
https://github.com/achillean/shodan-python/pull/118
- For listing python3 modules, please use trailing slashes:
%{python3_sitelib}/%{pypi_name}/ %{python3_sitelib}/%{pypi_name}-*.egg-info/
This will prevent upgrade issues if these ever change from directories to files.
You could also use something like this instead of the second line:
%{python3_sitelib}/%{pypi_name}-%{version}-py%{python3_version}.egg-info/
Then you'd not even need the glob.
Fixed.
Let me know if these points are helpful, and when the package is ready for final review.
Thank you for your feedback.
* Tue Jan 28 2020 Fabian Affolter mail@fabian-affolter.ch - 1.21.3-2 - Fix ownership - Improve the check workflow (rhbz#1795077)
Updated files: Spec URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan.spec SRPM URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan-1.21.3-2.fc31.src....
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #3 from Fabio Valentini decathorpe@gmail.com --- (In reply to Fabian Affolter from comment #2)
%{url} could be an option. If the reviewer wants that then I will add it. Otherwise I prefer to keep things "copy-and-pastable" for humans in the spec file.
Right, but since non-copy-pastable %{version} and %{name} are already in there, using %{url} won't make it worse :)
Also, why are you not using %{pypi_source} directly? Do the pypi sources miss some files? If so, adding a comment why you're using the GitHub tarball instead would be helpful for anybody who's looking at the package.
GitHub is the upstream location for the source and not PyPI. As a long as the project is providing proper tarballs I think that we should stick to the original upstream location and not a third-party delivery mechanism.
That's fair, and your decision.
- You could deduplicate the %description, with something like:
%global _description %{expand: The official Python library and CLI for Shodan Shodan is a search engine for Internet-connected devices. Google lets you search for websites, Shodan lets you search for devices. This library provides developers easy access to all of the data stored in Shodan in order to automate tasks and integrate into existing tools.}
Then you can use:
%description %{_description}
and
%description python3-%{pypi_name} %{_description}
Again, this removes the possibility to re-use the description outside of the spec file with copy-and-paste. It makes sense if you have a dozen of subpackages and want to use the same text but for regular packages it doesn't add much value from my point of view.
How so? It just moves the definition a few lines up in the .spec file ...
- Since you're shipping both python3-shodan and shodan binary packages, you
could just name the source package shodan directly. But that's a matter of style, I guess. If you'll keep the source package named python-shodan, then "%files -n python-%{pypi_name}-doc" is superfluous and could be just "%files doc".
From my point of view, it's at the packager's discretion. This comes down to the decision if it is more a tool or a lib. In most cases the CLI part is an add-on to the lib.
Correct. Still, "%files -n python-%{pypi_name}-doc" is superfluous and could be replaced with "%files doc".
- I guess you're removing a stray shebang line from worldmap.py in %prep?
Adding a descriptive comment would be great.
I really think that we should not add comments to every standard/common building block like remove shebang, fix permissions, remove the egg, remove the left-overs from the docs generation, etc.
That's your decision, it was only a suggestion. Not everybody can sight-read sed scripts ;)
- For listing python3 modules, please use trailing slashes:
%{python3_sitelib}/%{pypi_name}/ %{python3_sitelib}/%{pypi_name}-*.egg-info/
This will prevent upgrade issues if these ever change from directories to files.
You could also use something like this instead of the second line:
%{python3_sitelib}/%{pypi_name}-%{version}-py%{python3_version}.egg-info/
Then you'd not even need the glob.
Fixed.
Great.
Let me know if these points are helpful, and when the package is ready for final review.
Thank you for your feedback.
- Tue Jan 28 2020 Fabian Affolter mail@fabian-affolter.ch - 1.21.3-2
- Fix ownership
- Improve the check workflow (rhbz#1795077)
Updated files: Spec URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan.spec SRPM URL: https://fab.fedorapeople.org/packages/SRPMS/python-shodan-1.21.3-2.fc31.src. rpm
I'll run fedora-review for the formal review now.
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fabio Valentini decathorpe@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |POST Flags|fedora-review? |fedora-review+
--- Comment #4 from Fabio Valentini decathorpe@gmail.com --- ================================================================================ Note that the linked .spec file didn't match the contents of the .src.rpm file, either you didn't update one of the uploads, or fedora-review downloaded the wrong file (see the diff at the bottom). Just make sure you use the latest version on import.
There's also rpmlint warnings about CRLF line endings in some files. You could fix these with sed or dos2unix.
Otherwise, package looks good. APPROVED (fix the two points above before import) ================================================================================
Package Review ==============
Legend: [x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated [ ] = Manual review needed
===== MUST items =====
Generic: [x]: Package is licensed with an open-source compatible license and meets other legal requirements as defined in the legal section of Packaging Guidelines. [x]: License field in the package spec file matches the actual license. [x]: License file installed when any subpackage combination is installed. [x]: Package does not own files or directories owned by other packages. [x]: Package contains no bundled libraries without FPC exception. [x]: Changelog in prescribed format. [x]: Sources contain only permissible code or content. [-]: Package contains desktop file if it is a GUI application. [-]: Development files must be in a -devel package [x]: Package uses nothing in %doc for runtime. [x]: Package consistently uses macros (instead of hard-coded directory names). [x]: Package is named according to the Package Naming Guidelines. [x]: Package does not generate any conflict. [x]: Package obeys FHS, except libexecdir and /usr/target. [-]: If the package is a rename of another package, proper Obsoletes and Provides are present. [x]: Requires correct, justified where necessary. [x]: Spec file is legible and written in American English. [-]: Package contains systemd file(s) if in need. [-]: Package is not known to require an ExcludeArch tag. [x]: Large documentation must go in a -doc subpackage. Large could be size (~1MB) or number of files. [x]: Package complies to the Packaging Guidelines [x]: Package successfully compiles and builds into binary rpms on at least one supported primary architecture. [x]: Package installs properly. [x]: Rpmlint is run on all rpms the build produces. Note: There are rpmlint messages (see attachment). [x]: If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package is included in %license. [x]: Package requires other packages for directories it uses. [x]: Package must own all directories that it creates. [x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT [x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the beginning of %install. [x]: Macros in Summary, %description expandable at SRPM build time. [x]: Dist tag is present. [x]: Package does not contain duplicates in %files. [x]: Permissions on files are set properly. [x]: Package must not depend on deprecated() packages. [x]: Package use %makeinstall only when make install DESTDIR=... doesn't work. [x]: Package is named using only allowed ASCII characters. [x]: Package does not use a name that already exists. [x]: Package is not relocatable. [x]: Sources used to build the package match the upstream source, as provided in the spec URL. [x]: Spec file name must match the spec package %{name}, in the format %{name}.spec. [x]: File names are valid UTF-8. [x]: Packages must not store files under /srv, /opt or /usr/local
Python: [x]: Python eggs must not download any dependencies during the build process. [x]: A package which is used by another package via an egg interface should provide egg info. [x]: Package meets the Packaging Guidelines::Python [x]: Package contains BR: python2-devel or python3-devel [x]: Packages MUST NOT have dependencies (either build-time or runtime) on packages named with the unversioned python- prefix unless no properly versioned package exists. Dependencies on Python packages instead MUST use names beginning with python2- or python3- as appropriate. [x]: Python packages must not contain %{pythonX_site(lib|arch)}/* in %files [x]: Binary eggs must be removed in %prep
===== SHOULD items =====
Generic: [-]: If the source package does not include license text(s) as a separate file from upstream, the packager SHOULD query upstream to include it. [x]: Final provides and requires are sane (see attachments). [x]: Fully versioned dependency in subpackages if applicable. [?]: Package functions as described. [x]: Latest version is packaged. [x]: Package does not include license text files separate from upstream. [-]: Sources are verified with gpgverify first in %prep if upstream publishes signatures. [-]: Description and summary sections in the package spec file contains translations for supported Non-English languages, if available. [x]: Package should compile and build into binary rpms on all supported architectures. [?]: %check is present and all tests pass. [x]: Packages should try to preserve timestamps of original installed files. [x]: Reviewer should test that the package builds in mock. [x]: Buildroot is not present [x]: Package has no %clean section with rm -rf %{buildroot} (or $RPM_BUILD_ROOT) [x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin. [x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file [x]: Sources can be downloaded from URI in Source: tag [x]: SourceX is a working URL. [x]: Spec use %global instead of %define unless justified.
===== EXTRA items =====
Generic: [!]: Spec file according to URL is the same as in SRPM. Note: Spec file as given by url is not the same as in SRPM (see attached diff). See: (this test has no URL) [x]: Rpmlint is run on all installed packages. Note: There are rpmlint messages (see attachment).
Rpmlint ------- Checking: python3-shodan-1.21.3-2.fc32.noarch.rpm python-shodan-doc-1.21.3-2.fc32.noarch.rpm shodan-1.21.3-2.fc32.noarch.rpm python-shodan-1.21.3-2.fc32.src.rpm python-shodan-doc.noarch: W: summary-not-capitalized C python-shodan documentation python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/api.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/basic-search.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/cert-stream.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/query-summary.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/tutorial.rst.txt shodan.noarch: W: name-repeated-in-summary C Shodan shodan.noarch: W: no-documentation shodan.noarch: W: no-manual-page-for-binary shodan 4 packages and 0 specfiles checked; 0 errors, 9 warnings.
Rpmlint (installed packages) ---------------------------- python-shodan-doc.noarch: W: summary-not-capitalized C python-shodan documentation python-shodan-doc.noarch: W: invalid-url URL: https://github.com/achillean/shodan-python <urlopen error [Errno -2] Name or service not known> python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/api.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/basic-search.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/cert-stream.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/examples/query-summary.rst.txt python-shodan-doc.noarch: W: wrong-file-end-of-line-encoding /usr/share/doc/python-shodan-doc/html/_sources/tutorial.rst.txt python3-shodan.noarch: W: invalid-url URL: https://github.com/achillean/shodan-python <urlopen error [Errno -2] Name or service not known> shodan.noarch: W: name-repeated-in-summary C Shodan shodan.noarch: W: invalid-url URL: https://github.com/achillean/shodan-python <urlopen error [Errno -2] Name or service not known> shodan.noarch: W: no-documentation shodan.noarch: W: no-manual-page-for-binary shodan 3 packages and 0 specfiles checked; 0 errors, 12 warnings.
Source checksums ---------------- https://github.com/achillean/shodan-python/archive/1.21.3/shodan-1.21.3.tar.... : CHECKSUM(SHA256) this package : 469ba078a1995f6fd2ae1127f9faa26103c27b2758f6629ec95aad8f00d2775a CHECKSUM(SHA256) upstream package : 469ba078a1995f6fd2ae1127f9faa26103c27b2758f6629ec95aad8f00d2775a
Requires -------- python3-shodan (rpmlib, GLIBC filtered): python(abi) python3.8dist(click) python3.8dist(click-plugins) python3.8dist(colorama) python3.8dist(requests) python3.8dist(setuptools) python3.8dist(xlsxwriter)
python-shodan-doc (rpmlib, GLIBC filtered):
shodan (rpmlib, GLIBC filtered): /usr/bin/python3 python3-shodan
Provides -------- python3-shodan: python-shodan python3-shodan python3.8dist(shodan) python3dist(shodan)
python-shodan-doc: python-shodan-doc
shodan: shodan
Diff spec file in url and in SRPM --------------------------------- --- /home/deca/Downloads/1795077-python-shodan/srpm/python-shodan.spec 2020-01-28 15:51:58.076556540 +0100 +++ /home/deca/Downloads/1795077-python-shodan/srpm-unpacked/python-shodan.spec 2020-01-28 13:43:41.000000000 +0100 @@ -82,8 +82,8 @@ %doc README.rst %{python3_sitelib}/%{pypi_name}/ -%{python3_sitelib}/%{pypi_name}-*.egg-info/ +%{python3_sitelib}/%{pypi_name}-*.egg-info
%files -n %{pypi_name} -%{_bindir}/%{pypi_name} +%{_bindir}/shodan
%files -n python-%{pypi_name}-doc
Generated by fedora-review 0.7.4 (54fa030) last change: 2019-12-07 Command line :/usr/bin/fedora-review -m fedora-rawhide-x86_64 -b 1795077 Buildroot used: fedora-rawhide-x86_64 Active plugins: Shell-api, Generic, Python Disabled plugins: Ocaml, R, Perl, C/C++, SugarActivity, fonts, Haskell, PHP, Java Disabled flags: EPEL6, EPEL7, DISTTAG, BATCH, EXARCH
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #5 from Gwyn Ciesla gwync@protonmail.com --- (fedscm-admin): The Pagure repository was created at https://src.fedoraproject.org/rpms/python-shodan
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|POST |MODIFIED
--- Comment #6 from Fedora Update System updates@fedoraproject.org --- FEDORA-2020-0761f5dedd has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-0761f5dedd
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #7 from Fedora Update System updates@fedoraproject.org --- FEDORA-2020-63f57a07ee has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-63f57a07ee
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|MODIFIED |ON_QA
--- Comment #8 from Fedora Update System updates@fedoraproject.org --- python-shodan-1.21.3-3.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-0761f5dedd
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #9 from Fedora Update System updates@fedoraproject.org --- python-shodan-1.21.3-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-63f57a07ee
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
Fedora Update System updates@fedoraproject.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|ON_QA |CLOSED Resolution|--- |ERRATA Last Closed| |2020-02-12 01:40:46
--- Comment #10 from Fedora Update System updates@fedoraproject.org --- python-shodan-1.21.3-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.
https://bugzilla.redhat.com/show_bug.cgi?id=1795077
--- Comment #11 from Fedora Update System updates@fedoraproject.org --- python-shodan-1.21.3-3.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.
package-review@lists.fedoraproject.org