[Bug 2150574] New: Review Request: python-conda-package-streaming - Extract metadata from remote conda packages without downloading whole file
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Bug ID: 2150574 Summary: Review Request: python-conda-package-streaming - Extract metadata from remote conda packages without downloading whole file Product: Fedora Version: rawhide Hardware: All OS: Linux Status: NEW Component: Package Review Severity: medium Assignee: nobody@fedoraproject.org Reporter: orion@nwra.com QA Contact: extras-qa@fedoraproject.org CC: package-review@lists.fedoraproject.org Blocks: 1948558 Target Milestone: --- Classification: Fedora
Spec URL: https://orion.fedorapeople.org/python-conda-package-streaming.spec SRPM URL: https://orion.fedorapeople.org/python-conda-package-streaming-0.7.0-1.fc38.s... Description: Download conda metadata from packages without transferring entire file. Get metadata from local .tar.bz2 packages without reading entire files.
Uses enhanced pip lazy_wheel to fetch a file out of .conda with no more than 3 range requests, but usually 2.
Uses tar = tarfile.open(fileobj=...) to stream remote .tar.bz2. Closes the HTTP request once desired files have been seen.
Fedora Account System Username: orion
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1948558 [Bug 1948558] python-conda-package-handling-2.0.2 is available
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Wayne Sun gsun@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |gsun@redhat.com, | |mhroncok@redhat.com Doc Type|--- |If docs needed, set a value
--- Comment #1 from Wayne Sun gsun@redhat.com --- This is informal review as I'm not sponsored.
1) The upstream is BSD 3-Clause License so in the spec the SPDX name should be BSD-3-Clause rather than BSD, detail could be found in: https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuideline... https://docs.fedoraproject.org/en-US/legal/allowed-licenses/
2) [OPINION] not use the %{srcname} in URL for easier access 3) [OPINION] define variable for description 4) [OPINION] use two empty lines between blocks from %prep to the end 5) no need to specify %license under %doc as it's handled by %{pyproject_files}
Please check the fedora example file with 2,3,4,5: https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_example_s...
6) Use %pytest in %check which no need to specify the $PYTHONPATH https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_pytest
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #2 from Wayne Sun gsun@redhat.com --- With run rpmlint on the src rpm:
# rpmlint results/python-conda-package-streaming-0.7.0-1.fc38.src.rpm =================================================================== rpmlint session starts =================================================================== rpmlint: 2.2.0 configuration: /usr/lib/python3.10/site-packages/rpmlint/configdefaults.toml /etc/xdg/rpmlint/fedora-legacy-licenses.toml /etc/xdg/rpmlint/fedora-spdx-licenses.toml /etc/xdg/rpmlint/fedora.toml /etc/xdg/rpmlint/licenses.toml /etc/xdg/rpmlint/scoring.toml /etc/xdg/rpmlint/users-groups.toml /etc/xdg/rpmlint/warn-on-functions.toml checks: 32, packages: 1
python-conda-package-streaming.src: W: strange-permission conda-package-streaming-0.7.0.tar.gz 660 python-conda-package-streaming.src: W: strange-permission python-conda-package-streaming.spec 660 ==================================== 1 packages and 0 specfiles checked; 0 errors, 2 warnings, 0 badness; has taken 0.7 s ====================================
there is complain about permission, according to: https://fedoraproject.org/wiki/Common_Rpmlint_issues#strange-permission
it recommend update spec which I don't think necessary.
while build srpm on my laptop and check with rpmlint don't have this issue:
# rpmbuild -bs srpm/python-conda-package-streaming.spec setting SOURCE_DATE_EPOCH=1670025600 Wrote: /root/rpmbuild/SRPMS/python-conda-package-streaming-0.7.0-1.fc36.src.rpm
# rpmlint /root/rpmbuild/SRPMS/python-conda-package-streaming-0.7.0-1.fc36.src.rpm =================================================================== rpmlint session starts =================================================================== rpmlint: 2.2.0 configuration: /usr/lib/python3.10/site-packages/rpmlint/configdefaults.toml /etc/xdg/rpmlint/fedora-legacy-licenses.toml /etc/xdg/rpmlint/fedora-spdx-licenses.toml /etc/xdg/rpmlint/fedora.toml /etc/xdg/rpmlint/licenses.toml /etc/xdg/rpmlint/scoring.toml /etc/xdg/rpmlint/users-groups.toml /etc/xdg/rpmlint/warn-on-functions.toml checks: 32, packages: 1
==================================== 1 packages and 0 specfiles checked; 0 errors, 0 warnings, 0 badness; has taken 0.7 s ====================================
so it's not the issue with spec, the src rpm might not build properly.
@Miro any comment on this?
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #3 from Miro Hrončok mhroncok@redhat.com --- PYTHONPATH=%{buildroot}%{python3_sitelib} py.test-%{python3_version} -> does %pytest work here?
The deselected tests could use an explanatory comment.
python-conda-package-streaming.src: W: strange-permission conda-package-streaming-0.7.0.tar.gz 660 python-conda-package-streaming.src: W: strange-permission python-conda-package-streaming.spec 660 ... @Miro any comment on this?
I think such permissions will be lost anyway when importing to dist-git and the lookaside cache.
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #4 from Orion Poplawski orion@nwra.com --- Spec URL: https://orion.fedorapeople.org/python-conda-package-streaming.spec SRPM URL: https://orion.fedorapeople.org/python-conda-package-streaming-0.7.0-2.fc38.s...
* Wed Dec 07 2022 Orion Poplawski orion@nwra.com - 0.7.0-2 - Use macro for description - Use %%pytest macro - Fix license - Add comments about deselected tests
Yeah, permissions are just due to my restrictive umask, but I do not believe will affect others or cause issues
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #5 from Miro Hrončok mhroncok@redhat.com --- # THe deslected tests seem to require some kind of package server
Typo THe. Typo deselected.
Question out of curiosity (not a blocker for the package review): In %check, you set CONDA_EXE to %{_bindir}/conda. What is the default value?
Also, I see upstream has a [test] extra that can be buildrequired with:
%pyproject_buildrequires -x test
Instead of
# For tests BuildRequires: python%{python3_pkgversion}-boto3 BuildRequires: python%{python3_pkgversion}-bottle BuildRequires: python%{python3_pkgversion}-pytest BuildRequires: python%{python3_pkgversion}-pytest-mock
However, it does not include conda. Should this require conda for tests in upstream?
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #6 from Orion Poplawski orion@nwra.com --- (In reply to Miro Hrončok from comment #5)
# THe deslected tests seem to require some kind of package server
Typo THe. Typo deselected.
Fixed
Question out of curiosity (not a blocker for the package review): In %check, you set CONDA_EXE to %{_bindir}/conda. What is the default value?
Well, /etc/profile.d/conda.sh from the conda package sets it to /usr/bin/conda, but that doesn't get sourced in the package build. I suppose I could do:
. /etc/profile.d/conda.sh
instead. Done.
Also, I see upstream has a [test] extra that can be buildrequired with:
%pyproject_buildrequires -x test
Instead of
# For tests BuildRequires: python%{python3_pkgversion}-boto3 BuildRequires: python%{python3_pkgversion}-bottle BuildRequires: python%{python3_pkgversion}-pytest BuildRequires: python%{python3_pkgversion}-pytest-mock
However, it does not include conda. Should this require conda for tests in upstream?
Ah, thanks - still getting used to the pyproject stuff. It also adds a dep for:
python3dist(boto3-stubs) is needed by python-conda-package-streaming-0.7.0-2.fc38.noarch python3dist(boto3-stubs[essential]) is needed by python-conda-package-streaming-0.7.0-2.fc38.noarch
which doesn't appear to be strictly necessary and isn't packaged, so I've removed that. As, for conda, yeah upstream probably should. I've filed: https://github.com/conda/conda-package-streaming/pull/52
* Thu Dec 08 2022 Orion Poplawski orion@nwra.com - 0.7.0-3 - Use test extras for build requires - Add bootstrap conditional
Spec URL: https://orion.fedorapeople.org/python-conda-package-streaming.spec SRPM URL: https://orion.fedorapeople.org/python-conda-package-streaming-0.7.0-3.fc38.s...
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #7 from Orion Poplawski orion@nwra.com --- Anyone?
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Miro Hrončok mhroncok@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(gsun@redhat.com)
--- Comment #8 from Miro Hrončok mhroncok@redhat.com --- I assumed Wayne Sun will finish this review now when they are sponsored.
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Wayne Sun gsun@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Assignee|nobody@fedoraproject.org |gsun@redhat.com
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Wayne Sun gsun@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |POST Flags|needinfo?(gsun@redhat.com) |fedora-review+
--- Comment #9 from Wayne Sun gsun@redhat.com --- The package depends on requests and zstandard as specified in pyproject.toml: https://github.com/conda/conda-package-streaming/blob/main/pyproject.toml#L2...
Also the test dependency on conda is added in: https://github.com/conda/conda-package-streaming/blob/main/pyproject.toml#L3...
and in spec updated with: %pyproject_buildrequires -x test
The provide and require info is expected with mock build.
Package Review ==============
Legend: [x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated [ ] = Manual review needed
Issues: ======= - If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package is included in %license. Note: License file LICENSE is not marked as %license See: https://docs.fedoraproject.org/en-US/packaging- guidelines/LicensingGuidelines/#_license_text
The file LICENSE should have already been included as using %{pyproject_files}. This is an issue of the fedora review tool, and already been reported:
https://pagure.io/FedoraReview/issue/468
===== MUST items =====
Generic: [x]: Package is licensed with an open-source compatible license and meets other legal requirements as defined in the legal section of Packaging Guidelines. [x]: License field in the package spec file matches the actual license. Note: Checking patched sources after %prep for licenses. Licenses found: "Unknown or generated", "MIT License BSD 3-Clause License", "*No copyright* BSD 3-Clause License". [x]: Package contains no bundled libraries without FPC exception. [x]: Changelog in prescribed format. [x]: Sources contain only permissible code or content. [-]: Package contains desktop file if it is a GUI application. [-]: Development files must be in a -devel package [x]: Package uses nothing in %doc for runtime. [x]: Package consistently uses macros (instead of hard-coded directory names). [x]: Package is named according to the Package Naming Guidelines. [x]: Package does not generate any conflict. [x]: Package obeys FHS, except libexecdir and /usr/target. [-]: If the package is a rename of another package, proper Obsoletes and Provides are present. [x]: Requires correct, justified where necessary. [x]: Spec file is legible and written in American English. [-]: Package contains systemd file(s) if in need. [x]: Package is not known to require an ExcludeArch tag. [-]: Large documentation must go in a -doc subpackage. Large could be size (~1MB) or number of files. Note: Documentation size is 10240 bytes in 1 files. [x]: Package complies to the Packaging Guidelines [x]: Package successfully compiles and builds into binary rpms on at least one supported primary architecture. [x]: Package installs properly. [x]: Rpmlint is run on all rpms the build produces. Note: There are rpmlint messages (see attachment). [x]: Package requires other packages for directories it uses. [x]: Package must own all directories that it creates. [x]: Package does not own files or directories owned by other packages. [x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT [x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the beginning of %install. [x]: Macros in Summary, %description expandable at SRPM build time. [x]: Dist tag is present. [x]: Package does not contain duplicates in %files. [x]: Permissions on files are set properly. [x]: Package must not depend on deprecated() packages. [x]: Package use %makeinstall only when make install DESTDIR=... doesn't work. [x]: Package is named using only allowed ASCII characters. [x]: Package does not use a name that already exists. [x]: Package is not relocatable. [x]: Sources used to build the package match the upstream source, as provided in the spec URL. [x]: Spec file name must match the spec package %{name}, in the format %{name}.spec. [x]: File names are valid UTF-8. [x]: Packages must not store files under /srv, /opt or /usr/local
Python: [x]: Python eggs must not download any dependencies during the build process. [x]: A package which is used by another package via an egg interface should provide egg info. [x]: Package meets the Packaging Guidelines::Python [x]: Package contains BR: python2-devel or python3-devel [x]: Packages MUST NOT have dependencies (either build-time or runtime) on packages named with the unversioned python- prefix unless no properly versioned package exists. Dependencies on Python packages instead MUST use names beginning with python2- or python3- as appropriate. [x]: Python packages must not contain %{pythonX_site(lib|arch)}/* in %files [x]: Binary eggs must be removed in %prep
===== SHOULD items =====
Generic: [-]: If the source package does not include license text(s) as a separate file from upstream, the packager SHOULD query upstream to include it. [x]: Final provides and requires are sane (see attachments). [?]: Package functions as described. [x]: Latest version is packaged. [x]: Package does not include license text files separate from upstream. [-]: Sources are verified with gpgverify first in %prep if upstream publishes signatures. Note: gpgverify is not used. [?]: Package should compile and build into binary rpms on all supported architectures. [x]: %check is present and all tests pass. [?]: Packages should try to preserve timestamps of original installed files. [x]: Reviewer should test that the package builds in mock. [x]: Buildroot is not present [x]: Package has no %clean section with rm -rf %{buildroot} (or $RPM_BUILD_ROOT) [x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin. [x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file [x]: Sources can be downloaded from URI in Source: tag [x]: SourceX is a working URL. [x]: Spec use %global instead of %define unless justified.
===== EXTRA items =====
Generic: [x]: Rpmlint is run on all installed packages. Note: No rpmlint messages. [x]: Spec file according to URL is the same as in SRPM.
Rpmlint ------- Checking: python3-conda-package-streaming-0.7.0-3.fc38.noarch.rpm python-conda-package-streaming-0.7.0-3.fc38.src.rpm =============================================== rpmlint session starts =============================================== rpmlint: 2.4.0 configuration: /usr/lib/python3.11/site-packages/rpmlint/configdefaults.toml /etc/xdg/rpmlint/fedora-legacy-licenses.toml /etc/xdg/rpmlint/fedora-spdx-licenses.toml /etc/xdg/rpmlint/fedora.toml /etc/xdg/rpmlint/scoring.toml /etc/xdg/rpmlint/users-groups.toml /etc/xdg/rpmlint/warn-on-functions.toml rpmlintrc: [PosixPath('/tmp/tmpqwibc_kq')] checks: 31, packages: 2
python-conda-package-streaming.src: W: strange-permission conda-package-streaming-0.7.0.tar.gz 660 python-conda-package-streaming.src: W: strange-permission python-conda-package-streaming.spec 660 ================ 2 packages and 0 specfiles checked; 0 errors, 2 warnings, 0 badness; has taken 1.0 s ================
Rpmlint (installed packages) ---------------------------- ============================ rpmlint session starts ============================ rpmlint: 2.4.0 configuration: /usr/lib/python3.11/site-packages/rpmlint/configdefaults.toml /etc/xdg/rpmlint/fedora-legacy-licenses.toml /etc/xdg/rpmlint/fedora-spdx-licenses.toml /etc/xdg/rpmlint/fedora.toml /etc/xdg/rpmlint/scoring.toml /etc/xdg/rpmlint/users-groups.toml /etc/xdg/rpmlint/warn-on-functions.toml checks: 31, packages: 1
1 packages and 0 specfiles checked; 0 errors, 0 warnings, 0 badness; has taken 0.0 s
Source checksums ---------------- https://github.com/conda/conda-package-streaming/archive/v0.7.0/conda-packag... : CHECKSUM(SHA256) this package : 87be2b1c365a0e04d15579b9137b139f0837c52198bdba21c5ac071fc13efd75 CHECKSUM(SHA256) upstream package : 87be2b1c365a0e04d15579b9137b139f0837c52198bdba21c5ac071fc13efd75
Requires -------- python3-conda-package-streaming (rpmlib, GLIBC filtered): python(abi) python3.11dist(requests) python3.11dist(zstandard)
Provides -------- python3-conda-package-streaming: python-conda-package-streaming python3-conda-package-streaming python3.11-conda-package-streaming python3.11dist(conda-package-streaming) python3dist(conda-package-streaming)
Generated by fedora-review 0.9.0 (6761b6c) last change: 2022-08-23 Command line :/usr/bin/fedora-review -b 2150574 -m fedora-rawhide-x86_64 Buildroot used: fedora-rawhide-x86_64 Active plugins: Python, Generic, Shell-api Disabled plugins: R, Ocaml, fonts, Java, SugarActivity, Haskell, C/C++, PHP, Perl Disabled flags: EPEL6, EPEL7, DISTTAG, BATCH, EXARCH
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #10 from Miro Hrončok mhroncok@redhat.com --- (In reply to Wayne Sun from comment #9)
Issues:
- If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package is included in %license. Note: License file LICENSE is not marked as %license See: https://docs.fedoraproject.org/en-US/packaging- guidelines/LicensingGuidelines/#_license_text
The file LICENSE should have already been included as using %{pyproject_files}.
This package uses flit as a build backend, hence the LICENSE file is included in %{pyproject_files} but not marked as %license, because flit does not support that (yet).
To workaround you can do:
%files -n python%{python3_pkgversion}-%{srcname} -f %{pyproject_files} %doc README.md %license %{python3_sitelib}...dist-info/LICENSE
That will produce a harmless warning about a file listed twice but will work.
This is an issue of the fedora review tool, and already been reported:
Not in this case.
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
--- Comment #11 from Fedora Admin user for bugzilla script actions fedora-admin-xmlrpc@fedoraproject.org --- The Pagure repository was created at https://src.fedoraproject.org/rpms/python-conda-package-streaming
https://bugzilla.redhat.com/show_bug.cgi?id=2150574
Orion Poplawski orion@nwra.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |RAWHIDE Status|POST |CLOSED Last Closed| |2023-01-15 03:04:21
--- Comment #12 from Orion Poplawski orion@nwra.com --- Thank you everyone. This has been checked in and built.
package-review@lists.fedoraproject.org
-
bugzilla＠redhat.com