[Bug 2125789] New: Review Request: simdutf - Unicode validation and transcoding at billions of characters per second
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Bug ID: 2125789 Summary: Review Request: simdutf - Unicode validation and transcoding at billions of characters per second Product: Fedora Version: rawhide Status: NEW Component: Package Review Assignee: nobody@fedoraproject.org Reporter: aekoroglu@linux.intel.com QA Contact: extras-qa@fedoraproject.org CC: package-review@lists.fedoraproject.org Target Milestone: --- Classification: Fedora
SPEC Url: https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r... SRPM Url: https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r...
Description Unicode (UTF8, UTF16, UTF32) validation and transcoding at billions of characters per second using SSE2, AVX2, NEON, AVX-512.
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Benson Muite benson_muite@emailplus.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Doc Type|--- |If docs needed, set a value Flags| |fedora-review? CC| |benson_muite@emailplus.org Assignee|nobody@fedoraproject.org |benson_muite@emailplus.org
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #1 from Benson Muite benson_muite@emailplus.org --- It builds on Copr: https://copr.fedorainfracloud.org/coprs/fed500/simdutf/build/4832009/ However, tests fail on Celeron processors with Haswell architectures that do not have AVX2 instructions. Issue filed at: https://github.com/simdutf/simdutf/issues/168
Maybe a patch is required? A similar robustness improvement can be applied to https://github.com/simdjson/simdjson
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #2 from Benson Muite benson_muite@emailplus.org --- Checking the header only library on Celeron indicates architecture detection works ok. Simply using ctest will fail on an x86_64 processor that does not support AVX instructions. Ran fedora-review on a package without tests.
Package Review ==============
Legend: [x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated [ ] = Manual review needed
===== MUST items =====
C/C++: [x]: Package does not contain kernel modules. [x]: Package contains no static executables. [x]: If your application is a C or C++ application you must list a BuildRequires against gcc, gcc-c++ or clang. [x]: Header files in -devel subpackage, if present. [x]: ldconfig not called in %post and %postun for Fedora 28 and later. [x]: Package does not contain any libtool archives (.la) [x]: Rpath absent or only used for internal libs. [x]: Development (unversioned) .so files in -devel subpackage, if present.
Generic: [x]: Package is licensed with an open-source compatible license and meets other legal requirements as defined in the legal section of Packaging Guidelines. [!]: License field in the package spec file matches the actual license. Note: Checking patched sources after %prep for licenses. Licenses found: "Unknown or generated", "*No copyright* Apache License 2.0", "MIT License", "Unicode strict Apache License 2.0", "Open Software License 3.0", "*No copyright* Boost Software License 1.0", "GNU Lesser General Public License, Version 2.1", "BSD 3-Clause License", "*No copyright* Open Software License 3.0". 262 files have unknown license. Detailed output of licensecheck in /home/FedoraPackaging/reviews/simdutf/2125789-simdutf/srpm- unpacked/review-simdutf/licensecheck.txt [x]: License file installed when any subpackage combination is installed. [!]: If the package is under multiple licenses, the licensing breakdown must be documented in the spec. [x]: %build honors applicable compiler flags or justifies otherwise. [x]: Package contains no bundled libraries without FPC exception. [x]: Changelog in prescribed format. [x]: Sources contain only permissible code or content. [-]: Package contains desktop file if it is a GUI application. [x]: Development files must be in a -devel package [x]: Package uses nothing in %doc for runtime. [x]: Package consistently uses macros (instead of hard-coded directory names). [x]: Package is named according to the Package Naming Guidelines. [x]: Package does not generate any conflict. [x]: Package obeys FHS, except libexecdir and /usr/target. [-]: If the package is a rename of another package, proper Obsoletes and Provides are present. [x]: Requires correct, justified where necessary. [x]: Spec file is legible and written in American English. [-]: Package contains systemd file(s) if in need. [x]: Useful -debuginfo package or justification otherwise. [?]: Package is not known to require an ExcludeArch tag. [-]: Large documentation must go in a -doc subpackage. Large could be size (~1MB) or number of files. Note: Documentation size is 20480 bytes in 2 files. [x]: Package complies to the Packaging Guidelines [x]: Package successfully compiles and builds into binary rpms on at least one supported primary architecture. [x]: Package installs properly. [x]: Rpmlint is run on all rpms the build produces. Note: There are rpmlint messages (see attachment). [x]: If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package is included in %license. [x]: Package requires other packages for directories it uses. [x]: Package must own all directories that it creates. [x]: Package does not own files or directories owned by other packages. [x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT [x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the beginning of %install. [x]: Macros in Summary, %description expandable at SRPM build time. [x]: Dist tag is present. [x]: Package does not contain duplicates in %files. [x]: Permissions on files are set properly. [x]: Package must not depend on deprecated() packages. [x]: Package use %makeinstall only when make install DESTDIR=... doesn't work. [x]: Package is named using only allowed ASCII characters. [x]: Package does not use a name that already exists. [x]: Package is not relocatable. [x]: Sources used to build the package match the upstream source, as provided in the spec URL. [x]: Spec file name must match the spec package %{name}, in the format %{name}.spec. [x]: File names are valid UTF-8. [x]: Packages must not store files under /srv, /opt or /usr/local
===== SHOULD items =====
Generic: [-]: If the source package does not include license text(s) as a separate file from upstream, the packager SHOULD query upstream to include it. [x]: Final provides and requires are sane (see attachments). [?]: Fully versioned dependency in subpackages if applicable. Note: No Requires: %{name}%{?_isa} = %{version}-%{release} in simdutf- devel [x]: Package functions as described. [x]: Latest version is packaged. [x]: Package does not include license text files separate from upstream. [x]: Patches link to upstream bugs/comments/lists or are otherwise justified. [-]: Sources are verified with gpgverify first in %prep if upstream publishes signatures. Note: gpgverify is not used. [x]: Package should compile and build into binary rpms on all supported architectures. [?]: %check is present and all tests pass. [x]: Packages should try to preserve timestamps of original installed files. [x]: Reviewer should test that the package builds in mock. [x]: Buildroot is not present [x]: Package has no %clean section with rm -rf %{buildroot} (or $RPM_BUILD_ROOT) [x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin. [x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file [x]: Sources can be downloaded from URI in Source: tag [x]: SourceX is a working URL. [x]: Spec use %global instead of %define unless justified.
===== EXTRA items =====
Generic: [!]: Spec file according to URL is the same as in SRPM. Note: Spec file as given by url is not the same as in SRPM (see attached diff). See: (this test has no URL) [x]: Rpmlint is run on debuginfo package(s). Note: There are rpmlint messages (see attachment). [x]: Rpmlint is run on all installed packages. Note: There are rpmlint messages (see attachment). [x]: Large data in /usr/share should live in a noarch subpackage if package is arched.
Rpmlint ------- Cannot parse rpmlint output:
Rpmlint (debuginfo) ------------------- Cannot parse rpmlint output:
Rpmlint (installed packages) ---------------------------- Cannot parse rpmlint output:
Source checksums ---------------- https://github.com/simdutf/simdutf/archive/v1.0.1/simdutf-1.0.1.tar.gz : CHECKSUM(SHA256) this package : e7832ba58fb95fe00de76dbbb2f17d844a7ad02a6f5e3e9e5ce9520e820049a0 CHECKSUM(SHA256) upstream package : e7832ba58fb95fe00de76dbbb2f17d844a7ad02a6f5e3e9e5ce9520e820049a0
Requires -------- simdutf (rpmlib, GLIBC filtered): libc.so.6()(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libstdc++.so.6()(64bit) libstdc++.so.6(CXXABI_1.3)(64bit) rtld(GNU_HASH)
simdutf-devel (rpmlib, GLIBC filtered): cmake-filesystem(x86-64) libsimdutf.so.1()(64bit)
simdutf-debuginfo (rpmlib, GLIBC filtered):
simdutf-debugsource (rpmlib, GLIBC filtered):
Provides -------- simdutf: libsimdutf.so.1()(64bit) simdutf simdutf(x86-64)
simdutf-devel: cmake(simdutf) simdutf-devel simdutf-devel(x86-64)
simdutf-debuginfo: debuginfo(build-id) libsimdutf.so.1.0.0-1.0.1-1.fc38.x86_64.debug()(64bit) simdutf-debuginfo simdutf-debuginfo(x86-64)
simdutf-debugsource: simdutf-debugsource simdutf-debugsource(x86-64)
Diff spec file in url and in SRPM --------------------------------- --- /home/FedoraPackaging/simdutf/2125789-simdutf/srpm-unpacked/simdutf.spec 2022-09-11 17:42:59 .228210856 +0300 +++ /home/FedoraPackaging/simdutf/2125789-simdutf/srpm-unpacked/review-simdutf/srpm-unpacked/simdut f.spec 2022-09-11 17:44:26.000000000 +0300 @@ -1,2 +1,11 @@ +## START: Set by rpmautospec +## (rpmautospec version 0.3.0) +%define autorelease(e:s:pb:n) %{?-p:0.}%{lua: + release_number = 1; + base_release_number = tonumber(rpm.expand("%{?-b*}%{!?-b:1}")); + print(release_number + base_release_number - 1); +}%{?-e:.%{-e*}}%{?-s:.%{-s*}}%{!?-n:%{?dist}} +## END: Set by rpmautospec + Name: simdutf Version: 1.0.1 @@ -49,3 +58,4 @@
%changelog -%autochangelog +* Sun Sep 11 2022 John Doe packager@example.com 1.0.1-1 +- Uncommitted changes
Generated by fedora-review 0.8.0 (e988316) last change: 2022-04-07 Command line :/usr/bin/fedora-review -n simdutf Buildroot used: fedora-rawhide-x86_64 Active plugins: Generic, Shell-api, C/C++ Disabled plugins: PHP, Ocaml, Perl, fonts, R, Java, Haskell, SugarActivity, Python Disabled flags: EPEL6, EPEL7, DISTTAG, BATCH, EXARCH
Comments: a) licensecheck.txt contains further licenses, please add a breakdown in the spec file:
*No copyright* Apache License 2.0 --------------------------------- simdutf-1.0.1/LICENSE-APACHE simdutf-1.0.1/README.md
*No copyright* Boost Software License 1.0 ----------------------------------------- simdutf-1.0.1/benchmarks/competition/utf8lut/LICENSE
*No copyright* Open Software License 3.0 ---------------------------------------- simdutf-1.0.1/benchmarks/competition/u8u16/src/libu8u16.w
BSD 3-Clause License -------------------- simdutf-1.0.1/include/simdutf/internal/isadetection.h
GNU Lesser General Public License, Version 2.1 ---------------------------------------------- simdutf-1.0.1/benchmarks/competition/utf8sse4/fromutf8-sse.cpp
MIT License ----------- simdutf-1.0.1/LICENSE-MIT simdutf-1.0.1/benchmarks/competition/utf8lut/src/core/Dfa.h
Open Software License 3.0 ------------------------- simdutf-1.0.1/benchmarks/competition/u8u16/OSL3.0.txt
Unicode strict Apache License 2.0 --------------------------------- simdutf-1.0.1/benchmarks/competition/llvm/ConvertUTF.cpp simdutf-1.0.1/benchmarks/competition/llvm/ConvertUTF.h
b) There is an option to have a header only library, which can be useful for development. Might it be worth also packaging this as a subpackage? c) License files are in both the main package and the -devel package. As the -devel package should pull in the main package, it should be ok to just have the license files in the main package. d) It is also possible to include address sanitization with libasan. This could be done in a subpackage, if it would be helpful for users. e) For architecture detection on x86, may want to modify the testing helper function since tests will fail on x86 processors without AVX2 instructions. The tests are not packaged and the header only library does runtime detection correctly, but should there be an x86 machine on Koji without AVX2, building will fail on that machine.
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #3 from Ali Erdinc Koroglu aekoroglu@linux.intel.com --- Checked licensing issues and fixed it and AddressSanitizer enabled. No need to have architecture detection for tests* + header only sub package
SPEC Url: https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r... SRPM Url: https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r...
* : https://fedoraproject.org/wiki/Changes/x86-64_micro-architecture_update https://pagure.io/releng/issue/8513 https://pagure.io/fedora-infrastructure/issue/7968
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #4 from Benson Muite benson_muite@emailplus.org --- Thanks for the information on architectures. ConvertUTF.cpp and ConvertUTF.h maybe problematic. As per discussion at https://github.com/AmokHuginnsson/replxx/issues/12 can they be replaced by https://github.com/nemtrif/utfcpp which is packaged https://packages.fedoraproject.org/pkgs/utf8cpp/utf8cpp-devel/
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Benson Muite benson_muite@emailplus.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Blocks| |182235 (FE-Legal)
--- Comment #5 from Benson Muite benson_muite@emailplus.org --- Filed an issue upstream on licenses of included files https://github.com/simdutf/simdutf/issues/170
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=182235 [Bug 182235] Fedora Legal Tracker
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #6 from Benson Muite benson_muite@emailplus.org --- Apache-2.0 should be ASL 2.0 https://docs.fedoraproject.org/en-US/legal/allowed-licenses/
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #7 from Benson Muite benson_muite@emailplus.org --- It seems ConvertUTF.cpp and ConvertUTF.h can be removed/replaced https://github.com/simdutf/simdutf/issues/170
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Benson Muite benson_muite@emailplus.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(aekoroglu@linux.i | |ntel.com)
--- Comment #8 from Benson Muite benson_muite@emailplus.org --- For packaging, ConvertUTF.cpp and ConvertUTF.h are not needed as they are only used in the benchmarks. These are not run or packaged. Thus, the folder benchmarks can be removed, and lines 23 to 27 removed from the topmost CMakeLists.txt, with those changes I think it should be ok.
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Ali Erdinc Koroglu aekoroglu@linux.intel.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(aekoroglu@linux.i |needinfo- |ntel.com) |
--- Comment #9 from Ali Erdinc Koroglu aekoroglu@linux.intel.com --- Version updated to 2.0.2 and SIMDUTF_BENCHMARKS=OFF added, thank you.
SPEC Url : https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r... SRPM Url : https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r...
PS: Fedora is using SPDX license identifier for the new packages since July 2022 https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuideline...
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #10 from Benson Muite benson_muite@emailplus.org --- Can you add rm -r benchmarks after %autosetup -n %{name}-%{version} -p1 in the spec file. This will ensure troublesome content is not installed in error.
If the benchmark directory is removed, only licenses left are Apache 2 and BSD 3 Clause. All files are under Apache 2, except simdutf-2.0.2/include/simdutf/internal/isadetection.h which is BSD 3 Clause Can this information be added to the spec file? Licenses breakdown is required.
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #11 from Ali Erdinc Koroglu aekoroglu@linux.intel.com --- Done, thank you.
Spec Url : https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r... SRPM Url : https://download.copr.fedorainfracloud.org/results/aekoroglu/fedora/fedora-r...
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Benson Muite benson_muite@emailplus.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |POST Flags|fedora-review? |fedora-review+ Blocks|182235 (FE-Legal) |
--- Comment #12 from Benson Muite benson_muite@emailplus.org --- Approved.
Two minor points that can be addressed when importing: 1) %license LICENSE-APACHE LICENSE-MIT should just be %license LICENSE-APACHE
MIT licensed material was removed
2) after the line
License: Apache-2.0 AND BSD-3-Clause add #All files are under Apache 2, except simdutf-2.0.2/include/simdutf/internal/isadetection.h which is BSD 3 Clause
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=182235 [Bug 182235] Fedora Legal Tracker
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #13 from Ali Erdinc Koroglu aekoroglu@linux.intel.com --- You're right fixing it, thanks.
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
--- Comment #14 from Gwyn Ciesla gwync@protonmail.com --- (fedscm-admin): The Pagure repository was created at https://src.fedoraproject.org/rpms/simdutf
https://bugzilla.redhat.com/show_bug.cgi?id=2125789
Ali Erdinc Koroglu aekoroglu@linux.intel.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |RAWHIDE Status|POST |CLOSED Last Closed| |2022-11-14 07:03:35
package-review@lists.fedoraproject.org
-
bugzilla＠redhat.com