From: Prarit Bhargava prarit@redhat.com
redhat/kernel.spec.template: Parallelize compression
This line in the kernel.spec file:
find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz;
implies that ${RPM_BUILD_NCPUS} xz instances are run. This is not true, and this line actually applies ${RPM_BUILD_NCPUS} to a single instance of xz. This means that the compression has been done one module at a time (ie, in serial) rather than in parallel as is implied by the code.
Use xz's -n option to assign one cpu per process and parallelize the compression.
Suggested-by: "Herton R. Krzesinski" herton@redhat.com Signed-off-by: Prarit Bhargava prarit@redhat.com
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100755 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -2472,7 +2472,7 @@ find Documentation -type d | xargs chmod u+w fi \ fi \ if [ "%{zipmodules}" -eq "1" ]; then \ - find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz; \ + find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -n 1 -P${RPM_BUILD_NCPUS} -r xz; \ fi \ %{nil}
-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2254
From: Prarit Bhargava on gitlab.com https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2254#note_1248227...
Oh! I had no idea about -n! Thanks for that. I've updated the MR with a much simpler approach.
I tested this by doing a build with 'xz -v' and I can see that it works as expected:
``` <snip> 25.1 % 420.1 KiB / 1,848.0 KiB = 0.227 0:01 99.1 % 24 B / 240.0 KiB = 0.000 22.6 % 480.1 KiB / 1,872.0 KiB = 0.256 86.1 % 360.1 KiB / 1,960.0 KiB = 0.184 0:01 59.7 % 720.1 KiB / 4,056.0 KiB = 0.178 0:02 5.9 % 480.1 KiB / 2,152.0 KiB = 0.223 0:01 56.7 % 660.1 KiB / 3,536.0 KiB = 0.187 0:01 58.9 % 960.1 KiB / 5,800.0 KiB = 0.166 0:02 27.2 % 420.1 KiB / 1,800.0 KiB = 0.233 51.3 % 900.1 KiB / 5,128.0 KiB = 0.176 0:02 90.6 % 300.0 KiB / 1,560.0 KiB = 0.192 0.8 % 24 B / 8,192 B = 0.003 91.3 % 180.0 KiB / 936.0 KiB = 0.192 87.2 % 840.1 KiB / 4,864.0 KiB = 0.173 0:02 77.0 % 780.1 KiB / 5,400.0 KiB = 0.144 0:02 71.8 % 1,020.1 KiB / 5,304.0 KiB = 0.192 0:02 12.8 % 1,140.1 KiB / 5,032.0 KiB = 0.227 0:02 1.0 % 24 B / 8,192 B = 0.003 2.9 % 24 B / 16.0 KiB = 0.001 0.9 % 24 B / 8,192 B = 0.003 1.1 % 24 B / 16.0 KiB = 0.001 7.3 % 24 B / 24.0 KiB = 0.001 3.4 % 24 B / 16.0 KiB = 0.001 1.0 % 24 B / 16.0 KiB = 0.001 1.6 % 24 B / 16.0 KiB = 0.001 <snip> ```
kernel@lists.fedoraproject.org
-
Prarit Bhargava (via Email Bridge)