Missing expected images:
Workstation live i386 Kde live x86_64 Xfce raw-xz armhfp Workstation live x86_64 Kde live i386
Failed openQA tests: 52/79 (x86_64), 14/15 (i386), 1/2 (arm)
New failures (same test did not fail in Rawhide-20161130.n.0):
ID: 50248 Test: x86_64 Server-boot-iso install_default URL: https://openqa.fedoraproject.org/tests/50248 ID: 50249 Test: x86_64 Server-boot-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/50249 ID: 50250 Test: x86_64 Server-dvd-iso install_default_upload URL: https://openqa.fedoraproject.org/tests/50250 ID: 50251 Test: x86_64 Server-dvd-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/50251 ID: 50257 Test: x86_64 Server-dvd-iso install_repository_nfs_variation URL: https://openqa.fedoraproject.org/tests/50257 ID: 50258 Test: x86_64 Server-dvd-iso install_repository_nfs_graphical URL: https://openqa.fedoraproject.org/tests/50258 ID: 50267 Test: x86_64 Server-dvd-iso install_updates_nfs URL: https://openqa.fedoraproject.org/tests/50267 ID: 50270 Test: i386 Server-dvd-iso install_default URL: https://openqa.fedoraproject.org/tests/50270 ID: 50271 Test: x86_64 universal install_package_set_minimal URL: https://openqa.fedoraproject.org/tests/50271 ID: 50274 Test: x86_64 universal install_repository_http_variation URL: https://openqa.fedoraproject.org/tests/50274 ID: 50275 Test: x86_64 universal install_repository_http_graphical URL: https://openqa.fedoraproject.org/tests/50275 ID: 50276 Test: x86_64 universal install_mirrorlist_graphical URL: https://openqa.fedoraproject.org/tests/50276 ID: 50277 Test: x86_64 universal install_delete_pata URL: https://openqa.fedoraproject.org/tests/50277 ID: 50278 Test: x86_64 universal install_delete_pata@uefi URL: https://openqa.fedoraproject.org/tests/50278 ID: 50280 Test: x86_64 universal install_scsi_updates_img URL: https://openqa.fedoraproject.org/tests/50280 ID: 50281 Test: x86_64 universal install_multi URL: https://openqa.fedoraproject.org/tests/50281 ID: 50282 Test: x86_64 universal install_multi@uefi URL: https://openqa.fedoraproject.org/tests/50282 ID: 50283 Test: x86_64 universal install_simple_encrypted URL: https://openqa.fedoraproject.org/tests/50283 ID: 50284 Test: x86_64 universal install_simple_free_space URL: https://openqa.fedoraproject.org/tests/50284 ID: 50285 Test: x86_64 universal install_multi_empty URL: https://openqa.fedoraproject.org/tests/50285 ID: 50286 Test: x86_64 universal install_software_raid URL: https://openqa.fedoraproject.org/tests/50286 ID: 50287 Test: x86_64 universal install_delete_partial URL: https://openqa.fedoraproject.org/tests/50287 ID: 50288 Test: x86_64 universal install_btrfs URL: https://openqa.fedoraproject.org/tests/50288 ID: 50289 Test: x86_64 universal install_ext3 URL: https://openqa.fedoraproject.org/tests/50289 ID: 50290 Test: x86_64 universal install_xfs URL: https://openqa.fedoraproject.org/tests/50290 ID: 50291 Test: x86_64 universal install_lvmthin URL: https://openqa.fedoraproject.org/tests/50291 ID: 50292 Test: x86_64 universal install_no_swap URL: https://openqa.fedoraproject.org/tests/50292 ID: 50293 Test: x86_64 universal install_iscsi URL: https://openqa.fedoraproject.org/tests/50293 ID: 50295 Test: x86_64 universal install_simple_encrypted@uefi URL: https://openqa.fedoraproject.org/tests/50295 ID: 50296 Test: x86_64 universal install_simple_free_space@uefi URL: https://openqa.fedoraproject.org/tests/50296 ID: 50297 Test: x86_64 universal install_multi_empty@uefi URL: https://openqa.fedoraproject.org/tests/50297 ID: 50298 Test: x86_64 universal install_software_raid@uefi URL: https://openqa.fedoraproject.org/tests/50298 ID: 50300 Test: x86_64 universal install_btrfs@uefi URL: https://openqa.fedoraproject.org/tests/50300 ID: 50301 Test: x86_64 universal install_ext3@uefi URL: https://openqa.fedoraproject.org/tests/50301 ID: 50302 Test: x86_64 universal install_xfs@uefi URL: https://openqa.fedoraproject.org/tests/50302 ID: 50303 Test: x86_64 universal install_lvmthin@uefi URL: https://openqa.fedoraproject.org/tests/50303 ID: 50304 Test: x86_64 universal install_no_swap@uefi URL: https://openqa.fedoraproject.org/tests/50304 ID: 50309 Test: x86_64 universal upgrade_kde_64bit URL: https://openqa.fedoraproject.org/tests/50309 ID: 50314 Test: x86_64 universal upgrade_2_kde_64bit URL: https://openqa.fedoraproject.org/tests/50314 ID: 50316 Test: x86_64 universal install_updates_img_local URL: https://openqa.fedoraproject.org/tests/50316 ID: 50317 Test: x86_64 universal install_shrink_ext4 URL: https://openqa.fedoraproject.org/tests/50317 ID: 50318 Test: x86_64 universal install_shrink_ntfs URL: https://openqa.fedoraproject.org/tests/50318 ID: 50319 Test: x86_64 universal install_european_language URL: https://openqa.fedoraproject.org/tests/50319 ID: 50320 Test: x86_64 universal install_cyrillic_language URL: https://openqa.fedoraproject.org/tests/50320 ID: 50325 Test: i386 universal install_package_set_minimal URL: https://openqa.fedoraproject.org/tests/50325 ID: 50326 Test: i386 universal install_repository_http_graphical URL: https://openqa.fedoraproject.org/tests/50326 ID: 50327 Test: i386 universal install_scsi_updates_img URL: https://openqa.fedoraproject.org/tests/50327 ID: 50328 Test: i386 universal install_simple_encrypted URL: https://openqa.fedoraproject.org/tests/50328 ID: 50329 Test: i386 universal install_software_raid URL: https://openqa.fedoraproject.org/tests/50329 ID: 50330 Test: i386 universal install_btrfs URL: https://openqa.fedoraproject.org/tests/50330 ID: 50331 Test: i386 universal install_ext3 URL: https://openqa.fedoraproject.org/tests/50331 ID: 50332 Test: i386 universal install_lvmthin URL: https://openqa.fedoraproject.org/tests/50332 ID: 50336 Test: x86_64 universal install_delete_partial@uefi URL: https://openqa.fedoraproject.org/tests/50336
Old failures (same test failed in Rawhide-20161130.n.0):
ID: 50243 Test: x86_64 Workstation-boot-iso install_default URL: https://openqa.fedoraproject.org/tests/50243 ID: 50244 Test: x86_64 Workstation-boot-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/50244 ID: 50245 Test: i386 Workstation-boot-iso install_default URL: https://openqa.fedoraproject.org/tests/50245 ID: 50246 Test: arm Minimal-raw_xz-raw.xz install_arm_image_deployment_upload URL: https://openqa.fedoraproject.org/tests/50246 ID: 50269 Test: i386 Server-boot-iso install_default URL: https://openqa.fedoraproject.org/tests/50269 ID: 50272 Test: x86_64 universal install_anaconda_text URL: https://openqa.fedoraproject.org/tests/50272 ID: 50294 Test: x86_64 universal install_package_set_kde URL: https://openqa.fedoraproject.org/tests/50294 ID: 50307 Test: x86_64 universal upgrade_desktop_64bit URL: https://openqa.fedoraproject.org/tests/50307 ID: 50310 Test: x86_64 universal upgrade_desktop_encrypted_64bit URL: https://openqa.fedoraproject.org/tests/50310 ID: 50312 Test: x86_64 universal upgrade_2_desktop_64bit URL: https://openqa.fedoraproject.org/tests/50312 ID: 50315 Test: x86_64 universal upgrade_2_desktop_encrypted_64bit URL: https://openqa.fedoraproject.org/tests/50315 ID: 50333 Test: i386 universal upgrade_desktop_32bit URL: https://openqa.fedoraproject.org/tests/50333 ID: 50334 Test: i386 universal upgrade_2_desktop_32bit URL: https://openqa.fedoraproject.org/tests/50334 ID: 50335 Test: i386 universal install_package_set_kde URL: https://openqa.fedoraproject.org/tests/50335
Passed openQA tests: 13/79 (x86_64), 1/15 (i386)
New passes (same test did not pass in Rawhide-20161130.n.0):
ID: 50241 Test: x86_64 Everything-boot-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/50241 ID: 50311 Test: x86_64 universal upgrade_2_minimal_64bit URL: https://openqa.fedoraproject.org/tests/50311
Skipped openQA tests: 15 of 96
Installed system changes in test x86_64 Everything-boot-iso install_default: System load changed from 0.04 to 0.19 Previous test data: https://openqa.fedoraproject.org/tests/50142#downloads Current test data: https://openqa.fedoraproject.org/tests/50240#downloads
On Sat, 2016-12-03 at 07:03 +0000, Fedora compose checker wrote:
Installed system changes in test x86_64 Everything-boot-iso install_default: System load changed from 0.04 to 0.19 Previous test data: https://openqa.fedoraproject.org/tests/50142#downloads Current test data: https://openqa.fedoraproject.org/tests/50240#downloads
Aha! So welcome to the shiny new check-compose feature, check-compose fans! (*are* there any check-compose fans?)
For a few weeks now, the 'default install' openQA tests have collected some system information data after completing the install process. It's not rocket science - they just log in and run:
top -i -n20 -b rpm -qa --queryformat "%{NAME}\n" free df systemctl -t service --no-pager | grep -o ".*.service"
and store the output.
check-compose now analyzes those results and includes what it thinks are interesting changes in the output between the 'previous' compose and the new one. It'll include a note when:
* A filesystem's contents change in size by more than 10% * A non-tmpfs filesystem's size changes at all * Used memory or swap changes by more than 10% * The installed package set changes at all * The running service set changes at all * System load after the 60 second 'top' run changes by more than 0.1 * Average CPU usage during the 60 second 'top' run changes by more than 10% * Peak process count during the top run changes by more than 10%
The 'top' ones are the trickiest ones to decide on, I'm still kinda fiddling with that. The idea is to catch if, say, some rogue process starts eating a bunch of CPU or I/O time when the system should be 'idle', but making sure it's a fair comparison is a bit difficult. I might have to fiddle around with exactly when the top run happens and stuff. Please excuse any not-terribly-useful output from the 'top' comparison. (In this case, I just changed the 'top' run from happening *after* all the other tests to happening *before* all the other tests; on 20161130.n.0 it ran after, on 20161202.n.2 it ran before, so it's not really an apples-to-apples comparison).
When it prints any 'system information' lines for a particular test 'scenario', it'll print links to the download page for both the 'before' and 'after' tests, so you can look at the raw tool output and see if it's worth further investigation.
Hope folks find this useful! We'll see how it goes.
You can send bug reports, pull requests etc. to the check-compose project on Pagure:
https://pagure.io/fedora-qa/check-compose