No missing expected images.
Failed openQA tests: 82/101 (x86_64), 3/17 (i386), 1/2 (arm)
New failures (same test did not fail in Rawhide-20161022.n.0):
ID: 43539 Test: i386 universal install_simple_encrypted URL: https://openqa.fedoraproject.org/tests/43539 ID: 43558 Test: x86_64 universal support_server URL: https://openqa.fedoraproject.org/tests/43558 ID: 43559 Test: x86_64 Server-dvd-iso support_server URL: https://openqa.fedoraproject.org/tests/43559 ID: 43565 Test: x86_64 Server-boot-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/43565 ID: 43566 Test: x86_64 Server-dvd-iso install_default@uefi URL: https://openqa.fedoraproject.org/tests/43566 ID: 43581 Test: x86_64 universal install_repository_http_variation URL: https://openqa.fedoraproject.org/tests/43581 ID: 43582 Test: x86_64 universal install_repository_http_graphical URL: https://openqa.fedoraproject.org/tests/43582 ID: 43583 Test: x86_64 universal install_mirrorlist_graphical URL: https://openqa.fedoraproject.org/tests/43583 ID: 43584 Test: x86_64 universal install_delete_pata URL: https://openqa.fedoraproject.org/tests/43584 ID: 43585 Test: x86_64 universal install_kickstart_user_creation URL: https://openqa.fedoraproject.org/tests/43585 ID: 43586 Test: x86_64 universal install_scsi_updates_img URL: https://openqa.fedoraproject.org/tests/43586 ID: 43587 Test: x86_64 universal install_multi URL: https://openqa.fedoraproject.org/tests/43587 ID: 43592 Test: x86_64 universal install_delete_pata@uefi URL: https://openqa.fedoraproject.org/tests/43592 ID: 43593 Test: x86_64 universal install_multi@uefi URL: https://openqa.fedoraproject.org/tests/43593 ID: 43620 Test: x86_64 universal install_simple_encrypted URL: https://openqa.fedoraproject.org/tests/43620 ID: 43621 Test: x86_64 universal install_multi_empty URL: https://openqa.fedoraproject.org/tests/43621 ID: 43623 Test: x86_64 universal install_software_raid URL: https://openqa.fedoraproject.org/tests/43623 ID: 43624 Test: x86_64 universal install_delete_partial URL: https://openqa.fedoraproject.org/tests/43624 ID: 43625 Test: x86_64 universal install_kickstart_hdd URL: https://openqa.fedoraproject.org/tests/43625 ID: 43626 Test: x86_64 universal upgrade_minimal_64bit URL: https://openqa.fedoraproject.org/tests/43626 ID: 43628 Test: x86_64 universal upgrade_server_64bit URL: https://openqa.fedoraproject.org/tests/43628 ID: 43629 Test: x86_64 universal upgrade_kde_64bit URL: https://openqa.fedoraproject.org/tests/43629 ID: 43631 Test: x86_64 universal install_package_set_minimal URL: https://openqa.fedoraproject.org/tests/43631 ID: 43632 Test: x86_64 universal install_anaconda_text URL: https://openqa.fedoraproject.org/tests/43632 ID: 43644 Test: x86_64 universal install_simple_encrypted@uefi URL: https://openqa.fedoraproject.org/tests/43644 ID: 43645 Test: x86_64 universal install_multi_empty@uefi URL: https://openqa.fedoraproject.org/tests/43645 ID: 43646 Test: x86_64 universal install_software_raid@uefi URL: https://openqa.fedoraproject.org/tests/43646 ID: 43647 Test: x86_64 universal install_delete_partial@uefi URL: https://openqa.fedoraproject.org/tests/43647 ID: 43660 Test: x86_64 Workstation-live-iso desktop_update_graphical URL: https://openqa.fedoraproject.org/tests/43660 ID: 43661 Test: x86_64 Workstation-live-iso desktop_terminal URL: https://openqa.fedoraproject.org/tests/43661 ID: 43662 Test: x86_64 Workstation-live-iso desktop_browser URL: https://openqa.fedoraproject.org/tests/43662 ID: 43693 Test: x86_64 universal install_btrfs URL: https://openqa.fedoraproject.org/tests/43693 ID: 43694 Test: x86_64 universal install_ext3 URL: https://openqa.fedoraproject.org/tests/43694 ID: 43696 Test: x86_64 universal install_xfs URL: https://openqa.fedoraproject.org/tests/43696 ID: 43697 Test: x86_64 universal install_lvmthin URL: https://openqa.fedoraproject.org/tests/43697 ID: 43698 Test: x86_64 Workstation-live-iso base_selinux URL: https://openqa.fedoraproject.org/tests/43698 ID: 43699 Test: x86_64 Workstation-live-iso base_services_start URL: https://openqa.fedoraproject.org/tests/43699 ID: 43700 Test: x86_64 universal install_updates_img_local URL: https://openqa.fedoraproject.org/tests/43700 ID: 43701 Test: x86_64 Workstation-live-iso base_update_cli URL: https://openqa.fedoraproject.org/tests/43701 ID: 43702 Test: x86_64 Workstation-live-iso base_service_manipulation URL: https://openqa.fedoraproject.org/tests/43702 ID: 43703 Test: x86_64 universal install_shrink_ext4 URL: https://openqa.fedoraproject.org/tests/43703 ID: 43704 Test: x86_64 universal install_shrink_ntfs URL: https://openqa.fedoraproject.org/tests/43704 ID: 43705 Test: x86_64 universal install_cyrillic_language URL: https://openqa.fedoraproject.org/tests/43705 ID: 43706 Test: x86_64 universal install_kickstart_firewall_disabled URL: https://openqa.fedoraproject.org/tests/43706 ID: 43707 Test: x86_64 universal install_european_language URL: https://openqa.fedoraproject.org/tests/43707 ID: 43708 Test: x86_64 universal install_kickstart_firewall_configured URL: https://openqa.fedoraproject.org/tests/43708 ID: 43709 Test: x86_64 universal install_kickstart_nfs URL: https://openqa.fedoraproject.org/tests/43709 ID: 43717 Test: x86_64 universal install_ext3@uefi URL: https://openqa.fedoraproject.org/tests/43717 ID: 43720 Test: x86_64 universal install_xfs@uefi URL: https://openqa.fedoraproject.org/tests/43720 ID: 43721 Test: x86_64 universal install_btrfs@uefi URL: https://openqa.fedoraproject.org/tests/43721 ID: 43722 Test: x86_64 universal install_lvmthin@uefi URL: https://openqa.fedoraproject.org/tests/43722 ID: 43748 Test: x86_64 Server-dvd-iso base_selinux URL: https://openqa.fedoraproject.org/tests/43748 ID: 43749 Test: x86_64 Server-dvd-iso server_firewall_default URL: https://openqa.fedoraproject.org/tests/43749 ID: 43750 Test: x86_64 Server-dvd-iso base_services_start URL: https://openqa.fedoraproject.org/tests/43750 ID: 43751 Test: x86_64 Server-dvd-iso base_update_cli URL: https://openqa.fedoraproject.org/tests/43751 ID: 43752 Test: x86_64 Server-dvd-iso base_service_manipulation URL: https://openqa.fedoraproject.org/tests/43752 ID: 43753 Test: x86_64 Server-dvd-iso server_role_deploy_database_server URL: https://openqa.fedoraproject.org/tests/43753 ID: 43754 Test: x86_64 Server-dvd-iso server_cockpit_default URL: https://openqa.fedoraproject.org/tests/43754 ID: 43759 Test: x86_64 Server-dvd-iso server_cockpit_basic URL: https://openqa.fedoraproject.org/tests/43759 ID: 43771 Test: x86_64 Server-dvd-iso realmd_join_sssd URL: https://openqa.fedoraproject.org/tests/43771 ID: 43774 Test: x86_64 universal install_no_swap URL: https://openqa.fedoraproject.org/tests/43774 ID: 43775 Test: x86_64 universal install_package_set_kde URL: https://openqa.fedoraproject.org/tests/43775 ID: 43777 Test: x86_64 universal upgrade_2_minimal_64bit URL: https://openqa.fedoraproject.org/tests/43777 ID: 43779 Test: x86_64 universal upgrade_2_server_64bit URL: https://openqa.fedoraproject.org/tests/43779 ID: 43784 Test: x86_64 universal install_no_swap@uefi URL: https://openqa.fedoraproject.org/tests/43784 ID: 43792 Test: x86_64 KDE-live-iso desktop_update_graphical URL: https://openqa.fedoraproject.org/tests/43792 ID: 43795 Test: x86_64 KDE-live-iso desktop_notifications_postinstall URL: https://openqa.fedoraproject.org/tests/43795 ID: 43796 Test: x86_64 KDE-live-iso desktop_terminal URL: https://openqa.fedoraproject.org/tests/43796 ID: 43804 Test: x86_64 KDE-live-iso base_selinux URL: https://openqa.fedoraproject.org/tests/43804 ID: 43806 Test: x86_64 KDE-live-iso base_service_manipulation URL: https://openqa.fedoraproject.org/tests/43806 ID: 43807 Test: x86_64 KDE-live-iso base_services_start URL: https://openqa.fedoraproject.org/tests/43807 ID: 43808 Test: x86_64 KDE-live-iso base_update_cli URL: https://openqa.fedoraproject.org/tests/43808
Old failures (same test failed in Rawhide-20161022.n.0):
ID: 43457 Test: arm Minimal-raw_xz-raw.xz install_arm_image_deployment_upload URL: https://openqa.fedoraproject.org/tests/43457 ID: 43544 Test: i386 universal upgrade_desktop_32bit URL: https://openqa.fedoraproject.org/tests/43544 ID: 43545 Test: i386 universal upgrade_2_desktop_32bit URL: https://openqa.fedoraproject.org/tests/43545 ID: 43622 Test: x86_64 universal install_simple_free_space URL: https://openqa.fedoraproject.org/tests/43622 ID: 43627 Test: x86_64 universal upgrade_desktop_64bit URL: https://openqa.fedoraproject.org/tests/43627 ID: 43630 Test: x86_64 universal upgrade_desktop_encrypted_64bit URL: https://openqa.fedoraproject.org/tests/43630 ID: 43643 Test: x86_64 universal install_simple_free_space@uefi URL: https://openqa.fedoraproject.org/tests/43643 ID: 43663 Test: x86_64 Workstation-live-iso desktop_notifications_postinstall URL: https://openqa.fedoraproject.org/tests/43663 ID: 43710 Test: x86_64 universal install_iscsi URL: https://openqa.fedoraproject.org/tests/43710 ID: 43763 Test: x86_64 Server-dvd-iso server_role_deploy_domain_controller URL: https://openqa.fedoraproject.org/tests/43763 ID: 43778 Test: x86_64 universal upgrade_2_desktop_64bit URL: https://openqa.fedoraproject.org/tests/43778 ID: 43780 Test: x86_64 universal upgrade_2_desktop_encrypted_64bit URL: https://openqa.fedoraproject.org/tests/43780 ID: 43781 Test: x86_64 universal upgrade_2_kde_64bit URL: https://openqa.fedoraproject.org/tests/43781 ID: 43794 Test: x86_64 KDE-live-iso desktop_browser URL: https://openqa.fedoraproject.org/tests/43794
Passed openQA tests: 12/101 (x86_64), 14/17 (i386)
Skipped openQA tests: 8 of 120
On Mon, 2016-10-24 at 20:19 +0000, Fedora compose checker wrote:
No missing expected images.
Failed openQA tests: 82/101 (x86_64), 3/17 (i386), 1/2 (arm)
Executive summary: I broke it, yo.
On Mon, 2016-10-24 at 16:49 -0700, Adam Williamson wrote:
On Mon, 2016-10-24 at 20:19 +0000, Fedora compose checker wrote:
No missing expected images.
Failed openQA tests: 82/101 (x86_64), 3/17 (i386), 1/2 (arm)
Executive summary: I broke it, yo.
Slightly more detailed version of what I think happened:
We had a bit of an issue with how openQA registers and then cleans up 'assets' (which are things like ISOs and hard disk images) that I fixed last week. There was a bit of remaining cleanup work to do which I was going to do after these tests ran (because of boring details, it wasn't possible to fully complete the cleanup until some tests ran a full two days after I installed the fixed patches).
Unfortunately the mess resulted in the asset cleanup code deleting the assets that were actually needed for these tests before most of them had even run: it decided it needed to free up space, but it couldn't find older assets that were downloaded during the buggy period, so it could only find the newer assets to delete.
There also turned out to be a bug in one of the fixes I submitted which meant openQA wouldn't properly log the name of the asset it was looking for but couldn't find, which confused the issue a bit.
I've now cleaned up the consequences of the initial asset registration bugs, so hopefully openQA should no longer decide it needs to delete assets that we haven't finished using yet, and sent a fix for the logging bug. I've re-sent the 20161024.n.0 Rawhide tests and will send a new compose check report manually when they complete (assuming they work properly this time).
We possibly could improve the asset cleanup code to check for pending jobs that use the assets it's about to delete and not delete them in that case, that might be a useful safety precaution in case of weird issues like this in future...