11:26 p.m.
dustymabe reported a new issue against the project: `releng` that you are following:
``
Seeing a lot of `memory allocation failure` errors for image builds in f26:
- https://koji.fedoraproject.org/koji/taskinfo?taskID=19420757
- https://koji.fedoraproject.org/koji/taskinfo?taskID=19420748
Not sure what this is about. I ran an image build locally and it succeeded.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
9:19 p.m.
dustymabe added a new comment to an issue you are following:
``
according to @puiterwijk we will need [this upstream
commit](https://github.com/clalancette/oz/commit/32a4edd3124e7acb2dac02ff...
pulled back into our oz and patched on our servers. see the [upstream
PR](https://github.com/clalancette/oz/pull/231) for comments.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
8:31 a.m.
The issue: `f26 image build failures: `memory allocation failure`` of project: `releng`
has been assigned to `imcleod` by mohanboddu.
https://pagure.io/releng/issue/6776
10:41 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
dustymabe added a new comment to an issue you are following:
``
updated the title to be more descriptive
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
10:44 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
The issue: `f26 image build failures: `security library: memory allocation failure`` of
project: `releng` has been assigned to `dgregor` by ausil.
https://pagure.io/releng/issue/6776
10:44 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
ausil added a new comment to an issue you are following:
``
```
buildhw-03.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-08.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-04.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-02.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-07.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-06.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-05.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-10.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-01.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-09.phx2.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
buildhw-aarch64-01.arm.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
120 factory-build-e453c145-6a28-4591-abd5-d1bf508d6d15 running
121 factory-build-0f9b5bae-b13c-4ec4-98c9-b75cfc8bf513 running
124 factory-build-93c8030d-f8f9-4337-948d-14dc05793963 running
128 factory-build-c5d1f408-fef2-402a-bb25-19ae067e5089 running
135 factory-build-06a229f7-bdd3-4c8e-9449-c85b6fe04112 running
136 factory-build-2ad50b68-db21-4055-8faf-9dff6515354f running
buildhw-aarch64-03.arm.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
115 factory-build-af4645ee-29f1-4a13-8031-587ca6453d9e running
117 factory-build-cafbca2d-db0c-4ffe-b523-745eba60ee2f running
118 factory-build-208c68b2-53f3-4056-b94d-229682822cd2 running
119 factory-build-0dab4805-e819-4d6f-b513-f8b3629241bb running
120 factory-build-7aa92663-372f-427f-ad30-45724bcca6a3 running
121 factory-build-b91c7f85-4612-4219-a9e0-1f99c87fa977 running
123 factory-build-99869671-9156-41d2-82c6-2663ff44858e running
126 factory-build-49f691c6-685a-4abd-8ede-5600515c4bad running
128 factory-build-2faaef25-31fc-4460-95ab-46a97ed04fd2 running
129 factory-build-f9115156-a0b5-45a2-b623-8a6cc07eaf95 running
132 factory-build-98a70020-c396-4c05-ae50-ed1da0f685f9 running
buildhw-aarch64-02.arm.fedoraproject.org | SUCCESS | rc=0 >>
Id Name State
----------------------------------------------------
130 factory-build-2e7c1034-8fdc-433c-bd0c-db6dc08d17bd running
133 factory-build-b2feb358-d0da-4026-b15d-d627a254f223 running
134 factory-build-9130ae49-736a-492a-90bb-65abf97d16ea running
135 factory-build-0b4f76ba-79d0-4a30-86b8-8ac7fe723829 running
139 factory-build-4cc234ca-e4e9-4117-abcf-cbd1efe94530 running
141 factory-build-19c05ac0-900a-4340-b498-d2878093a733 running
142 factory-build-5e4b9412-8c86-4609-bef6-5f3c69c58330 running
```
We have some issues with vms not being cleaned up.
https://github.com/redhat-imaging/imagefactory/issues/397 sets the tone for that.
There is another issue where we are getting
```
ApplianceError: Image status is FAILED: (35, 'security library: memory allocation
failure.')
```
which has caused this issue to be filed. It has not been fully debugged, we suspect its
due to nss, however it is a different error to what we have seen in the past. the issue
started happening after the updates being applied to builders on the second of May. The
updated package set was:
```
SDL2-2.0.5-3.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
python2-idna-2.5-1.fc25.noarch Tue 02 May 2017 10:32:31 PM UTC
libwsman1-2.6.2-8.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
json-c-0.12.1-1.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
gpgme-1.8.0-10.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
fedora-repos-25-3.noarch Tue 02 May 2017 10:32:31 PM UTC
emacs-filesystem-25.2-1.fc25.noarch Tue 02 May 2017 10:32:31 PM UTC
dmidecode-3.0-7.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
bluez-libs-5.44-1.fc25.x86_64 Tue 02 May 2017 10:32:31 PM UTC
bash-completion-2.5-1.fc25.noarch Tue 02 May 2017 10:32:31 PM UTC
tzdata-2017b-1.fc25.noarch Tue 02 May 2017 10:32:30 PM UTC
telnet-0.17-68.fc25.x86_64 Tue 02 May 2017 10:32:30 PM UTC
strace-4.16-1.fc25.x86_64 Tue 02 May 2017 10:32:30 PM UTC
python2-pyasn1-0.2.3-1.fc25.noarch Tue 02 May 2017 10:32:30 PM UTC
python2-boto-2.45.0-3.fc25.noarch Tue 02 May 2017 10:32:30 PM UTC
lua-libs-5.3.4-1.fc25.x86_64 Tue 02 May 2017 10:32:30 PM UTC
xkeyboard-config-2.20-2.fc25.noarch Tue 02 May 2017 10:32:29 PM UTC
screen-4.5.1-1.fc25.x86_64 Tue 02 May 2017 10:32:29 PM UTC
pulseaudio-libs-10.0-2.fc25.x86_64 Tue 02 May 2017 10:32:29 PM UTC
make-4.1-6.fc25.x86_64 Tue 02 May 2017 10:32:29 PM UTC
libgusb-0.2.10-1.fc25.x86_64 Tue 02 May 2017 10:32:29 PM UTC
libcroco-0.6.11-3.fc25.x86_64 Tue 02 May 2017 10:32:29 PM UTC
iproute-tc-4.10.0-1.fc25.x86_64 Tue 02 May 2017 10:32:28 PM UTC
python3-sssdconfig-1.15.2-2.fc25.noarch Tue 02 May 2017 10:32:27 PM UTC
python2-dnf-1.1.10-6.fc25.noarch Tue 02 May 2017 10:32:27 PM UTC
dnf-plugins-core-0.1.21-5.fc25.noarch Tue 02 May 2017 10:32:27 PM UTC
ca-certificates-2017.2.11-1.1.fc25.noarch Tue 02 May 2017 10:32:27 PM UTC
zsh-5.2-6.fc25.x86_64 Tue 02 May 2017 10:32:26 PM UTC
polkit-0.113-8.fc25.x86_64 Tue 02 May 2017 10:32:26 PM UTC
acpid-2.0.28-2.fc25.x86_64 Tue 02 May 2017 10:32:26 PM UTC
os-prober-1.74-1.fc25.x86_64 Tue 02 May 2017 10:32:25 PM UTC
at-spi2-core-2.22.1-1.fc25.x86_64 Tue 02 May 2017 10:32:25 PM UTC
selinux-policy-targeted-3.13.1-225.13.fc25.noarch Tue 02 May 2017 10:32:11 PM UTC
policycoreutils-python-utils-2.5-20.fc25.x86_64 Tue 02 May 2017 10:32:09 PM UTC
openssh-server-7.4p1-4.fc25.x86_64 Tue 02 May 2017 10:32:09 PM UTC
krb5-workstation-1.14.4-7.fc25.x86_64 Tue 02 May 2017 10:32:09 PM UTC
glusterfs-fuse-3.10.1-1.fc25.x86_64 Tue 02 May 2017 10:32:09 PM UTC
glusterfs-api-3.10.1-1.fc25.x86_64 Tue 02 May 2017 10:32:09 PM UTC
dnf-yum-1.1.10-6.fc25.noarch Tue 02 May 2017 10:32:09 PM UTC
dnf-automatic-1.1.10-6.fc25.noarch Tue 02 May 2017 10:32:09 PM UTC
vim-enhanced-8.0.586-1.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
sudo-1.8.19p2-1.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
nfs-utils-2.1.1-5.rc2.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
NetworkManager-1.4.4-4.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
kernel-4.10.13-200.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
iptables-services-1.6.0-3.fc25.x86_64 Tue 02 May 2017 10:32:08 PM UTC
rsyslog-8.26.0-1.fc25.x86_64 Tue 02 May 2017 10:32:07 PM UTC
osinfo-db-20170423-1.fc25.noarch Tue 02 May 2017 10:32:07 PM UTC
gtk3-3.22.12-2.fc25.x86_64 Tue 02 May 2017 10:32:07 PM UTC
oz-0.15.0-5.fc25.noarch Tue 02 May 2017 10:32:06 PM UTC
koji-builder-1.12.0-2.fc25.noarch Tue 02 May 2017 10:32:06 PM UTC
euca2ools-3.4.1-1.fc25.noarch Tue 02 May 2017 10:32:06 PM UTC
python2-libguestfs-1.36.3-2.fc25.x86_64 Tue 02 May 2017 10:32:05 PM UTC
publicsuffix-list-dafsa-20170206-1.fc25.noarch Tue 02 May 2017 10:32:05 PM UTC
mock-1.3.4-1.fc25.noarch Tue 02 May 2017 10:32:05 PM UTC
libpsl-0.17.0-1.fc25.x86_64 Tue 02 May 2017 10:32:05 PM UTC
libguestfs-1.36.3-2.fc25.x86_64 Tue 02 May 2017 10:32:05 PM UTC
GeoIP-GeoLite-data-2017.04-1.fc25.noarch Tue 02 May 2017 10:32:05 PM UTC
GeoIP-1.6.10-1.fc25.x86_64 Tue 02 May 2017 10:32:05 PM UTC
distribution-gpg-keys-1.11-1.fc25.noarch Tue 02 May 2017 10:32:05 PM UTC
augeas-libs-1.8.0-1.fc25.x86_64 Tue 02 May 2017 10:32:05 PM UTC
python-requestbuilder-0.7.1-1.fc25.noarch Tue 02 May 2017 10:32:04 PM UTC
libXi-1.7.9-1.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
libnetfilter_conntrack-1.0.6-2.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
liblogging-stdlog-1.0.6-1.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
libepoxy-1.4.1-1.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
iptables-1.6.0-3.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
hwdata-0.299-1.fc25.noarch Tue 02 May 2017 10:32:04 PM UTC
dosfstools-4.1-1.fc25.x86_64 Tue 02 May 2017 10:32:04 PM UTC
kernel-modules-4.10.13-200.fc25.x86_64 Tue 02 May 2017 10:32:00 PM UTC
kernel-core-4.10.13-200.fc25.x86_64 Tue 02 May 2017 10:31:59 PM UTC
linux-firmware-20170313-72.git695f2d6d.fc25.noarch Tue 02 May 2017 10:31:57 PM UTC
mesa-libglapi-13.0.4-3.fc25.x86_64 Tue 02 May 2017 10:31:53 PM UTC
mesa-libGL-13.0.4-3.fc25.x86_64 Tue 02 May 2017 10:31:53 PM UTC
libglvnd-glx-0.2.999-14.20170308git8e6e102.fc25.x86_64 Tue 02 May 2017 10:31:53 PM UTC
vim-common-8.0.586-1.fc25.x86_64 Tue 02 May 2017 10:31:52 PM UTC
python-multilib-conf-1.2-1.fc25.noarch Tue 02 May 2017 10:31:52 PM UTC
python2-multilib-1.2-1.fc25.noarch Tue 02 May 2017 10:31:52 PM UTC
pyOpenSSL-16.2.0-1.fc25.noarch Tue 02 May 2017 10:31:52 PM UTC
perl-libs-5.24.1-385.fc25.x86_64 Tue 02 May 2017 10:31:52 PM UTC
mesa-libwayland-egl-13.0.4-3.fc25.x86_64 Tue 02 May 2017 10:31:52 PM UTC
koji-1.12.0-2.fc25.noarch Tue 02 May 2017 10:31:52 PM UTC
vim-minimal-8.0.586-1.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
vim-filesystem-8.0.586-1.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
pango-1.40.5-1.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
openssh-clients-7.4p1-4.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
libnfsidmap-0.27-1.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
json-glib-1.2.6-1.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
git-core-2.9.3-3.fc25.x86_64 Tue 02 May 2017 10:31:50 PM UTC
systemd-container-231-14.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
rpcbind-0.2.4-5.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
openssh-7.4p1-4.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
openldap-2.4.44-10.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
nss-tools-3.29.3-1.1.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
nss-sysinit-3.29.3-1.1.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
nss-pem-1.0.3-3.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
nss-3.29.3-1.1.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
NetworkManager-libnm-1.4.4-4.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
mesa-libEGL-13.0.4-3.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
libglvnd-egl-0.2.999-14.20170308git8e6e102.fc25.x86_64 Tue 02 May 2017 10:31:49 PM UTC
policycoreutils-python3-2.5-20.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
ntfsprogs-2016.2.22-4.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
mesa-libgbm-13.0.4-3.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
libkadm5-1.14.4-7.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
krb5-libs-1.14.4-7.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
gssproxy-0.7.0-3.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
gnutls-3.5.11-1.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
cups-libs-2.2.0-8.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
audit-libs-python3-2.7.6-1.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
audit-2.7.6-1.fc25.x86_64 Tue 02 May 2017 10:31:48 PM UTC
glusterfs-3.10.1-1.fc25.x86_64 Tue 02 May 2017 10:31:47 PM UTC
dnf-1.1.10-6.fc25.noarch Tue 02 May 2017 10:31:47 PM UTC
crypto-policies-20160921-4.gitf3018dd.fc25.noarch Tue 02 May 2017 10:31:47 PM UTC
systemd-udev-231-14.fc25.x86_64 Tue 02 May 2017 10:31:42 PM UTC
selinux-policy-3.13.1-225.13.fc25.noarch Tue 02 May 2017 10:31:42 PM UTC
policycoreutils-2.5-20.fc25.x86_64 Tue 02 May 2017 10:31:42 PM UTC
libdrm-2.4.79-1.fc25.x86_64 Tue 02 May 2017 10:31:42 PM UTC
systemd-pam-231-14.fc25.x86_64 Tue 02 May 2017 10:31:41 PM UTC
systemd-231-14.fc25.x86_64 Tue 02 May 2017 10:31:41 PM UTC
dbus-1.11.12-1.fc25.x86_64 Tue 02 May 2017 10:31:41 PM UTC
coreutils-8.25-17.fc25.x86_64 Tue 02 May 2017 10:31:41 PM UTC
python3-libs-3.5.3-5.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
python3-dnf-plugins-core-0.1.21-5.fc25.noarch Tue 02 May 2017 10:31:40 PM UTC
python3-distro-1.0.3-1.fc25.noarch Tue 02 May 2017 10:31:40 PM UTC
python3-3.5.3-5.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
python2-hawkey-0.6.4-3.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
libsemanage-python3-2.5-9.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
gtk-update-icon-cache-3.22.12-2.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
gdk-pixbuf2-modules-2.36.6-1.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
coreutils-common-8.25-17.fc25.x86_64 Tue 02 May 2017 10:31:40 PM UTC
system-python-libs-3.5.3-5.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
system-python-3.5.3-5.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
python3-hawkey-0.6.4-3.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
python3-dnf-1.1.10-6.fc25.noarch Tue 02 May 2017 10:31:39 PM UTC
libglvnd-0.2.999-14.20170308git8e6e102.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
file-libs-5.29-4.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
file-5.29-4.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
dnf-conf-1.1.10-6.fc25.noarch Tue 02 May 2017 10:31:39 PM UTC
cryptsetup-libs-1.7.5-1.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
cryptsetup-1.7.5-1.fc25.x86_64 Tue 02 May 2017 10:31:39 PM UTC
polkit-libs-0.113-8.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
p11-kit-trust-0.23.2-3.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
ntfs-3g-2016.2.22-4.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
nss-softokn-freebl-3.29.5-1.0.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
nss-softokn-3.29.5-1.0.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
libsemanage-2.5-9.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
libidn2-2.0.2-1.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
glusterfs-client-xlators-3.10.1-1.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
gdbm-1.13-1.fc25.x86_64 Tue 02 May 2017 10:31:37 PM UTC
p11-kit-0.23.2-3.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
nss-util-3.29.5-1.0.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
libsolv-0.6.26-3.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
libseccomp-2.3.2-1.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
iptables-libs-1.6.0-3.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
iproute-4.10.0-1.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
info-6.1-4.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
hawkey-0.6.4-3.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
grep-2.27-2.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
glusterfs-libs-3.10.1-1.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
gdk-pixbuf2-2.36.6-1.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
dbus-libs-1.11.12-1.fc25.x86_64 Tue 02 May 2017 10:31:36 PM UTC
systemd-libs-231-14.fc25.x86_64 Tue 02 May 2017 10:31:35 PM UTC
glib2-2.50.3-1.fc25.x86_64 Tue 02 May 2017 10:31:35 PM UTC
audit-libs-2.7.6-1.fc25.x86_64 Tue 02 May 2017 10:31:35 PM UTC
```
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
10:47 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
ausil added a new comment to an issue you are following:
``
แบhen there is too many wild running vms we have had image failures due to being unable to
allocate memory for the virtual machine. that error message is:
```
ApplianceError: Image status is FAILED: internal error: process exited while connecting to
monitor: 2017-05-06T11:35:28.826779Z qemu-system-arm: cannot set up guest memory
'mach-virt.ram': Cannot allocate memory
```
as can be seen in task https://koji.fedoraproject.org/koji/taskinfo?taskID=19420033
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
1:34 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
ausil added a new comment to an issue you are following:
``
```
May 10 12:28:00 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:00,713
[INFO] {23494} koji.TaskManager:1077 Attempting to take task 19487598
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,057
[INFO] {23494} koji.TaskManager:678 pids: {19487568: 7113, 19487598: 7114}
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,204
[INFO] {23494} koji.TaskManager:680 open task: {'waiting': None, 'id':
19487598, 'weight': 2.0}
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,204
[INFO] {23494} koji.TaskManager:680 open task: {'waiting': True, 'id':
19487568, 'weight': 1.0}
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]:
/usr/lib64/python2.7/optparse.py:1013: PendingDeprecationWarning: The KSOption class is
deprecated and will be removed in pykickstart-3.
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]: option =
self.option_class(*args, **kwargs)
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:06,538
[INFO] {7114} koji.build.BaseTaskHandler:3572 dispatching a baseimg builder
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:07,050
[WARNING] {7114} koji.TaskManager:1179 TRACEBACK: Traceback (most recent call last):
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/daemon.py", line 1160, in runTask
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: response =
(handler.run(),)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/tasks.py", line 158, in run
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: return
koji.util.call_with_argcheck(self.handler, self.params, self.opts)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/util.py", line 156, in
call_with_argcheck
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: return func(*args,
**kwargs)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3839, in handler
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: uses this to figure
out what plugin to run
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3473, in do_images
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: an absolute path to
the modified XML
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3581, in _buildBase
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]:
self.logger.info('dispatching %s image builder' % format)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3537, in _checkImageState
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: f_dict['raw']
= False
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: ApplianceError: Image
status is FAILED: (35, 'security library: memory allocation failure.')
```
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
1:39 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
ausil added a new comment to an issue you are following:
``
```
May 10 12:28:00 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:00,713
[INFO] {23494} koji.TaskManager:1077 Attempting to take task 19487598
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,057
[INFO] {23494} koji.TaskManager:678 pids: {19487568: 7113, 19487598: 7114}
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,204
[INFO] {23494} koji.TaskManager:680 open task: {'waiting': None, 'id':
19487598, 'weight': 2.0}
May 10 12:28:03 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:03,204
[INFO] {23494} koji.TaskManager:680 open task: {'waiting': True, 'id':
19487568, 'weight': 1.0}
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]:
/usr/lib64/python2.7/optparse.py:1013: PendingDeprecationWarning: The KSOption class is
deprecated and will be removed in pykickstart-3.
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]: option =
self.option_class(*args, **kwargs)
May 10 12:28:06 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:06,538
[INFO] {7114} koji.build.BaseTaskHandler:3572 dispatching a baseimg builder
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: 2017-05-10 12:28:07,050
[WARNING] {7114} koji.TaskManager:1179 TRACEBACK: Traceback (most recent call last):
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/daemon.py", line 1160, in runTask
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: response =
(handler.run(),)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/tasks.py", line 158, in run
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: return
koji.util.call_with_argcheck(self.handler, self.params, self.opts)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/lib/python2.7/site-packages/koji/util.py", line 156, in
call_with_argcheck
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: return func(*args,
**kwargs)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3839, in handler
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: uses this to figure
out what plugin to run
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3473, in do_images
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: an absolute path to
the modified XML
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3581, in _buildBase
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]:
self.logger.info('dispatching %s image builder' % format)
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: File
"/usr/sbin/kojid", line 3537, in _checkImageState
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: f_dict['raw']
= False
May 10 12:28:07 buildhw-06.phx2.fedoraproject.org kojid[23494]: ApplianceError: Image
status is FAILED: (35, 'security library: memory allocation failure.')
```
the builder traceback is in koji code
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
3:17 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
breilly added a new comment to an issue you are following:
``
I've created a patch for this issue from an upstream commit and tested it locally
using mockbuild.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
3:25 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
dustymabe added a new comment to an issue you are following:
``
hey @breilly, thanks - that should help us find out if the issue is related to oz's
initialization of the nss library or not.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
4:42 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
ausil added a new comment to an issue you are following:
``
someone applied updates to the builders today
```
python-fedora-0.9.0-3.fc25.noarch Wed 10 May 2017 04:46:57 PM UTC
python2-openidc-client-0-3.20170327git5456800.fc25.noarch Wed 10 May 2017 04:46:57 PM
UTC
kernel-4.10.14-200.fc25.x86_64 Wed 10 May 2017 04:46:57 PM UTC
kernel-modules-4.10.14-200.fc25.x86_64 Wed 10 May 2017 04:46:54 PM UTC
kernel-core-4.10.14-200.fc25.x86_64 Wed 10 May 2017 04:46:52 PM UTC
rsync-3.1.2-3.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
python-pycurl-7.43.0-5.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
python3-pycurl-7.43.0-5.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
publicsuffix-list-dafsa-20170424-1.fc25.noarch Wed 10 May 2017 02:34:12 AM UTC
mesa-libwayland-egl-17.0.5-2.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
libsolv-0.6.27-1.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
libICE-1.0.9-9.fc25.x86_64 Wed 10 May 2017 02:34:12 AM UTC
emacs-filesystem-25.2-2.fc25.noarch Wed 10 May 2017 02:34:12 AM UTC
ca-certificates-2017.2.14-1.0.fc25.noarch Wed 10 May 2017 02:34:12 AM UTC
nss-util-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nss-tools-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nss-sysinit-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nss-softokn-freebl-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nss-softokn-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nss-3.30.2-1.0.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
nspr-4.14.0-2.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
mesa-libglapi-17.0.5-2.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
mesa-libGL-17.0.5-2.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
mesa-libgbm-17.0.5-2.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
mesa-libEGL-17.0.5-2.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
libX11-common-1.6.5-1.fc25.noarch Wed 10 May 2017 02:34:11 AM UTC
libX11-1.6.5-1.fc25.x86_64 Wed 10 May 2017 02:34:11 AM UTC
```
after restarting kojid it seems things are happy again
https://koji.fedoraproject.org/koji/taskinfo?taskID=19490843
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
6:13 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
dustymabe added a new comment to an issue you are following:
``
I was talking to @kevin last night about "reverting" to older nss to see if it
solved the problem. He noticed that there was newer nss so he suggested we "go
forward" and try that first before trying to "go back". so he applied the
updates. The image updates from last night failed, though, so we assumed that the updates
did not fix the problem.
Are you saying that the new updates did "fix the problem", but that kojid
wasn't restarted and that's why the image builds from last night failed?
If so can you kick a .1 f25 atomic host build? the task you linked above is f26 and f26 is
broken in other ways unfortunately :(
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
8:33 p.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
dustymabe added a new comment to an issue you are following:
``
the .1 compose got further than the memory issue we were seeing, which is good news.
http://kojipkgs.fedoraproject.org/compose/twoweek/Fedora-Atomic-25-201705...
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
10:53 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
dustymabe added a new comment to an issue you are following:
``
Since brendan didn't have commit access to oz (yet) we took the patches and applied
them to oz in dist-git http://pkgs.fedoraproject.org/cgit/rpms/oz.git/log/?h=f25. We also
built a version that can be tagged into infra when needed:
https://koji.fedoraproject.org/koji/buildinfo?buildID=889265
we did not yet install oz into the builders, but dennis did restart kojid after an update
that included updating the nss rpms again and we had a successful build last night. We
should install the new oz so that we don't have non-deterministic behavior when it
comes to initializing the nss library. We can wait until we start seeing the problem
again, or we could do it now.
Either way this immediate issue is addressed and this ticket can be closed.
``
To reply, visit the link below or just reply to this email
https://pagure.io/releng/issue/6776
11:07 a.m.
New subject: [releng] Issue #6776: f26 image build failures: `security library: memory
allocation failure`
The status of the issue: `f26 image build failures: `security library: memory allocation
failure`` of project: `releng` has been updated to: Closed as Fixed by mohanboddu.
https://pagure.io/releng/issue/6776
2551
days inactive
2555
days old
rel-eng@lists.fedoraproject.org
15 comments
4 participants
participants (4)
-
Brendan Reilly -
Dennis Gilmore
-
Dusty Mabe
-
Mohan Boddu