Fedora 26's rpm SIGILLs

[Ralf Corsepius]

Hi,

Since today's dnf update, rpm is dumping cores due to SIGILLS on my PIII.

No further comment from me - It would be an offence.

Ralf

[Florian Weimer]

On 10/30/2017 04:21 PM, Ralf Corsepius wrote:
> Since today's dnf update, rpm is dumping cores due to SIGILLS on my PIII.

Can you provide additional details?

Thanks,
Florian

[Ralf Corsepius]

On 10/30/2017 04:39 PM, Florian Weimer wrote:
> On 10/30/2017 04:21 PM, Ralf Corsepius wrote:
> > Since today's dnf update, rpm is dumping cores due to SIGILLS on my PIII.
>
> Can you provide additional details?

No, I regret,

- technically I can't. I am currently trying to recover this machine 
from backups.
- ATM, I do not feel in a mood to behave helpful.

Ralf

[Alexander Ploumistos]

I had upgraded my i686 systems to f27, so I could not test on f26, but
I decided to give it a try on my Athlon.

I upgraded from kernel-4.13.9-300.fc27.i686 to
kernel-4.13.5-300.fc27.i686, dnf-2.7.3-1.fc27.noarch to
dnf-2.7.5-1.fc27.noarch and rpm-4.14.0-0.rc2.3.fc27.i686 to
rpm-4.14.0-2.fc27.i686. There was no crash at any point and rpm
doesn't misbehave in any way.

In the mirrors my systems connect to, f26 is on rpm-4.13.0.2-1, while
rpm-4.13.0.1-7 was the previous version and it had been at that
version since August.

I also checked the upstream tracker and I could not find any similar
bug report - either open or closed.

Can anyone else take a look?

[GMail]

On Mon, 30 Oct 2017 16:21:38 +0100
Ralf Corsepius  wrote:
> Hi,
>
> Since today's dnf update, rpm is dumping cores due to SIGILLS on my PIII.
>
> No further comment from me - It would be an offence.
>
> Ralf
> _______________________________________________
> X86 mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

Any clue on how we can replicate this? Moreso, are you sure this is i686-specific?

[Ralf Corsepius]

On 10/30/2017 10:19 PM, GMail wrote:
> On Mon, 30 Oct 2017 16:21:38 +0100
> Ralf Corsepius  wrote:
> > Hi,
> >
> > Since today's dnf update, rpm is dumping cores due to SIGILLS on my PIII.
> >
> > No further comment from me - It would be an offence.
> >
> > Ralf
> > _______________________________________________
> > X86 mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
>
> Any clue on how we can replicate this? Moreso, are you sure this is i686-specific?

The reproducer was
"rpm -qa <something>"

e.g. rpm -qa 'kernel*'

Fortunately, I meanwhile managed to get a couple of the coredumps off 
the machine and uploaded them to https://corsepiu.fedorapeople.org/rpm/

Some additional remarks:

- My attempts to restore from backup (ca. 3 weeks old) seemingly are 
being killed by some nss* related update:

# rpm -U <some rpm>
error: Failed to initialize NSS library

ATM, my guess is, the architectural issues with nss-softkn* which are 
effectively showstopping fc27 on my PIII, now have been propagated to 
f26 and killing it, too.

- On my PIII, I am (for months) suffering from a kernel bug (Presumably 
the NIC driver) which render all Fedora kernels > 4.9.14 unusable for me.

Now, my guess is, the rpm SIGILLs I experienced yesterday originate from 
the "dnf update" having tripped of boths (or more) bugs simultaneously, 
all in all resulting in a corrupted installation.

Ralf

[Florian Weimer]

On 10/31/2017 05:09 AM, Ralf Corsepius wrote:
> Fortunately, I meanwhile managed to get a couple of the coredumps off 
> the machine and uploaded them to https://corsepiu.fedorapeople.org/rpm/

These files aren't readable:

-rw-r-----. 1 corsepiu corsepiu 324542 Oct 31 03:55 
core.rpm.0.17abf534e68e47a5b5083faf5456eedd.1197.1509376379000000.lz4
-rw-r-----. 1 corsepiu corsepiu 323956 Oct 31 03:55 
core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1171.1509376197000000.lz4
-rw-r-----. 1 corsepiu corsepiu 324793 Oct 31 03:55 
core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1175.1509376201000000.lz4
-rw-r-----. 1 corsepiu corsepiu 324938 Oct 31 03:55 
core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1179.1509376203000000.lz4

I doubt that restoring from backup will be necessary unless there is 
actual corruption.  If this is indeed a build problem, you should be 
able to upgrade the affected packages using rpm --root from rescue media 
once we've fixed the problem.

Thanks,
Florian

[Ralf Corsepius]

On 10/31/2017 11:41 AM, Florian Weimer wrote:
> On 10/31/2017 05:09 AM, Ralf Corsepius wrote:
> > Fortunately, I meanwhile managed to get a couple of the coredumps off 
> > the machine and uploaded them to https://corsepiu.fedorapeople.org/rpm/
>
> These files aren't readable:

Urgh, they should be readable now ;)
> -rw-r-----. 1 corsepiu corsepiu 324542 Oct 31 03:55 
> core.rpm.0.17abf534e68e47a5b5083faf5456eedd.1197.1509376379000000.lz4
> -rw-r-----. 1 corsepiu corsepiu 323956 Oct 31 03:55 
> core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1171.1509376197000000.lz4
> -rw-r-----. 1 corsepiu corsepiu 324793 Oct 31 03:55 
> core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1175.1509376201000000.lz4
> -rw-r-----. 1 corsepiu corsepiu 324938 Oct 31 03:55 
> core.rpm.0.c8d8c97534bc4d8e84f203294ef440b0.1179.1509376203000000.lz4
>
> I doubt that restoring from backup will be necessary unless there is 
> actual corruption.  If this is indeed a build problem, you should be 
> able to upgrade the affected packages using rpm --root from rescue media 
> once we've fixed the problem.

This machine is in a multiboot configuration.

The way, I am restoring it, is rsync'ing the backup from LAN from inside 
a different installation (fc25).

Ralf

[Florian Weimer]

On 10/31/2017 02:40 PM, Ralf Corsepius wrote:
> On 10/31/2017 11:41 AM, Florian Weimer wrote:
> > On 10/31/2017 05:09 AM, Ralf Corsepius wrote:
> > > Fortunately, I meanwhile managed to get a couple of the coredumps off 
> > > the machine and uploaded them to https://corsepiu.fedorapeople.org/rpm/
> >
> > These files aren't readable:
>
> Urgh, they should be readable now ;)

The culprit is nss-softokn-freebl:

#0  0xb7445b2a in SHA1_Update (ctx=0x1918478,
     dataIn=0xb7ec9f20 <rpm_header_magic> "\216\255\350\001", len=8)
     at sha_fast.c:101
#1  0xb75e3047 in NSC_DigestUpdate (hSession=2,
     pPart=0xb7ec9f20 <rpm_header_magic> "\216\255\350\001", ulPartLen=8)
     at pkcs11c.c:1795
#2  0xb7a6fbdd in PK11_DigestOp () from /lib/libnss3.so

Dump of assembler code for function SHA1_Update:
    0xb7445af0 <+0>:	sub    $0x2c,%esp
    0xb7445af3 <+3>:	mov    %ebx,0x1c(%esp)
    0xb7445af7 <+7>:	mov    0x38(%esp),%ebx
    0xb7445afb <+11>:	mov    %esi,0x20(%esp)
    0xb7445aff <+15>:	mov    0x34(%esp),%esi
    0xb7445b03 <+19>:	mov    %edi,0x24(%esp)
    0xb7445b07 <+23>:	mov    %ebp,0x28(%esp)
    0xb7445b0b <+27>:	test   %ebx,%ebx
    0xb7445b0d <+29>:	je     0xb7445b77 <SHA1_Update+135>
    0xb7445b0f <+31>:	mov    0x30(%esp),%eax
    0xb7445b13 <+35>:	xor    %edx,%edx
    0xb7445b15 <+37>:	movd   %ebx,%xmm0
    0xb7445b19 <+41>:	movd   %edx,%xmm1
    0xb7445b1d <+45>:	punpckldq %xmm1,%xmm0
    0xb7445b21 <+49>:	movq   0x40(%eax),%xmm2
    0xb7445b26 <+54>:	mov    0x30(%esp),%eax
=> 0xb7445b2a <+58>:	paddq  %xmm2,%xmm0

I'll file a bug.

Thanks,
Florian

[Florian Weimer]

On 10/31/2017 02:53 PM, Florian Weimer wrote:
> I'll file a bug.

Here it is: https://bugzilla.redhat.com/show_bug.cgi?id=1507938

Thanks,
Florian

[Ralf Corsepius]

On 10/31/2017 03:05 PM, Florian Weimer wrote:
> On 10/31/2017 02:53 PM, Florian Weimer wrote:
> > I'll file a bug.
>
> Here it is: https://bugzilla.redhat.com/show_bug.cgi?id=1507938

Thanks,

Time to -1
https://bodhi.fedoraproject.org/updates/FEDORA-2017-c0d71e8998
to prevent this regression from infecting fc25 as well?

Ralf

[Jeff Backus]

On Tue, Oct 31, 2017 at 9:53 AM, Florian Weimer  wrote:
> The culprit is nss-softokn-freebl:

Yeah, this had come up before, as you noticed in the BZ.

I thought this fix had already made it out, but I guess they are
waiting to rebase NSS. Seems like a good reason to include 3.34 in
F26.

Any word yet on whether this will happen? I see that Ralf asked on the
BZ, but didn't see a response.

jeff

[Ralf Corsepius]

On 10/31/2017 07:03 PM, Jeff Backus wrote:
> On Tue, Oct 31, 2017 at 9:53 AM, Florian Weimer  wrote:
> > The culprit is nss-softokn-freebl:
>
> Yeah, this had come up before, as you noticed in the BZ.
>
> I thought this fix had already made it out, but I guess they are
> waiting to rebase NSS. Seems like a good reason to include 3.34 in
> F26.

FWIW: Meanwhile I can confirm that, Ueno claim of f25 also already being 
been broken by updates is true.

Current nss-softokn-freebl (and thus thunderbird) doesn't comply to 
Fedora's architectural requirements in all released Fedoras.
The initial versions (in release) do.

Ralf

[Jeff Backus]

On Tue, Oct 31, 2017 at 2:22 PM, Ralf Corsepius  wrote:
> On 10/31/2017 07:03 PM, Jeff Backus wrote:
> > On Tue, Oct 31, 2017 at 9:53 AM, Florian Weimer 
> > wrote:
> > > The culprit is nss-softokn-freebl:
> >
> > Yeah, this had come up before, as you noticed in the BZ.
> >
> > I thought this fix had already made it out, but I guess they are
> > waiting to rebase NSS. Seems like a good reason to include 3.34 in
> > F26.
>
> FWIW: Meanwhile I can confirm that, Ueno claim of f25 also already being
> been broken by updates is true.
>
> Current nss-softokn-freebl (and thus thunderbird) doesn't comply to Fedora's
> architectural requirements in all released Fedoras.
> The initial versions (in release) do.

Thanks for the confirmation. We'll have to keep an eye out for anyone
else running into the problem.

It sounds like a simple downgrade of nss-softokn-freebl (and anything
that uses as a dependency) is sufficient to work around the issue. Do
we know if this is correct?

jeff