Python misbehaves on Xen/Fedora 17/AMD FX-4170 Processor

[W. Michael Petullo]

I am attempting to run Xen on an AMD FX-4170 Quad-Core Processor.
I have installed Fedora 17 to serve as Dom0.

I am finding that Python does not run well on this
configuration. Specifically, "import random" causes the Python interpreter
to terminate with "Illegal instruction". This means that xend and yum will
not run.

I filed a bug under yum [1], but just realized that things work if I
boot Fedora 17 on bare metal.

Under Xen, /proc/cpuinfo reports the following flags:

fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall
nx mmxext fxsr_opt lm constant_tsc rep_good nopl nonstop_tsc extd_apicid
aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor
lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
xop fma4 perfctr_core arat cpb hw_pstate

Has anyone had similar trouble?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=829011

-- 
Mike

:wq

[Konrad Rzeszutek Wilk]

On Thu, Jun 28, 2012 at 02:29:08PM -0500, W. Michael Petullo wrote:
>  I am attempting to run Xen on an AMD FX-4170 Quad-Core Processor.
>  I have installed Fedora 17 to serve as Dom0.
>  
>  I am finding that Python does not run well on this
>  configuration. Specifically, "import random" causes the Python interpreter
>  to terminate with "Illegal instruction". This means that xend and yum will
>  not run.
>  
>  I filed a bug under yum [1], but just realized that things work if I
>  boot Fedora 17 on bare metal.
>  
>  Under Xen, /proc/cpuinfo reports the following flags:
>  
>  fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall
>  nx mmxext fxsr_opt lm constant_tsc rep_good nopl nonstop_tsc extd_apicid
>  aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor
>  lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
>  xop fma4 perfctr_core arat cpb hw_pstate
>  
>  Has anyone had similar trouble? 
No. But it sounds like the AVX disaster.

https://bugzilla.redhat.com/show_bug.cgi?id=801650

>  
>  [1] https://bugzilla.redhat.com/show_bug.cgi?id=829011
>  
>  -- 
>  Mike
>  
>  :wq
>  --
>  xen mailing list
>  xen(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/xen

[W. Michael Petullo]

> > I am attempting to run Xen on an AMD FX-4170 Quad-Core
> Processor.
> > I have installed Fedora 17 to serve as Dom0.
> > 
> > I am finding that Python does not run well on this
> > configuration. Specifically, "import random" causes the Python interpreter
> > to terminate with "Illegal instruction". This means that xend and yum will
> > not run.
> > 
> > I filed a bug under yum [1], but just realized that things work if I
> > boot Fedora 17 on bare metal.
> > 
> > Under Xen, /proc/cpuinfo reports the following flags:
> > 
> > fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall
> > nx mmxext fxsr_opt lm constant_tsc rep_good nopl nonstop_tsc extd_apicid
> > aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor
> > lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
> > xop fma4 perfctr_core arat cpb hw_pstate
> > 
> > Has anyone had similar trouble? 
 
>  No. But it sounds like the AVX disaster.
>  
>  https://bugzilla.redhat.com/show_bug.cgi?id=801650 
It seems to be.

I booted bare metal and used yum to install gdb and the debug symbols
for Python and glibc.

Then I rebooted in Xen. I found that gdb itself crashed when the debug
symbols were installed, so I removed them. I ran gdb again and obtained
a partial backtrace on "import random":

(gdb) ba
#0  0x00007ffff71474ec in __ieee754_exp_fma4 () from /lib64/libm.so.6
#1  0x00007ffff009e415 in ?? () from /usr/lib64/python2.7/lib-dynload/math.so
#2  0x00007ffff7b00053 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#3  0x00007ffff7b00b2f in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#4  0x00007ffff7b00c02 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
#5  0x00007ffff7b1017d in PyImport_ExecCodeModuleEx () from /lib64/libpython2.7.so.1.0
[...]

After looking a little closer, it seems that __ieee754_exp_fma4 is
executing "vmovsd %xmm0,-0x20(%rsp)".

I thought booting Xen with xsave=1 might help, but this causes Xen to
crash on this hardware. As mentioned above, it looks like there has been
work in glibc to properly detect AVX (my understanding is that you have
to check that both the processor and OS supports it). Is it possible
that this glibc work missed something?

-- 
Mike

:wq

[Pasi Kärkkäinen]

On Mon, Jul 02, 2012 at 10:04:13AM -0500, W. Michael Petullo wrote:
>  
>  I thought booting Xen with xsave=1 might help, but this causes Xen to
>  crash on this hardware.
>  
Hopefully you'll open a thread about this crash on xen-devel..

Thanks,

-- Pasi

[Konrad Rzeszutek Wilk]

On Mon, Jul 02, 2012 at 10:04:13AM -0500, W. Michael Petullo wrote:
>  >> I am attempting to run Xen on an AMD FX-4170 Quad-Core
> Processor.
>  >> I have installed Fedora 17 to serve as Dom0.
>  >> 
>  >> I am finding that Python does not run well on this
>  >> configuration. Specifically, "import random" causes the Python
> interpreter
>  >> to terminate with "Illegal instruction". This means that xend and yum
> will
>  >> not run.
>  >> 
>  >> I filed a bug under yum [1], but just realized that things work if I
>  >> boot Fedora 17 on bare metal.
>  >> 
>  >> Under Xen, /proc/cpuinfo reports the following flags:
>  >> 
>  >> fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall
>  >> nx mmxext fxsr_opt lm constant_tsc rep_good nopl nonstop_tsc extd_apicid
>  >> aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor
>  >> lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
>  >> xop fma4 perfctr_core arat cpb hw_pstate
>  >> 
>  >> Has anyone had similar trouble?
>   
>  > No. But it sounds like the AVX disaster.
>  > 
>  > https://bugzilla.redhat.com/show_bug.cgi?id=801650
>  
>  It seems to be.
>  
>  I booted bare metal and used yum to install gdb and the debug symbols
>  for Python and glibc.
>  
>  Then I rebooted in Xen. I found that gdb itself crashed when the debug
>  symbols were installed, so I removed them. I ran gdb again and obtained
>  a partial backtrace on "import random":
>  
>  (gdb) ba
>  #0  0x00007ffff71474ec in __ieee754_exp_fma4 () from /lib64/libm.so.6
>  #1  0x00007ffff009e415 in ?? () from /usr/lib64/python2.7/lib-dynload/math.so
>  #2  0x00007ffff7b00053 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
>  #3  0x00007ffff7b00b2f in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
>  #4  0x00007ffff7b00c02 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
>  #5  0x00007ffff7b1017d in PyImport_ExecCodeModuleEx () from /lib64/libpython2.7.so.1.0
>  [...]
>  
>  After looking a little closer, it seems that __ieee754_exp_fma4 is
>  executing "vmovsd %xmm0,-0x20(%rsp)".
>  
>  I thought booting Xen with xsave=1 might help, but this causes Xen to
>  crash on this hardware. As mentioned above, it looks like there has been 
Right. That is b/c of Fedora's incorrect extra patch that neuters xsave.

>  work in glibc to properly detect AVX (my understanding is that you
> have
>  to check that both the processor and OS supports it). Is it possible
>  that this glibc work missed something? 
Very likely. There was another fix to it that got added in Debian (or Ubuntu)
that checked for FMA4 on AMD (which had a different CPUID).

Can you open a Red Hat bugzilla please? And CC me on it: ketuzsezr(a)darnok.org

>  
>  -- 
>  Mike
>  
>  :wq

[W. Michael Petullo]

> >>> Has anyone had similar trouble? 
> >> No. But it sounds like the AVX disaster.
> >> 
> >> https://bugzilla.redhat.com/show_bug.cgi?id=801650 
> > It seems to be.
> >
> > I booted bare metal and used yum to install gdb and the debug symbols
> > for Python and glibc.
> > 
> > Then I rebooted in Xen. I found that gdb itself crashed when the debug
> > symbols were installed, so I removed them. I ran gdb again and obtained
> > a partial backtrace on "import random":
> > 
> > (gdb) ba
> > #0  0x00007ffff71474ec in __ieee754_exp_fma4 () from /lib64/libm.so.6
> > #1  0x00007ffff009e415 in ?? () from /usr/lib64/python2.7/lib-dynload/math.so
> > #2  0x00007ffff7b00053 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
> > #3  0x00007ffff7b00b2f in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
> > #4  0x00007ffff7b00c02 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
> > #5  0x00007ffff7b1017d in PyImport_ExecCodeModuleEx () from
> /lib64/libpython2.7.so.1.0
> > [...]
> > 
> > After looking a little closer, it seems that __ieee754_exp_fma4 is
> > executing "vmovsd %xmm0,-0x20(%rsp)".
> > 
> > I thought booting Xen with xsave=1 might help, but this causes Xen to
> > crash on this hardware. As mentioned above, it looks like there has been 
 
>  Right. That is b/c of Fedora's incorrect extra patch that neuters
> xsave. 
 
> > work in glibc to properly detect AVX (my understanding is that
> you have
> > to check that both the processor and OS supports it). Is it possible
> > that this glibc work missed something? 
 
>  Very likely. There was another fix to it that got added in Debian (or
> Ubuntu)
>  that checked for FMA4 on AMD (which had a different CPUID).
>  
>  Can you open a Red Hat bugzilla please? And CC me on it: ketuzsezr(a)darnok.org 
The issue in glibc has been fixed by Jeff Law. Please see:

	https://admin.fedoraproject.org/updates/glibc-2.15-51.fc17

Python works fine with this update, pushed today.

-- 
Mike

:wq