Kernel crash on wandboard quad, Fedora 24 Beta

[Steven Falco]

Recently I upgraded my wandboard quad to Fedora 24 beta.  Since I did that, I can
no longer reboot the machine without getting a kernel crash.  This happens 100%
of the time.

I filed a bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1336127

You can see the whole reboot log in the above bug - here is part of the log to
give a flavor of what is happening.  It looks like the bug originates when the
USB driver for the root device is shut down.  I do not have any external USB
devices connected to the machine.

[190129.856042] systemd-shutdown[1]: Rebooting.
[190129.864389] ci_hdrc ci_hdrc.1: remove, state 4
[190129.868951] usb usb1: USB disconnect, device number 1
[190129.874989] ci_hdrc ci_hdrc.1: USB bus 1 deregistered
[190129.880465] Unable to handle kernel NULL pointer dereference at virtual address
000000e0
[190129.888651] pgd = ed4d0000
[190129.891450] [000000e0] *pgd=00000000
[190129.895136] Internal error: Oops: 5 [#1] SMP ARM
[190129.899845] Modules linked in: rpcsec_gss_krb5 snd_soc_fsl_ssi snd_soc_fsl_spdif
imx_pcm_fiq snd_soc_fsl_asrc caam_jr snd_soc_imx_sgtl5000 dw_hdmi_ahb_audio coda
snd_soc_sgtl5000 imx_pcm_dma snd_soc_imx_spdif snd_soc_fsl_asoc_card snd_soc_core
imx_ipuv3_crtc snd_pcm_dmaengine snd_ac97_codec ac97_bus snd_seq v4l2_mem2mem
snd_seq_device snd_pcm videobuf2_vmalloc dw_hdmi_imx snd_timer ci_hdrc_imx nvmem_imx_ocotp
snd videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 ci_hdrc videobuf2_core dw_hdmi
nvmem_core udc_core v4l2_common snd_soc_imx_audmux soundcore caam imx_ipu_v3 usbmisc_imx
imx_sdma videodev imx2_wdt imx_thermal media etnaviv nfsd mmc_block at803x ahci_imx
i2c_imx libahci_platform sdhci_esdhc_imx fec sdhci_pltfm ptp sdhci pps_core mmc_core
rtc_snvs imxdrm phy_mxs_usb drm_kms_helper anatop_regulator syscopyarea sysfillrect
sysimgblt fb_sys_fops drm
[190129.976821] CPU: 2 PID: 1 Comm: systemd-shutdow Not tainted 4.5.4-300.fc24.armv7hl #1
[190129.984744] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[190129.991366] task: ee920000 ti: ee91a000 task.ti: ee91a000
[190129.996866] PC is at usb_hcd_irq+0xc/0x40
[190130.000978] LR is at __free_irq+0x1cc/0x290
[190130.005256] pc : [<c06e59fc>]    lr : [<c02a2454>]    psr: a00b0093
[190130.005256] sp : ee91bd68  ip : 00000000  fp : c066431c
[190130.016914] r10: ed718018  r9 : 600b0013  r8 : ed718018
[190130.022229] r7 : 200b0013  r6 : 0000011c  r5 : dfaf4380  r4 : eea9df00
[190130.028847] r3 : ed719018  r2 : 00000000  r1 : 00000000  r0 : 0000011c
[190130.035466] Flags: NzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
[190130.042779] Control: 10c5387d  Table: 3d4d004a  DAC: 00000051
[190130.048616] Process systemd-shutdow (pid: 1, stack limit = 0xee91a220)

In the ARM status meeting, a request was made to report issues with kernel 4.6,
so I tried upgrading to the 4.6.0-1.fc25.armv7hl build.  This did _not_ fix the
issue.  It still crashes 100% of the time when rebooting.

When this crash happens, the machine hangs, and I have to press the reset button to
get the machine running again.

This machine was working properly with Fedora 23.  Reboots always worked correctly.

	Steve

[Peter Robinson]

On Wed, May 18, 2016 at 1:52 PM, Steven Falco <stevenfalco(a)gmail.com&gt; wrote:
>  Recently I upgraded my wandboard quad to Fedora 24 beta.  Since I did
> that, I can
>  no longer reboot the machine without getting a kernel crash.  This happens 100%
>  of the time.
>
>  I filed a bug:
>
>  https://bugzilla.redhat.com/show_bug.cgi?id=1336127
>
>  You can see the whole reboot log in the above bug - here is part of the log to
>  give a flavor of what is happening.  It looks like the bug originates when the
>  USB driver for the root device is shut down.  I do not have any external USB
>  devices connected to the machine.
>
>  [190129.856042] systemd-shutdown[1]: Rebooting.
>  [190129.864389] ci_hdrc ci_hdrc.1: remove, state 4
>  [190129.868951] usb usb1: USB disconnect, device number 1
>  [190129.874989] ci_hdrc ci_hdrc.1: USB bus 1 deregistered
>  [190129.880465] Unable to handle kernel NULL pointer dereference at virtual address
> 000000e0
>  [190129.888651] pgd = ed4d0000
>  [190129.891450] [000000e0] *pgd=00000000
>  [190129.895136] Internal error: Oops: 5 [#1] SMP ARM
>  [190129.899845] Modules linked in: rpcsec_gss_krb5 snd_soc_fsl_ssi snd_soc_fsl_spdif
> imx_pcm_fiq snd_soc_fsl_asrc caam_jr snd_soc_imx_sgtl5000 dw_hdmi_ahb_audio coda
> snd_soc_sgtl5000 imx_pcm_dma snd_soc_imx_spdif snd_soc_fsl_asoc_card snd_soc_core
> imx_ipuv3_crtc snd_pcm_dmaengine snd_ac97_codec ac97_bus snd_seq v4l2_mem2mem
> snd_seq_device snd_pcm videobuf2_vmalloc dw_hdmi_imx snd_timer ci_hdrc_imx nvmem_imx_ocotp
> snd videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 ci_hdrc videobuf2_core dw_hdmi
> nvmem_core udc_core v4l2_common snd_soc_imx_audmux soundcore caam imx_ipu_v3 usbmisc_imx
> imx_sdma videodev imx2_wdt imx_thermal media etnaviv nfsd mmc_block at803x ahci_imx
> i2c_imx libahci_platform sdhci_esdhc_imx fec sdhci_pltfm ptp sdhci pps_core mmc_core
> rtc_snvs imxdrm phy_mxs_usb drm_kms_helper anatop_regulator syscopyarea sysfillrect
> sysimgblt fb_sys_fops drm
>  [190129.976821] CPU: 2 PID: 1 Comm: systemd-shutdow Not tainted 4.5.4-300.fc24.armv7hl
> #1
>  [190129.984744] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
>  [190129.991366] task: ee920000 ti: ee91a000 task.ti: ee91a000
>  [190129.996866] PC is at usb_hcd_irq+0xc/0x40
>  [190130.000978] LR is at __free_irq+0x1cc/0x290
>  [190130.005256] pc : [<c06e59fc>]    lr : [<c02a2454>]    psr: a00b0093
>  [190130.005256] sp : ee91bd68  ip : 00000000  fp : c066431c
>  [190130.016914] r10: ed718018  r9 : 600b0013  r8 : ed718018
>  [190130.022229] r7 : 200b0013  r6 : 0000011c  r5 : dfaf4380  r4 : eea9df00
>  [190130.028847] r3 : ed719018  r2 : 00000000  r1 : 00000000  r0 : 0000011c
>  [190130.035466] Flags: NzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
>  [190130.042779] Control: 10c5387d  Table: 3d4d004a  DAC: 00000051
>  [190130.048616] Process systemd-shutdow (pid: 1, stack limit = 0xee91a220)
>
>  In the ARM status meeting, a request was made to report issues with kernel 4.6,
>  so I tried upgrading to the 4.6.0-1.fc25.armv7hl build.  This did _not_ fix the
>  issue.  It still crashes 100% of the time when rebooting.
>
>  When this crash happens, the machine hangs, and I have to press the reset button to
>  get the machine running again.
>
>  This machine was working properly with Fedora 23.  Reboots always worked correctly.
This is a kernel issue, not a release issue, so if you upgrade F-23 to
the 4.5.x kernel you'd likely get the same issue. I've got it on my
todo list to look at.

[Steven Falco]

On 05/18/2016 09:01 AM, Peter Robinson wrote:
>  On Wed, May 18, 2016 at 1:52 PM, Steven Falco
> <stevenfalco(a)gmail.com&gt; wrote:
> > Recently I upgraded my wandboard quad to Fedora 24 beta.  Since I did that, I can
> > no longer reboot the machine without getting a kernel crash.  This happens 100%
> > of the time.
> >
> > I filed a bug:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1336127
> >
> > You can see the whole reboot log in the above bug - here is part of the log to
> > give a flavor of what is happening.  It looks like the bug originates when the
> > USB driver for the root device is shut down.  I do not have any external USB
> > devices connected to the machine.
> >
> > [190129.856042] systemd-shutdown[1]: Rebooting.
> > [190129.864389] ci_hdrc ci_hdrc.1: remove, state 4
> > [190129.868951] usb usb1: USB disconnect, device number 1
> > [190129.874989] ci_hdrc ci_hdrc.1: USB bus 1 deregistered
> > [190129.880465] Unable to handle kernel NULL pointer dereference at virtual address
> 000000e0
> > [190129.888651] pgd = ed4d0000
> > [190129.891450] [000000e0] *pgd=00000000
> > [190129.895136] Internal error: Oops: 5 [#1] SMP ARM
> > [190129.899845] Modules linked in: rpcsec_gss_krb5 snd_soc_fsl_ssi snd_soc_fsl_spdif
> imx_pcm_fiq snd_soc_fsl_asrc caam_jr snd_soc_imx_sgtl5000 dw_hdmi_ahb_audio coda
> snd_soc_sgtl5000 imx_pcm_dma snd_soc_imx_spdif snd_soc_fsl_asoc_card snd_soc_core
> imx_ipuv3_crtc snd_pcm_dmaengine snd_ac97_codec ac97_bus snd_seq v4l2_mem2mem
> snd_seq_device snd_pcm videobuf2_vmalloc dw_hdmi_imx snd_timer ci_hdrc_imx nvmem_imx_ocotp
> snd videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 ci_hdrc videobuf2_core dw_hdmi
> nvmem_core udc_core v4l2_common snd_soc_imx_audmux soundcore caam imx_ipu_v3 usbmisc_imx
> imx_sdma videodev imx2_wdt imx_thermal media etnaviv nfsd mmc_block at803x ahci_imx
> i2c_imx libahci_platform sdhci_esdhc_imx fec sdhci_pltfm ptp sdhci pps_core mmc_core
> rtc_snvs imxdrm phy_mxs_usb drm_kms_helper anatop_regulator syscopyarea sysfillrect
> sysimgblt fb_sys_fops drm
> > [190129.976821] CPU: 2 PID: 1 Comm: systemd-shutdow Not tainted
> 4.5.4-300.fc24.armv7hl #1
> > [190129.984744] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
> > [190129.991366] task: ee920000 ti: ee91a000 task.ti: ee91a000
> > [190129.996866] PC is at usb_hcd_irq+0xc/0x40
> > [190130.000978] LR is at __free_irq+0x1cc/0x290
> > [190130.005256] pc : [<c06e59fc>]    lr : [<c02a2454>]    psr: a00b0093
> > [190130.005256] sp : ee91bd68  ip : 00000000  fp : c066431c
> > [190130.016914] r10: ed718018  r9 : 600b0013  r8 : ed718018
> > [190130.022229] r7 : 200b0013  r6 : 0000011c  r5 : dfaf4380  r4 : eea9df00
> > [190130.028847] r3 : ed719018  r2 : 00000000  r1 : 00000000  r0 : 0000011c
> > [190130.035466] Flags: NzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
> > [190130.042779] Control: 10c5387d  Table: 3d4d004a  DAC: 00000051
> > [190130.048616] Process systemd-shutdow (pid: 1, stack limit = 0xee91a220)
> >
> > In the ARM status meeting, a request was made to report issues with kernel 4.6,
> > so I tried upgrading to the 4.6.0-1.fc25.armv7hl build.  This did _not_ fix the
> > issue.  It still crashes 100% of the time when rebooting.
> >
> > When this crash happens, the machine hangs, and I have to press the reset button to
> > get the machine running again.
> >
> > This machine was working properly with Fedora 23.  Reboots always worked correctly.
>  
>  This is a kernel issue, not a release issue, so if you upgrade F-23 to
>  the 4.5.x kernel you'd likely get the same issue. I've got it on my
>  todo list to look at.
>   
I understand.  If I can help resolve it, please let me know what you'd like me to do.

	Steve