Updated to 19; now kernel panic on boot. No log

[sean darcy]

Updated to F19, kernel-3.12.5-200.fc19.x86_64.

On boot I get a kernel panic around [2.2]. The trace that's on the 
screen is about smp, timers, hpet. Can't read quickly enough to see what 
actually happened.

/var/log/messages has no log of the kernel boot. Shouldn't there be one?

rsyslogd.conf seems to be set to put it all in syslog:

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

I can boot with the F18 kernel. syslog there shows logging started right 
at boot:

Dec 29 18:04:06 new-gateway rsyslogd: [origin software="rsyslogd" 
swVersion="7.2.6" x-pid="424"
x-info="http://www.rsyslog.com"] start
Dec 29 18:04:06 new-gateway systemd-cgroups-agent[226]: Failed to get 
D-Bus connection: Failed to connect to socket 
/org/freedesktop/systemd1/private: Conn
ection refused
Dec 29 18:04:06 new-gateway systemd[1]: systemd 204 running in system 
mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP 
+GCRYPT +ACL +XZ)
Dec 29 18:04:06 new-gateway systemd[1]: Set hostname to 
<new-gateway.11hidden>.
Dec 29 18:04:06 new-gateway systemd[1]: Starting LVM2 metadata daemon...
Dec 29 18:04:06 new-gateway systemd-sysctl[248]: Overwriting earlier 
assignment of kernel/sysrq in file '/etc/sysctl.conf'.
Dec 29 18:04:06 new-gateway systemd[1]: Started Apply Kernel Variables.
Dec 29 18:04:06 new-gateway systemd[1]: Started Setup Virtual Console.
Dec 29 18:04:06 new-gateway systemd[1]: Started Create static device 
nodes in /dev.
Dec 29 18:04:06 new-gateway systemd[1]: Started LVM2 metadata daemon.
Dec 29 18:04:06 new-gateway systemd[1]: Starting udev Kernel Device 
Manager...
Dec 29 18:04:06 new-gateway systemd[1]: Mounted POSIX Message Queue File 
System.
Dec 29 18:04:06 new-gateway kernel: [    0.000000] Initializing cgroup 
subsys cpuset
Dec 29 18:04:06 new-gateway kernel: [    0.000000] Initializing cgroup 
subsys cpu
Dec 29 18:04:06 new-gateway kernel: [    0.000000] Initializing cgroup 
subsys cpuacct
Dec 29 18:04:06 new-gateway kernel: [    0.000000] Linux version 
3.11.10-100.fc18.x86_64 (mockbuild@bkernel02) (gcc version 4.7.2 
20121109 (Red Hat 4.7.2-8
) (GCC) ) #1 SMP Mon Dec 2 20:28:38 UTC 2013


Or does the error about the D-Bus connection cause logging to fail?

Any help appreciated.

sean

[Chris Murphy]

On Dec 29, 2013, at 5:11 PM, Sean Darcy <seandarcy2(a)gmail.com&gt; wrote:

>  Updated to F19, kernel-3.12.5-200.fc19.x86_64.
>  
>  On boot I get a kernel panic around [2.2]. The trace that's on the screen is about
> smp, timers, hpet. Can't read quickly enough to see what actually happened. 
The kernel panic halts the startup process, or it continues to startup? If you have shell
access, use 'ip addr' to determine if you have network access. If not insert and
mount a  USB stick. Then use 'journalctl -b -o short-monotonic >
/mnt/journal.log' to write out the journal for the current boot to the USB stick. Then
you can reboot to the working kernel and post it, or if you have network access you can
use 'fpaste <filename>', which if successful will return a URL which you
should note. Then post that.

>  
>  /var/log/messages has no log of the kernel boot. Shouldn't there be one? 
If the kp happens early and halts startup, then no, because rootfs is still mounted
read-only. 


>  I can boot with the F18 kernel. syslog there shows logging started
> right at boot: 
Logging starts in any case right at boot, but it can't be made persistent until the
volume the log is to be written to is mounted rw. And that happens well after the kernel
and initramfs are running, after udev initalizes, and after several dracut stages have
executed, systemd has spawned a whole bunch of start jobs, including fsck based on fstab,
and then after fsck completes you get a rw mounted /sysroot.


Chris Murphy

[sean darcy]

On 12/30/2013 11:38 AM, Chris Murphy wrote:
>  On Dec 29, 2013, at 5:11 PM, Sean Darcy <seandarcy2(a)gmail.com&gt; wrote:
>
> > Updated to F19, kernel-3.12.5-200.fc19.x86_64.
> >
> > On boot I get a kernel panic around [2.2]. The trace that's on the screen is
> about smp, timers, hpet. Can't read quickly enough to see what actually happened.
>
>  The kernel panic halts the startup process, or it continues to startup? If you have shell
> access, use 'ip addr' to determine if you have network access. If not insert and
> mount a  USB stick. Then use 'journalctl -b -o short-monotonic >
> /mnt/journal.log' to write out the journal for the current boot to the USB stick. Then
> you can reboot to the working kernel and post it, or if you have network access you can
> use 'fpaste <filename>', which if successful will return a URL which you
> should note. Then post that.
>
> >
> > /var/log/messages has no log of the kernel boot. Shouldn't there be one?
>
>  If the kp happens early and halts startup, then no, because rootfs is still mounted
> read-only.
>
>
> > I can boot with the F18 kernel. syslog there shows logging started right at boot:
>
>  Logging starts in any case right at boot, but it can't be made persistent until the
> volume the log is to be written to is mounted rw. And that happens well after the kernel
> and initramfs are running, after udev initalizes, and after several dracut stages have
> executed, systemd has spawned a whole bunch of start jobs, including fsck based on fstab,
> and then after fsck completes you get a rw mounted /sysroot.
>
>
>  Chris Murphy
>  
Sadly, it's a real kernel panic. Everything stops. I can't even reboot. 
It's a good thing there's a power cord!

I'll copy what's on the screen, and submit it as a bug report. Maybe 
someone smarter than I am can figure out what actually caused it.

Looking at the FC18 kernel, it seems to happen about when the serial 
driver is initialized. Is there a way to disable the serial driver? I 
tried serial=0 on the kernel command line. That didn't seem to work.

sean

[Chris Murphy]

On Dec 30, 2013, at 3:41 PM, Sean Darcy <seandarcy2(a)gmail.com&gt; wrote:
>  
>  Sadly, it's a real kernel panic. Everything stops. I can't even reboot. It's
> a good thing there's a power cord!
>  
>  I'll copy what's on the screen, and submit it as a bug report. Maybe someone
> smarter than I am can figure out what actually caused it. 
It's better and easier to take a photo, cell phone JPG is fine. Also include the
results from lspci -nnvv (using the working kernel).

>  
>  Looking at the FC18 kernel, it seems to happen about when the serial driver is
> initialized. Is there a way to disable the serial driver? I tried serial=0 on the kernel
> command line. That didn't seem to work. 
With the F18 kernel successfully booted, use lspci -nnvv to find the kernel being loaded
for serial support and then blacklist it on kernel command line when attempting to boot
the F19 kernel that's giving you problems.


Chris Murphy

[sean darcy]

On 12/30/2013 06:39 PM, Chris Murphy wrote:
>  On Dec 30, 2013, at 3:41 PM, Sean Darcy <seandarcy2(a)gmail.com&gt; wrote:
> >
> > Sadly, it's a real kernel panic. Everything stops. I can't even reboot.
> It's a good thing there's a power cord!
> >
> > I'll copy what's on the screen, and submit it as a bug report. Maybe someone
> smarter than I am can figure out what actually caused it.
>
>  It's better and easier to take a photo, cell phone JPG is fine. Also include the
> results from lspci -nnvv (using the working kernel).
>  
Great idea.

> >
> > Looking at the FC18 kernel, it seems to happen about when the serial driver is
> initialized. Is there a way to disable the serial driver? I tried serial=0 on the kernel
> command line. That didn't seem to work.
>
>  With the F18 kernel successfully booted, use lspci -nnvv to find the kernel being loaded
> for serial support and then blacklist it on kernel command line when attempting to boot
> the F19 kernel that's giving you problems.
>
>  
[root@new-gateway ~]# lspci -nnvv | grep -C 2 ial
			[PN] Part number: BCM95787m
			[EC] Engineering changes: 106679-15
			[SN] Serial number: 0123456789
			[MN] Manufacture ID: 31 34 65 34
			[RV] Reserved: checksum good, 29 byte(s) reserved
--
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=01
			Status:	NegoPending- InProgress-
	Capabilities: [160 v1] Device Serial Number 00-1d-72-ff-fe-05-4a-68
	Kernel driver in use: tg3


Not sure what to do with this.


>  Chris Murphy
>

[Chris Murphy]

On Dec 30, 2013, at 8:26 PM, Sean Darcy <seandarcy2(a)gmail.com&gt; wrote:

>  	Kernel driver in use: tg3
>  
>  
>  Not sure what to do with this. 
Reboot, at grub menu highlight the f20 kernel option, hit e to edit, find the linux
command line, remove quiet rhgb, add rd.blacklist=tg3 and then F10 to boot.

Chris Murphy