High Volume of Syslog Messages on Fedora 16 Dom0

[Tim Flink]

I just upgraded an older Opteron machine that I've been using as Xen
Dom0 from RHEL5 to Fedora 16.

When I boot with Xen, my syslog is full of messages (several per
second) like the following:

kernel: [    1.226209] powernow-k8: fid trans failed, fid 0x2, curr 0x0
kernel: [    1.226271] powernow-k8: transition frequency failed

Things seem to be working (I can use xm list) but I would rather not
have my system log be quite that noisy. Unfortunately, I'm quite sure
where to go from here to debug and fix the issue; any suggestions?

Tim

Fedora 16
3.1.0-0.rc8.git0.1.fc16.x86_64
xen-4.1.1-6
2x Dual Core AMD Opteron(tm) Processor 865 HE

[M A Young]

On Fri, 7 Oct 2011, Tim Flink wrote:
> I just upgraded an older Opteron machine that I've been using as Xen
> Dom0 from RHEL5 to Fedora 16.
>
> When I boot with Xen, my syslog is full of messages (several per
> second) like the following:
>
> kernel: [    1.226209] powernow-k8: fid trans failed, fid 0x2, curr 0x0
> kernel: [    1.226271] powernow-k8: transition frequency failed
>
> Things seem to be working (I can use xm list) but I would rather not
> have my system log be quite that noisy. Unfortunately, I'm quite sure
> where to go from here to debug and fix the issue; any suggestions?

There is a bug on this issue at
https://bugzilla.redhat.com/show_bug.cgi?id=739159

Michael Young

[Tim Flink]

On Fri, 7 Oct 2011 14:11:04 +0100 (BST)
M A Young  wrote:
> On Fri, 7 Oct 2011, Tim Flink wrote:
>
> There is a bug on this issue at
> https://bugzilla.redhat.com/show_bug.cgi?id=739159

Thanks for the pointer. For some reason, I didn't find that when I
searched through bugzilla earlier.

The kernel maintainers claim that isn't a kernel bug and that it's a
Xen issue. I have filed the bug upstream as:

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1789

Thanks,

Tim

[Marko Ristola]

Hi.

I found out that disabling AMD Cool and Quiet from BIOS
will remove the flood of these messages under Xen Dom0:
If BIOS support for switching CPU frequencies is turned off,
Linux kernel won't try to alter the frequency.

My computer has Fedora 16, basic desktop computer.
The hardware is old, and thus I haven't bothered to request features
for this old hardware. Only problem with the hardware is, that
XEN Hypervisor scheduler doesn't use HPET. I think hypervisor scheduler
does busy looping, instead of using HALT instruction.
This was a problem on last hot Summer, but now it is cold Autum.

Without Xen Hypervisor on Linux kernel, I use "hpet=force".
Linux Kernel finds out by the knowledge of old hardware,
that HPET is propably there, even though ACPI doesn't advertice it.

"hpet=force" was on XEN Hypervisor a few years ago, but the code has been removed.
I don't know if it is coming back.

I can test old hardware so that XEN hypervisor and DOM0 messages go into a serial cable.
Not every day though.

Regards,
Marko Ristola

[Konrad Rzeszutek Wilk]

On Fri, Oct 07, 2011 at 10:31:37PM +0300, Marko Ristola wrote:
> Hi.
>
> I found out that disabling AMD Cool and Quiet from BIOS
> will remove the flood of these messages under Xen Dom0:
> If BIOS support for switching CPU frequencies is turned off,
> Linux kernel won't try to alter the frequency.
>
> My computer has Fedora 16, basic desktop computer.
> The hardware is old, and thus I haven't bothered to request features
> for this old hardware. Only problem with the hardware is, that
> XEN Hypervisor scheduler doesn't use HPET. I think hypervisor scheduler
> does busy looping, instead of using HALT instruction.

The default operation is to use 'hlt' (from arch/x86/domain.c in
Xen 4.1.1 hypervisor):

static void default_idle(void)
{
    local_irq_disable();
    if ( cpu_is_haltable(smp_processor_id()) )
        safe_halt();
    else 
        local_irq_enable();
}

HPET does not have anything to do with HALT. HPET is used to
figure out the time.
> This was a problem on last hot Summer, but now it is cold Autum.
>
> Without Xen Hypervisor on Linux kernel, I use "hpet=force".
> Linux Kernel finds out by the knowledge of old hardware,
> that HPET is propably there, even though ACPI doesn't advertice it.
>
> "hpet=force" was on XEN Hypervisor a few years ago, but the code has been removed.
> I don't know if it is coming back.

Unless somebody posts a patch - then no.
> I can test old hardware so that XEN hypervisor and DOM0 messages go into a serial cable.

That is appreciated. The Xen ACPI cpufreq patches that would push up the _P states to
the hypervisor are not yet ready - but when they are ready it would be very good to try
it out on your hardware.

Note, the ACPI cpufreq patches enable the hypervisor to use different
methods that just doing 'hlt'. They can use 'mwait' or some other form
of putting the CPU to sleep.. and naturally also change the frequency and those
fancy things.

[Marko Ristola]

Hi Konrad.

Thank you for your time, and sorry about the confusion
that I might have caused with Xen scheduler and cpuidle.
I've misunderstood them completely.

On 10/13/2011 08:42 PM, Konrad Rzeszutek Wilk wrote:
> On Fri, Oct 07, 2011 at 10:31:37PM +0300, Marko Ristola wrote:
> > for this old hardware. Only problem with the hardware is, that
> > XEN Hypervisor scheduler doesn't use HPET. I think hypervisor scheduler
> > does busy looping, instead of using HALT instruction.
> The default operation is to use 'hlt' (from arch/x86/domain.c in
> Xen 4.1.1 hypervisor):
>
> static void default_idle(void)
> {
>      local_irq_disable();
>      if ( cpu_is_haltable(smp_processor_id()) )
>          safe_halt();
>      else
>          local_irq_enable();
> }
>
> HPET does not have anything to do with HALT. HPET is used to
> figure out the time.

Oops. I'm very sorry.

I have following lines:
"(XEN) Platform timer is 3.579MHz ACPI PM Timer".
"(XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'."
I have no problem with these now, so let's go on.
> > "hpet=force" was on XEN Hypervisor a few years ago, but the code has been removed.
> > I don't know if it is coming back.
>
> Unless somebody posts a patch - then no.

Is it okay to grab the code bits from Linux kernel and create a patch?
I need equal semantics from the Kernel, to be able to verify that the patch works.
> > I can test old hardware so that XEN hypervisor and DOM0 messages go into a serial cable.
>
> That is appreciated. The Xen ACPI cpufreq patches that would push up the _P states to
> the hypervisor are not yet ready - but when they are ready it would be very good to try
> it out on your hardware.
>
> Note, the ACPI cpufreq patches enable the hypervisor to use different
> methods that just doing 'hlt'. They can use 'mwait' or some other form
> of putting the CPU to sleep.. and naturally also change the frequency and those
> fancy things.

Sounds very good: power saving features will just work.

Regards,
Marko Ristola