8 a.m.
This bugzilla is absurd:
https://bugzilla.redhat.com/show_bug.cgi?id=1043212
I've just added a new entry to it because last night's
disruption corrupted the binary format journal files
and sent rsyslogd into a 100% cpu loop even after
a reboot.
It seems highly likely that all these bugs are also
symptoms of the same problem:
https://bugzilla.redhat.com/show_bug.cgi?id=1057811
https://bugzilla.redhat.com/show_bug.cgi?id=1057618
https://bugzilla.redhat.com/show_bug.cgi?id=1057614
and maybe even this one as well (if it turns out
it only happens if the system has been up across
a run of cron.daily):
https://bugzilla.redhat.com/show_bug.cgi?id=1023820
It seems pretty clear no testing of systemd has been
done on systems that are up 24/7 and need to survive
a run of cron.daily :-(.
Enterprise customers are gonna love RHEL 7...
12:30 p.m.
On Tue, 28 Jan 2014 11:21:15 -0700
Pete Travis wrote:
This is concerning. It should not be possible to enable a masked
service.
Are you running a customized version of systemd? If not, have you run a
`rpm -V systemd` or similar, tested memory and disks, etc?
I think the rpm scripts for those services simply went to a
lot of trouble to force the services to run. They may not
have even had systemd units when I created the script, and
I was just relying on systemd to forward the request to
chkconfig.
12:21 p.m.
On Jan 28, 2014 11:13 AM, "Tom Horsley" <horsley1953(a)gmail.com> wrote:
On Tue, 28 Jan 2014 10:54:33 -0700
Pete Travis wrote:
> I read in this bug report that you are using a bespoke yum plugin to
> disable KSM and ksmtuned. Logically, the more your local configuration
> differs from default, the more difficult it will be to troubleshoot
> affected software. Have you considered masking unwanted services, ie
> `systemctl mask KSM.service` ?
Disable, mask, it wouldn't have made any difference. Whenever
a yum update dragged in new versions of those services, it
always enabled them regardless of the administrator requested
setting, thus the big hammer to re-disable them every time
yum ran.
This is concerning. It should not be possible to enable a masked service.
Are you running a customized version of systemd? If not, have you run a
`rpm -V systemd` or similar, tested memory and disks, etc?
--Pete
12:13 p.m.
On Tue, 28 Jan 2014 10:54:33 -0700
Pete Travis wrote:
I read in this bug report that you are using a bespoke yum plugin to
disable KSM and ksmtuned. Logically, the more your local configuration
differs from default, the more difficult it will be to troubleshoot
affected software. Have you considered masking unwanted services, ie
`systemctl mask KSM.service` ?
Disable, mask, it wouldn't have made any difference. Whenever
a yum update dragged in new versions of those services, it
always enabled them regardless of the administrator requested
setting, thus the big hammer to re-disable them every time
yum ran.
It appears, however, that those services do not even exist
in fedora 20. (Perhaps the whole KSM thing was dropped
as a bad idea? Maybe they still lurk around under a different
name? I have no idea).
The important thing for the bugzilla is that running two
"systemctl disable" commands in a row is one of the
many triggers to make systemd go funny in the head
and screw up lots of stuff (even does it disabling services
that don't exist :-).
11:54 a.m.
On Jan 25, 2014 7:01 AM, "Tom Horsley" <horsley1953(a)gmail.com> wrote:
This bugzilla is absurd:
https://bugzilla.redhat.com/show_bug.cgi?id=1043212
....
I read in this bug report that you are using a bespoke yum plugin to
disable KSM and ksmtuned. Logically, the more your local configuration
differs from default, the more difficult it will be to troubleshoot
affected software. Have you considered masking unwanted services, ie
`systemctl mask KSM.service` ?
--Pete
3:38 p.m.
On Sat, 25 Jan 2014 12:33:40 -0800
Joe Zeff wrote:
On 01/25/2014 12:04 PM, Ed Greshko wrote:
> On 01/25/14 22:00, Tom Horsley wrote:
>> It seems pretty clear no testing of systemd has been
>> done on systems that are up 24/7 and need to survive
>> a run of cron.daily
>
> My system runs 24/7. I've got 0 problems.
>
Yup! Running projects for WCG and Einstein@home if nothing else.
My current theory is a race condition. My system at work
is near identical as far as installed software, and it hasn't
had anything like the problems my home system has.
But at work, it is an old slow DELL with an old slow
disk. At home it is a speedy new cpu with an SSD system
disk. Everything is near instantaneous at home, including
encountering all these problems :-).
2:33 p.m.
On 01/25/2014 12:04 PM, Ed Greshko wrote:
On 01/25/14 22:00, Tom Horsley wrote:
> It seems pretty clear no testing of systemd has been
> done on systems that are up 24/7 and need to survive
> a run of cron.daily
My system runs 24/7. I've got 0 problems.
Yup! Running projects for WCG and Einstein@home if nothing else.
1:16 p.m.
On Sat, 25 Jan 2014 09:00:49 -0500
Tom Horsley <horsley1953(a)gmail.com> wrote:
This bugzilla is absurd:
https://bugzilla.redhat.com/show_bug.cgi?id=1043212
I've just added a new entry to it because last night's
disruption corrupted the binary format journal files
and sent rsyslogd into a 100% cpu loop even after
a reboot.
:(
I don't find it absurd... I think it's just difficult to pin down.
I have fedora on all my servers here at home and have never seen this
issue. ;(
I do make sure prelink is not installed on any of my boxes, and it
sounds like that might have been a vector for some people seeing it.
kevin
12:26 p.m.
> Enterprise customers are gonna love RHEL 7...
<SARCASM>
But Tom, surely most desktop users of Fedora don't leave their systems up
24x7. The easy solution is to remove cron.d daily. After all, the scripts
it runs will probably just try to e-mail the results and there's no MTA
since desktop users don't need that either.
</SARCASM>
And I've just added this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1057883
Apparently whatever screws up the system during cron.daily
can also happen during a yum update.
I should just reboot my system after every command I run I guess.
12:20 p.m.
On Sat, Jan 25, 2014 at 06:13:18PM +0000, David G. Miller wrote:
Tom Horsley <horsley1953 <at> gmail.com> writes:
>
> This bugzilla is absurd:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1043212
>
> I've just added a new entry to it because last night's
> disruption corrupted the binary format journal files
> and sent rsyslogd into a 100% cpu loop even after
> a reboot.
<SNIP>
> It seems pretty clear no testing of systemd has been
> done on systems that are up 24/7 and need to survive
> a run of cron.daily .
>
> Enterprise customers are gonna love RHEL 7...
<SARCASM>
But Tom, surely most desktop users of Fedora don't leave their systems up
24x7. The easy solution is to remove cron.d daily. After all, the scripts
it runs will probably just try to e-mail the results and there's no MTA
since desktop users don't need that either.
</SARCASM>
I saw the sarcasm tags, so this reply is prolly unneeded. but,...
While I don't run Fedora on my desktop (I do run it on a couple of
portables that are only on when needed), but rather Centos, I do leave
my desktop up 24x7, since it is the mail server for a private domain.
(not to mention, it also grinds away at Folding at Home 24x7.)
--
-------------------------------------------------------------------------------
.---- Fred Smith /
( /__ ,__. __ __ / __ : /
/ / / /__) / / /__) .+' Home: fredex(a)fcshome.stoneham.ma.us
/ / (__ (___ (__(_ (___ / :__ 781-438-5471
-------------------------------- Jude 1:24,25 ---------------------------------
12:13 p.m.
Tom Horsley <horsley1953 <at> gmail.com> writes:
This bugzilla is absurd:
https://bugzilla.redhat.com/show_bug.cgi?id=1043212
I've just added a new entry to it because last night's
disruption corrupted the binary format journal files
and sent rsyslogd into a 100% cpu loop even after
a reboot.
<SNIP>
It seems pretty clear no testing of systemd has been
done on systems that are up 24/7 and need to survive
a run of cron.daily .
Enterprise customers are gonna love RHEL 7...
<SARCASM>
But Tom, surely most desktop users of Fedora don't leave their systems up
24x7. The easy solution is to remove cron.d daily. After all, the scripts
it runs will probably just try to e-mail the results and there's no MTA
since desktop users don't need that either.
</SARCASM>
Cheers,
Dave
3741
days inactive
3744
days old
11 comments
7 participants
participants (7)
-
David G. Miller (aka DaveAtFraud) -
Ed Greshko -
Fred Smith -
Joe Zeff -
Kevin Fenzi -
Pete Travis -
Tom Horsley