Hi folks! :-)
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response. It takes several seconds (cca 15-20) to open an "open file" dialog in a text editor, or a new terminal window or such. The window takes 2-4 seconds to gain focus after I click on it. If I move the mouse over a typical toolbar, it takes a second or two for the particular button to get highlighted, and the highlighting "rectangle" follows the mouse like 2-3 buttons behind. Clicking on a new message in KMail takes 1-3 seconds to actually display it. Most GUI stuff becomes increasingly (and annoyingly) slow.
There is enough processor power, according to top and other diagnostic tools. Besides, compiz happens to remain completely responsive --- cube, this, that, all effects are fast and snappy. Mplayer runs flawlessly. But the rest of the GUI starts to drag. The text selection in an editor follows the mouse with 1-3 seconds delay. And such stuff...
On a freshly (re)booted system everything is fast and snappy for a couple of days, and then it gradually starts to get slower and slower. This coincides with increased usage of swap, which tends to rise from 0 to 1.2 GB in two weeks period. I suspect that swapping is partially the culprit for performance degradation, but I am unable to determine what app is using all that swap, and for what.
I would appreciate any pointers where to start looking for this swap drainage. Also, if I try to run the system without swap (disabling it manually right after boot), it starts to choke after some time, and I need to reboot it.
The hardware has 2GB RAM, 4 GB swap, core 2 duo @ 1.5 GHz.
I use the machine for usual desktop activities. Typically I have several apps running non-stop: firefox (3-4 tabs), kmail, ktorrent, kile, couple of konsole instances, two skype instances, cairo-dock, xmms and okular. The desktop is 64bit F12, KDE, running compiz/emerald. I have several usual plasmoids on the desktop, nothing too fancy (clock, graphs for cpu, temperature, battery, network and RAM usage).
Occasionally I start other stuff (krusader, quake3, Wolfram's Mathematica, other infrequent things like photo editing stuff etc.) but I shut them down after usage. I notice the slowdown even if I don't use any of that.
The symptoms appear like something is leaking memory --- slowly (noticable only after two weeks of continuous running), but cumulatively. This triggers gradual swap usage, which then gradually decreases performance and responsiveness of the whole desktop.
I tried shutting down all desktop apps, and this releases some swap (but not all?!), but the system remains in a bad shape, which is visible if I (re)start any app again.
Things like mplayer run flawlessly, but it takes cca 15-20 seconds to start the Konsole from which I can invoke mplayer.
Once I left the system running unattended for a month (I wasn't at home), and I accessed it only via ssh a couple of times (without problems). When I got back, the desktop was so slow and unresponsive that I couldn't even wait for it to do a regular shutdown/restart, and instead I pulled the plug and rebooted it fresh.
When I try to shut down a multi-tab app (firefox, kile, konsole), it takes so much time to close itself that I get a dialog saying that the app is not responding (and offering to terminate it). If I just wait, eventually both the dialog and the app close themselves. It can take them from 5 seconds to half a minute do close, depending on the shape of the rest of the desktop.
I'm at a loss where and how to look for memory and performance drain. My current uptime is
$ uptime 02:04:55 up 14 days, 4:54, 6 users, load average: 0.62, 0.55, 0.39
and I can feel the slowdown quite easily. I'd be happy to hear any advice on how to troubleshoot this, before I get pissed off and reboot the system again. I'll also gladly provide any additional info.
Thanks, :-) Marko
On Sun, Oct 24, 2010 at 02:38:23AM +0100, Marko Vojinovic wrote:
Hi folks! :-)
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response. It takes several seconds (cca 15-20) to open an "open file" dialog in a text editor, or a new terminal window or such.
there was a lengthy thread in a redhat list earlier this month about someone with several servers, SOME of which had similar-sounding problems. I refer you to one in which the OP describes how he solved it, in case it may be helpful to you:
https://www.redhat.com/archives/redhat-list/2010-October/msg00055.html
Good luck!
The window takes 2-4 seconds to gain focus after I click on it. If I move the mouse over a typical toolbar, it takes a second or two for the particular button to get highlighted, and the highlighting "rectangle" follows the mouse like 2-3 buttons behind. Clicking on a new message in KMail takes 1-3 seconds to actually display it. Most GUI stuff becomes increasingly (and annoyingly) slow.
There is enough processor power, according to top and other diagnostic tools. Besides, compiz happens to remain completely responsive --- cube, this, that, all effects are fast and snappy. Mplayer runs flawlessly. But the rest of the GUI starts to drag. The text selection in an editor follows the mouse with 1-3 seconds delay. And such stuff...
On a freshly (re)booted system everything is fast and snappy for a couple of days, and then it gradually starts to get slower and slower. This coincides with increased usage of swap, which tends to rise from 0 to 1.2 GB in two weeks period. I suspect that swapping is partially the culprit for performance degradation, but I am unable to determine what app is using all that swap, and for what.
I would appreciate any pointers where to start looking for this swap drainage. Also, if I try to run the system without swap (disabling it manually right after boot), it starts to choke after some time, and I need to reboot it.
The hardware has 2GB RAM, 4 GB swap, core 2 duo @ 1.5 GHz.
I use the machine for usual desktop activities. Typically I have several apps running non-stop: firefox (3-4 tabs), kmail, ktorrent, kile, couple of konsole instances, two skype instances, cairo-dock, xmms and okular. The desktop is 64bit F12, KDE, running compiz/emerald. I have several usual plasmoids on the desktop, nothing too fancy (clock, graphs for cpu, temperature, battery, network and RAM usage).
Occasionally I start other stuff (krusader, quake3, Wolfram's Mathematica, other infrequent things like photo editing stuff etc.) but I shut them down after usage. I notice the slowdown even if I don't use any of that.
The symptoms appear like something is leaking memory --- slowly (noticable only after two weeks of continuous running), but cumulatively. This triggers gradual swap usage, which then gradually decreases performance and responsiveness of the whole desktop.
I tried shutting down all desktop apps, and this releases some swap (but not all?!), but the system remains in a bad shape, which is visible if I (re)start any app again.
Things like mplayer run flawlessly, but it takes cca 15-20 seconds to start the Konsole from which I can invoke mplayer.
Once I left the system running unattended for a month (I wasn't at home), and I accessed it only via ssh a couple of times (without problems). When I got back, the desktop was so slow and unresponsive that I couldn't even wait for it to do a regular shutdown/restart, and instead I pulled the plug and rebooted it fresh.
When I try to shut down a multi-tab app (firefox, kile, konsole), it takes so much time to close itself that I get a dialog saying that the app is not responding (and offering to terminate it). If I just wait, eventually both the dialog and the app close themselves. It can take them from 5 seconds to half a minute do close, depending on the shape of the rest of the desktop.
I'm at a loss where and how to look for memory and performance drain. My current uptime is
$ uptime 02:04:55 up 14 days, 4:54, 6 users, load average: 0.62, 0.55, 0.39
and I can feel the slowdown quite easily. I'd be happy to hear any advice on how to troubleshoot this, before I get pissed off and reboot the system again. I'll also gladly provide any additional info.
Thanks, :-) Marko
-- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
On Sunday, October 24, 2010 04:06:30 fred smith wrote:
On Sun, Oct 24, 2010 at 02:38:23AM +0100, Marko Vojinovic wrote:
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response. It takes several seconds (cca 15-20) to open an "open file" dialog in a text editor, or a new terminal window or such.
there was a lengthy thread in a redhat list earlier this month about someone with several servers, SOME of which had similar-sounding problems. I refer you to one in which the OP describes how he solved it, in case it may be helpful to you:
https://www.redhat.com/archives/redhat-list/2010-October/msg00055.html
Thanks, I looked at the thread, but it seems to be a hardware problem in their case. I'm not experiencing their symptoms with the hardware clock.
Thanks anyway! :-) Marko
On Sun, 24 Oct 2010 02:38:23 +0100 Marko Vojinovic vvmarko@gmail.com wrote:
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response.
<snip>
The symptoms appear like something is leaking memory --- slowly (noticable only after two weeks of continuous running), but cumulatively.
This probably isn't much help, but what happens if you log out when it starts to slow, stop X, and then start X? I mean, go to runlevel 3, then back to runlevel 5. If everything is OK again, it tells you that the problem is in X. If it isn't, the problem is in the system part, the OS.
On Sunday, October 24, 2010 04:18:02 stan wrote:
On Sun, 24 Oct 2010 02:38:23 +0100
Marko Vojinovic vvmarko@gmail.com wrote:
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response.
<snip>
The symptoms appear like something is leaking memory --- slowly (noticable only after two weeks of continuous running), but cumulatively.
This probably isn't much help, but what happens if you log out when it starts to slow, stop X, and then start X? I mean, go to runlevel 3, then back to runlevel 5. If everything is OK again, it tells you that the problem is in X. If it isn't, the problem is in the system part, the OS.
Just tried it, logged out and logged back in. Didn't go to runlevel 3, X gets restarted with only logging out and in, AFAIK.
Anyway, you seem to be right! Restarting X purged most of the swap, from 1.3 GB it went down to 31.4 MB. And the system regained responsiveness.
So apparently something inside X is leaking memory. But X has many components --- compiz, emerald, KDE, Xorg, intel driver, etc., so any idea how do I diagnose this further?
I mean, I can restore the system either by reboot or logging out/in, but I'd like to know what is the actual cause of this end eliminate *that*. Hopefully I would like to correct whatever is wrong so that I don't need to get into the situation of having to reboot or relogin myself again.
I am open to suggestions on how to proceed in troubleshooting this further. If anything, my system will become slow again in a week or two... ;-)
Thanks for the help, any further advice appreciated!
Best, :-) Marko
Marko Vojinovic wrote:
On Sunday, October 24, 2010 04:18:02 stan wrote:
On Sun, 24 Oct 2010 02:38:23 +0100
Marko Vojinovic vvmarko@gmail.com wrote:
I am experiencing a gradual performance drop problem --- having a machine running 24/7, after some time (say, two weeks) the system becomes increasingly slow, in terms of desktop response.
<snip>
The symptoms appear like something is leaking memory --- slowly (noticable only after two weeks of continuous running), but cumulatively.
This probably isn't much help, but what happens if you log out when it starts to slow, stop X, and then start X? I mean, go to runlevel 3, then back to runlevel 5. If everything is OK again, it tells you that the problem is in X. If it isn't, the problem is in the system part, the OS.
Just tried it, logged out and logged back in. Didn't go to runlevel 3, X gets restarted with only logging out and in, AFAIK.
Anyway, you seem to be right! Restarting X purged most of the swap, from 1.3 GB it went down to 31.4 MB. And the system regained responsiveness.
So apparently something inside X is leaking memory. But X has many components --- compiz, emerald, KDE, Xorg, intel driver, etc., so any idea how do I diagnose this further?
You can not be sure that the memory is used by X; it could be one of the graphical apps running in your X session. By restarting X you restarted all of them too.
Keep an eye on the output of this command and see if something increases suspiciously after a day or two.
ps ww -e k -rss o pid,pmem,command,rss,size,vsz|head -20
I have a similar problem; my session has been up for more than 130 days and it is getting slower and slower, especially on desktop switching (but still usable). It looks like X is leaking/fragmenting/doingsomethingwrong, but my configuration is very peculiar (F10 with KDE3 and nvidia binary driver, many many applications running), so I have no hope to track it down and have it solved.
On Sun, 24 Oct 2010 05:54:56 +0100 Marko Vojinovic vvmarko@gmail.com wrote:
Just tried it, logged out and logged back in. Didn't go to runlevel 3, X gets restarted with only logging out and in, AFAIK.
You're probably right. I think of X as this huge complex beast that sits between the OS and me, and think of it as a black box. :-)
Anyway, you seem to be right! Restarting X purged most of the swap, from 1.3 GB it went down to 31.4 MB. And the system regained responsiveness.
Excellent, easier than rebooting, and eliminates the underlying OS as a problem.
So apparently something inside X is leaking memory. But X has many components --- compiz, emerald, KDE, Xorg, intel driver, etc., so any idea how do I diagnose this further?
I think this is the more likely place too, but as Roberto said, it could be an application within X leaking memory also.
I mean, I can restore the system either by reboot or logging out/in, but I'd like to know what is the actual cause of this end eliminate *that*. Hopefully I would like to correct whatever is wrong so that I don't need to get into the situation of having to reboot or relogin myself again.
I am open to suggestions on how to proceed in troubleshooting this further. If anything, my system will become slow again in a week or two... ;-)
I haven't chased anything like this because I usually shut down overnight. I used to leave my systems up for months (I never noticed any slowing so my use case must be different), but I read an article that about 10% of the power used in the US is for electronics just idling so they are instantly available, and decided to do my part by going down when not using for any length of time. Not advocating, just explaining.
Because it takes so long to manifest, it is probably something running constantly, but with a very small leak. I think there are tools that allow snapshots of jobs on the system and their memory footprint. Make a script and schedule that job to run every 12 hours or so, taking a snapshot of every active job on the system. The key thing here is that the system thinks that memory is being used, even though the job that is using it won't be aware it is using it because it is leaking. So you can track it through the view of the memory on the system.
You can run the command Roberto suggested in the shell script, appending it to a file, and run a parse script on that file, having it flag any growing jobs. Should be easy to write in one of the scripting languages. Or schedule the parse program as part of the shell script and have *it* write a file that contains a meaningful summary of the output.
There is probably a tool designed for this specific purpose, though I'm not aware of it.
On Sunday, October 24, 2010 12:36:52 stan wrote:
There is probably a tool designed for this specific purpose, though I'm not aware of it.
On Saturday, October 23, 2010 21:38:23 Marko Vojinovic wrote:
I'm at a loss where and how to look for memory and performance drain.
Just run top(1) to see what's eating memory (if that's the problem). Typing 'M' will sort processes by memory usage.
On 10/23/2010 09:54 PM, Marko Vojinovic wrote:
Anyway, you seem to be right! Restarting X purged most of the swap, from 1.3 GB it went down to 31.4 MB. And the system regained responsiveness.
Whatever is using swap is probably the application using the most RAM. Take a look at 'System Monitor' or the command line tool 'top'. Either can sort processes by memory. Close programs one at a time until swap use drops to find the culprit.
AFAIK, you can't check swap use directly, so that's your only path. However, if you have enough "free" memory (which is "free" + "buffers" + "cache"), you can force everything in swap back to ram. Open a terminal, "su -" to log in as the root account. Enter "swapoff -a" to disable swap and force its contents to RAM. Once that happens, you can use 'System Monitor' or 'top' to see what applications are using the most RAM. If you're looking at 'top', pay attention to the "RES" column.
Marko Vojinovic <vvmarko <at> gmail.com> writes:
...
Hi,
$ yum info sysstat
http://sebastien.godard.pagesperso-orange.fr/ Features. Documentation. Tutorials. FAQ
$ man sar
Btw, because you used ssh service, have you run rkhunter and chkrootkit, both available from Fedora repo ?
JB
-
Fred Smith -
Garry Williams -
Gordon Messmer -
JB -
Marko Vojinovic -
Roberto Ragusa -
stan