I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects. As a developer, I pretty much have netbeans open all day and am forever tailing log files, so I hit this problem pretty much straight away.
After some digging around I found a script to list the current count of watchers per process. The half-dozen worst offenders were:-
INOTIFY WATCHER COUNT PID CMD ---------------------------------------- 4903 4620 /data/Applications/java/jdk-15/bin/java -Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans 3573 3032 /usr/libexec/tracker-miner-fs-3 64 2022 /usr/lib/systemd/systemd --user 47 2663 /usr/libexec/gsd-xsettings 27 34846 /usr/share/atom/atom --type=renderer --enable-experimental-web-platform-features --field-trial-handle=123480363484759119 18 2812 /usr/bin/gnome-software --gapplication-service
So netbeans + tracker-miner had pretty much consumed the entire pool within an hour of starting the system.
I've bumped the figure up to 65K by adding an entry to /etc/sysctl.conf and asked on #fedora if anyone knew of the side effects of having a high number - the only answers found were that approx 1K of unswapable kernel memory is used per watcher. So this increase only puts an additional 57MB of such memory usage onto my system.
Are there any other side effects of this to worry about?
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
-- Gargoyle
On Wed, Nov 18, 2020 09:13:54 +0000, Gargoyle wrote:
I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects. As a developer, I pretty much have netbeans open all day and am forever tailing log files, so I hit this problem pretty much straight away.
After some digging around I found a script to list the current count of watchers per process. The half-dozen worst offenders were:-
INOTIFY WATCHER COUNT PID CMD
4903 4620 /data/Applications/java/jdk-15/bin/java -Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans 3573 3032 /usr/libexec/tracker-miner-fs-3 64 2022 /usr/lib/systemd/systemd --user 47 2663 /usr/libexec/gsd-xsettings 27 34846 /usr/share/atom/atom --type=renderer --enable-experimental-web-platform-features --field-trial-handle=123480363484759119 18 2812 /usr/bin/gnome-software --gapplication-service
So netbeans + tracker-miner had pretty much consumed the entire pool within an hour of starting the system.
I've bumped the figure up to 65K by adding an entry to /etc/sysctl.conf and asked on #fedora if anyone knew of the side effects of having a high number
- the only answers found were that approx 1K of unswapable kernel memory is
used per watcher. So this increase only puts an additional 57MB of such memory usage onto my system.
Are there any other side effects of this to worry about?
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
To add: Dropbox users often run into this issue, which has interesting side effects, like PDF readers not updating when the file is changed:
https://ask.fedoraproject.org/t/pdf-readers-stops-doing-automatic-refresh-on...
On Wed, Nov 18, 2020 at 10:49 AM Ankur Sinha sanjay.ankur@gmail.com wrote:
On Wed, Nov 18, 2020 09:13:54 +0000, Gargoyle wrote:
I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects. As a developer, I pretty much have netbeans open all day and am forever tailing log files, so I hit this problem pretty much straight away.
After some digging around I found a script to list the current count of watchers per process. The half-dozen worst offenders were:-
INOTIFY WATCHER COUNT PID CMD
4903 4620 /data/Applications/java/jdk-15/bin/java -Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans 3573 3032 /usr/libexec/tracker-miner-fs-3 64 2022 /usr/lib/systemd/systemd --user 47 2663 /usr/libexec/gsd-xsettings 27 34846 /usr/share/atom/atom --type=renderer --enable-experimental-web-platform-features --field-trial-handle=123480363484759119 18 2812 /usr/bin/gnome-software --gapplication-service
So netbeans + tracker-miner had pretty much consumed the entire pool within an hour of starting the system.
I've bumped the figure up to 65K by adding an entry to /etc/sysctl.conf and asked on #fedora if anyone knew of the side effects of having a high number
- the only answers found were that approx 1K of unswapable kernel memory is
used per watcher. So this increase only puts an additional 57MB of such memory usage onto my system.
Are there any other side effects of this to worry about?
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
To add: Dropbox users often run into this issue, which has interesting side effects, like PDF readers not updating when the file is changed:
https://ask.fedoraproject.org/t/pdf-readers-stops-doing-automatic-refresh-on...
This is also an issue for syncthing when inotify support is enabled for sync'ed folders. I had to bump that setting to "fs.inotify.max_user_watches=204800" on my main machine because otherwise it ran out of file watches really really quickly. I think bumping it to 2^16 would be a good default value to make sure most users don't bump against the really low 8K limit ...
Fabio
On 11/18/20 10:13 AM, Gargoyle wrote:
I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects. As a developer, I pretty much have netbeans open all day and am forever tailing log files, so I hit this problem pretty much straight away.
After some digging around I found a script to list the current count of watchers per process. The half-dozen worst offenders were:-
INOTIFY WATCHER COUNT PID CMD
4903 4620 /data/Applications/java/jdk-15/bin/java -Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans 3573 3032 /usr/libexec/tracker-miner-fs-3 64 2022 /usr/lib/systemd/systemd --user 47 2663 /usr/libexec/gsd-xsettings 27 34846 /usr/share/atom/atom --type=renderer --enable-experimental-web-platform-features --field-trial-handle=123480363484759119 18 2812 /usr/bin/gnome-software --gapplication-service
So netbeans + tracker-miner had pretty much consumed the entire pool within an hour of starting the system.
I've bumped the figure up to 65K by adding an entry to /etc/sysctl.conf and asked on #fedora if anyone knew of the side effects of having a high number - the only answers found were that approx 1K of unswapable kernel memory is used per watcher. So this increase only puts an additional 57MB of such memory usage onto my system.
Are there any other side effects of this to worry about?
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
See also https://bugzilla.redhat.com/show_bug.cgi?id=1575156
I wonder how was this fixed.
On 18/11/2020 09:58, Miro Hrončok wrote:
On 11/18/20 10:13 AM, Gargoyle wrote:
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
See also https://bugzilla.redhat.com/show_bug.cgi?id=1575156
I wonder how was this fixed.
Seems I must have run into it on my work machine as well, due to vscode by the looks of it, and I increased the limit significantly as a result.
Tom
Am 18.11.20 um 10:13 schrieb Gargoyle:
4903 4620 /data/Applications/java/jdk-15/bin/java -Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans
The first action that comes in mind is to reduce the amount of notifiers in use.
When my Android Studio runs, it opens a shitload of files for no good reason. No idea who big your list of open files in your ide is, but my actual open source files are maybe 20 max. No need to have thousands of notifiers in use for this.
And for your question: the more notifiers are running, the slower your filesystem will react: for any access the list of notfiers needs to be checked. Depending on the implementation, a rising number may have an exponential increasing impact.
For normal Desktop use, a limit of 8192 is more than enough IMHO.
best regards, Marius
st 18. 11. 2020 v 10:16 odesílatel Gargoyle g@rgoyle.com napsal:
I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects. As a developer, I pretty much have netbeans open all day and am forever tailing log files, so I hit this problem pretty much straight away.
After some digging around I found a script to list the current count of watchers per process. The half-dozen worst offenders were:-
INOTIFY WATCHER COUNT PID CMD
4903 4620 /data/Applications/java/jdk-15/bin/java-Djdk.home=/data/Applications/java/jdk-15 -classpath /data/Applications/netbeans 3573 3032 /usr/libexec/tracker-miner-fs-3 64 2022 /usr/lib/systemd/systemd --user 47 2663 /usr/libexec/gsd-xsettings 27 34846 /usr/share/atom/atom --type=renderer --enable-experimental-web-platform-features --field-trial-handle=123480363484759119 18 2812 /usr/bin/gnome-software --gapplication-service
So netbeans + tracker-miner had pretty much consumed the entire pool within an hour of starting the system.
I've bumped the figure up to 65K by adding an entry to /etc/sysctl.conf and asked on #fedora if anyone knew of the side effects of having a high number - the only answers found were that approx 1K of unswapable kernel memory is used per watcher. So this increase only puts an additional 57MB of such memory usage onto my system.
Are there any other side effects of this to worry about?
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM. I've not rebooted my machine overnight and netbeans is now up to 13,000 watches and tracker-miner upto 4,500. So in less than 24 hours I have exceeded 16K watches.
Perhaps an new default value in the region of 32K - 64K would be more appropriate?
-- Gargoyle _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
I have the exact same experience with Goland and PyCharm. I raised the limit to 524288 using this guide[1] and it works like a charm.
[1]: https://confluence.jetbrains.com/display/IDEADEV/Inotify+Watches+Limit
On Wed, 18 Nov 2020 09:13:54 +0000 Gargoyle g@rgoyle.com wrote:
I've just made the switch from Ubuntu to Fedora 33 for my main desktop system. Of the few teething problems I am going through, the initial 8192 limit for fs.inotify.max_user_watches was probably causing the most side effects.
If not, then perhaps the default should be significantly increased with the aim of removing the problem for new users with modern desktop/laptop machines which probably have 4GB+ RAM.
A dynamic default is being tackled directly at kernel level. There is a currently in-progress patch on linux-fsdevel for this, first revision is at https://patchwork.kernel.org/project/linux-fsdevel/patch/20201026204418.2319.... For further context, see https://github.com/coreos/fedora-coreos-tracker/issues/637
Ciao, Luca
On 18/11/2020 13:28, Luca BRUNO wrote:
A dynamic default is being tackled directly at kernel level. There is a currently in-progress patch on linux-fsdevel for this, first revision is at https://patchwork.kernel.org/project/linux-fsdevel/patch/20201026204418.2319.... For further context, see https://github.com/coreos/fedora-coreos-tracker/issues/637
Great info, thanks Luca.
It seems like the same issues are identified on that thread as by others on this one. IDE's and sync'y things being almost universally problematic with the low default.
I'm not familiar with kernel dev. How long do these kind of things take to trickle into a distro? Would it still be worth targeting something for F34? Could a file dropped into /etc/sysctl.d/10-max_user_watches.conf be a quick win?
So far the only drawbacks mentioned are an increase in RAM usage when a lot of watches are in use and _maybe_ a filesystem slowdown (which so far hasn't been raised by anyone on the kernel.org thread).
My biggest offender, netbeans (17k watchers), is currently using 1.6G of RAM. Another 50MB really would be a drop in the ocean.
G.
-
Ankur Sinha -
Fabio Valentini -
Gargoyle -
Luca BRUNO -
Marius Schwarz -
Miro Hrončok -
Ondrej Budai -
Tom Hughes