Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
Rahul
On 8/9/07, Rahul Sundaram sundaram@fedoraproject.org wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
Rahul
I have often thought an entire release (or every ~3-4) should consist of nothing but bug fixes and code optimization. Even at Ingo's 2.3% it is worth a healthy discussion, which has already happened! If much faster machines could see 5-10% that would be a major improvement IMO.
I say lets try it! How hard would it be to test? ext3 is the biggest issue, but there is a work-a-round?
Dr. Diesel wrote:
On 8/9/07, Rahul Sundaram sundaram@fedoraproject.org wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
Rahul
I have often thought an entire release (or every ~3-4) should consist of nothing but bug fixes and code optimization. Even at Ingo's 2.3% it is worth a healthy discussion, which has already happened! If much faster machines could see 5-10% that would be a major improvement IMO.
I say lets try it! How hard would it be to test? ext3 is the biggest issue, but there is a work-a-round?
I'm not sure anyone has mentioned what are probably the biggest uses of atime: (1) a debugging step to see if a file is actually being read by some process. For example you edit some config file, start the service and the change you expect didn't happen. If atime didn't change, the file wasn't read, perhaps due to some other directive passed earlier. Likewise with executables that you think should be run, or libraries that should be used. (2) as an indication that files have never been used and can probably be deleted. Since most backup operations act as a read, this tends to not be very useful.
Personally I'd trade these for better performance but I wouldn't want the change to be a surprise.
Les Mikesell wrote:
I'm not sure anyone has mentioned what are probably the biggest uses of atime:
(2) as an indication that files have never been used and can probably be deleted. Since most backup operations act as a read, this tends to not be very useful.
Serious backup systems (programs designed for the purpose) save and restore the old value of atime. 'tar' has --atime-preserve. cpio has --reset-access-time.
John Reiser wrote:
Les Mikesell wrote:
I'm not sure anyone has mentioned what are probably the biggest uses of atime:
(2) as an indication that files have never been used and can probably be deleted. Since most backup operations act as a read, this tends to not be very useful.
Serious backup systems (programs designed for the purpose) save and restore the old value of atime. 'tar' has --atime-preserve. cpio has --reset-access-time.
Unfortunately those do more damage than good, since incremental backups need to be based on ctime changes and resetting the atime updates ctime.
Les Mikesell wrote:
John Reiser wrote:
Les Mikesell wrote:
I'm not sure anyone has mentioned what are probably the biggest uses of atime:
(2) as an indication that files have never been used and can probably be deleted. Since most backup operations act as a read, this tends to not be very useful.
Serious backup systems (programs designed for the purpose) save and restore the old value of atime. 'tar' has --atime-preserve. cpio has --reset-access-time.
Unfortunately those do more damage than good, since incremental backups need to be based on ctime changes and resetting the atime updates ctime.
That's why my backup scripts temporarily remount the filesystem with the "noatime" option and restore the previous state of that option afterward.
After reading the article on kernel list, I tried out noatime on root partition. I also added 'data=writeback'. Guess what? Unbootable system.
I had assumed (wrongly) that mount was smart enough that for something so critical as mounting / it would just ignore a bad option. Wrong.
IMO, this should REALLY be fixed.
On 10/08/07, Neal Becker ndbecker2@gmail.com wrote:
After reading the article on kernel list, I tried out noatime on root partition. I also added 'data=writeback'. Guess what? Unbootable system.
I had assumed (wrongly) that mount was smart enough that for something so critical as mounting / it would just ignore a bad option. Wrong.
IMO, this should REALLY be fixed.
No, mount reasonably assumes that if a user is fiddling about with mount options for the root file system then user can resolve an unbootable system because option is bad.
Chris
Rahul Sundaram wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
I'm using noatime on all Ext3 filesystems on a bunch of Fedora 7, CentOS 5 and Ubuntu 7.04 machines - desktops, workstations, VMware guests, servers with a variety of network services.
Zero issues so far.
On 8/10/07, Florin Andrei florin@andrei.myip.org wrote:
Rahul Sundaram wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
I'm using noatime on all Ext3 filesystems on a bunch of Fedora 7, CentOS 5 and Ubuntu 7.04 machines - desktops, workstations, VMware guests, servers with a variety of network services.
Zero issues so far.
I use it on all my ext3 filesystems too no problems at all ... so +1
Alexandru Ciobanu wrote:
Rahul Sundaram wrote:
Thoughts on disabling it?
I've been using noatime,nodiratime on servers, desktops, notebooks (both ext3 and reiserfs) since I've switched from Slackware with no issues. So +1.
noatime is a superset of nodiratime so you don't need both FYI.
Rahul
2007/8/9, Rahul Sundaram sundaram@fedoraproject.org:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
actually for various reasons i think relatime is a much better and cleaner solution in my eyes.
regards, Rudolf Kastl
Rahul
-- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Rudolf Kastl wrote:
2007/8/9, Rahul Sundaram sundaram@fedoraproject.org:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
actually for various reasons i think relatime is a much better and cleaner solution in my eyes.
I would appreciate more details. If you got various details, specify them.
Rahul
On Fri, Aug 10, 2007 at 04:38:33AM +0530, Rahul Sundaram wrote:
actually for various reasons i think relatime is a much better and cleaner solution in my eyes.
I would appreciate more details. If you got various details, specify them.
Ingo's approach to relatime is very nice. Basically treating the atime as having a day sized granularity plus relative changes for comparisons with ctime/mtime - that means all the tmpwatch tools continue to work, HSM continues to work and mutt is happy
On 8/10/07, Alan Cox alan@redhat.com wrote:
On Fri, Aug 10, 2007 at 04:38:33AM +0530, Rahul Sundaram wrote:
actually for various reasons i think relatime is a much better and cleaner solution in my eyes.
I would appreciate more details. If you got various details, specify
them.
Ingo's approach to relatime is very nice. Basically treating the atime as having a day sized granularity plus relative changes for comparisons with ctime/mtime - that means all the tmpwatch tools continue to work, HSM continues to work and mutt is happy
that seems to way to go imho not only for fedora but also upstream
2007/8/10, dragoran drago01@gmail.com:
On 8/10/07, Alan Cox alan@redhat.com wrote:
On Fri, Aug 10, 2007 at 04:38:33AM +0530, Rahul Sundaram wrote:
actually for various reasons i think relatime is a much better and cleaner solution in my eyes.
I would appreciate more details. If you got various details, specify
them.
Ingo's approach to relatime is very nice. Basically treating the atime as having a day sized granularity plus relative changes for comparisons with ctime/mtime - that means all the tmpwatch tools continue to work, HSM continues to work and mutt is happy
that seems to way to go imho not only for fedora but also upstream
turning off features that have design flaws in my eyes simply isnt a fix, but only a mere attempt to cure the symptom from a pure philosophical point of view. surely alot of more experienced users do use and enjoy those workarounds but at the end of the day we are crippling features instead of fixing issues. workarounds just slow down fixes though.
regards, Rudolf Kastl
-- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Rudolf Kastl wrote:
turning off features that have design flaws in my eyes simply isnt a fix, but only a mere attempt to cure the symptom from a pure philosophical point of view. surely alot of more experienced users do use and enjoy those workarounds but at the end of the day we are crippling features instead of fixing issues. workarounds just slow down fixes though.
I agree with this...
Before we take a step like turning off atime, I think those who wish to do so need to provide good justification for it, well beyond "I heard it may go faster" or "it feels faster to me" - we need some good tests, some actual, measurable difference shown on various workloads, at a minimum.
The only hard numbers presented in the lkml thread (unless I missed some...) didn't actually show a *huge* difference IIRC.
I'd also suggest that we at least identify that if atime does have a measurable difference, whether this is true in general, or if this is somewhat more specific to a particular filesystem which could be fixed... etc.
IOW, if this change is made, I think it should be done with eyes wide open - what is the measurable advantage, under what workloads, for what filesystems, etc - as well as what the potential drawbacks are.
Thanks,
-Eric
On Fri, 10 Aug 2007 11:04:33 -0500, Eric Sandeen esandeen@redhat.com wrote:
IOW, if this change is made, I think it should be done with eyes wide open - what is the measurable advantage, under what workloads, for what filesystems, etc - as well as what the potential drawbacks are.
Turning off atime is not new and even with ancient UNIXes it provided miniscule gains in outright throughput, although I suspect it can be possible to create a contrieved benchmark to demonstrate improvement. I do not expect noatime to be justifiable on the grounds of performance.
However, if dropping atime allows to reduce the number of hard drive spin-ups, I'm all for it.
-- Pete
On Fri, 2007-08-10 at 03:26 +0530, Rahul Sundaram wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
That said, I feel "noatime" might be suitable for selected partitions, but not as a general solution.
Also, I never understood what noatime solves what mounting partitions read-only can't do more properly.
Ralf
On 8/9/07, Ralf Corsepius rc040203@freenet.de wrote:
On Fri, 2007-08-10 at 03:26 +0530, Rahul Sundaram wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
Thoughts on disabling it?
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
That said, I feel "noatime" might be suitable for selected partitions, but not as a general solution.
Also, I never understood what noatime solves what mounting partitions read-only can't do more properly.
Ralf
Considering how easy it is for knowledgeable users to make disable atime, I don't see it as necessary to make this so by default.
On 8/9/07, Ralf Corsepius rc040203@freenet.de wrote:
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
If it's such a fundamental feature that should be kept around, why have NFS optimization documents always recommended disabling atime updates especially on servers where there is a lot of throughput?
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
Those that need atime will eventually figure out how to turn it on. The potential for a better user experience as well possible power savings seems to outweigh the fundamental feature argument.
James Hubbard
James Hubbard wrote:
On 8/9/07, Ralf Corsepius rc040203@freenet.de wrote:
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
If it's such a fundamental feature that should be kept around, why have NFS optimization documents always recommended disabling atime updates especially on servers where there is a lot of throughput?
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
Those that need atime will eventually figure out how to turn it on. The potential for a better user experience as well possible power savings seems to outweigh the fundamental feature argument.
+1
If it's such a fundamental feature that should be kept around, why have NFS optimization documents always recommended disabling atime updates especially on servers where there is a lot of throughput?
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
Those that need atime will eventually figure out how to turn it on. The potential for a better user experience as well possible power savings seems to outweigh the fundamental feature argument.
+1
On Fri, 2007-08-10 at 07:59 -0400, James Hubbard wrote:
On 8/9/07, Ralf Corsepius rc040203@freenet.de wrote:
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
If it's such a fundamental feature that should be kept around, why have NFS optimization documents always recommended disabling atime updates especially on servers where there is a lot of throughput?
I don't know.
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
That's what people call "arrogance of the masses". Let's kill that 1%, if 99% don't care!
<sarcasm> It's the same argument why people argue against utf-8, work as root (don't need uid/gids) and don't want SELinux?
Let's remove all of this from the kernel, single seat/single user systems don't need all this at all. </sarcasm>
Those that need atime will eventually figure out how to turn it on. The potential for a better user experience as well possible power savings seems to outweigh the fundamental feature argument.
A friend of mine experimented with atime/noatime yesterday:
These were his results:
Test case: A heavy weight compiler-job
Default /etc/fstab real 5m18.226s user 4m44.557s sys 1m17.193s User+Sys: 365.750
Rebooted -- all filesystems noatime,nodiratime real 5m4.256s user 4m36.841s sys 1m8.364s User+Sys: 346.750
new / old = .9465
[Fedora-7, i386 on an AMD Athlon(tm) 64 X2 Dual Core Processor 3800+]
Way off from the figures the proponents of notime are reporting.
Ralf
On Fri, 10 Aug 2007 15:09:11 +0200 Ralf Corsepius rc040203@freenet.de wrote:
On Fri, 2007-08-10 at 07:59 -0400, James Hubbard wrote:
On 8/9/07, Ralf Corsepius rc040203@freenet.de wrote:
IMO, disabling atime by default, just because 99% of applications, don't use it, is short-sighted. It basically ditches a fundamental feature of unix filessystems and converts there behavior to "DOS'ish".
If it's such a fundamental feature that should be kept around, why have NFS optimization documents always recommended disabling atime updates especially on servers where there is a lot of throughput?
I don't know.
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
That's what people call "arrogance of the masses". Let's kill that 1%, if 99% don't care!
<sarcasm> It's the same argument why people argue against utf-8, work as root (don't need uid/gids) and don't want SELinux?
Let's remove all of this from the kernel, single seat/single user systems don't need all this at all.
</sarcasm>
Those that need atime will eventually figure out how to turn it on. The potential for a better user experience as well possible power savings seems to outweigh the fundamental feature argument.
A friend of mine experimented with atime/noatime yesterday:
These were his results:
Test case: A heavy weight compiler-job
Default /etc/fstab real 5m18.226s user 4m44.557s sys 1m17.193s User+Sys: 365.750
Rebooted -- all filesystems noatime,nodiratime real 5m4.256s user 4m36.841s sys 1m8.364s User+Sys: 346.750
new / old = .9465
[Fedora-7, i386 on an AMD Athlon(tm) 64 X2 Dual Core Processor 3800+]
Way off from the figures the proponents of notime are reporting.
Well... a heavy compile job isn't going to reflect that much because you're writing to the disk anyway.
Perhaps try doing a 'find / -exec cat {} > /dev/null ;' with and without atime.
josh
On Fri, 2007-08-10 at 10:33 -0500, Josh Boyer wrote:
On Fri, 10 Aug 2007 15:09:11 +0200 Ralf Corsepius rc040203@freenet.de wrote:
Way off from the figures the proponents of notime are reporting.
Well... a heavy compile job isn't going to reflect that much because you're writing to the disk anyway.
Well, not necessarily. As others have pointed out before (the kernel devs were referring to kernel compilation as example), compilation involves a lot of read-access to header files (typically in the order of 100s per source file).
Perhaps try doing a 'find / -exec cat {} > /dev/null ;' with and without atime.
Worth a try. I'll propose this to the person having experimented on this.
Ralf
Ralf Corsepius wrote:
A friend of mine experimented with atime/noatime yesterday:
These were his results:
Test case: A heavy weight compiler-job
Default /etc/fstab real 5m18.226s user 4m44.557s sys 1m17.193s User+Sys: 365.750
Rebooted -- all filesystems noatime,nodiratime real 5m4.256s user 4m36.841s sys 1m8.364s User+Sys: 346.750
new / old = .9465
+1 for real numbers!!!
Though the test would seem better if there was an explicit reboot before the first test, to make sure caches are in basically the same state.
[Fedora-7, i386 on an AMD Athlon(tm) 64 X2 Dual Core Processor 3800+]
Way off from the figures the proponents of notime are reporting.
I didn't see that many people reporting numbers. But are you kidding, 5+% is HUGE! (both sarcasm and sincerity). I mean, for something that is basically FREE.
And you also failed to remind people of the other key HUGE benefit (I think, if I've been following this correctly)- that (laptop) drives would never get needless writes for every file read. I.e. if I understand correctly, reading a file thats in cache won't cause the drive to need to be spun up to update the atime.
-dmc
On Fri, 10 Aug 2007 15:09:11 +0200, Ralf Corsepius wrote:
On Fri, 2007-08-10 at 07:59 -0400, James Hubbard wrote:
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
That's what people call "arrogance of the masses". Let's kill that 1%, if 99% don't care!
Nobody's killing anybody, but the system should have sensible defaults that suit most people, and be able to be changed where appropriate for the minority.
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
I've disabled atime on all filesystems on all my machines for well over 5 years, pretty much as the first step after install, and have never once run into an issue. (Ok, so I never use mutt, but probably most users don't use mutt and since it has workarounds this can't be a reason by itself to keep atime.) I've missed the functionality probably less than once a year, and that's as a developer/sysadmin too.
<sarcasm> It's the same argument why people argue against utf-8, work as root (don't need uid/gids) and don't want SELinux?
Let's remove all of this from the kernel, single seat/single user systems don't need all this at all.
</sarcasm>
No it's not, that's all completely unrelated.
A friend of mine experimented with atime/noatime yesterday: These were his results: Test case: A heavy weight compiler-job new / old = .9465
That's over 5% performance improvement, a huge amount for such a simple change. I'll bet most of the difficult kernel optimisations are winning less than that.
Now imagine the number of files touched on a boot, or logging into the GUI and how much difference it makes there. (For hard numbers on the 10,000s of files touched read Dave Jones's blog.)
Personally I can't believe its taken this long for people to sit up and take notice of atime which was pretty much a design failure from the start.
Cheers,
Martin.
On Fri, 2007-08-10 at 21:43 +0000, Martin Ebourne wrote:
On Fri, 10 Aug 2007 15:09:11 +0200, Ralf Corsepius wrote:
On Fri, 2007-08-10 at 07:59 -0400, James Hubbard wrote:
Just because it's a fundamental feature doesn't mean that it has to be used. Fundamentally, my CPU can run at 2GHz all of the time that doesn't mean that it should. If 99% of the applications can do without it and probably 99% of the people can as well, why not go ahead and get disable it.
That's what people call "arrogance of the masses". Let's kill that 1%, if 99% don't care!
Nobody's killing anybody,
You would - You would kill all applications which rely on atime.
Not that many applications will do so, but it's not hard to imagine one.
An example would be apps cleaning up/removing files based on atime.
<sarcasm> It's the same argument why people argue against utf-8, work as root (don't need uid/gids) and don't want SELinux?
Let's remove all of this from the kernel, single seat/single user systems don't need all this at all.
</sarcasm>
No it's not, that's all completely unrelated.
Not? I disagree.
You want a "fundamental feature, most users don't need/want" to be disabled to gain performance. What I listed above is along the same rationale.
A friend of mine experimented with atime/noatime yesterday: These were his results: Test case: A heavy weight compiler-job new / old = .9465
That's over 5% performance improvement,
Well, whether 5% is "much" is arguable.
On many occasions, you won't even notice because it will get lost in the jitter other factors impose.
Take out SELinux, gain "x %". Don't use kde, gain "x %". Use "lean mail browser", gain "x %". Use gcc-<version>, gain another 20%. Don't have "big application" running in the background, gain "x %". Buy a better harddisk, gain "x %".
a huge amount for such a simple change. I'll bet most of the difficult kernel optimisations are winning less than that.
Agreed, but with regard to compiler turn-around times, 5% is below the jitter of other factors, such as differences in compilation speed between different versions of GCC and differences in the impact of compilation-flags being used in GCC.
Now imagine the number of files touched on a boot, or logging into the GUI and how much difference it makes there. (For hard numbers on the 10,000s of files touched read Dave Jones's blog.)
Make /usr, /bin, /sbin etc. read-only, consider noatime on /home, atime on /var, /tmp would be a reasonable approach.
Personally I can't believe its taken this long for people to sit up and take notice of atime which was pretty much a design failure from the start.
Well, IIRC, noatime exists for circa a decade. Also, IIRC, this is not the first time of such a proposal. I recall SuSE devs having recommended it to me many years ago (ca. 2000) when facing "jerky" desktop behavior at that time.
Ralf
On Sat, Aug 11, 2007 at 06:58:56AM +0200, Ralf Corsepius wrote:
You would - You would kill all applications which rely on atime. Not that many applications will do so, but it's not hard to imagine one. An example would be apps cleaning up/removing files based on atime.
Ingo's relatime work handles this
Alan
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
Rahul
Rahul Sundaram napsal(a):
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
The patch changes behavior.
We're not in that much of a rush, are we? Why don't we wait to see what, if any, relatime variant is committed to the upstream kernel? relatime design decisions will happen on lkml, not on this list. Mirek
Miloslav Trmac wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
The patch changes behavior.
We're not in that much of a rush, are we? Why don't we wait to see what, if any, relatime variant is committed to the upstream kernel? relatime design decisions will happen on lkml, not on this list.
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
This file might have sections for different fs types so you could apply appropriate settings, like read/write block sizes on all nfs mounts. It would be a nice touch if the mounting action would retry failing mounts without the options to recover from typos or options that conflict with ones specified in fstab, but since that could cause surprises itself it should be a setting within the file.
As usual, documentation for this file would be nice but optional. A gui to manage it would be nice but optional. The file could be introduced in one fedora version with no default options but a choice to add noatime during the install. Then in the next or some future version there would be more real-world experience about what default to use and you'd have an easy way to override the default in any case. This is something that could be useful regardless of what the kernel does with relatime.
Les Mikesell wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people - especially the home use - has little or no use for it whatsoever. Its a bit like the way mount fails on a broken fstab, it assumes that if you are messing with the fstab you know what you are doing. Equally anyone who _needs_ atime knows what they are doing and how to enable it.
Any sort of fancy /etc/sysconfig trick is more effort than is needed, when the only change needed to undo it is to remove an option from the fstab.
Just because something was always that way doesn't mean it needs to stay that way - and whatever the numbers, noatime *does* improve performance.
Benjamin Lewis wrote:
Les Mikesell wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
Rahul
Rahul Sundaram wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
But whether it is a noticeable difference or not will depend on the number of files they access, either with automated build procedures or GUI file managers that like to peek inside every file. People who keep their directory trees sparse and don't venture out of their home directory in a GUI much will probably not know the difference except that tmpwatch may not clean up correctly and their mailer may have to do more work to decide if a message is unread (etc.).
-- Les Mikesell lesmikesell@gmail.com
Les Mikesell wrote:
Rahul Sundaram wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
But whether it is a noticeable difference or not will depend on the number of files they access, either with automated build procedures or GUI file managers that like to peek inside every file. People who keep their directory trees sparse and don't venture out of their home directory in a GUI much will probably not know the difference except that tmpwatch may not clean up correctly and their mailer may have to do more work to decide if a message is unread (etc.).
In my experience, viewing a folder of images as thumbnails is significantly slower with atime. Many users will be viewing folders of images methinks. In addition, most GUIs read a large (read huge) number of files when they load, so load times for the GUI should be reduced also.
-- Les Mikesell lesmikesell@gmail.com
EDIT: Sorry if anyone gets this twice, forgot to forward it to the list..
Benjamin Lewis wrote:
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
But whether it is a noticeable difference or not will depend on the number of files they access, either with automated build procedures or GUI file managers that like to peek inside every file. People who keep their directory trees sparse and don't venture out of their home directory in a GUI much will probably not know the difference except that tmpwatch may not clean up correctly and their mailer may have to do more work to decide if a message is unread (etc.).
In my experience, viewing a folder of images as thumbnails is significantly slower with atime.
Are you sure that your viewer didn't cache the thumbnail images on the first run so you were timing something different after changing the option?
Many users will be viewing folders of images methinks. In addition, most GUIs read a large (read huge) number of files when they load, so load times for the GUI should be reduced also.
Some users will, some won't. The atime change should be going though the disk write cache and thus only have a real-time hit when there is enough disk activity that the cache writes conflict with read operations.
Les Mikesell wrote:
Benjamin Lewis wrote:
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
But whether it is a noticeable difference or not will depend on the number of files they access, either with automated build procedures or GUI file managers that like to peek inside every file. People who keep their directory trees sparse and don't venture out of their home directory in a GUI much will probably not know the difference except that tmpwatch may not clean up correctly and their mailer may have to do more work to decide if a message is unread (etc.).
In my experience, viewing a folder of images as thumbnails is significantly slower with atime.
Are you sure that your viewer didn't cache the thumbnail images on the first run so you were timing something different after changing the option?
I did clear the caches, yes.
Many users will be viewing folders of images methinks. In addition, most GUIs read a large (read huge) number of files when they load, so load times for the GUI should be reduced also.
Some users will, some won't. The atime change should be going though the disk write cache and thus only have a real-time hit when there is enough disk activity that the cache writes conflict with read operations.
Benjamin Lewis wrote:
In my experience, viewing a folder of images as thumbnails is significantly slower with atime. Many users will be viewing folders of images methinks. In addition, most GUIs read a large (read huge) number of files when they load, so load times for the GUI should be reduced also.
in my experience... should be...
Does anyone have any numbers? This thread has been long on assertions, and very short on measured results.
noatime may well be faster in various situations. Is anyone who is advocating noatime willing to demonstrate the speed gain quantitatively?
For such a long thread on performance tweaking why are there so few benchmarks or timing results?
-Eric
Eric Sandeen wrote:
Benjamin Lewis wrote:
In my experience, viewing a folder of images as thumbnails is significantly slower with atime. Many users will be viewing folders of images methinks. In addition, most GUIs read a large (read huge) number of files when they load, so load times for the GUI should be reduced also.
in my experience... should be...
Does anyone have any numbers? This thread has been long on assertions, and very short on measured results.
noatime may well be faster in various situations. Is anyone who is advocating noatime willing to demonstrate the speed gain quantitatively?
For such a long thread on performance tweaking why are there so few benchmarks or timing results?
-Eric
I believe there were some quantitative numbers earlier in the thread, and I remember someone mentioning a 5% increase.
Benjamin Lewis wrote:
Eric Sandeen wrote:
For such a long thread on performance tweaking why are there so few benchmarks or timing results?
-Eric
I believe there were some quantitative numbers earlier in the thread, and I remember someone mentioning a 5% increase.
There was one post of one workload, yes. And a write-heavy one, at that.
-Eric
Eric Sandeen wrote:
Benjamin Lewis wrote:
Eric Sandeen wrote:
For such a long thread on performance tweaking why are there so few benchmarks or timing results?
-Eric
I believe there were some quantitative numbers earlier in the thread, and I remember someone mentioning a 5% increase.
There was one post of one workload, yes. And a write-heavy one, at that.
I for one would like to see numbers for
- boot time - time to gdm-autologged-in firefox-autostartonsession-viewing-local-htmlpage
- number of laptop drive spin-ups per hour - i.e. boot like above, then leave system idle for a few hours - also try leaving other common apps open when idle, i.e. thunderbird,...
- time for kernel compile - i.e. same as original test, but with what I commented, i.e. reboot before both tests to ensure accuracy
- time for find / -exec grep "somestring" '{}' ';'
- time for tar copy of rootfs to a different filesystem
- time for reconstruction of f7/f8 media (livecd and traditional) via livecd-tools and pungi (or revisor)
- anything anybody else adds to this list...
- oh, and did I forget to re-emphasize the laptop spin-up test...
-dmc
Eric Sandeen wrote:
Douglas McClendon wrote:
- oh, and did I forget to re-emphasize the laptop spin-up test...
Doesn't laptop mode & friends remount with noatime already? It's at least an option in those scripts...
I have no idea, that's why I was hoping someone would test it for me :)
Actually, I just looked at my rpm -qa and yum list available. Where did laptop mode go? (I did grep -i laptop)
If fedora already does remount with noatime by default on laptops in some fashion, that would seem to be a pretty strong argument in favor of making it default all the time, given any significant performance benefit. (and the fact that I haven't heard any convincing use-case of it that doesn't have a better workaround).
-dmc
On Sat, 2007-08-11 at 23:21 +0530, Rahul Sundaram wrote:
Benjamin Lewis wrote:
Les Mikesell wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it. The majority of people
- especially the home use - has little or no use for it whatsoever.
The crux is _nobody_ really know if he really uses or might have a future use-case for it.
Also it's very hard to find out if notime breaks a use-case, because it in most won't cause applications to barf, but to silently "mis-behave".
In best case, after some amount of time, he will notice that something won't work "as advertised".
The majority of people will see a performance boost they will appreciate regardless of whether they know about atime or not.
Please provide figures of the boost you have experienced personally for the test cases you have experienced.
Ralf
Benjamin Lewis wrote:
Les Mikesell wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it.
Yes, just like the kind of person who _needs_ networking knows how to issue ifconfig commands directly to set it up. That doesn't mean that a general purpose way to set it up with the most likely default and a GUI to change it is not an improvement.
The majority of people
- especially the home use - has little or no use for it whatsoever. Its
a bit like the way mount fails on a broken fstab, it assumes that if you are messing with the fstab you know what you are doing. Equally anyone who _needs_ atime knows what they are doing and how to enable it.
Except that they may have applications currently in use that rely on the decades old, documented behavior and should not have these broken as a surprise. Let one release go where you encourage people to break these with their own choice and report it, then you'll know what to expect when you break it with the default.
Any sort of fancy /etc/sysconfig trick is more effort than is needed, when the only change needed to undo it is to remove an option from the fstab.
atime is not the only mount option that people need to change and a one-off hack for every little thing is not as nice as a general purpose solution that exactly matches the approach of the gazillion other things under /etc/sysconfig, put there for the same purpose.
Just because something was always that way doesn't mean it needs to stay that way - and whatever the numbers, noatime *does* improve performance.
Agreed, but RedHat-style administration puts changes like this under user control with files under /etc/sysconfig and sometimes provides a GUI tool to modify it. People who don't want this level/style of control are probably using some other OS.
Les Mikesell wrote:
Benjamin Lewis wrote:
Les Mikesell wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it.
Yes, just like the kind of person who _needs_ networking knows how to issue ifconfig commands directly to set it up. That doesn't mean that a general purpose way to set it up with the most likely default and a GUI to change it is not an improvement.
That's not a fair comparison.
The majority of people
- especially the home use - has little or no use for it whatsoever. Its
a bit like the way mount fails on a broken fstab, it assumes that if you are messing with the fstab you know what you are doing. Equally anyone who _needs_ atime knows what they are doing and how to enable it.
Except that they may have applications currently in use that rely on the decades old, documented behavior and should not have these broken as a surprise. Let one release go where you encourage people to break these with their own choice and report it, then you'll know what to expect when you break it with the default.
I thought this only really affected some backup applications - emphasis on the _some_ part, mutt and tmpwatch - both of which have patches to resolve their issues. In any case, with proper release notes this would not be a surprise.
Any sort of fancy /etc/sysconfig trick is more effort than is needed, when the only change needed to undo it is to remove an option from the fstab.
atime is not the only mount option that people need to change and a one-off hack for every little thing is not as nice as a general purpose solution that exactly matches the approach of the gazillion other things under /etc/sysconfig, put there for the same purpose.
I was referring to the amount of effort required to make an /etc/sysconfig switch work.
Just because something was always that way doesn't mean it needs to stay that way - and whatever the numbers, noatime *does* improve performance.
Agreed, but RedHat-style administration puts changes like this under user control with files under /etc/sysconfig and sometimes provides a GUI tool to modify it. People who don't want this level/style of control are probably using some other OS.
Benjamin Lewis wrote:
I'd expect the RedHat-style approach to this to be: add some file under /etc/sysconfig like mount-options that contains options that can be merged with the ones in /etc/fstab and all of the magic automounting bits (this is probably as important on usb flash drives as anywhere).
[snip]
There is something which leap out at me as soon a I saw this: the kind of person who _needs_ atime, knows how to set it.
Yes, just like the kind of person who _needs_ networking knows how to issue ifconfig commands directly to set it up. That doesn't mean that a general purpose way to set it up with the most likely default and a GUI to change it is not an improvement.
That's not a fair comparison.
Right - typos in the ifconfig commands normally won't break console access, where a typo in /etc/fstab will make your system unbootable.
The majority of people
- especially the home use - has little or no use for it whatsoever. Its
a bit like the way mount fails on a broken fstab, it assumes that if you are messing with the fstab you know what you are doing. Equally anyone who _needs_ atime knows what they are doing and how to enable it.
Except that they may have applications currently in use that rely on the decades old, documented behavior and should not have these broken as a surprise. Let one release go where you encourage people to break these with their own choice and report it, then you'll know what to expect when you break it with the default.
I thought this only really affected some backup applications - emphasis on the _some_ part,
No, backups never use atime - it tells you when someone last read a file, not when something changed. Atime would be used for debugging (did an app read this file before crashing - or at all?) or determining if a file has not been read since it was written (is a mailbox/maildir file new/unread?), or if a file is unecessary and can be deleted since nothing has read it for some long interval).
mutt and tmpwatch - both of which have patches to resolve their issues.
That's all anyone has found, but (a) this sounds like a top-of-the-head guess instead of someone grepping the whole of fedora extra source and (b) doesn't consider apps someone has written or obtained elsewhere that use the documented filesystem spec.
In any case, with proper release notes this would not be a surprise.
Do people read those? Especially if they skip the version where the change is made?
Any sort of fancy /etc/sysconfig trick is more effort than is needed, when the only change needed to undo it is to remove an option from the fstab.
atime is not the only mount option that people need to change and a one-off hack for every little thing is not as nice as a general purpose solution that exactly matches the approach of the gazillion other things under /etc/sysconfig, put there for the same purpose.
I was referring to the amount of effort required to make an /etc/sysconfig switch work.
One-off hacks might be easy. That's not a good excuse for a distribution used by a lot of people to do them. Anyone who has used RH/fedora for any length of time is used to bits and pieces of the stock config files being extracted into files under /etc/sysconfig for local management so that would be expected in this case as well.
Miloslav Trmac wrote:
Rahul Sundaram napsal(a):
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
The patch changes behavior.
We're not in that much of a rush, are we? Why don't we wait to see what, if any, relatime variant is committed to the upstream kernel? relatime design decisions will happen on lkml, not on this list. Mirek
This is not the first time these discussions have happened. Not a question of rush but one of finally making a decision. In the same lkml thread I see the redirection to fedora-devel list and I made no attempt to discussion the design here. If relatime is what we want, lets switch to that by default.
Rahul
On Sat, 2007-08-11 at 21:34 +0530, Rahul Sundaram wrote:
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
And how do you "fix" the "find"-based cron job users might have implemented to clean up /var/cache from files not being used for time span "x"?
Ralf Corsepius wrote:
On Sat, 2007-08-11 at 21:34 +0530, Rahul Sundaram wrote:
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
And how do you "fix" the "find"-based cron job users might have implemented to clean up /var/cache from files not being used for time span "x"?
Any change will have some corner cases where it will cause problems. We can make defaults work good without atime and for other instances there is always the release notes as usual. We are only talking about changing this for a new release so users would have time to adopt if necessary.
For most users, it is just a performance boost.
Rahul
On Sun, 2007-08-12 at 08:46 +0530, Rahul Sundaram wrote:
Ralf Corsepius wrote:
On Sat, 2007-08-11 at 21:34 +0530, Rahul Sundaram wrote:
Matthew Miller wrote:
On Fri, Aug 10, 2007 at 09:43:18PM +0000, Martin Ebourne wrote:
The minority here is having atime, hardly anyone ever uses it compared to the cost of having it. It's easy to reenable it for the few who really find it useful and want to pay for it.
tmpwatch runs by default. Everyone uses that.
Has a simple patch already which I mentioned in my original mail.
And how do you "fix" the "find"-based cron job users might have implemented to clean up /var/cache from files not being used for time span "x"?
Any change will have some corner cases where it will cause problems.
This is not a corner case. This is the OS having dropped a fundamental feature.
We can make defaults work good without atime and for other instances there is always the release notes as usual. We are only talking about changing this for a new release so users would have time to adopt if necessary.
For most users, it is just a performance boost.
Implement "aging"/"time-based cleanup of /var/cache/yum" into yum and it will affect everybody.
Ralf Corsepius wrote:
This is not a corner case. This is the OS having dropped a fundamental feature.
A feature that only a couple of programs use in any way in the Fedora repository is by no means a fundamental feature. Even if it is, we aren't taking away the feature and only turning it off by default for a performance boost. Those who want it will turn it by default.
Implement "aging"/"time-based cleanup of /var/cache/yum" into yum and it will affect everybody.
We don't need to worry about non existent features.
Rahul
On Sun, 2007-08-12 at 09:04 +0530, Rahul Sundaram wrote:
Ralf Corsepius wrote:
This is not a corner case. This is the OS having dropped a fundamental feature.
A feature that only a couple of programs use in any way in the Fedora repository is by no means a fundamental feature.
It is a _documented_, _standardized_ feature of the OS, being in existance ever since Unix exists.
Even if it is, we aren't taking away the feature and only turning it off by default for a performance boost. Those who want it will turn it by default.
Once again, you are not able to know if you want it nor if you will want it.
Have you ever tried it or are you just FUDing around?
Implement "aging"/"time-based cleanup of /var/cache/yum" into yum and it will affect everybody.
We don't need to worry about non existent features.
Bummer, that's what I call "wide-sight" and "leadership quality".
Ralf
Ralf Corsepius wrote:
It is a _documented_, _standardized_ feature of the OS, being in existance ever since Unix exists.
Either it's used and useful or it isn't. Linux has introduced several features that Unix hasn't and not implemented some Unix has including some "documented and standardized" features.
Once again, you are not able to know if you want it nor if you will want it.
Have you ever tried it or are you just FUDing around?
I don't see how FUD applies in any way here. If people don't know whether they want it or not now, they will learn when they need to.
Implement "aging"/"time-based cleanup of /var/cache/yum" into yum and it will affect everybody.
We don't need to worry about non existent features.
Bummer, that's what I call "wide-sight" and "leadership quality".
Instead of resorting to adhominem attacks as usual try and address the subject in hand.
Rahul
On Sun, 2007-08-12 at 12:05 +0530, Rahul Sundaram wrote:
Ralf Corsepius wrote:
It is a _documented_, _standardized_ feature of the OS, being in existance ever since Unix exists.
Either it's used and useful or it isn't. Linux has introduced several features that Unix hasn't and not implemented some Unix has including some "documented and standardized" features.
Once again, you are not able to know if you want it nor if you will want it.
Have you ever tried it or are you just FUDing around?
I don't see how FUD applies in any way here. If people don't know whether they want it or not now, they will learn when they need to.
Provide the results of the tests you exercised yourself. So far you haven't.
All you did was to agitate and reiterate populist propaganda.
My checks so far have not provided any indication noatime solves anything, also I can not reproduce the 50% speedup some people claim to see.
We don't need to worry about non existent features.
Bummer, that's what I call "wide-sight" and "leadership quality".
Instead of resorting to adhominem attacks as usual try and address the subject in hand.
The subject at hand are: - You are wanting to cripple Fedora, by sacrificing a little used feature. Thereby you are closing out certain types of future developments and current usages. I call this short-sightedness and narrow-mindedness. You are wanting Fedora to diverge away from UNIX. You are conventing Fedora into WinIx.
- I think you are not qualified to judge the technical side-effects of what you are proposing and not qualified to take a leading role in fedora. I was glad to see you failed the elections.
Ralf Corsepius wrote:
whether they want it or not now, they will learn when they need to.
Provide the results of the tests you exercised yourself. So far you haven't.
I gave you a reference. If you want more data, do the tests yourself.
All you did was to agitate and reiterate populist propaganda.
My checks so far have not provided any indication noatime solves anything, also I can not reproduce the 50% speedup some people claim to see.
5% results shown even in your numbers are good enough.
- I think you are not qualified to judge the technical side-effects of
what you are proposing and not qualified to take a leading role in fedora. I was glad to see you failed the elections.
Again resorting to adhominem attacks. End of discussion.
Rahul
On Sun, 2007-08-12 at 12:33 +0530, Rahul Sundaram wrote:
Ralf Corsepius wrote:
whether they want it or not now, they will learn when they need to.
Provide the results of the tests you exercised yourself. So far you haven't.
I gave you a reference.
All you did was to agitate and reiterate populist propaganda.
My checks so far have not provided any indication noatime solves anything, also I can not reproduce the 50% speedup some people claim to see.
5% results shown even in your numbers are good enough.
At a very questionable price.
Disabling SELinux provides the same amount of speed up.
- I think you are not qualified to judge the technical side-effects of
what you are proposing and not qualified to take a leading role in fedora. I was glad to see you failed the elections.
Again resorting to adhominem attacks.
Nope, resistance against a tyrannic dictator.
Ralf Corsepius wrote: umbers are good enough.
At a very questionable price.
Disabling SELinux provides the same amount of speed up.
They aren't comparable. atime isn't a security feature and it is hardly used by any software at all.
Nope, resistance against a tyrannic dictator.
Stop your name calling Ralf. I haven't dictated anything here. It is very shameful that you derail all the conversations with these kind of personal attacks unrelated to the subject either on list or offlist very frequently. Learn to be constructive or if you can't, atleast stay out of the discussion.
Rahul
On Sun, 2007-08-12 at 13:09 +0530, Rahul Sundaram wrote:
Ralf Corsepius wrote: umbers are good enough.
At a very questionable price.
Disabling SELinux provides the same amount of speed up.
They aren't comparable. atime isn't a security feature and it is hardly used by any software at all.
* atime is a feature necessary to provide certain features to applications, such as time-based clean ups. * lack of atime in a filesystem cause functional regressions.
SELinux is completely optional and doesn't cause functional regression.
Nope, resistance against a tyrannic dictator.
Stop your name calling Ralf. I haven't dictated anything here.
You are. You've asked for opinions and so far have been completely ignoring any counterarguments and demonstrated your lack of understanding them.
It is very shameful that you derail all the conversations with these kind of personal attacks unrelated to the subject either on list or offlist very frequently.
Well, actually I think Fedora is derailing due to its leadership committing one mistake after the other.
Ralf Corsepius wrote:
You are. You've asked for opinions and so far have been completely ignoring any counterarguments and demonstrated your lack of understanding them.
I can't possibly dictate anything since I am not the person making the decisions but your arguments don't sound convincing to me. Disagreeing with your opinion does not equate to lack of understanding or dictatorship. You should just stop name calling people unnecessarily.
Rahul
On 12/08/07, Ralf Corsepius rc040203@freenet.de wrote:
SELinux is completely optional and doesn't cause functional regression.
Stops warning or error messages from "modprobe" being logged to my X session log (~/X.log) if I'm in enforcing mode, because of the security context?
Well, actually I think Fedora is derailing due to its leadership committing one mistake after the other.
That's another issue :)
Why not default to "noatime" for all bar /tmp, and create that as a separate partition / logical volume by default? Also, please disable tmpwatch, as OFTEN I have found it removed files from /tmp (even with atime ON) that were still needed, due to an X session being open for more than a day or so. Clear /tmp on reboot instead.
Bill Crawford wrote:
On 12/08/07, Ralf Corsepius rc040203@freenet.de wrote:
SELinux is completely optional and doesn't cause functional regression.
Stops warning or error messages from "modprobe" being logged to my X session log (~/X.log) if I'm in enforcing mode, because of the security context?
Well, actually I think Fedora is derailing due to its leadership committing one mistake after the other.
That's another issue :)
Why not default to "noatime" for all bar /tmp, and create that as a separate partition / logical volume by default?
Because nobody has shown conclusively, at least no on this thread, that noatime actually helps enough to matter, IMHO.
As far as I'm concerned, nobody has done the necessary legwork to justify this change.
I have at least one case where noatime actually slows things down. With 66 million inodes on an ext3 filesystem, "find" across the filesystem with a fresh mount / cold cache was a few seconds slower with noatime. Odd result, but it shows at least that this change shouldn't be made based on a hunch, but only after looking at some real results.
-Eric
[root@bear-05 ~]# for I in `seq 1 10` do mount /dev/sdb12 /mnt/test time find /mnt/test > /dev/null umount /dev/sdb12 done
real 0m58.355s user 0m0.384s sys 0m1.212s
real 0m53.485s user 0m0.400s sys 0m1.236s
real 0m53.900s user 0m0.348s sys 0m1.288s
real 0m53.919s user 0m0.328s sys 0m1.080s
real 0m54.835s user 0m0.288s sys 0m1.316s
real 0m55.544s user 0m0.348s sys 0m1.332s
real 0m53.221s user 0m0.376s sys 0m1.332s
real 0m54.252s user 0m0.308s sys 0m1.300s
real 0m54.379s user 0m0.344s sys 0m1.192s
real 0m54.430s user 0m0.364s sys 0m1.188s [root@bear-05 ~]# for I in `seq 1 10`; do mount -o noatime /dev/sdb12 /mnt/test; time find /mnt/test > /dev/null; umount /dev/sdb12; done
real 0m58.461s user 0m0.380s sys 0m1.052s
real 0m57.830s user 0m0.336s sys 0m1.180s
real 0m57.997s user 0m0.356s sys 0m1.236s
real 0m58.123s user 0m0.348s sys 0m1.228s
real 0m58.565s user 0m0.352s sys 0m1.132s
real 0m59.683s user 0m0.332s sys 0m1.292s
real 0m58.457s user 0m0.408s sys 0m1.096s
real 0m58.406s user 0m0.344s sys 0m1.240s
real 0m58.678s user 0m0.332s sys 0m1.192s
real 0m58.934s user 0m0.376s sys 0m1.308s
On 8/21/07, Eric Sandeen esandeen@redhat.com wrote:
[root@bear-05 ~]# for I in `seq 1 10`; do mount -o noatime /dev/sdb12 /mnt/test; time find /mnt/test > /dev/null; umount /dev/sdb12; done
Hmm. If you do the find so quickly after the mount, isn't it possible that the kernel is still reading in some metadata structures asynchronously, or checking the journal, etc?
Colin Walters wrote:
On 8/21/07, *Eric Sandeen* <esandeen@redhat.com mailto:esandeen@redhat.com> wrote:
[root@bear-05 ~]# for I in `seq 1 10`; do mount -o noatime /dev/sdb12 /mnt/test; time find /mnt/test > /dev/null; umount /dev/sdb12; doneHmm. If you do the find so quickly after the mount, isn't it possible that the kernel is still reading in some metadata structures asynchronously, or checking the journal, etc?
Nope, things like journal replay are done before the filesystem becomes available for use.
-Eric
On 8/22/07, Eric Sandeen esandeen@redhat.com wrote:
As far as I'm concerned, nobody has done the necessary legwork to justify this change.
I have at least one case where noatime actually slows things down. With 66 million inodes on an ext3 filesystem, "find" across the filesystem with a fresh mount / cold cache was a few seconds slower with noatime. Odd result, but it shows at least that this change shouldn't be made based on a hunch, but only after looking at some real results.
-Eric
Find will not benchmark atime, as it doesn't open the files for reading. If you want to test it you would need to read the files as well.
Out of curiosity, I ran some benchmarks:
# find /data -type f 2>/dev/null | wc -l 105602 # cat readblock.c #include <unistd.h> #include <fcntl.h>
int main(int argc, char **argv) { int i, fd; char buf[1024]; for(i=1; i<argc; i++) { fd = open(argv[i], O_RDONLY); if(fd >= 0) { read(fd, buf, 1024); close(fd); } } return 0; } # make readblock cc moveblock.c -o moveblock # umount /data # sync && echo 3 > /proc/sys/vm/drop_caches # mount /data # time find /data -type f -exec ./readblock '{}' '+'
real 7m28.955s user 0m0.343s sys 0m12.870s # umount /data # sync && echo 3 > /proc/sys/vm/drop_caches # mount -o noatime /data # time find /data -type f -exec ./readblock '{}' '+'
real 6m53.724s user 0m0.327s sys 0m12.023s
An 8% improvement, and that's with cleared caches. A bigger improvement would be expected when there is cache hits, since it doesn't have to read or write anything to the disk.
Reading a block or so from every file is not an uncommon operation. It happens when you use grep, it happens when you load a directory in nautilus (and all the file types are found by scanning the first block of the file), it happens when you open your photo managing application (thumbnails), and it happens when icons are loaded for menus. 8% makes a difference in the desktop and on the server.
n0dalus.
n0dalus wrote:
On 8/22/07, Eric Sandeen esandeen@redhat.com wrote:
As far as I'm concerned, nobody has done the necessary legwork to justify this change.
I have at least one case where noatime actually slows things down. With 66 million inodes on an ext3 filesystem, "find" across the filesystem with a fresh mount / cold cache was a few seconds slower with noatime. Odd result, but it shows at least that this change shouldn't be made based on a hunch, but only after looking at some real results.
-Eric
Find will not benchmark atime, as it doesn't open the files for reading.
It will update the atime of the dirs it reads.
[esandeen@neon ext4]$ stat /tmp | grep Access | grep -v Uid Access: 2007-08-21 12:56:16.582910381 -0500 [esandeen@neon ext4]$ find /tmp > /dev/null [esandeen@neon ext4]$ stat /tmp | grep Access | grep -v Uid Access: 2007-08-21 12:56:56.762072721 -0500
I know it's a strange test for atime, but it did seem to have a detrimental effect on this particular workload (think updatedb).
Yes, there are certainly better tests, and I'm not saying "find" is a definitive test for noatime, it was more of a curiosity.
I am saying, people who want this should prove the gains they claim. Thanks for posting your numbers :)
-Eric
On 21/08/07, Eric Sandeen esandeen@redhat.com wrote:
I have at least one case where noatime actually slows things down. With 66 million inodes on an ext3 filesystem, "find" across the filesystem with a fresh mount / cold cache was a few seconds slower with noatime. Odd result, but it shows at least that this change shouldn't be made based on a hunch, but only after looking at some real results.
It's mainly of benefit where the atime updates would interleave with read traffic on the same disk, thus causing extra seeks, for example. Experiential (yeah, I know :)) evidence is that when there's a fair amount of disk traffic anyway, things like recursive grep or find finish faster, but also *other things* go faster when run at the same time. An isolated "benchmark" such as yours never shows this sort of effect. Whether the cache is warm or cold isn't so relevent; more's to the point, the savings show best on the second and subsequent accesses when the data to be read *is* cached and therefore there's no disk traffic at all.
Rahul Sundaram wrote:
Ralf Corsepius wrote:
whether they want it or not now, they will learn when they need to.
Provide the results of the tests you exercised yourself. So far you haven't.
I gave you a reference. If you want more data, do the tests yourself.
OK, I ran my own test doing an rpmbuild of a custom kernel. My %_topdir and %_tmppath point to a filesystem that is normally mounted noatime (it's primarily used as a news spool and work area for backups), and my /usr is normally mounted read-only. I tried several combinations and found no meaningful difference.
Build tree mounted noatime, /usr mounted read-only: real 32m28.765s user 43m11.627s sys 4m36.532s
Build tree mounted atime, /usr mounted read-only: real 32m39.343s user 42m55.705s sys 4m41.920s
Build tree mounted atime, /usr mounted rw,atime: real 32m26.383s user 43m1.042s sys 4m42.099s
Repeating with build tree noatime, /usr read-only: real 32m13.625s user 42m59.867s sys 4m39.544s
The tiny differences are totally masked by differences in the amount of time GPG took for key generation. The builds were done on a 3.0 GHz Pentium 4 with Hyperthreading enabled and 1 GB of RAM. The system was essentially idle except for the builds.
As for enabling atime when I find I want it, that would likely be when I want to see what files haven't been used in the last 6 months and I realize I should have enabled atime 6 months ago. Oops, too late now.
Once upon a time, Robert Nichols rnicholsNOSPAM@comcast.net said:
OK, I ran my own test doing an rpmbuild of a custom kernel.
This is being pitched as a "big gain for the desktop user". How many desktop users build a kernel? Where is the gain in the typical desktop usage pattern?
Someone said that Linux only follows POSIX where it makes sense, but when has Linux followed POSIX for years and _then_ turned away?
How many users truly see a security gain (real difference, not theoretical) out of SELinux? Why isn't that turned off for that last ounce of performance? I've spent a lot more time having to figure out what SELinux was doing and how to fix problems.
I use atime regularly, whether when using mutt or looking for what files have been accessed (to decide what is being used, what can be deleted, etc.). Okay, so there's a hack to make mutt work; how does it affect performance? stat() is about as simple as it gets for checking when a mailbox has been updated vs. when it was read. What about bash (and the other shells) that print a "you've got new mail" message? What else is there in the distribution that uses atime (has anyone checked)?
Sure, I can re-enable it on my systems, or I can go use something else that doesn't randomly disable standard Unix behavior because it is inconvenient.
Chris Adams wrote:
Once upon a time, Robert Nichols rnicholsNOSPAM@comcast.net said:
OK, I ran my own test doing an rpmbuild of a custom kernel.
This is being pitched as a "big gain for the desktop user". How many desktop users build a kernel? Where is the gain in the typical desktop usage pattern?
It's about the only file-intensive, fairly long-running, repeatable task I do on my system, and I don't do it often enough to really care how long it takes (within reason). And, if you actually read the rest of my message, you'll see I didn't find anything to pitch as a "big gain," or indeed, any gain.
Le dimanche 12 août 2007 à 05:02 +0200, Ralf Corsepius a écrit :
And how do you "fix" the "find"-based cron job users might have implemented to clean up /var/cache from files not being used for time span "x"?
Ralf,
Everyone can find pathological cases
atime killed one of my flash cards before I learnt to disable it by default (nowadays the system is supposed to detect flash and disable atime, no idea how foolproof the check is)
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
Going through the LKML thread on http://kerneltrap.org/node/14148 I learnt that Ubuntu has already turned it off, and in principle both Alan Cox and Ingo Molnar want it to be that way for Fedora 8 too.
Happy hacking, Debarshi
Debarshi 'Rishi' Ray wrote:
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
Going through the LKML thread on http://kerneltrap.org/node/14148 I learnt that Ubuntu has already turned it off, and in principle both Alan Cox and Ingo Molnar want it to be that way for Fedora 8 too.
Ubuntu doesn't have a decade+ history of putting user-controled settings under /etc/sysconfig and providing python scripts named redhat-config-xxx or system-config-xxx to manage them. Their style is to pretend they know more than you do about how your machine should be configured. Even if they are right, there is something to be said for consistency.
On 8/12/07, Les Mikesell lesmikesell@gmail.com wrote:
Debarshi 'Rishi' Ray wrote:
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
Going through the LKML thread on http://kerneltrap.org/node/14148 I learnt that Ubuntu has already turned it off, and in principle both Alan Cox and Ingo Molnar want it to be that way for Fedora 8 too.
Ubuntu doesn't have a decade+ history of putting user-controled settings under /etc/sysconfig and providing python scripts named redhat-config-xxx or system-config-xxx to manage them. Their style is to pretend they know more than you do about how your machine should be configured. Even if they are right, there is something to be said for consistency.
+1
Would it make sense to add it as an option to first-boot? Personally it is something I will try, but I have reservations about enforcing on everybody.
Andrew Parker wrote:
On 8/12/07, Les Mikesell lesmikesell@gmail.com wrote:
Debarshi 'Rishi' Ray wrote:
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
Going through the LKML thread on http://kerneltrap.org/node/14148 I learnt that Ubuntu has already turned it off, and in principle both Alan Cox and Ingo Molnar want it to be that way for Fedora 8 too.
Ubuntu doesn't have a decade+ history of putting user-controled settings under /etc/sysconfig and providing python scripts named redhat-config-xxx or system-config-xxx to manage them. Their style is to pretend they know more than you do about how your machine should be configured. Even if they are right, there is something to be said for consistency.
+1
Would it make sense to add it as an option to first-boot? Personally it is something I will try, but I have reservations about enforcing on everybody.
I have to wonder what they've done with the guy who put all those other settings under /etc/sysconfig or why he didn't do this one too a long time ago.
On Sun, 2007-08-12 at 11:08 +0200, Nicolas Mailhot wrote:
Le dimanche 12 août 2007 à 05:02 +0200, Ralf Corsepius a écrit :
And how do you "fix" the "find"-based cron job users might have implemented to clean up /var/cache from files not being used for time span "x"?
Ralf,
Everyone can find pathological cases
Well, what you call pathological cases, I call "generalization".
That said, I consider notime to lack generality, because it violates standards (POSIX), breaks applications and disables a feature of the OS.
atime killed one of my flash cards before I learnt to disable it by default (nowadays the system is supposed to detect flash and disable atime, no idea how foolproof the check is)
May-be notime should be made the default on flash drives?
atime is a legacy ass-backwards default That no one dared touching it so far does not make it less stupid We're making more user-impacting changes with less cause every release
Let's just disable atime asap so more testers check it before F8, and forget about it
As I already said many times before. It's trivial to implement "atime-based find" scripts:
You'll hardly find any in a default installation, because they hardly make much sense as part of a distro (exept. tmpwatch), but they make a lot of sense as part of individual installation.
E.g. * A cron job removing files not having been accessed for more than a month in /var/cache/yum. * Users wanting to remove "sound/data/image" files they have not listened/read/looked at for time xxx inside of their /home ...
Similar stuff also can easily be integrated into applications. When abandoning atime you'd break these applications and would force future applications to administrate time-stamps as part of the application instead of them being able to utilize the OSes resources.
Let's just disable atime asap so more testers check it before F8, and forget about it
IMO, atime should be the default. Instead, users should learn to use partitions (instead of lumping together everything into one partition and to apply individual mount-flags to them, for those people thinking notime is helpful to them - I don't find noatime helpful.
Ralf
On Mon, Aug 13, 2007 at 08:41:08AM +0200, Ralf Corsepius wrote:
That said, I consider notime to lack generality, because it violates standards (POSIX), breaks applications and disables a feature of the OS.
POSIX doesn't seem to mind.
As I already said many times before. It's trivial to implement "atime-based find" scripts:
And relatime is day accurate still
notime is helpful to them - I don't find noatime helpful.
I doubt it finds you helpful either ;)
Alan
On Fri, Aug 10, 2007 at 03:26:02 +0530, Rahul Sundaram sundaram@fedoraproject.org wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
mutt can also use maildir format which has some advantages over mbox. I don't know how commonly people using mutt use maildir instead of mbox though.
Thoughts on disabling it?
I disable it on my machines, though I use a couple with slow disks and not a lot of memory.
On Fri, Aug 10, 2007 at 10:45:00AM -0500, Bruno Wolff III wrote:
On Fri, Aug 10, 2007 at 03:26:02 +0530, Rahul Sundaram sundaram@fedoraproject.org wrote:
Hi
http://kerneltrap.org/node/14148
Looks like we get a good performance boost and only tmpwatch and mutt with mbox seems to be affected. A simple patch to tmpwatch has been posted on the same thread.
mutt can also use maildir format which has some advantages over mbox. I don't know how commonly people using mutt use maildir instead of mbox though.
mutt handles mbox with noatime if check_mbox_size option is enabled in config. There are some limitations, it doesn't work well when running two instances of mutt simultaneously, but it's usable.
-
Alan Cox -
Alexandru Ciobanu -
Aly Dharshi -
Andrew Parker -
Andy Lawrence -
Arthur Pemberton -
Benjamin Lewis -
Bill Crawford -
Bruno Wolff III -
Chris Adams -
Christopher Brown -
Colin Walters -
Debarshi Ray -
Douglas McClendon -
drago01 -
Eric Sandeen -
Florin Andrei -
James Hubbard -
John Reiser -
Josh Boyer -
Les Mikesell -
Martin Ebourne -
Matthew Miller -
Miloslav Trmač -
Miroslav Lichvar -
n0dalus -
Neal Becker -
Nicolas Mailhot -
Pete Zaitcev -
Rahul Sundaram -
Ralf Corsepius -
Robert Nichols -
Rudolf Kastl