fopen and memory usage

Gilboa Davara gilboad at gmail.com
Thu Sep 27 07:26:23 UTC 2007


On Wed, 2007-09-26 at 13:58 -0400, aragonx at dcsnow.com wrote:
> >> No, it doesn't. What purpose would that serve?  It will cost you a
> >> buffer
> >> for the I/O (perhaps 8192 bytes) and a FILE control structure.
> >
> > If I remember rightly its nearer 64K nowdays simply because disks are
> > faster, memory is cheaper and its a good buffer size.
> >
> > Basically the C library will make an intelligent decision on buffering
> > which you should assume is right exept in very special cases. That value
> > may change depending on library version, kernel, system memory, OS and a
> > million other things but its there in the library to save you having to
> > worry about it in the app.
> >
> > If you need to do specific buffering see "man setbuf"
> 
> Thank you for your replies.  I think I have a clear understanding of what
> is going on but let me air what I think, then you can tell me how stupid I
> am.  :)
> 
> Okay, so I use fopen and it gives me a stream.  Now when I call fread (say
> for 100K) on my large file (2G) it will try to read the first 64K
> regardless of size and populate *ptr.  If size is larger than 64K (100K
> here) it will then read another 64K?  Does it use the same buffer space
> (effectively only using 64K) and is that buffer space outside the memory
> footprint of my application?
> 
> Again, thank you for all of your help.
> 

Side question... do you -require- buffering?
Using fXXXX on large files may produce noticeable performance hit.
If you don't, open (2) may produce (far) better results. (As the price
of having to do string parsing/new line searching yourself)

- Gilboa





More information about the users mailing list