NFS Buffering

[JB]

JB <jb.1234abcd <at> gmail.com> writes:

> ...

How nfsiod's number of read-ahead/write-behind worker threads is determined
and configured ?

$ cat linux-2.6.34.i686/fs/nfs/internal.h
/*
 * NFS internal definitions
 */
...
/* Maximum number of readahead requests
 * FIXME: this should really be a sysctl so that users may tune it to suit
 *        their needs. People that do NFS over a slow network, might for
 *        instance want to reduce it to something closer to 1 for improved
 *        interactive response.
 */
#define NFS_MAX_READAHEAD       (RPC_DEF_SLOT_TABLE - 1)
...

$ cat linux-2.6.34.i686/include/linux/sunrpc/xprt.h
/*
 ... 
 *  Declarations for the RPC transport interface.
 ...
*/
...
#define RPC_DEF_SLOT_TABLE      (16U)
...

So, the max # of workers is 16 - 1 = 15

See: linux-2.6.34.i686/Documentation/kernel-parameters.txt
        ...
        sunrpc.tcp_slot_table_entries=
        sunrpc.udp_slot_table_entries=
                        [NFS,SUNRPC]
                        Sets the upper limit on the number of simultaneous
                        RPC calls that can be sent from the client to a
                        server. Increasing these values may allow you to
                        improve throughput, but will also increase the
                        amount of memory reserved for use by the client.
        ...

$ lsmod |grep -i sunrpc
sunrpc                163601  1
$ $ modinfo sunrpc
filename:       /lib/modules/2.6.34.7-56.fc13.i686/kernel/net/sunrpc/sunrpc.ko
license:        GPL
srcversion:     274F8EC6B56054A06EDF2A4
depends:        
vermagic:       2.6.34.7-56.fc13.i686 SMP mod_unload 686 
parm:           min_resvport:portnr
parm:           max_resvport:portnr
parm:           tcp_slot_table_entries:slot_table_size
parm:           udp_slot_table_entries:slot_table_size

$ ls /sys/module/sunrpc/parameters/
max_resvport  pool_mode               udp_slot_table_entries
min_resvport  tcp_slot_table_entries
$ cat /sys/module/sunrpc/parameters/udp_slot_table_entries
16
$ cat /sys/module/sunrpc/parameters/tcp_slot_table_entries 
16

Managing NFS and NIS, 2nd Edition.
By Hal Stern, Mike Eisler and Ricardo Labiaga
18.5. NFS async thread tuning.
...
If you are running eight NFS async threads on an NFS client, then the client
will generate eight NFS write requests at once when it is performing
a sequential write to a large file. The eight requests are handled by the NFS
async threads. ... when a Solaris process issues a new write requests while
all the NFS async threads are blocked waiting for a reply from the server,
the write request is queued in the kernel and the requesting process returns
successfully without blocking. The requesting process does not issue an RPC to
the NFS server itself, only the NFS async threads do. When an NFS async thread
RPC call completes, it proceeds to grab the next request from the queue and
sends a new RPC to the server. It may be necessary to reduce the number of NFS
requests if a server cannot keep pace with the incoming NFS write requests.
...

And for those who are not into NFS ...

Lesley Gore - It's My Party (1965)
http://www.youtube.com/watch?v=XsYJyVEUaC4

JB