On 10/22/2016 02:39 PM, bruce wrote:
On Sat, Oct 22, 2016 at 4:00 PM, jd1008 jd1008@gmail.com wrote:
On 10/22/2016 12:14 PM, Gordon Messmer wrote:
On 10/22/2016 06:13 AM, bruce wrote:
I've thought of having a "pid" file on the nfs, where each clientApp would reset/lock/use the pidFile to then get/use/delete the file as required, but this seems tb slow.. but doable.
Probably. You'd have a bunch of clients periodically checking for the lock file before they do work. You'll lose some time to idling when one application finishes, before another application gets the lock and starts working. You may be better off having one process scheduling work, starting up or signaling an application when the data is available. You'll avoid the complexity of the locking, and possibly reduce idle time.
So, you are suggesting: One process on the nfs server that knows about all the files in question and waits for clients to ask for a file (any file????), returns back to that nfs client an nfs handle for some specific file chosen by the server, creates some flag somewhere (on the server) that the file has been served to username xyz, uid cde, on client a.b.c. When is finished with that file, does something like ask the server to delete that file, or says to the server I am done with that file; at which point server might delete the file per client request, or simply remove the flag that the file is in use by username xyz, uid cde, on client a.b.c, ... and so on. Well, now you have another wait problem for each request for some unserved file. In such a serial service of requests - clients might time out, and can err out or simply retry. What if the list of files is in the thousands? Serially serving said files would be very time consuming.
Parallel server processes: The server process might be coded so that for each request, it forks a child process to serve the request. Now, child processes have to compete for a lock on the list of files to be served, one of the children will succeed, and set the flag (mark username xyz, uid cde, on client a.b.c has file /filename/), serve the file, unlock the list and exit. Same client that requested the file must then request the server to delete the file or tell the server it is done with the file - again requiring the child server process (not same child process that served the file, but a new child spawned to handle the request) to lock the list + delete the file or simply remove the flag of the client.
On the surface of it, it sounds like this is a slow process. But it is not.
A sever process that spawns children that compete for the list is a good way to serve said files.
Hey JD!
Thanks for the reply.
I thought about having some sort of "nfs/server" side process that iterates through the list of files, and then determines when a "client" is ready, and pushes the file to the client.
In having the clients, pull the file (set the pidFILE), there's the possibility that the pidFile, could be messed up, along with a few other minor issues, but it would be simple to implement.
Thanks
-b
Hi Bruce, Client's .pid files are not shared. They only exist on the client. So, client A would inevitable end up processing the same file as client B.
I suggest you write a small daemon on the nfs server that does what I suggest. The "list" is simply an array of strings (array of pointers to null terminated character strings.). This array should be in an include file that also exists on the client that will build the client application code. Once built it can be installed on all clients. the server process will also include this file. The server process should 1. setup the socket for listening to requests. 2. loop listen for incoming requests accept the request (which contains data of the nature of the request) spawn a thread and pass to it the acceptance socket (the data of which the thread will process). endloop
3. The thread will parse the request to see if it is a request for a file or a request to close a file or a request to delete a file. The thread will use sychronization primitives to lock the list mentioned above and perform the request. If the request is for a file, then it needs to open that file for reading, and return an nfs handle to the client, do some overhead, like create the flag of client info (username, uid, client ip), then unlock the list and return. The parent process will not wait for the child thread, as it must simply accept connections and spawn the same thread to handle the request. Multiply (I do not mean mathematical 'multiply', I mean more than one), spawned threads are independent of each other. They only share the the list and the list's lock primitives.
The client will use that handle as a file descriptor and process the file. When client is done, client will request to a: close the file or b: close and delete the file.
That's it.