off topic: combined output of concurrent processes

[Michael D. Setzer II]

On 14 Apr 2012 at 9:45, John Wendel wrote:

Date sent:      	Sat, 14 Apr 2012 09:45:50 -0700
From:           	John Wendel <jwendel10 at comcast.net>
To:             	users at lists.fedoraproject.org
Subject:        	Re: off topic: combined output of concurrent 
processes

> On 04/14/2012 08:20 AM, Amadeus W.M. wrote:
> >> If you really would like to get output in sequence, write to a
> >> pipe, and have a reader process drain the pipe to a logfile.  It's
> >> pretty easy; look at "mknod" with the 'p' option, or "mkfifo".  I'd
> >> still suggest tagging each output line with an identifier and
> >> sequence number.
> >>
> > For the sake of the argument, assume I echo 500 As, 500 Bs and 500
> > Cs.
> >
> > I don't care which process the output is coming from. It doesn't
> > matter which order the As, Bs and Cs are output. All I care about is
> > that I don't get 349As followed by 245Bs, etc. I want to see blocks
> > of 500 each.
> >
> > I don't see how echoing into a pipe would change the problem.
> > Theoretically, if several processes (e.g. echo) are running in the
> > background, e.g. on a round robin basis, then potentially I could
> > see random sequences of As, Bs and Cs. It doesn't seem to be the
> > case in practice though. So which is it?
> >
> > This has to do with the operating system internals, it's not a
> > trivial question.
> >
> Actually it is semi-non-trivial. :-)
> 
> Unix/Linux makes the following guarantee ...
> 
> Multiple processes that open the same file for writing each maintain
> their own file positions, so they may overwrite the output of another
> process, unless the processes all open the file with the "O_APPEND"
> option. With the "O_APPEND" option, the system guarantees that the
> entire data from a single write by a process will be written to the
> end of the file as a indivisible block and will not be mixed with the
> output from another process. Without "O_APPEND", data from multiple
> processes may be intermixed in any order, or may seem to "disappear"
> (is overwritten by other data).
> 
> Of course, if you didn't write the code that is doing the output,
> you'll have to examine the source to see if it uses the "O_APPEND"
> open option. This may be non-trivial.
> 

Interesting thread. Did the following modified test to see what I 
would get, and at the end it does get 100 100 600 for the As, Bs, 
and Cs by using a file for the output instead of stdout. 

The resulting file does a somewhat random pattern of the 5 
character blocks. This was run on my quad core system.

#!/bin/bash
rm ./outtest
i=0
while [ $i -lt 100 ];
do
    echo "AAAAA" >>./outtest &    # e.g. 500 As
    echo "BBBBB" >>./outtest & 
    echo "CCCCC" >>./outtest & 

    i=$(($i+1))
done
sleep 5
grep ^A ./outtest |wc
grep ^B ./outtest |wc
grep ^C ./outtest |wc



> Regards,
> 
> John
> 
> -- 
> users mailing list
> users at lists.fedoraproject.org
> To unsubscribe or change subscription options:
> https://admin.fedoraproject.org/mailman/listinfo/users
> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have
> a question? Ask away: http://ask.fedoraproject.org


+----------------------------------------------------------+
  Michael D. Setzer II -  Computer Science Instructor      
  Guam Community College  Computer Center                  
  mailto:mikes at kuentos.guam.net                            
  mailto:msetzerii at gmail.com
  http://www.guam.net/home/mikes
  Guam - Where America's Day Begins                        
  G4L Disk Imaging Project maintainer 
  http://sourceforge.net/projects/g4l/
+----------------------------------------------------------+

http://setiathome.berkeley.edu (Original)
Number of Seti Units Returned:  19,471
Processing time:  32 years, 290 days, 12 hours, 58 minutes
(Total Hours: 287,489)

BOINC at HOME CREDITS
SETI        12076418.310558   |   EINSTEIN     7674404.639852
ROSETTA      4421834.189334   |   ABC         12313717.425001