parallel bash scripts

Wed Mar 28 22:20:07 UTC 2012

On 03/27/2012 10:35 PM, bruce wrote:
> Hi mark.
>
> I have a series of php apps that are async, and modify different tbls
> within a mysql db. The php apps also perform other tasks as well.
>
> The process needs to be able to periodically run the php apps. The
> shell scripts are used to run the individual php apps. During testing,
> I wanted to be able to run the apps, in a background manner, as well
> as display the output for debugging.
>
> The system has ~5-10 php apps that will be running.
>
> The overall goal for all of this is part of a targeted web crawler,
> where the php apps perform backend processing to manage the data from
> the crawling process as it gets stored in the database.
>
> So the process should fire up all of the processes, and they should
> then run every X secs... At least thats the way I ultimately want it
> to work. But I'm unable to figure out how to do this, and at the same
> time, display the output for the debugging process...
>
> Hopefully I've done a better job of explaining some of what this is for..
>
>
>
> On Tue, Mar 27, 2012 at 10:23 PM, Mark LaPierre<marklapier at aol.com>  wrote:
>> I don't understand what you are trying to do.
>>
>> Do you want to run these commands in series?  Run one, wait for it to
>> complete, then start the other?
>>
>> If that's where you want to go then leave off the trailing ampersand but
>> keep the redirection.  The output of the first program will appear in the
>> terminal due to the redirection.  When the first program ends the second
>> program will start.  That one needs redirection of standard out to standard
>> error too so that it's output will appear in the terminal as well.
>>
>> Solution with trailing ampersand is appropriate for endless loops.
>> Solution without trailing ampersand is appropriate for serial operation.
>>
>> I put the loop counter in the demo program just to stop the thing without
>> having to us ps and kill.  Without the loop counter they will continue to
>> print inter-twined messages forever.
>>
>> Does the program webservice_setup.sh start programs itself that you are
>> trying to read the output from?  Are they correctly redirected to send their
>> standard output to standard error?
>>
>>
>> On 03/27/2012 10:01 PM, bruce wrote:
>>>
>>> Hi mark.
>>>
>>> the following is part of what i have tried... and what isn't working.
>>> i changed the test shell scripts to no longer have infinite loops.
>>> they simply execute the php app one time.
>>>
>>> however, it still doesn't work.
>>>
>>> in the initial shell script, it calls a php app that does some mysql
>>> processes. the php app has a number of echo/print msgs to track what's
>>> going on for testing..  when running as a background process the shell
>>> script/php app isn't working.
>>>
>>> the cmdline sits.. when i select the "return" key.. the process is
>>> diplayed as being stopped...
>>>
>>> if i remove the background "&" the shell script runs...
>>>
>>> #### -- not working
>>> /apps/parseapp2/webservice_setup.sh    1>&2&
>>>
>>> #### -- works
>>> /apps/parseapp2/webservice_setup.sh
>>>
>>> /apps/parseapp2/webservice_setup.sh
>>> -------------------------------
>>> echo "starting the osu webservice test"
>>> mysql -uroot1 -pfoo<    /apps/parseapp2/setup_tblDb.sql
>>> mysql -uroot1 -pfoo<    /apps/parseapp2/webservice/webservice_tbls.sql
>>> /apps/parseapp2/parse_tbl_setup.php
>>> echo "started the osu webservice test"
>>>
>>>
>>> thoughts???
>>>
>>> thanks
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 27, 2012 at 9:37 PM, Mark LaPierre<marklapier at aol.com>    wrote:
>>>>
>>>> Check out the redirection at the end of each command.  1>&2 redirects the
>>>> standard out of your child command to the standard error which then
>>>> appears
>>>> in the parent shell.  At the end the last&    launches your command into a
>>>>
>>>> background shell and then moves on to launch the next command.  The
>>>> redirections don't care if the command ever terminates.
>>>>
>>>> The result is that both commands are launched and the parent shell
>>>> terminates leaving the standard error attached to the terminal that the
>>>> parent was launched in.
>>>>
>>>>
>>>> On 03/27/2012 09:08 PM, bruce wrote:
>>>>>
>>>>>
>>>>> marklapier at aol.com
>>>>>
>>>>> hey mark....
>>>>>
>>>>> what you have, appears to be pretty close to what i had... except my
>>>>> tests never ended... the loops are infinite...
>>>>>
>>>>> can i do a fpaste and have you take a look at what i have?
>>>>>
>>>>> -btuce
>>>>>
>>>>>
>>>>> On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre<marklapier at aol.com>
>>>>>   wrote:
>>>>>>
>>>>>>
>>>>>> On 03/27/2012 05:25 PM, bruce wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> hi.
>>>>>>>
>>>>>>> got a couple of test bash scripts.
>>>>>>>
>>>>>>> dog.sh, cat.sh
>>>>>>> each script runs the underlying php in an endless loop.
>>>>>>>
>>>>>>> I'm trying to figure out how to run the scripts in parallel, from the
>>>>>>> same parent shell script. something like:
>>>>>>>
>>>>>>> test.sh
>>>>>>>
>>>>>>>
>>>>>>> where dog.sh would be :
>>>>>>> --------------------------------
>>>>>>> while true
>>>>>>> do
>>>>>>>    pgrep dog
>>>>>>>    if [ $? -ne 0 ]
>>>>>>>    then
>>>>>>>      /dog.php
>>>>>>>    fi
>>>>>>> sleep 5
>>>>>>> done
>>>>>>>
>>>>>>> my current tests, run dog.sh, which runs the dog.php ... but the test
>>>>>>> never gets to run cat.sh
>>>>>>>
>>>>>>> thoughts/comments...
>>>>>>>
>>>>>>> thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hey Bruce,
>>>>>>
>>>>>> Do you mean to run these subprograms in parallel or in series?
>>>>>>
>>>>>> cat.sh
>>>>>> #! /bin/bash
>>>>>>
>>>>>> CAT=0
>>>>>> until [ $CAT -eq 10 ]
>>>>>> do
>>>>>> echo "Inside a dog it's too dark to read. $CAT"
>>>>>> CAT=$[$CAT + 1]
>>>>>> sleep 2
>>>>>> done
>>>>>>
>>>>>>
>>>>>> dog.sh
>>>>>> #! /bin/bash
>>>>>>
>>>>>> DOG=0
>>>>>> until [ $DOG -eq 10 ]
>>>>>> do
>>>>>> echo "Next to a dog a book is man's best friend. $DOG"
>>>>>> DOG=$[$DOG + 1]
>>>>>> sleep 2
>>>>>> done
>>>>>>
>>>>>>
>>>>>> test.sh
>>>>>> #! /bin/sh
>>>>>>
>>>>>> /home/mlapier/test/dog.sh 1>&2&
>>>>>> /home/mlapier/test/cat.sh 1>&2&
>>>>>>
>>>>>>
>>>>>> [mlapier at mushroom test]$ ./test.sh
>>>>>> [mlapier at mushroom test]$ Next to a dog a book is man's best friend. 0
>>>>>> Inside a dog it's too dark to read. 0
>>>>>> Next to a dog a book is man's best friend. 1
>>>>>> Inside a dog it's too dark to read. 1
>>>>>> Next to a dog a book is man's best friend. 2
>>>>>> Inside a dog it's too dark to read. 2
>>>>>> Next to a dog a book is man's best friend. 3
>>>>>> Inside a dog it's too dark to read. 3
>>>>>> Next to a dog a book is man's best friend. 4
>>>>>> Inside a dog it's too dark to read. 4
>>>>>> Next to a dog a book is man's best friend. 5
>>>>>> Inside a dog it's too dark to read. 5
>>>>>> Next to a dog a book is man's best friend. 6
>>>>>> Inside a dog it's too dark to read. 6
>>>>>> Next to a dog a book is man's best friend. 7
>>>>>> Inside a dog it's too dark to read. 7
>>>>>> Next to a dog a book is man's best friend. 8
>>>>>> Inside a dog it's too dark to read. 8
>>>>>> Next to a dog a book is man's best friend. 9
>>>>>> Inside a dog it's too dark to read. 9
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> users mailing list
>>>>>> users at lists.fedoraproject.org
>>>>>> To unsubscribe or change subscription options:
>>>>>> https://admin.fedoraproject.org/mailman/listinfo/users
>>>>>> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>>> Have a question? Ask away: http://ask.fedoraproject.org
>>>>
>>>>
>>>>
>>