OT What would be the best script type

[slim]

To do the following?

I just got one of those fancy-dancy gmail accounts, and while I aint
crazy about the email side of it, the one gig storage gave me an idea.

what Id like to do, is make a script that will gather certain dir. and
files for back up, split some tar.gz files at 9.8 meg each and email
them to my gmail account... the gmail account has a 10 meg limit on a
single message hence the need to split the file...

I can think of a way to do this via php or perl but I was thinking a
shell script would do it more efficiently... what do you all think?

Thanks
slim

[Mike Fedyk]

slim wrote:
>  To do the following?
>  
>  I just got one of those fancy-dancy gmail accounts, and while I aint
>  crazy about the email side of it, the one gig storage gave me an idea.
>  
>  what Id like to do, is make a script that will gather certain dir. and
>  files for back up, split some tar.gz files at 9.8 meg each and email
>  them to my gmail account... the gmail account has a 10 meg limit on a
>  single message hence the need to split the file... 
Well, since you're going to have to encode it, your files will have to 
be split into approx 4.9MB.  Base64 may be a bit more efficient though.

You could split the files, encode, mail, and keep the subjects for 
indexes pretty well with maybe a 50 line shell script, but that doesn't 
count dealing with http, ssl, and logging into the gmail account.

I would imagine that google wouldn't want you using gmail for a file 
store, so they'd probably add some dynamics into the data submission and 
retrieval systems...

Mike

[slim]

On Tue, 2004-06-29 at 02:36, Mike Fedyk wrote:
>  slim wrote:
>  > To do the following?
>  > 
>  > I just got one of those fancy-dancy gmail accounts, and while I aint
>  > crazy about the email side of it, the one gig storage gave me an idea.
>  > 
>  > what Id like to do, is make a script that will gather certain dir. and
>  > files for back up, split some tar.gz files at 9.8 meg each and email
>  > them to my gmail account... the gmail account has a 10 meg limit on a
>  > single message hence the need to split the file...
>  
>  Well, since you're going to have to encode it, your files will have to 
>  be split into approx 4.9MB.  Base64 may be a bit more efficient though.
>  
>  You could split the files, encode, mail, and keep the subjects for 
>  indexes pretty well with maybe a 50 line shell script, but that doesn't 
>  count dealing with http, ssl, and logging into the gmail account.
>  
>  I would imagine that google wouldn't want you using gmail for a file 
>  store, so they'd probably add some dynamics into the data submission and 
>  retrieval systems...
>  
>  Mike
>   
the file would be an attachment to an email so no login would be
required. basically the script would make a backup tar.gz it then just
attach it to a e-mail sent to gmail.

According to the gmail tos, it actually encourages this type of
attachment, I guess to test it out.  it just wont accept .exe files

thanks for your reply
slim

[Mike Klinke]

On Tuesday 29 June 2004 01:38, slim wrote:
>  To do the following?
>
>  I just got one of those fancy-dancy gmail accounts, and while I
>  aint crazy about the email side of it, the one gig storage gave me
>  an idea.
>
>  what Id like to do, is make a script that will gather certain dir.
>  and files for back up, split some tar.gz files at 9.8 meg each and
>  email them to my gmail account...  
 
Slim,

Just out of curiosity I wonder what advantage you find in storing such 
small amounts of back-up data on a machine you don't physically 
control unless, of course, it's neiter sensitive in nature nor very 
valuable to you.

Regards,  Mike Klinke

[slim]

On Tue, 2004-06-29 at 08:33, Mike Klinke wrote:
>  On Tuesday 29 June 2004 01:38, slim wrote:
>  > To do the following?
>  >
>  > I just got one of those fancy-dancy gmail accounts, and while I
>  > aint crazy about the email side of it, the one gig storage gave me
>  > an idea.
>  >
>  > what Id like to do, is make a script that will gather certain dir.
>  > and files for back up, split some tar.gz files at 9.8 meg each and
>  > email them to my gmail account... 
>   
>  Slim,
>  
>  Just out of curiosity I wonder what advantage you find in storing such 
>  small amounts of back-up data on a machine you don't physically 
>  control unless, of course, it's neiter sensitive in nature nor very 
>  valuable to you.
>  
>  Regards,  Mike Klinke 
Mike,
I dont physically control the servers as they are many, many miles
from me. :) I just was wanting to experiment writing scripts. I back up
all my data for real to a different partition on each server and keep
masters here at home.

[Ow Mun Heng]

On Tue, 2004-06-29 at 10:33, slim wrote:
>  On Tue, 2004-06-29 at 08:33, Mike Klinke wrote:
>  > On Tuesday 29 June 2004 01:38, slim wrote:
>  > > To do the following?
>  > >
>  > > I just got one of those fancy-dancy gmail accounts, and while I
>  > > aint crazy about the email side of it, the one gig storage gave me
>  > > an idea.
>  > >
>  > > what Id like to do, is make a script that will gather certain dir.
>  > > and files for back up, split some tar.gz files at 9.8 meg each and
>  > > email them to my gmail account... 
>  >  
>  > Slim,
>  > 
>  > Just out of curiosity I wonder what advantage you find in storing such 
>  > small amounts of back-up data on a machine you don't physically 
>  > control unless, of course, it's neiter sensitive in nature nor very 
>  > valuable to you.
>  > 
>  > Regards,  Mike Klinke
>  
>  Mike,
>  I dont physically control the servers as they are many, many miles
>  from me. :) I just was wanting to experiment writing scripts. I back up
>  all my data for real to a different partition on each server and keep
>  masters here at home. 
Well, you can either write a bash shell or perl script to do that. It's
faily simple depending on your needs (and what's your profienciency in
writing it. I'm more comfy with bash). To get them to be mailed to your
gmail account, you can get logwatch/cront do just that. 

And your question was...?/

--

[slim]

>  Well, you can either write a bash shell or perl script to do that.
> It's
>  faily simple depending on your needs (and what's your profienciency in
>  writing it. I'm more comfy with bash). To get them to be mailed to your
>  gmail account, you can get logwatch/cront do just that. 
>  
>  And your question was...?/ 
Hi, I was not asking any real question, just asking opinions
on what scripting language would do it most efficiently.

I have very small skills in perl and php and I can whip up small shell
scripts for what little I need. I want to to this to learn only.
Thanks for the reply, I'll check out the cront in more detail.

[Reuben D. Budiardja]

On Tuesday 29 June 2004 02:38 am, slim wrote:
>  To do the following?
>
>  I just got one of those fancy-dancy gmail accounts, and while I aint
>  crazy about the email side of it, the one gig storage gave me an idea.
>
>  what Id like to do, is make a script that will gather certain dir. and
>  files for back up, split some tar.gz files at 9.8 meg each and email
>  them to my gmail account... the gmail account has a 10 meg limit on a
>  single message hence the need to split the file...
>
>  I can think of a way to do this via php or perl but I was thinking a
>  shell script would do it more efficiently... what do you all think? 
That should not be too hard, certainly do-able with something like PHP. But in 
shell script, all you need would be the  following commands:

tar
gzip (maybe able to combine tar and gzip)
split 
mutt (to send file as attachment. I'm not sure if you can do it easily with 
'mail').

You may need to loop over the resulting pieces file from split, then call 
'mutt' inside the script.

I'd do it in PHP, since I know it better than shell (or bash), and just do 
system() or exex() for calling tar, gzip and split. Efficiency should not be 
a problem with PHP running in command line mode. 

RDB
-- 
Reuben D. Budiardja
Department of Physics and Astronomy
The University of Tennessee, Knoxville, TN
---------------------------------------------------------
"To be a nemesis, you have to actively try to destroy 
something, don't you? Really, I'm not out to destroy 
Microsoft. That will just be a completely unintentional 
side effect."
                 - Linus Torvalds -

[slim]

On Tue, 2004-06-29 at 12:50, Reuben D. Budiardja wrote:
>  On Tuesday 29 June 2004 02:38 am, slim wrote:
>  > To do the following?
>  >
>  > I just got one of those fancy-dancy gmail accounts, and while I aint
>  > crazy about the email side of it, the one gig storage gave me an idea.
>  >
>  > what Id like to do, is make a script that will gather certain dir. and
>  > files for back up, split some tar.gz files at 9.8 meg each and email
>  > them to my gmail account... the gmail account has a 10 meg limit on a
>  > single message hence the need to split the file...
>  >
>  > I can think of a way to do this via php or perl but I was thinking a
>  > shell script would do it more efficiently... what do you all think?
>  
>  That should not be too hard, certainly do-able with something like PHP. But in 
>  shell script, all you need would be the  following commands:
>  
>  tar
>  gzip (maybe able to combine tar and gzip)
>  split 
>  mutt (to send file as attachment. I'm not sure if you can do it easily with 
>  'mail').
>  
>  You may need to loop over the resulting pieces file from split, then call 
>  'mutt' inside the script.
>  
>  I'd do it in PHP, since I know it better than shell (or bash), and just do 
>  system() or exex() for calling tar, gzip and split. Efficiency should not be 
>  a problem with PHP running in command line mode. 
>  
>  RDB
>  -- 
>  Reuben D. Budiardja
>  Department of Physics and Astronomy
>  The University of Tennessee, Knoxville, TN
>  ---------------------------------------------------------
>  "To be a nemesis, you have to actively try to destroy 
>  something, don't you? Really, I'm not out to destroy 
>  Microsoft. That will just be a completely unintentional 
>  side effect."
>                   - Linus Torvalds -
>   
Yep I was looking at some ways in php to do this. but I cant get by
the fact that I'll more than likely need two scripts.

one to do the backup, zip and split and one to retrieve the files and
send via e-mail...

Thanks

[Reuben D. Budiardja]

On Tuesday 29 June 2004 02:06 pm, slim wrote:
>  On Tue, 2004-06-29 at 12:50, Reuben D. Budiardja wrote:
>  > On Tuesday 29 June 2004 02:38 am, slim wrote: 
<snip>

>  > > what Id like to do, is make a script that will gather
> certain dir. and
>  > > files for back up, split some tar.gz files at 9.8 meg each and email
>  > > them to my gmail account. 
<sni>
>  > I'd do it in PHP, since I know it better than shell (or
> bash), and just
>  > do system() or exex() for calling tar, gzip and split. Efficiency should
>  > not be a problem with PHP running in command line mode.
>  >
>
>  Yep I was looking at some ways in php to do this. but I cant get by
>  the fact that I'll more than likely need two scripts.
>
>  one to do the backup, zip and split and one to retrieve the files and
>  send via e-mail... 
I fail to understand why you need two scripts. If you use exec() or system() 
in PHP, the script won't continue until it returns.

RDB
-- 
Reuben D. Budiardja
Department of Physics and Astronomy
The University of Tennessee, Knoxville, TN

[Jeff Vian]

On Tue, 2004-06-29 at 13:06, slim wrote:
>  On Tue, 2004-06-29 at 12:50, Reuben D. Budiardja wrote:
>  > On Tuesday 29 June 2004 02:38 am, slim wrote:
>  > > To do the following?
>  > >
>  > > I just got one of those fancy-dancy gmail accounts, and while I aint
>  > > crazy about the email side of it, the one gig storage gave me an idea.
>  > >
>  > > what Id like to do, is make a script that will gather certain dir. and
>  > > files for back up, split some tar.gz files at 9.8 meg each and email
>  > > them to my gmail account... the gmail account has a 10 meg limit on a
>  > > single message hence the need to split the file...
>  > >
>  > > I can think of a way to do this via php or perl but I was thinking a
>  > > shell script would do it more efficiently... what do you all think?
>  > 
>  > That should not be too hard, certainly do-able with something like PHP. But in 
>  > shell script, all you need would be the  following commands:
>  > 
>  > tar
>  > gzip (maybe able to combine tar and gzip)
>  > split 
>  > mutt (to send file as attachment. I'm not sure if you can do it easily with 
>  > 'mail').
>  > 
>  > You may need to loop over the resulting pieces file from split, then call 
>  > 'mutt' inside the script.
>  > 
>  > I'd do it in PHP, since I know it better than shell (or bash), and just do 
>  > system() or exex() for calling tar, gzip and split. Efficiency should not be 
>  > a problem with PHP running in command line mode. 
>  > 
>  > RDB
>  > -- 
>  > Reuben D. Budiardja
>  > Department of Physics and Astronomy
>  > The University of Tennessee, Knoxville, TN
>  > ---------------------------------------------------------
>  > "To be a nemesis, you have to actively try to destroy 
>  > something, don't you? Really, I'm not out to destroy 
>  > Microsoft. That will just be a completely unintentional 
>  > side effect."
>  >                  - Linus Torvalds -
>  > 
>  
>  Yep I was looking at some ways in php to do this. but I cant get by
>  the fact that I'll more than likely need two scripts.
>  
>  one to do the backup, zip and split and one to retrieve the files and
>  send via e-mail...
>  
>  Thanks
>  
>   
I would expect that you would (for programming simplicity) need at least
2 scripts.  One that handles the creation / splitting, and one that is
called later with the number/names of the files created for sending.

One approach I would use would be similar to what mondoarchive does. 
Create fixed size (nearly) files on the fly while processing and then
you do not even need to do the split, but could pass the directory name
as an argument and simply send all in the directory  by mail.