wget --reject option? downloads then deletes - bug/feature?

Morgan Read mstuff at read.org.nz
Tue Oct 17 02:57:48 UTC 2006


Hello

I'm running wget with option --reject, expecting that the files are skipped from
downloading, but instead they're downloaded then deleted.  The whole point of
using the option was to avoid downloading a database which runs to over 12,000
files (before I terminated wget!).

Is this correct behaviour?  Does anyone know a command to download the contents
(linked to by some page) of some directory, but with out some files defined by
some file pattern?

Below is the command as run, prior to being terminated (ctrl-c).

Thanks,
Morgan.

##########################
[morgan at morgansmachine ~]$ wget -r -E -k -nc -p -w 1 --random-wait
--reject="*table*" -I /naftadatabase
http://www.worldtradelaw.net/nafta/naftamain.htm
--20:14:31--  http://www.worldtradelaw.net/nafta/naftamain.htm
           => `www.worldtradelaw.net/nafta/naftamain.htm'
Resolving www.worldtradelaw.net... 65.123.204.61
Connecting to www.worldtradelaw.net|65.123.204.61|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7,095 (6.9K) [text/html]

100%[====================================>] 7,095         23.89K/s

20:14:32 (23.82 KB/s) - `www.worldtradelaw.net/nafta/naftamain.htm' saved
[7095/7095]

Loading robots.txt; please ignore errors.
--20:14:34--  http://www.worldtradelaw.net/robots.txt
           => `www.worldtradelaw.net/robots.txt'
Reusing existing connection to www.worldtradelaw.net:80.
HTTP request sent, awaiting response... 200 OK
Length: 30 [text/plain]

100%[====================================>] 30            --.--K/s

20:14:34 (751.20 KB/s) - `www.worldtradelaw.net/robots.txt' saved [30/30]

--20:14:34--  http://www.worldtradelaw.net/naftadatabase/nafta19.asp
           => `www.worldtradelaw.net/naftadatabase/nafta19.asp'
Reusing existing connection to www.worldtradelaw.net:80.
HTTP request sent, awaiting response... 200 OK
Length: 46,960 (46K) [text/html]

100%[====================================>] 46,960        73.69K/s

20:14:36 (73.51 KB/s) - `www.worldtradelaw.net/naftadatabase/nafta19.asp.html'
saved [46960/46960]

--20:14:37--  http://www.worldtradelaw.net/naftadatabase/naftaecc.asp

...
<snip>
...

--20:15:12--  http://www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:1;
          => `www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:1;'
Reusing existing connection to www.worldtradelaw.net:80.
HTTP request sent, awaiting response... 200 OK
Length: 4,411 (4.3K) [text/html]

100%[====================================>] 4,411         --.--K/s

20:15:13 (378.82 KB/s) -
`www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:1;.html' saved [4411/4411]

Removing www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:1;.html since
it should be rejected.
--20:15:13--  http://www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:2;
          => `www.worldtradelaw.net/naftadatabase/nafta20.asp?table1=s:2;'
Reusing existing connection to www.worldtradelaw.net:80.
HTTP request sent, awaiting response...
[morgan at morgansmachine ~]$
##########################
-- 
Morgan Read
NEW ZEALAND
<mailto:mstuffATreadDOTorgDOTnz>

fedora: Freedom Forever!
http://fedoraproject.org/wiki/Overview

"By choosing not to ship any proprietary or binary drivers, Fedora does differ
from other distributions. ..."
Quote: Max Spevik
       http://interviews.slashdot.org/article.pl?sid=06/08/17/177220



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 251 bytes
Desc: OpenPGP digital signature
Url : http://lists.fedoraproject.org/pipermail/users/attachments/20061017/06d7bcdc/attachment-0002.bin 


More information about the users mailing list