hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done > wl1
on my aspire one (which is running rawhide) it kept crashing my system down with no reasons shown in any of the logs, so i tried it on my desktop machine which is an athlonx2 6000 with 2GB of ram and a raid0 array, with 2GB of swap and it too crashed the terminal but it did give me an error and it turned out to be an oom error so i tried upping the swap partition to 10GB and i'm still getting an oom crash, can anyone suggest a way of getting the command to dump the output to disk each time it gets close to running out of memory? or is this command always doomed to fail?
tia phil
2009/10/4 psmith psmith@fedoraproject.org:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done > wl1
Consider that for a second...
You are trying to generate a list of all possible combinations of an 8 character word composed of only uppercase letters - that's 26^8 combinations (208 Billion). Each word is 8 bytes long, which I make to be 1670616516608 bytes... or to put it another way, 1.5 TB
So, you've got 4 GB of virtual memory and you are trying to fit an 1555GB array into it. Simple mathematics says no. Dumping the arguments before it dies is pointless, because it hasn't even got as far as expanding arguments yet.
You need to think of another was to do this and I humbly suggest that Bash should not be high on your list.
On 04/10/09 11:17, Sharpe, Sam J wrote:
2009/10/4 psmithpsmith@fedoraproject.org:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done> wl1
Consider that for a second...
You are trying to generate a list of all possible combinations of an 8 character word composed of only uppercase letters - that's 26^8 combinations (208 Billion). Each word is 8 bytes long, which I make to be 1670616516608 bytes... or to put it another way, 1.5 TB
So, you've got 4 GB of virtual memory and you are trying to fit an 1555GB array into it. Simple mathematics says no. Dumping the arguments before it dies is pointless, because it hasn't even got as far as expanding arguments yet.
You need to think of another was to do this and I humbly suggest that Bash should not be high on your list.
thanks for the reality check sam :o
do you have any suggestions what/where i should be looking to get my required output?
phil
On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
On 04/10/09 11:17, Sharpe, Sam J wrote:
2009/10/4 psmithpsmith@fedoraproject.org:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done> wl1
Consider that for a second...
You are trying to generate a list of all possible combinations of an 8 character word composed of only uppercase letters - that's 26^8 combinations (208 Billion). Each word is 8 bytes long, which I make to be 1670616516608 bytes... or to put it another way, 1.5 TB
So, you've got 4 GB of virtual memory and you are trying to fit an 1555GB array into it. Simple mathematics says no. Dumping the arguments before it dies is pointless, because it hasn't even got as far as expanding arguments yet.
You need to think of another was to do this and I humbly suggest that Bash should not be high on your list.
thanks for the reality check sam :o
do you have any suggestions what/where i should be looking to get my required output?
As Sam says, your initial attempt is trying to produce all 1.5TB at once, which is why it blows up, but that's going to happen anyway even if you program the same thing in C. IOW it's not an issue with the Shell per se, but with the approach to the problem.
Assuming you actually want all 1.5TB of data (really?) you'd be better consuming it as it's produced, perhaps via a pipe. Of course that depends on what you're doing with it, which you haven't said.
poc
Patrick O'Callaghan wrote:
On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
do you have any suggestions what/where i should be looking to get my required output?
As Sam says, your initial attempt is trying to produce all 1.5TB at once, which is why it blows up, but that's going to happen anyway even if you program the same thing in C. IOW it's not an issue with the Shell per se, but with the approach to the problem.
Assuming you actually want all 1.5TB of data (really?) you'd be better consuming it as it's produced, perhaps via a pipe. Of course that depends on what you're doing with it, which you haven't said.
poc
One way to do it would be to break it down into nested loops. I am not sure you could do it in bash - I am not sure how bash handles loops. You can try something like: (Not tested)
for a in {A..Z}; \ do for b in {A..Z}; \ do for c in {A..Z}; \ do for d in {A..Z}; \ do for e in {A..Z}; \ do for f in {A..Z}; \ echo $a$b$c$d$e$f ; \ done ; done ; done ; done ; done ; done ; done ; done
Mikkel
you might consider piping your echo $w | xargs...
On 10/04/2009 02:42 AM, psmith wrote:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done > wl1
on my aspire one (which is running rawhide) it kept crashing my system down with no reasons shown in any of the logs, so i tried it on my desktop machine which is an athlonx2 6000 with 2GB of ram and a raid0 array, with 2GB of swap and it too crashed the terminal but it did give me an error and it turned out to be an oom error so i tried upping the swap partition to 10GB and i'm still getting an oom crash, can anyone suggest a way of getting the command to dump the output to disk each time it gets close to running out of memory? or is this command always doomed to fail?
tia phil
On 09-10-04 06:17:20, Sharpe, Sam J wrote:
2009/10/4 psmith psmith@fedoraproject.org:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo
$w;done >
wl1
Consider that for a second...
You are trying to generate a list of all possible combinations of an 8 character word composed of only uppercase letters - that's 26^8 combinations (208 Billion). Each word is 8 bytes long, which I make to be 1670616516608 bytes... or to put it another way, 1.5 TB
So, you've got 4 GB of virtual memory and you are trying to fit an 1555GB array into it. Simple mathematics says no. Dumping the arguments before it dies is pointless, because it hasn't even got as far as expanding arguments yet.
You need to think of another was to do this and I humbly suggest that Bash should not be high on your list.
Bash should be OK if Pathname Expansion is used instead of the more general Brace Expansion:
$ for w in [[:upper:]][[:upper:]][[:upper:]][[:upper:]][[:upper:]] [[:upper:]][[:upper:]][[:upper:]] ; do echo $w ; done
However, if there are no matches, the match string will be tried unless the shell options failglob is unset and nullglob is set:
$ ( shopt -s nullglob ; shopt -u failglob ; for w in [[:upper:]] [[:upper:]][[:upper:]][[:upper:]][[:upper:]][[:upper:]][[:upper:]] [[:upper:]] ; do echo $w ; done )
(Sorry for the wrapping.)
On 04/10/09 14:03, Patrick O'Callaghan wrote:
On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
On 04/10/09 11:17, Sharpe, Sam J wrote:
2009/10/4 psmithpsmith@fedoraproject.org:
hi list, whilst trying to run this bash command
for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done> wl1
Consider that for a second...
You are trying to generate a list of all possible combinations of an 8 character word composed of only uppercase letters - that's 26^8 combinations (208 Billion). Each word is 8 bytes long, which I make to be 1670616516608 bytes... or to put it another way, 1.5 TB
So, you've got 4 GB of virtual memory and you are trying to fit an 1555GB array into it. Simple mathematics says no. Dumping the arguments before it dies is pointless, because it hasn't even got as far as expanding arguments yet.
You need to think of another was to do this and I humbly suggest that Bash should not be high on your list.
thanks for the reality check sam :o
do you have any suggestions what/where i should be looking to get my required output?
As Sam says, your initial attempt is trying to produce all 1.5TB at once, which is why it blows up, but that's going to happen anyway even if you program the same thing in C. IOW it's not an issue with the Shell per se, but with the approach to the problem.
Assuming you actually want all 1.5TB of data (really?) you'd be better consuming it as it's produced, perhaps via a pipe. Of course that depends on what you're doing with it, which you haven't said.
poc
i'm doing some pen testing of my brother's companies network he wants me to see if it's possible to get in so I'd be using the output as a word list, and yes unfortunately i'll need all of it for comparison i'm not sure if the program (aircrack) takes data from a pipe as a word list, but i'll look into it
phil
phil
On 04/10/09 15:28, Mikkel wrote:
Patrick O'Callaghan wrote:
On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
do you have any suggestions what/where i should be looking to get my required output?
As Sam says, your initial attempt is trying to produce all 1.5TB at once, which is why it blows up, but that's going to happen anyway even if you program the same thing in C. IOW it's not an issue with the Shell per se, but with the approach to the problem.
Assuming you actually want all 1.5TB of data (really?) you'd be better consuming it as it's produced, perhaps via a pipe. Of course that depends on what you're doing with it, which you haven't said.
poc
One way to do it would be to break it down into nested loops. I am not sure you could do it in bash - I am not sure how bash handles loops. You can try something like: (Not tested)
for a in {A..Z}; \ do for b in {A..Z}; \ do for c in {A..Z}; \ do for d in {A..Z}; \ do for e in {A..Z}; \ do for f in {A..Z}; \ echo $a$b$c$d$e$f ; \ done ; done ; done ; done ; done ; done ; done ; done
Mikkel
thanks mikkel :-) this got it to output the data to the file as it was working though the command needed to be
for a in {A..Z}; \ do for b in {A..Z}; \ do for c in {A..Z}; \ do for d in {A..Z}; \ do for e in {A..Z}; \ do for f in {A..Z}; \ do for g in {A..Z}; \ do for h in {A..Z}; \ do echo $a$b$c$d$e$f$g$h; \ done;done;done;done;done;done;done;done; > wl1
now i've just got to wait for the new hard drives to arrive and also work out how to get it to start outputting a different file every 2GB or so :-)
thanks again phil
psmith wrote:
for a in {A..Z}; \ do for b in {A..Z}; \ do for c in {A..Z}; \ do for d in {A..Z}; \ do for e in {A..Z}; \ do for f in {A..Z}; \ do for g in {A..Z}; \ do for h in {A..Z}; \ do echo $a$b$c$d$e$f$g$h; \ done;done;done;done;done;done;done;done; > wl1
now i've just got to wait for the new hard drives to arrive and also work out how to get it to start outputting a different file every 2GB or so :-)
thanks again phil
Well, if you are willing to setting for a specific count, instead of size, you could change the last line to:
done;done;done;done;done;done;done;done; > wl1.$a$b$c$d
or
done;done;done;done;done;done;done;done; > $a$b$c$d.w11
Mikkel
Around 07:17pm on Tuesday, October 06, 2009 (UK time), psmith scrawled:
i'm doing some pen testing of my brother's companies network he wants me to see if it's possible to get in so I'd be using the output as a word list, and yes unfortunately i'll need all of it for comparison i'm not sure if the program (aircrack) takes data from a pipe as a word list, but i'll look into it
Have you considered how long it will take to try 208 billion attempts at cracking the system! Or have I misunderstood what you are doing?
Mikkel wrote:
psmith wrote:
for a in {A..Z}; \ do for b in {A..Z}; \ do for c in {A..Z}; \ do for d in {A..Z}; \ do for e in {A..Z}; \ do for f in {A..Z}; \ do for g in {A..Z}; \ do for h in {A..Z}; \ do echo $a$b$c$d$e$f$g$h; \ done;done;done;done;done;done;done;done; > wl1
now i've just got to wait for the new hard drives to arrive and also work out how to get it to start outputting a different file every 2GB or so :-)
thanks again phil
Well, if you are willing to setting for a specific count, instead of size, you could change the last line to:
done;done;done;done;done;done;done;done; > wl1.$a$b$c$d
or
done;done;done;done;done;done;done;done; > $a$b$c$d.w11
Mikkel
Oops - I was not thinking - change the last line to: done;done;done;done;done;done;done;done; and change the echo line to: do echo $a$b$c$d$e$f$g$h; >> $a$b$c$d.w11 \
Mikkel
On Tue, 2009-10-06 at 19:17 +0100, psmith wrote:
i'm doing some pen testing of my brother's companies network he wants me to see if it's possible to get in so I'd be using the output as a word list, and yes unfortunately i'll need all of it for comparison i'm not sure if the program (aircrack) takes data from a pipe as a word list, but i'll look into it
If by "getting in" you mean "guessing a password" and if the password system accepts only 8 upper-case letters, then the answer is "it depends how long the system takes to accept or reject an attempt". There are 26^8 possible combinations i.e. 208,827,064,576 and on average you need to check half of them, assuming they are really random. If each attempt takes 1 second, you get on average 6621 years for a hit on a specific password. OTOH if an attempt takes 1 microsecond, it's just under 60 hours.
Things to bear in mind:
* If these are user-generated passwords, they are not random. * If the intruder can check against multiple users at once (i.e. he doesn't care which one it is) the numbers drop dramatically. * If the system is at all well-designed it will block attempts after some number of failures from the same origin, e.g. 3. In any case, it should report bursts of failed attempts to the administrator. * Social engineering beats brute force a large percentage of the time.
IOW, and in the absence of more concrete information, I'd say you're not learning anything special by doing a brute-force check like this.
poc