generate binary triplets

0 visningar
Hoppa till det första olästa meddelandet

sill...@yahoo.com

oläst,
13 maj 2008 15:59:522008-05-13
till
The script below generates all triples of bytes in the range 0-49 so
the first 3 bytes are 00 00 01 and the last 3 are (in decimal) 49 49
49. Its so slow it could almost serve as a benchmark.

It took 14 minutes to run on my puny system.
I suspect (!) that the code is seriously flawed....
and would like suggestiond to improve its performance.

Feel free to make constructive comments on the code itself too.

I would prefer answers in bash but maybe some awk or perl might be
useful too.
Please stay with these only. I know its easily doable in a few lines
of C.

Thanks
Hal
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/bin/sh

m=50

for ((x=0; x<$m; x++)) ; do
for ((y=0; y<$m; y++)) ; do
for ((z=0; z<$m; z++)) ; do
echo -n -e $(printf "\\\x%x" $x $y $z)
done
done
done

John DuBois

oläst,
13 maj 2008 17:35:032008-05-13
till
In article <74ccdaf7-6322-40a4...@25g2000hsx.googlegroups.com>,

In ksh93:

for i in \\{0..49%03o}\\{0..49%03o}\\{0..49%03o}; do print -n "$i"; done

This takes 0.5s on my system.

John
--
John DuBois spc...@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

Michael Paoli

oläst,
14 maj 2008 02:04:092008-05-14
till
On May 13, 12:59 pm, silly...@yahoo.com wrote:
> The script below generates all triples of bytes in the range 0-49 so
> the first 3 bytes are 00 00 01 and the last 3 are (in decimal) 49 49
> 49. Its so slow it could almost serve as a benchmark.
>
> It took 14 minutes to run on my puny system.
> I suspect (!) that the code is seriously flawed....
> and would like suggestiond to improve its performance.
>
> I would prefer answers in bash but maybe some awk or perl might be
> useful too.
> #!/bin/sh
> m=50
> for ((x=0; x<$m; x++)) ; do
> for ((y=0; y<$m; y++)) ; do
> for ((z=0; z<$m; z++)) ; do
> echo -n -e $(printf "\\\x%x" $x $y $z)
> done
> done
> done

50^3 command substitutions is likely where your greatest performance
hit
is - I got about an 8x speedup when removing the command
substitution.
I got about another 35x speedup - for a total speedup of better than
275x, by doing the whole thing in perl.

$ expand -i4 < foo
#!/usr/bin/perl

my $m=50;

for(my $x=0;$x<$m;++$x){
for(my $y=0;$y<$m;++$y){
for(my $z=0;$z<$m;++$z){
print(chr($x),chr($y),chr($z));
}
}
}
$ time ./foo >>/dev/null

real 0m0.323s
user 0m0.300s
sys 0m0.020s
$

However, the ksh example:


for i in \\{0..49%03o}\\{0..49%03o}\\{0..49%03o}; do print -n "$i";
done

given in:
news:hfCdnZhnKdYalLfV...@speakeasy.net
still outperformed my perl example. In my tests, it gave times:
real 0m0.008s
user 0m0.010s
sys 0m0.000s
... well, it wasn't *that* fast the first time I did it (ksh binary
probably
got cached by the kernel) ... but first time through it was still
roughly
twice as fast as my perl example.

sill...@yahoo.com

oläst,
14 maj 2008 03:33:182008-05-14
till
> twice as fast as my perl example.- Hide quoted text -
>
> - Show quoted text -

I had problems doing it in bash without command substitution, how
would it look with it removed?

Hal

sill...@yahoo.com

oläst,
14 maj 2008 16:38:402008-05-14
till
> I got about an 8x speedup when removing the command substitution.

I got the impression that there was no chr() equivalent in bash -
without using command substitution. Look forward to seing how overcame
this in bash.

Just tried it in awk for completeness and it is twice as slow as perl,
- still good though.

BEGIN {
m=50

for (x=0; x<m; x++)
for(y=0; y<m; y++)
for(z=0; z<m; z++)
printf("%c%c%c",x,y,z)
}

Hal

mop2

oläst,
14 maj 2008 17:23:252008-05-14
till

If I understand what you want perhaps this helps.

$ cat s
#!/bin/bash

# time
t () {
[ $T ] && echo `date +%s.%N`-$T | bc && T= || T=`date +%s.%N`
}


t >&2
#-------
w=
for x in {0..49};do w="$w `printf %x $x`";done
for x in $w;do
for y in $w;do
for z in $w;do
printf "\x$x \x$y \x$z"
done
done
done
#-------
t >&2
#--------end s


$ ./s >output
2.799225844
$

$ stat -c %s output
625000
$

$ head -c16 output|od -t x1
0000000 00 20 00 20 00 00 20 00 20 01 00 20 00 20 02 00
0000020
$

$ tail -c16 output|od -t x1
0000000 2e 31 20 31 20 2f 31 20 31 20 30 31 20 31 20 31
0000020
$

sill...@yahoo.com

oläst,
15 maj 2008 03:24:102008-05-15
till
>
> If I understand what you want perhaps this helps.
>
>

Thanks for your efforts but it's an attampt to write binary bytes so
the output should be something like this based on th awk code above:

$ awk -f genchar2.awk | od -vAn -t x1 -w9 | head -5
00 00 00 00 00 01 00 00 02
00 00 03 00 00 04 00 00 05
00 00 06 00 00 07 00 00 08
00 00 09 00 00 0a 00 00 0b
00 00 0c 00 00 0d 00 00 0e

I think the answer for bash is to use an array so that you call
chr[$x] (etc) as required.

Hal

sill...@yahoo.com

oläst,
15 maj 2008 16:03:382008-05-15
till
This version, using the array chr[], is a 10 second event on my box.

Just for fun, any suggestions on how to make this even faster using
bash alone?

Hal
~~~~~~~~~~~~~

#!/bin/sh
m=50

for ((x=0; x<$m; x++)) ; do

chr[$x]=$(printf "\\\x%x" $x)
done

for ((x=0; x<$m; x++)) ; do

cx=${chr[$x]}


for ((y=0; y<$m; y++)) ; do

cy=${chr[$y]}


for ((z=0; z<$m; z++)) ; do

echo -n -e $cx$cy${chr[$z]}
done
done
done

mop2

oläst,
15 maj 2008 20:44:042008-05-15
till

Sorry, my original line was based on the original post.
With more recent info (without char space):
printf "\x$x\x$y\x$z"

Then:
$ ./s >out2
2.673443011

Or using command "time":
real 0m2.712s
user 0m2.408s
sys 0m0.292s

$ stat -c %s out2
375000

$ od -vAn -t x1 -w9 out2| head -5

sill...@yahoo.com

oläst,
16 maj 2008 15:04:562008-05-16
till
> Sorry, my original line was based on the original post.

which 'worked' if you ignore the time it took:
$ ./original-post.sh | od -vAn -t x1 -w9 | head -5


00 00 00 00 00 01 00 00 02
00 00 03 00 00 04 00 00 05
00 00 06 00 00 07 00 00 08
00 00 09 00 00 0a 00 00 0b
00 00 0c 00 00 0d 00 00 0e

> With more recent info (without char space):
> printf "\x$x\x$y\x$z"

to give...
...


w=
for x in {0..49};do w="$w `printf %x $x`";done

for x in $w;do
for y in $w;do
for z in $w;do
printf "\x$x\x$y\x$z"
done
done
done

...

Very nice!
- 3.5 seconds on my box.

Thanks for that, I think you win this one!

Off topic here, but is there a way you can do something like this...
t=49
for x in {0..$t}...

Hal.

Bill Marcum

oläst,
16 maj 2008 16:13:182008-05-16
till
On 2008-05-16, sill...@yahoo.com <sill...@yahoo.com> wrote:
>
> Thanks for that, I think you win this one!
>
> Off topic here, but is there a way you can do something like this...
> t=49
> for x in {0..$t}...
>
I don't know if it's portable, but
for x in $( seq 0 $t )

mop2

oläst,
16 maj 2008 21:09:492008-05-16
till

On Fri, 16 May 2008 16:04:56 -0300, <sill...@yahoo.com> wrote:

>> Sorry, my original line was based on the original post.
> which 'worked' if you ignore the time it took:

You are correct! My fault was about printf behavior.

> Off topic here, but is there a way you can do something like this...
> t=49
> for x in {0..$t}...

Yes, some possibilities:
$ t=9
$ for x in {0..$t};do printf $x;done
{0..9}$
$ eval "for x in {0..$t};do printf _\$x;done"
_0_1_2_3_4_5_6_7_8_9$
$ for x in `eval echo {0..$t}`;do printf _$x;done
_0_1_2_3_4_5_6_7_8_9$

The Bill Marcum's suggestion with "seq", I think, is more portable
than
this one.
The time for execution of that line is irrelevant in the script's
time.

ba sh

oläst,
13 maj 2008 21:14:592008-05-13
till

If I understand what you want perhaps this helps.

$ cat s
#!/bin/bash

# time
t () {
[ $T ] && echo `date +%s.%N`-$T | bc && T= || T=`date +%s.%N`
}


t >&2
#-------


w=
for x in {0..49};do w="$w `printf %x $x`";done
for x in $w;do
for y in $w;do
for z in $w;do
printf "\x$x \x$y \x$z"
done
done
done

Stephane CHAZELAS

oläst,
14 maj 2008 06:31:252008-05-14
till
2008-05-14, 00:33(-07), sill...@yahoo.com:
[...]

> I had problems doing it in bash without command substitution, how
> would it look with it removed?
[...]

awk 'BEGIN{
for (i=0;i<50;i++)
for (j=0;j<50;j++)
for (k=0;k<50;k++)
printf "%c%c%c", i, j, k}'

Remember bash is a shell, so a tool to call commands. The art of
shell scripting is the art to find the proper tool for a task.

Running tool thousands of time in a loop is generally not the
best approach.

--
Stéphane

Bash

oläst,
14 maj 2008 13:40:582008-05-14
till

>> On May 13, 12:59 pm, silly...@yahoo.com wrote:
>>
>>
>>
>>
>>
>> > The script below generates all triples of bytes in the range 0-49 so
>> > the first 3 bytes are 00 00 01 and the last 3 are (in decimal) 49 49
>> > 49. Its so slow it could almost serve as a benchmark.
>>
>> > It took 14 minutes to run on my puny system.
>> > I suspect (!) that the code is seriously flawed....
>> > and would like suggestiond to improve its performance.
>>
>> > I would prefer answers in bash but maybe some awk or perl might be
>> > useful too.
>> > #!/bin/sh
>> > m=50
>> > for ((x=0; x<$m; x++)) ; do
>> >    for ((y=0; y<$m; y++)) ; do
>> >       for ((z=0; z<$m; z++)) ; do
>> >          echo -n -e $(printf "\\\x%x" $x $y $z)
>> >       done
>> >    done
>> > done
>>

mop2

oläst,
16 maj 2008 20:47:432008-05-16
till
On Fri, 16 May 2008 16:04:56 -0300, <sill...@yahoo.com> wrote:

>> Sorry, my original line was based on the original post.
> which 'worked' if you ignore the time it took:

You are correct! My fault was about printf behavior.

> Off topic here, but is there a way you can do something like this...


> t=49
> for x in {0..$t}...

sill...@yahoo.com

oläst,
17 maj 2008 08:51:192008-05-17
till
After experimenting with expressions like...
$ echo {a..c}\ {a..c}\ {a..c}
a a a a a b a a c a b a a b b a b c a c a a c b a c c b a a b a b b a
c b b a b b b b b c b c a b c b b c c c a a c a b c a c c b a c b b c
b c c c a c c b c c c

... I Managed to get 'something like' John's ksh93 code working...

echo -n -e $(printf "\\\x%x" $(echo {0..49}\ {0..49}\ {0..49}))

- fastest so far for me using bash, just under 3.5 seconds.

Hal

sill...@yahoo.com

oläst,
18 maj 2008 04:36:382008-05-18
till

A minor mod to mop2's has the edge on speed for me...

#!/bin/bash
w=$(printf "\\\x%x " {0..49})

for x in $w;do
for y in $w;do
for z in $w;do

printf "$x$y$z"
done
done
done

Svara alla
Svara författaren
Vidarebefordra
0 nya meddelanden