word splitting issue
John Koy
I'm having a word splitting problem in the following:
-------------------
#!/bin/bash
cnvdraw=''
while read lin; do
set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
done <<ENDI
25 25 45 45
50 50 70 70
75 75 100 100
ENDI
convert imgfile.tif -fill none -stroke black ${cnvdraw} - | display -
-------------------
I want to obtain and execute the following as result:
convert imgfile.tif -fill none -stroke black -draw 'rectangle
25,25,45,45' -draw 'rectangle 50,50,70,70' -draw 'rectangle
75,75,100,100' - | display -
however, I get the following errors:
convert: Non-conforming drawing primitive definition `rectangle' @
draw.c/DrawImage/3123.
convert: unable to open image `25,25,45,45'': No such file or directory
@ blob.c/OpenBlob/2439.
convert: Non-conforming drawing primitive definition `rectangle' @
draw.c/DrawImage/3123.
convert: unable to open image `50,50,70,70'': No such file or directory
@ blob.c/OpenBlob/2439.
convert: Non-conforming drawing primitive definition `rectangle' @
draw.c/DrawImage/3123.
convert: unable to open image `75,75,100,100'': No such file or
directory @ blob.c/OpenBlob/2439.
convert: Non-conforming drawing primitive definition `rectangle' @
draw.c/DrawImage/3123.
How do I resolve that? (bash 4.0.33)
Dave Gibson
> Hello,
> I'm having a word splitting problem in the following:
>
> -------------------
> #!/bin/bash
>
> cnvdraw=''
>
> while read lin; do
> set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
> done <<ENDI
> 25 25 45 45
> 50 50 70 70
> 75 75 100 100
> ENDI
>
> convert imgfile.tif -fill none -stroke black ${cnvdraw} - | display -
eval "convert imgfile.tif -fill none -stroke black $cnvdraw - | display -"
Incidentally, it is possible to specify multiple variables with read,
removing the need to reset the positional parameters:
while read x0 y0 x1 y1 ; do
cnvdraw="$cnvdraw -draw 'rectangle $x0,$y0 $x1,$y1'"
Grant
>John Koy <John...@example.com> wrote:
>> Hello,
>> I'm having a word splitting problem in the following:
>>
>> -------------------
>> #!/bin/bash
>>
>> cnvdraw=''
>>
>> while read lin; do
>> set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
>> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
>> done <<ENDI
>> 25 25 45 45
>> 50 50 70 70
>> 75 75 100 100
>> ENDI
>>
>> convert imgfile.tif -fill none -stroke black ${cnvdraw} - | display -
>
>eval "convert imgfile.tif -fill none -stroke black $cnvdraw - | display -"
>
>Incidentally, it is possible to specify multiple variables with read,
>removing the need to reset the positional parameters:
>
>while read x0 y0 x1 y1 ; do
Yes but make room for the end of line crap, comments etc:
while read x0 y0 x1 y1 crap
do
...
> cnvdraw="$cnvdraw -draw 'rectangle $x0,$y0 $x1,$y1'"
>done << ENDI
>25 25 45 45
>50 50 70 70
>75 75 100 100
>ENDI
Grant.
--
http://bugsplatter.id.au
John Koy
> John Koy<John...@example.com> wrote:
>> Hello,
>> I'm having a word splitting problem in the following:
>>
>> -------------------
>> #!/bin/bash
>>
>> cnvdraw=''
>>
>> while read lin; do
>> set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
>> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
>> done<<ENDI
>> 25 25 45 45
>> 50 50 70 70
>> 75 75 100 100
>> ENDI
>>
>> convert imgfile.tif -fill none -stroke black ${cnvdraw} - | display -
>
> eval "convert imgfile.tif -fill none -stroke black $cnvdraw - | display -"
>
thanks, it works.
I have another concern: in real case the input is not a heredoc but a
file with thousands of lines, so ${cnvdraw} will possibly become like
40-50K (or more). What's the limit, if any, for a Bash variable size?
Dave Gibson
> On Sun, 27 Dec 2009 20:18:25 +0000, dave+news002@\
> gibson-hrd.abelgratis.co.uk.invalid (Dave Gibson) wrote:
>
>>John Koy <John...@example.com> wrote:
>>> Hello,
>>> I'm having a word splitting problem in the following:
>>>
>>> -------------------
>>> #!/bin/bash
>>>
>>> cnvdraw=''
>>>
>>> while read lin; do
>>> set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
>>> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
>>> done <<ENDI
>>> 25 25 45 45
>>> 50 50 70 70
>>> 75 75 100 100
>>> ENDI
>>Incidentally, it is possible to specify multiple variables with read,
>>removing the need to reset the positional parameters:
>>
>>while read x0 y0 x1 y1 ; do
>
> Yes but make room for the end of line crap, comments etc:
>
> while read x0 y0 x1 y1 crap
UNNECESSARY!!! THE OP IS CLEARLY DEALING WITH KNOWN GOOD INPUT!!!!
What? What other post? Input from where? Bugger.
> Grant.
Just out of curiousity, what do you think next week's lottery
numbers will be?
Dave Gibson
> Dave Gibson wrote:
>> John Koy<John...@example.com> wrote:
>>> Hello,
>>> I'm having a word splitting problem in the following:
>>>
>>> -------------------
>>> #!/bin/bash
>>>
>>> cnvdraw=''
>>>
>>> while read lin; do
>>> set ${lin}; x0=$1; y0=$2; x1=$3; y1=$4
>>> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
>>> done<<ENDI
>>> 25 25 45 45
>>> 50 50 70 70
>>> 75 75 100 100
>>> ENDI
>>>
>>> convert imgfile.tif -fill none -stroke black ${cnvdraw} - | display -
>>
>> eval "convert imgfile.tif -fill none -stroke black $cnvdraw - | display -"
>>
>
> thanks, it works.
> I have another concern: in real case the input is not a heredoc but a
> file with thousands of lines, so ${cnvdraw} will possibly become like
> 40-50K (or more). What's the limit, if any, for a Bash variable size?
I don't think there is a specific limit on variable size. The total
length of a command line may be restricted to as little as 32KB though.
The size of the input data is also likely to make a read loop very slow.
An alternative approach would be to process the input data into -draw
arguments and then pipe the result into convert as a response file
(denoted by a leading '@' on the filename -- '@-' for stdin).
Something like this:
awk '{ printf "rectangle %d,%d %d,%d\n", $1, $2, $3, $4 }' input.data |
convert imgfile.tif -fill none -stroke black -draw @- - | display -
Jon LaBadie
I never considered shell looping to be terribly slow for most of the uses
one makes of them, so I set up a test. My input was the above 4 lines
replicated to make a total of 10,000 lines. The resulting variable was a
little over 300 Kbytes. As Dave mentioned, passing that as a command line
argument could be a problem on some systems.
When I timed the script it took just 4.1 seconds. But then I remembered I
use ksh93 and the OP asked about bash. What a difference!! I tired of
waiting for 10,000 lines to finish and cut it back to 1,000 lines. This
took 4.9 seconds, slightly longer for 1/10 the input size. Trying with
larger inputs showed the time taken to increase non-linearly so that by
5,000 lines, the time taken for the loop was about 2 minutes. The results
were nearly the same for both bash3 and bash4.
John Koy
The bottleneck in the loop, for Bash, is not at I/O but var. assignment:
cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
It seems Bash is better at handling arrays in such situations. So, the
loop now has become sth like that:
------------------------------
lincount=0
while read x0 y0 x1 y1 rest; do
cnvdraw[${lincount}]=" -draw 'rectangle $x0,$y0,$x1,$y1'"
let lincount++
done <${infile}
eval "convert imgfile.tif -fill none -stroke black ${cnvdraw[*]} - |
display -"
------------------------------
It performed pretty fast for 6000 line input.
Chris F.A. Johnson
You don't need the lincount variable (since bash-3.1):
cnvdraw+=( "-draw 'rectangle $x0,$y0,$x1,$y1'" )
> let lincount++
> done <${infile}
>
> eval "convert imgfile.tif -fill none -stroke black ${cnvdraw[*]} - |
> display -"
> ------------------------------
>
> It performed pretty fast for 6000 line input.
>
>
>
--
Chris F.A. Johnson, author <http://shell.cfajohnson.com/>
===================================================================
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)
===== My code in this post, if any, assumes the POSIX locale =====
===== and is released under the GNU General Public Licence =====
Jon LaBadie
Within that assignment, I suspect the bottleneck is dynamic memory allocation,
malloc/realloc, or whatever bash and ksh use to handle memory allocation.
John Koy
[...]
>>
>> The bottleneck in the loop, for Bash, is not at I/O but var. assignment:
>> cnvdraw="${cnvdraw} -draw 'rectangle $x0,$y0,$x1,$y1'"
>>
>> It seems Bash is better at handling arrays in such situations. So, the
>> loop now has become sth like that:
>> ------------------------------
>> lincount=0
>> while read x0 y0 x1 y1 rest; do
>> cnvdraw[${lincount}]=" -draw 'rectangle $x0,$y0,$x1,$y1'"
>
> You don't need the lincount variable (since bash-3.1):
>
> cnvdraw+=( "-draw 'rectangle $x0,$y0,$x1,$y1'" )
>
It seems an array was not even necessary:
cnvdraw+=" -draw 'rectangle $x0,$y0,$x1,$y1'"
Thanks for all the responses.