for loop delim problem

[Bob Dylan]

Hi all,

Trying to get a pipe delimited file manipulation batch file set up and
am having a hard time getting the actual for loop to set my variables
(or getting the echo wrong or something else stupid).

There are three .txt files in c:\bob\nic of the format TOY*.txt, pipe
delimited with about 15 fields.

###############code###############
@echo off
c:
cd c:\bob\nlc

for /f "tokens=1-12* delims=|" %%a in ('dir /b /a-d TOY*.txt') do (
@echo %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %
%q %%r %%s %%t
)
#############code##################

############output################
TOY478435.txt %n %o %p %q %r %s %t
TOY478436.txt %n %o %p %q %r %s %t
TOY478437.txt %n %o %p %q %r %s %t

c:\Bob\NLC>type toy478435.txt
13|TOY5|dave coogan|Tel: |64 Dover Close|Blisworth|GB||GH99 5FD|D|
Thanks for your order|0.00|N|N|Contents fragile|1
############output###############

So i can see that the single quoted dir command fills the first and
that my twelve tokens are being initialised but for some reason the
file isn’t being read into the variables. If I can get the variables
set I will be able to write them to a new file in a different order.

Any help appreciated!
Thanks,
Bob

[tonysathre]

You're trying to echo tokens that don't exist. Your input file only
contains 13 tokens when delimited by a pipe.

C:\test>for /f "tokens=1-12* delims=|" %a in ('type test.txt') do echo
%a %b %c
%d %e %f %g %h %i %j %k %l %m >> output.txt

C:\test>echo 13 TOY5 dave coogan Tel: 64 Dover Close Blisworth GB
GH99 5FD D
1>>output.txt

C:\test>echo Thanks for your order 0.00 N N Contents fragile 1
1>>output
.txt

C:\test>type output.txt

13 TOY5 dave coogan Tel: 64 Dover Close Blisworth GB GH99 5FD D
Thanks for your order 0.00 N N Contents fragile 1

C:\test>

[foxidrive]

On 30/03/2012 10:38, Bob Dylan wrote:
> Hi all,
>
> Trying to get a pipe delimited file manipulation batch file set up and
> am having a hard time getting the actual for loop to set my variables
> (or getting the echo wrong or something else stupid).
>
> There are three .txt files in c:\bob\nic of the format TOY*.txt, pipe
> delimited with about 15 fields.
>

Try this style:

###############code###############
@echo off
c:
cd c:\bob\nlc

for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (
for /f "tokens=1-12* delims=|" %%b in ('type "%%a"') do (

@echo %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %
%q %%r %%s %%t
)

)




--
Mic

[Bob Dylan]

Hi Tony, Foxi,

Thanks for your help with this, that seems to have worked as
intended. Bit confused that it wasnt happy to just have TOY*.txt as
'set' but am so very pleased to have it doing what I envisaged now!
Thanks so much for your advice.

@echo off
:top
c:
cd c:\nl\input

for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (

for /f "tokens=1-17 delims=|" %%b in ('type "%%a"') do (
@echo %%b^|Toy Co^|%%f^|^|^|%%g^|%%i^|COUNTY^|%%d^|%%e > c:\nl\output\%
%a
@echo %%b^|Toy Co^|%%f^|^|^|%%g^|%%i^|COUNTY^|%%d^|%%e
del %%a
)
)
c:\nl\sleep 1
GOTO top

[Bob Dylan]

Hi all,

A bit further down the line have developed the script a bit more and
seems to be working well however when the input file has empty fields
(relatively rare and didnt come up in the test data) there is a double
delimiter which the for loop doesnt seem to pick up as a distinct
token/variable. This causes the following token to input file field
mapping to be one field out. This was predominantly happening for
files with %%i set as "IE" so I made a conditional ordering for that
which seemed to do the trick for that particular issue.
Unfortunately, throwing a few dozen more test files at it revealed
that sometimes other fields were blank too and so I dont think I'm
going to be able to account reliably for the changes through
additional 'else if' code.

The only solution I can think of is a way of getting the for loop to
recognise double vertical bar as a valid field (even if it does get
set as NULL / space or something)
or a way of finding occurrences of "||" and replacing them with "| |"
before using a for loop to interpret the files. Either that or use a
language that lets you more easily reference special characters.

Have considered using a program that I have used before called FART to
get called to replace all instances of || but I'm not sure how to feed
it the character in the first place. I could set some constants to
the find and replace strings but don't seem to be able to.

Any pointers much appreciated!

Please excuse some of the commenting,




@echo off
setlocal enableextensions enabledelayedexpansion
:top
c:
cd c:\nlc\input

for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (
for /f "tokens=1-17 delims=|" %%b in ('type "%%a"') do (

REM @echo b %%b c %%c d %%d e %%e f %%f g %%g h %%h i %%i j %%j k %%k
l %%l m %%m n %%n o %%o p %%p q %%q
set TELNO=%%e
if %%i == IE (
@echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
%date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%p^|1.0^|%%k^|0^| > c:\nlc
\output\%%a
) ELSE (
@echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
%date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^| > C:\nlc
\output\%%a
)
@echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
%date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|
del %%a
)
)
@echo %TIME%
c:\nlc\sleep 1
GOTO top

[foxidrive]

On 4/05/2012 07:31, Bob Dylan wrote:

> The only solution I can think of is a way of getting the for loop to
> recognise double vertical bar as a valid field (even if it does get
> set as NULL / space or something)
> or a way of finding occurrences of "||" and replacing them with "| |"
> before using a for loop to interpret the files. Either that or use a
> language that lets you more easily reference special characters.
>
> Have considered using a program that I have used before called FART to
> get called to replace all instances of || but I'm not sure how to feed
> it the character in the first place. I could set some constants to
> the find and replace strings but don't seem to be able to.

This will replace the || with | |

> @echo off
> setlocal enableextensions enabledelayedexpansion
> :top
> c:
> cd c:\nlc\input
> for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (

at this position in the batch file, use GnuSED to change the || to | |

sed "s/||/| |/g" "%%a" > "temptoyfile"


And then change this line to the following.

> for /f "tokens=1-17 delims=|" %%b in ('type "%%a"') do (

for /f "tokens=1-17 delims=|" %%b in ('type "temptoyfile"') do (

> REM @echo b %%b c %%c d %%d e %%e f %%f g %%g h %%h i %%i j %%j k %%k
> l %%l m %%m n %%n o %%o p %%p q %%q
> set TELNO=%%e
> if %%i == IE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%p^|1.0^|%%k^|0^| > c:\nlc
> \output\%%a
> ) ELSE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^| > C:\nlc
> \output\%%a
> )
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|
> del %%a
> )
> )

Add here:
del temptoyfile 2>nul

> @echo %TIME%
> c:\nlc\sleep 1
> GOTO top

--
Mic

[Todd Vargo]

I don't know what it is you are trying to do, but try it with the
following (untested) 4 lines inserted to replace "||" with "| |".

> @echo off
> setlocal enableextensions enabledelayedexpansion
> :top
> c:
> cd c:\nlc\input

echo> tmp.vbs s = Wscript.StdIn.ReadAll
echo> tmp.vbs Wscript.Echo Replace(s,"^|^|","^| ^|")

> for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (
> for /f "tokens=1-17 delims=|" %%b in (

'type "%%a"^|cscript /nologo tmp.vbs') do (

> REM @echo b %%b c %%c d %%d e %%e f %%f g %%g h %%h i %%i j %%j k %%k
> l %%l m %%m n %%n o %%o p %%p q %%q
> set TELNO=%%e
> if %%i == IE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%p^|1.0^|%%k^|0^|> c:\nlc
> \output\%%a
> ) ELSE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|> C:\nlc
> \output\%%a
> )
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|
> del %%a
> )
> )

del tmp.vbs

> @echo %TIME%
> c:\nlc\sleep 1
> GOTO top

--
Todd Vargo
(Post questions to group only. Remove "z" to email personal messages)

[foxidrive]

On 5/3/2012 5:31 PM, Bob Dylan wrote:

> or a way of finding occurrences of "||" and replacing them with "| |"
> before using a for loop to interpret the files. >

TV> I don't know what it is you are trying to do, but try it with the
TV> following (untested) 4 lines inserted to replace "||" with "| |".

I changed one of the added lines below, and it does work.

> @echo off
> setlocal enableextensions enabledelayedexpansion
> :top
> c:
> cd c:\nlc\input

echo> tmp.vbs s = Wscript.StdIn.ReadAll

echo>> tmp.vbs Wscript.Echo Replace(s,"||","| |")

> for /f "delims=" %%a in ('dir /b /a-d TOY*.txt') do (
> for /f "tokens=1-17 delims=|" %%b in (

'type "%%a"^|cscript /nologo tmp.vbs') do (

> REM @echo b %%b c %%c d %%d e %%e f %%f g %%g h %%h i %%i j %%j k %%k
> l %%l m %%m n %%n o %%o p %%p q %%q
> set TELNO=%%e
> if %%i == IE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%p^|1.0^|%%k^|0^|> c:\nlc
> \output\%%a
> ) ELSE (
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|> C:\nlc
> \output\%%a
> )
> @echo %%b^|%%c^|%%c^|%%d^|%%f^|%%g^|^|%%g^|%%h^|%%d^|!TELNO:~5,12!^|
> %date:~6,4%^-%date:~3,2%^-%date:~0,2%^|%%q^|1.0^|%%l^|0^|
> del %%a
> )
> )

del tmp.vbs

[Bob Dylan]

You consistent legend Foxi!

The two line VBS worked a treat, didnt have much luck getting the
batch file to create and delete it each itteration of the main loop
but saving the two lines as a permenant script seemed to do it
perfectly.

c:\NLC>type tmp.vbs
s = Wscript.StdIn.ReadAll
Wscript.Echo Replace(s,"||","| |")

[Todd Vargo]

On 5/4/2012 2:14 AM, foxidrive wrote:
> On 5/3/2012 5:31 PM, Bob Dylan wrote:
>
>> or a way of finding occurrences of "||" and replacing them with "| |"
>> before using a for loop to interpret the files.>
>
>
> TV> I don't know what it is you are trying to do, but try it with the
> TV> following (untested) 4 lines inserted to replace "||" with "| |".
>
> I changed one of the added lines below, and it does work.

Late night. Thanks for catching that, good eye.

[foxidrive]

Kudo's to Todd. I'll use that technique too in future. :)



--
Mic

[Todd Vargo]

Note, since the file name tmp.vbs is often used in batch file examples
to designate a temporary file, I suggest Bob give it a different name
for the permanent file to avoid it being overwritten by accident.
Something like openpipes.vbs or similar would be a better/safer name to use.