Snip A File
Reggie
file for part of a daily routine I have to go through. I get a raw email
file directly off of a mailserver that contains a base 64 encoded
attachment. I simply need to extract the base64 information and create a new
file just containing it. Here's a sample I will call b64.txt
(NOTE ---there is header info, etc. above the next line)
Content-Transfer-Encoding: BASE64
IyBCb3hFeGNoYW5nZQ0KIw0KIyA1MCB
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM
ZWFkZXIgIlRvIiBjb250YWlucyAiaXB
IHdlaWdodCAtOTk5DQppZiBoZWFkZXI
U1RTRVJWLkJVRkZBTE8uRURVIiB3ZWl
--Boundary-hlb1KcaIn6pYwLG9v2S8N--
What I need to end up with is a new file like this with no blank lines above
or below it, that I call bd64.txt:
IyBCb3hFeGNoYW5nZQ0KIw0KIyA1MCB
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM
ZWFkZXIgIlRvIiBjb250YWlucyAiaXB
IHdlaWdodCAtOTk5DQppZiBoZWFkZXI
U1RTRVJWLkJVRkZBTE8uRURVIiB3ZWl
There are up to 72 characters per line but there may be less.
This will be used on a Win2K machine.
Any help is appreciated.
Reggie
Marco Maier
<news:Blctc.1039$Yd3...@newsread3.news.atl.earthlink.net> :
@echo off
type nul >> b64-snip.txt
for /f "tokens=*" %%a in ('findstr /v "^-" b64.txt^|findstr /v "Content"')
do echo %%a >> b64-snip.txt
Marco Maier
type nul >> b64-snip.txt
echo %%a)>> b64-snip.txt
Reggie
news:hhw3trfuai0d$.MRC@MIR.SID...
Doesn't seem to work. It doesn't snip all the email header lines above
Content-Transfer-Encoding: BASE64.
It does snip off --BlatBoundary-PzPsIenJrCrLoDkkHJ4FZ--
Reggie
Marco Maier
<news:bAjtc.1267$Yd3...@newsread3.news.atl.earthlink.net> :
> Doesn't seem to work. It doesn't snip all the email header lines above
> Content-Transfer-Encoding: BASE64.
> It does snip off --BlatBoundary-PzPsIenJrCrLoDkkHJ4FZ--
it works here.
Marco Maier
> it works here.
however it is better:
type nul > b64-snip.txt
Reggie
"Marco Maier" <marcomaie...@Myahoo.it> wrote in message
Okay. input above "content" also contains "content" so I substituted it with
BASE64, which is not elsewhere in the header. I still end up with all header
info above Content-Transfer-Encoding: BASE64. See what you end up with if
you run the batch on this:
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7BIT
Content-description: Mail message body
Box Exchanger Update
--BlatBoundary-PzPsIenJrCrLoDkkHJ4FZ
Content-Type: text/plain; name=boxexchange.txt
Content-Disposition: attachment; filename="boxexchange.txt"
Content-Transfer-Encoding: BASE64
Marco Maier
> Okay. input above "content" also contains "content" so I substituted it with
> BASE64, which is not elsewhere in the header. I still end up with all header
> info above Content-Transfer-Encoding: BASE64. See what you end up with if
> you run the batch on this:
>
> Content-Type: text/plain; charset=US-ASCII
> Content-Transfer-Encoding: 7BIT
> Content-description: Mail message body
>
> Box Exchanger Update
> --BlatBoundary-PzPsIenJrCrLoDkkHJ4FZ
> Content-Type: text/plain; name=boxexchange.txt
> Content-Disposition: attachment; filename="boxexchange.txt"
>
> Content-Transfer-Encoding: BASE64
>
> IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM
> ZWFkZXIgIlRvIiBjb250YWlucyAiaXB
> IHdlaWdodCAtOTk5DQppZiBoZWFkZXI
> U1RTRVJWLkJVRkZBTE8uRURVIiB3ZWl
> --Boundary-hlb1KcaIn6pYwLG9v2S8N--
You should have been more precise.
Try this one:
@echo off
type nul > b64-snip.txt
for /f "tokens=* skip=10" %%a in ('findstr /v "^-" b64.txt^|findstr /v "Content"') do (
echo %%a)>> b64-snip.txt
foxidrive
> Okay. input above "content" also contains "content" so I substituted it with
> BASE64, which is not elsewhere in the header. I still end up with all header
> info above Content-Transfer-Encoding: BASE64. See what you end up with if
> you run the batch on this:
>
>
> Content-Type: text/plain; charset=US-ASCII
> Content-Transfer-Encoding: 7BIT
> Content-description: Mail message body
>
> Box Exchanger Update
> --BlatBoundary-PzPsIenJrCrLoDkkHJ4FZ
> Content-Type: text/plain; name=boxexchange.txt
> Content-Disposition: attachment; filename="boxexchange.txt"
>
> Content-Transfer-Encoding: BASE64
>
> IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM
> ZWFkZXIgIlRvIiBjb250YWlucyAiaXB
> IHdlaWdodCAtOTk5DQppZiBoZWFkZXI
> U1RTRVJWLkJVRkZBTE8uRURVIiB3ZWl
> --Boundary-hlb1KcaIn6pYwLG9v2S8N--
This filters out all lines that contain a space, plus lines containing
"--". I suspect "--" won't be in BASE64 encoding seeing that the
terminating line starts with it.
@echo off
find /v "--" "file.txt"|find /v " ">bd64.txt
Marco Maier
<news:1fety7tktqwmv$.1o19ck5pd5r00$.d...@40tude.net> :
> This filters out all lines that contain a space, plus lines containing
> "--". I suspect "--" won't be in BASE64 encoding seeing that the
> terminating line starts with it.
>
> @echo off
> find /v "--" "file.txt"|find /v " ">bd64.txt
---begin output-----
IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM
ZWFkZXIgIlRvIiBjb250YWlucyAiaXB
IHdlaWdodCAtOTk5DQppZiBoZWFkZXI
U1RTRVJWLkJVRkZBTE8uRURVIiB3ZWl
----end output--------
But he doesn't want spaces
foxidrive
But there are no spaces. :)
I knew about the empty lines but figure they may not be an issue - and they
can be filtered out through another pipe using findstr. If you feel keen
I'll let you add it. Thanks for the note Marco, as in my haste I forgot
to mention the blank lines.
Matthias Tacke
Here is my try, it will erroneously still get one word lines.
From command line:
for /F "tokens=*" %A in ('findstr "^[0-9a-zA-Z]*$" b64.txt') do @echo/%A
In a batch percent signs have to be doubled.
I don't know which additional chars are allowed in base64 since numbers
and chars upper/lower result in 62.
To be more precise one would have to trigger on "base64" feed the found
line number to more +x and stop with --Boundary .
--
Greetings
Matthias________________________________________
For help on nt commands enter in a cmd window:
W2K>HH windows.chm::ntcmds.htm XP>HH ntcmds.chm
Matthias Tacke
<snip>
>Here is my try, it will erroneously still get one word lines.
>From command line:
>for /F "tokens=*" %A in ('findstr "^[0-9a-zA-Z]*$" b64.txt') do @echo/%A
>
>In a batch percent signs have to be doubled.
>
>I don't know which additional chars are allowed in base64 since numbers
>and chars upper/lower result in 62.
>
>To be more precise one would have to trigger on "base64" feed the found
>line number to more +x and stop with --Boundary .
>
::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off
setlocal
Set File=B64.TXT
Set Out=B64-Strip.TXT
del /Q %Out% >NUL
set Trigger=BASE64
for /F "delims=[]" %%A in ('find /N "%Trigger%" %File%') do set start=%%A
for /F "tokens=*" %%A in ('more +%start% %file%') do Call :Check %%A
goto :eof
:Check
set Base64=%1
If /I "%Base64:~0,2%" NEQ "--" (>>%Out% echo/%1&goto :eof)
exit /B
::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
HTH
Marco Maier
> "Matthias Tacke" wrote:
> If /I "%Base64:~0,2%" NEQ "--" (>>%Out% echo/%1&goto :eof)
> exit /B
>::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Nice.Or he could try the following but I don't know if works in all cases:
findstr "^...............................$"[^ ] b64.txt
Marco Maier
ops:
findstr "^...............................$" b64.txt|find /v " "
foxidrive
> I don't know which additional chars are allowed in base64 since numbers
> and chars upper/lower result in 62.
FYI and FMI.
http://www.mhonarc.org/~ehood/MIME/2045/rfc2045.html
Table 1: The Base64 Alphabet
Value Encoding Value Encoding Value Encoding Value Encoding
0 A 17 R 34 i 51 z
1 B 18 S 35 j 52 0
2 C 19 T 36 k 53 1
3 D 20 U 37 l 54 2
4 E 21 V 38 m 55 3
5 F 22 W 39 n 56 4
6 G 23 X 40 o 57 5
7 H 24 Y 41 p 58 6
8 I 25 Z 42 q 59 7
9 J 26 a 43 r 60 8
10 K 27 b 44 s 61 9
11 L 28 c 45 t 62 +
12 M 29 d 46 u 63 /
13 N 30 e 47 v
14 O 31 f 48 w (pad) =
15 P 32 g 49 x
16 Q 33 h 50 y
Matthias Tacke
>Matthias Tacke wrote in message <news:c94ta3$rh5$05$1...@news.t-online.com> :
>
>> "Matthias Tacke" wrote:
>
>> If /I "%Base64:~0,2%" NEQ "--" (>>%Out% echo/%1&goto :eof)
>> exit /B
>>::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>
>
>Nice.
But didn't work correct. This does:
::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo off
setlocal
Set File=B64.TXT
Set Out=B64-Strip.TXT
del /Q %Out% >NUL 2>&1
set Trigger=BASE64
for /F "delims=[]" %%A in ('find /N "%Trigger%" %File%') do set start=%%A
for /F "tokens=*" %%A in ('more +%start% %file%') do call :Check %%A
goto :eof
:Check
set Base64=%1
If /I "%Base64:~0,2%" NEQ "--" (>>%Out% echo/%1&goto :eof)
set Out=NUL
::base64.cmd:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
>Or he could try the following but I don't know if works in all cases:
>
>findstr "^...............................$"[^ ] b64.txt
That works only with fixed length which the op meant may vary.
And AFAIK the . also matches white space.
foxidrive
> On Thu, 27 May 2004 12:56:15 GMT, Marco Maier wrote:
>
>> foxidrive wrote in message
>> <news:1fety7tktqwmv$.1o19ck5pd5r00$.d...@40tude.net> :
>>
>>> This filters out all lines that contain a space, plus lines containing
>>> "--". I suspect "--" won't be in BASE64 encoding seeing that the
>>> terminating line starts with it.
>>>
>>> @echo off
>>> find /v "--" "file.txt"|find /v " ">bd64.txt
>> ----end output--------
>> But he doesn't want spaces
> I knew about the empty lines but figure they may not be an issue - and they
> can be filtered out through another pipe using findstr.
Here it is.
@echo off
find /v "--" "file.txt"|find /v " "|findstr /c /v "^$">bd64.txt
Matthias Tacke
I had found it in the meantime but nevertheless thanks for the info.
Reggie
news:Blctc.1039$Yd3...@newsread3.news.atl.earthlink.net...
> I am absolutely no batch person but I think I could use some sort of a
batch
> file for part of a daily routine I have to go through. I get a raw email
> file directly off of a mailserver that contains a base 64 encoded
> attachment. I simply need to extract the base64 information and create a
new
> file just containing it. Here's a sample I will call b64.txt
>
> (NOTE ---there is header info, etc. above the next line)
> Content-Transfer-Encoding: BASE64
>
My thanks to all who participated in this thread. I have elected to give
Matthias' batch the green light. Pardon me if I wasn't clear mentioning
there was information prior to the line "Content-Transfer-Encoding: BASE64"
. I thought my "NOTE" in my first post would raise the red flag. Whatever, I
thank everyone for their input and help. So far the batch works okay and my
files decode okay when I apply a base64 decoder. If I encounter any
difficulties I will repost but I don't forsee any problems.
Sorry for the delay in replying. It was a "hi-ho, off to work I go" day.
Reggie
foxidrive
> On Thu, 27 May 2004 23:04:34 +1000, foxidrive wrote:
>
>> On Thu, 27 May 2004 12:56:15 GMT, Marco Maier wrote:
>>
>>> foxidrive wrote in message
>>> <news:1fety7tktqwmv$.1o19ck5pd5r00$.d...@40tude.net> :
>>>
>>>> This filters out all lines that contain a space, plus lines containing
>>>> "--". I suspect "--" won't be in BASE64 encoding seeing that the
>>>> terminating line starts with it.
>>>>
>>>> @echo off
>>>> find /v "--" "file.txt"|find /v " ">bd64.txt
>
>>> ----end output--------
>>> But he doesn't want spaces
>> I knew about the empty lines but figure they may not be an issue
and on rereading the original post I see now that blank lines are an issue
after all.
>> - and they can be filtered out through another pipe using findstr.
>
> Here it is.
How many times can I get it wrong? :)
The /c in my previous post didn't enable regexp, but it worked in that case
which is why I didn't spot the error.
This should do it... though from past mentions of findstr, regexp is the
default but it doesn't hurt to specify it here, so any future readers will
realise it's a regexp.
@echo off
find /v "--" "file.txt"|find /v " "|findstr /r /v "^$">bd64.txt
I do wonder why the OP is stripping the encoding to place in a file. If
it's for decoding then the two decoders I use ignore all spurious text in
an article anyway. There's no need to strip anything out.
Reggie
>
> I do wonder why the OP is stripping the encoding to place in a file. If
> it's for decoding then the two decoders I use ignore all spurious text in
> an article anyway. There's no need to strip anything out.
>
Because the OP's commandline decoder does not. :-) He receives a "Illegal
character '-' in input file" if he attempts to decode the raw email. Also,
being an illerate, the OP assumed that because his decoder does not decode
base64 from raw emails that decoders simply do not do this, unless, of
course, those functions are included in an email client, which cannot be
used in this instance. But thanks for the clue in and I'll look for a
decoder that will. This was all quite interesting anyhow, so the thread
wasn't total futility.
Reggie
Matthias Tacke
>
>I do wonder why the OP is stripping the encoding to place in a file. If
>it's for decoding then the two decoders I use ignore all spurious text in
>an article anyway. There's no need to strip anything out.
It's late, at least here. :-o
Just out of curiosity I played around with that base64 coding I found.
It's not to hard to implement in batch, padding still missing:
::base64conv.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@echo on
setlocal enabledelayedexpansion
set code=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=
set /A B1=10,B2=0x7f,B3=255
echo %B1% %B2% %B3%
:: byte-triplet to base64-quadruple
set /A Q1=(%B1% "&" 252) ">>" 2
set /A Q2=((%B1% "&" 3) "<<" 4) + ((%B2% "&" 240) ">>" 4)
set /A Q3=((%B2% "&" 15) "<<" 2) + ((%B3% "&" 192) ">>" 6)
set /A Q4=(%B3% "&" 63)
::
echo %Q1% %Q2% %Q3% %Q4%
echo !code:~%Q1%,1! !code:~%Q2%,1! !code:~%Q3%,1! !code:~%Q4%,1!
:: base64-quadruple to byte-triplet
:: yes, I know, to work a loop to find the pos in %code% should be here.
set /A B1=(%Q1% "<<" 2) + ((%Q2% "&" 48) ">>" 4)
set /A B2=((%Q2% "&" 15) "<<" 4) + ((%Q3% "&" 60) ">>" 2)
set /A B3=((%Q3% "&" 3) "<<" 6) + %Q4%
::
echo %B1% %B2% %B3%
::base64conv.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Reggie
> >
> Just out of curiosity I played around with that base64 coding I found.
> It's not to hard to implement in batch, padding still missing:
>
> ::base64conv.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Still being a dummy, are you saying it's possible to use this directly in my
batch somehow to decode the base64 content I am now extracting? This would
be great if it's so. Also still being a dummy, what is padding?
Reggie
Matthias Tacke
Hello Reggie.
In theory, yes it is possible. What I posted are the base math
conversions. Since 3 Bytes are converted to 4 base64 chars it's likely
the source length isn't a multiple of 3, so a partially filled quadruple
of base64 chars is padded with = chars.
Handling of special chars (below dec 32 and above 127, also redirection)
is problematic in batch. So you will need external help for reading and
writing binary.
There is no instr function which could get the offset into %code% for
any given base64 char, so every byte from base64 input stream requires a
counting loop what would make a batch quite slow.
My idea was more to convert between base64 and hex, what won't help you.
IMO a solution is out of the scope of batch, batch is not intended to
replace programming languages :-)
So, out of theory, I suggest using a tool written in asm,bas,c,c++,pas,
etc. (Just alphabetical order, not preference :-)
Marco Maier
> So, out of theory, I suggest using a tool written in asm,bas,c,c++,pas,
> etc. (Just alphabetical order, not preference :-)
I suggest him to google this keywords:
freeware "command line" base64 decode
foxidrive
> Just out of curiosity I played around with that base64 coding I found.
> It's not to hard to implement in batch, padding still missing:
>
>::base64conv.cmd::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Matthias, that's a very cunning use of math in batch - the first time I
have seen some of those operators used. I can't say that I understand it
but then math ain't my strong suit. :)
Matthias Tacke
Having a picture like this telling where the bits go it's not difficult.
:: lByte 1 lByte 2 lByte 3 l
:: l8 7 6 5 4 3 2 1l8 7 6 5 4 3 2 1l8 7 6 5 4 3 2 1l
:: l l l l l l l
:: l6 5 4 3 2 1l6 5 4 3 2 1l6 5 4 3 2 1l6 5 4 3 2 1l
:: l B64-1 l B64-2 l B64-3 l B64-4 l
The coding is simply bit masking with "&=and" followed by bit shifts to
fit into the new var. I.e. the first line & 252 (binary 11111100) masks
(retains) the upper 6 bits,to fit into b64-1 I have to divide by 4 or
what is equal ">>" 2 shift right two times.
A very long time ago I started with Z-80 assembler and a "ต-professor",
so a little bit of that is still present :-)
Reggie
> >> >
> So, out of theory, I suggest using a tool written in asm,bas,c,c++,pas,
> etc. (Just alphabetical order, not preference :-)
>
> --
And that's what I have done. I picked up uudeview.exe and it works perfectly
for me. When specific mail is delivered to the backup mailserver that is at
another location, it triggers a rule that runs a batch against it. Uudeview
decodes the base64 info and updates appropriate mail filtering data files on
the backup mailserver, keeping it current with the primary mailserver. Works
like a charm.
Thanks again for everyone's input.
Reggie