binary character in INBOX
tempest
can't explain how or why it happened, but for some reason, after
getting an email (hitting g within VM), vm started saying that it had
an error with an invalid string, i.e. stringp \?UTF-8\?, etc. I tried
to open the INBOX file in Accessories->Text Editor, and the program
wouldn't load the file since there were binary characters in the
file.
First, is it possible to find the binary characters? and salvage the
file?
Second, I'm worried that the error is the result of an I/O error. What
is the most likely reason for this error?
1. Is it possible that the email contained a binary character?
2. There is an I/O or memory or motherboard problem?
3. Is it possible that emacs-vm would do this? I have a hard time
believing this one.
The dilemma I face is to determine whether I should abandon my
machine, since this INBOX error could be a warning sign that other
files will become corrupted, or whether it's possible that emacs-vm
wouldn't handle a message with a binary character in it. My provider
is earthlink; they scan my email for viruses. Does anyone know if they
should have rejected the problem email given the binary character?
any thoughts greatly appreciated.
Kurt Hackenberg
tempest <joelv...@earthlink.net> wrote:
>I am using the emacs and vm that comes standard with fedora 10. I
>can't explain how or why it happened, but for some reason, after
>getting an email (hitting g within VM), vm started saying that it had
>an error with an invalid string, i.e. stringp \?UTF-8\?, etc. I tried
>to open the INBOX file in Accessories->Text Editor, and the program
>wouldn't load the file since there were binary characters in the
>file.
First, I suggest that you rename INBOX to something else. That way
you can resume receiving mail, and can work on that damaged file on
the side without it changing under you (with new mail arriving, and so
on). That assumes that later you'll know how to deal with having the
contents of that file off to the side -- handle it there, integrate
the messages back into INBOX, throw it away if there's nothing you
want...
>First, is it possible to find the binary characters? and salvage the
>file?
Sure, if you know the file format and how to use relevant tools. The
file format is either some variant of mbox or, unlikely, Babyl
(Rmail's format) -- those are all VM knows.
Tools -- let's see -- well, how about grep? Write a regular
expression? Some versions might be able to do that. Otherwise I
might write a little C program.
No, it's probably easier than that. New mail is added to the end of
the file. If you haven't changed the file since (like by getting more
new mail), then just look at the end of the file. Look over the whole
last message carefully -- or maybe all the messages added that last
time you got new mail. Emacs can handle binary characters; you could
edit the file directly with Emacs.
Fixing the file might be pretty easy with Emacs, once you find the
problem. Be careful to keep the file format correct. Keep a backup.
If you don't know what I'm talking about...hmm...is there anybody
around who might, who you can ask to look at the file?
>What is the most likely reason for this error?
>
>1. Is it possible that the email contained a binary character?
That's my guess. Note that some binary characters are allowed in
email. All ASCII characters are allowed, including control
characters. Characters with values greater than 127 are not allowed.
Either way, if VM can't handle it, that's probably a bug in VM.
>2. There is an I/O or memory or motherboard problem?
Unlikely, especially if no other problems anywhere in the system.
>The dilemma I face is to determine whether I should abandon my
>machine...
Way too soon for that. The thing to do is just find out what's in
that file.
Ralf Fassel
| I tried to open the INBOX file in Accessories->Text Editor, and the
| program wouldn't load the file since there were binary characters in
| the file.
You get what you deserve when trying to work around the
One True Editor :-)
Why don't you open the INBOX in emacs itself? Emacs deals with binary
content quite fine. Close VM so that INBOX is no longer in the buffer
list, then M-x find-file-literally RET ~/INBOX RET to avoid any fancy
guessing on emacs' side what the file might be good for
Then M-x re-search-forward RET [ C-q 2 0 0 - C-q 2 7 7 ]
This should find any character above ASCII 127.
C-s to search further.
E.g. I have some mails in INBOX which have
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
No problem in VM though..
HTH
R'
tempest
> Then M-x re-search-forward RET [ C-q 2 0 0 - C-q 2 7 7 ]
>
Thank you Raif and Kurt,
Here's my progress report.
I have followed your suggestions and done some sleuthing too.
1. I've looked for non-utf characters. The file is filled with them!
and justifiably so, because there are zip file attachments etc... and
accented characters and messages written in us-ascii, iso-8859-1,
iso-8859-15, windows-1252, and utf-8 etc.
2. I moved INBOX to another location and started with a clean INBOX
3. loaded the old inbox into evolution and that works fine, but saving
the messages to a new file and trying to get emacs vm to read it does
not work so well; and it does not appear that code has been written to
convert evolution style mailboxes to vm format.
In one test I did try to remove all of the non-utf characters using a
perl script. the vm-decode-mime-encoded-words-in-string: Wrong type
argument: stringp, =\?utf-8\?Q\? error disappears, but then I get a
wrong type argument: markerp, 55504 error.
This makes me believe that the file contains something in its
"information" portion/headers that emacs-vm does not like.
I ran M-x debug after the crash to try to figure out where in the file
the crash is occuring. I was lead to believe that a zip file
attachment was making it crash, and after some attempts to alter the
inbox to remove the archive, I found that the stringp error
disappeared but I can't avoid the markerp that then occurs.
I also tried to create inbox files out of a few messages and that
works fine. so theoretically I can salvage the file one message at a
time. but I have over 590 messages in the file!
What I need is a way to go through the inbox one message at a time and
have emacs vm tell me which message it doesn't like so I can remove
it. I suppose I can write a perl script to output each message using
"^From " as a deliminter. but then I'll have to read in each file as a
mail folder and see if emacs complains.
Does emacs vm has a way to quickly identify the problem message? or at
least can it output messages that are error free?
Ralf Fassel
| What I need is a way to go through the inbox one message at a time
| and have emacs vm tell me which message it doesn't like so I can
| remove it.
1) Divide the INBOX in half, try to load via VM.
2) a) If error: goto 1)
b) If no error: use second half, goto 1)
Should get you to the offending message quite quickly.
Messages are delimited by "^From " as you already know.
| 1. I've looked for non-utf characters. The file is filled with them!
| and justifiably so, because there are zip file attachments etc... and
| accented characters and messages written in us-ascii, iso-8859-1,
| iso-8859-15, windows-1252, and utf-8 etc.
? At least ZIP-Archives should never be attached in binary format but
almost always base64- or otherwise ASCII-armored. And the non-ASCII
messages are usually also in quoted-printable or some other form of
ASCII-encoding, since 8-bit transport is not guaranteed with SMTP...
R'
tempest
> 1) Divide the INBOX in half, try to load via VM.
> 2) a) If error: goto 1)
> b) If no error: use second half, goto 1)
>
ok. I saw that my inbox had 591 messages in it and the header x-vm-
message-order indicated that 560 of them were ordered. the remaining
31 were not even being tracked in this message-order. starting from
there i divided the inbox into its first 560 messages and remaining
31. The first 560 loaded fine. The remaining 31 bombed on the 30th
message. The 31st is ok.
Here are the contents of the 30th message. I believe it crashes when
it encounters the string =?utf-8?Q?
I receive email from Le Monde all the time. If this happens again, I'd
like to know how I can search for and replace the sequence that causes
emacs vm to crash.
From owner-al...@retour.listes.lemonde.fr Thu Dec 4 08:00:03
2008
X-VM-VHeader: ("Resent-" "From:" "Sender:" "To:" "Apparently-To:"
"Cc:" "Subject:" "Date:") nil
X-VM-Bookmark: 1
X-VM-Message-Order:
(1)
X-VM-Labels: nil
X-VM-Summary-Format: "%n %*%a %-17.17F %-3.3m %2d %4l/%-5c %I\"%s\"\n"
X-VM-IMAP-Retrieved: nil
X-VM-POP-Retrieved: nil
X-VM-Last-Modified: (18757 29003 729082)
X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil]
["4578" "Thursday" "4" "December" "2008" "13:54:02" "+0100" "Le
Monde.fr" "lis...@listes.lemonde.fr" nil "59" =?utf-8?Q?#
("Derni=C3=A8re?= minute : La BCE abaisse de 0,75 point son principal
taux directeur =?utf-8?Q?=C3=A0?= 2,50 %" 0 36 (vm-coding iso-latin-9
vm-charset "ISO-8859-15" vm-string t) 36 37 nil 37 76 (vm-coding iso-
latin-9 vm-charset "ISO-8859-15" vm-string t) 76 77 nil 77 85 (vm-
coding iso-latin-9 vm-charset "ISO-8859-15" vm-string t)) "^From:" nil
nil "12" nil nil (number " " mark " Le Monde.fr Dec 4
59/4578 " thread-indent "\"=?ISO-8859-15?Q?
Derni=E8re_minute_:__La_BCE_abaisse_de?= =?ISO-8859-15?Q?
0,75_point_son_principal_taux_directeur?= =?ISO-8859-15?Q?=E0_2,50 %?=
\"\n") nil nil nil nil nil nil nil]
nil)
Return-Path: <owner-al...@retour.listes.lemonde.fr>
Received: from lafontaine.localdomain (localhost.localdomain
[127.0.0.1])
by localhost.localdomain (8.14.3/8.14.3) with ESMTP id mB4D028K004249
for <joel@localhost>; Thu, 4 Dec 2008 08:00:03 -0500
Status: U
Received: from pop.earthlink.net [209.86.93.211]
by lafontaine.localdomain with POP3 (fetchmail-6.3.8)
for <joel@localhost> (single-drop); Thu, 04 Dec 2008 08:00:03 -0500
(EST)
Received: from mx-taint.atl.sa.earthlink.net ([127.0.0.1])
by mx-taint.atl.sa.earthlink.net (EarthLink SMTP Server) with SMTP id
1l8dKM1vJ3Nl36L0; Thu, 4 Dec 2008 07:56:24 -0500 (EST)
Received: from listes.lemonde.fr ([195.154.120.159])
by mx-taint.atl.sa.earthlink.net (EarthLink SMTP Server) with ESMTP
id 1l8dKL43x3Nl36L0
for <joelv...@earthlink.net>; Thu, 4 Dec 2008 07:56:23 -0500 (EST)
Received: by listes.lemonde.fr (Postfix)
id 5CA0B3A092B; Thu, 4 Dec 2008 13:55:04 +0100 (CET)
Delivered-To: alerte-html_1126...@listes.lemonde.fr
Received: by listes.lemonde.fr (Postfix, from userid 48)
id 1DF983A0A14; Thu, 4 Dec 2008 13:54:50 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/html; charset="iso-8859-15"
Reply-To: lis...@lemonde.fr
Content-Transfer-Encoding: quoted-printable
Message-Id: <2008120412545...@listes.lemonde.fr>
X-ELNK-Received-Info: spv=0;
X-ELNK-AV: 0
X-ELNK-Info: sbv=1; sbrc=-0; sbf=eb; sbw=000;
From: "Le Monde.fr" <lis...@listes.lemonde.fr>
Sender: "Le Monde.fr - lettre derniere minute (version HTML)" <alerte-
ht...@listes.lemonde.fr>
To: alert...@listes.lemonde.fr
Subject: =?ISO-8859-15?Q?
Derni=E8re=20minute=20:=20=20La=20BCE=20abaisse=20de?=
=?ISO-8859-15?Q?=200,75=20point=20son=20principal=20taux=20directeur?
=
=?ISO-8859-15?Q?=20=E0=202,50=20%?=
Date: Thu, 4 Dec 2008 13:54:02 +0100
<html><head></head><body>=0D=0A<style>=0D=0A.lm_Type_nl {font: bold
16px ver=
dana; color: #de0018 =09}=0D=0A.lm_Date {font: 16px verdana; color:
#de00=
18 }=0D=0A.lm_Titre =09{font: bold 14px verdana; padding-left: 20px=09}
=0D=
=0A.lm_Texte {font: 11px verdana }=0D=0A.lm_Infos {font: bold 11px
verda=
na; color: #87000A }=0D=0Aa.lm_Texte { text-decoration: underline;
color: #=
009 }=0D=0A</style>=0D=0A=0D=0A<!-- CONTENU -->=0D=0A<table
border=3D"0" ce=
llpadding=3D"0" cellspacing=3D"0" width=3D"100%">=0D=0A
<tbody><tr>=0D=0A =
<td bloc=3D"true" id=3D"toutderniereminute" style=3D"-moz-user-
focus: n=
ormal;">=0D=0A <table border=3D"0" cellpadding=3D"5"
cellspacing=3D"=
0" width=3D"600"> =0D=0A <tbody><tr>=0D=0A
<td wi=
dth=3D"8"></td>=0D=0A <td
width=3D"584">=0D=0A =
<div class=3D"lm_Type_nl" id=3D"bloctitre_alerte">LE MONDE.FR :
Derni=E8re=
minute</div>=0D=0A <div class=3D"lm_Date"
id=3D"date_du_jo=
ur">jeudi 4 d=E9cembre 2008</div>=0D=0A </
td>=0D=0A =
<td width=3D"8"></td>=0D=0A </tr>=0D=0A
<br>=0D=
=0A <tr>=0D=0A <td colspan=3D"3"
align=3D"left" bgc=
olor=3D"#edd9db" valign=3D"middle">=0D=0A <div
class=3D"lm_=
Titre" id=3D"titre_alerte"> La BCE abaisse de 0,75 point son principal
taux =
directeur =E0 2,50 %</div>=0D=0A </td>=0D=0A
</tr><=
tr>=0D=0A <td width=3D"8"></td>
=0D=0A <td =
width=3D"584">=0D=0A <div class=3D"lm_Texte"
id=3D"descr_al=
erte">La Banque centrale europ=E9enne a d=E9cid=E9, jeudi 4
d=E9cembre, d'ab=
aisser de 0,75 point de pourcentage =E0 2,50 % son principal taux
directeur.=
C'est la plus forte r=E9duction de son histoire. (AFP)</
div>=0D=0A =
</td>=0D=0A <td width=3D"8"></td>=0D=0A
</tr>=
=0D=0A <br>=0D=0A <tr>=0D=0A <td
width=
=3D"8"></td>=0D=0A <td
width=3D"584">=0D=0A <=
div class=3D"lm_Texte" id=3D"infos"><span class=3D"Infos">Plus
d'information=
s </span>sur Le Monde.fr dans quelques minutes. - <a href=3D"http://
www.lemo=
nde.fr/?xtor=3DEPR-32280012">http://www.lemonde.fr</a></
div>=0D=0A =
</td>=0D=0A <td width=3D"8"></td>=0D=0A
</tr>=
=0D=0A </tbody></table>=0D=0A </td>=0D=0A </tr>=0D=0A</
tbody>=
</table>=0D=0A=0D=0A=0D=0A<br><br><table width=3D"600" border=3D"0"
cellpadd=
ing=3D"0" cellspacing=3D"0"><tr>=0D=0A<td> </td></
tr><tr>=0D=0A<td><hr =
size=3D"1"></td></tr><tr>=0D=0A<td> </td></tr><tr>=0D=0A<td
style=3D"pa=
dding:3">=0D=0A=09<font face=3D"verdana" size=3D"1"><b>Informations :</
b>=
=0D=0A <br><br>Vous recevez cette lettre car vous êtes
inscrit(e)=
à la liste de diffusion 'alerte' du Monde.fr.=0D=0A=09
<br>Pour mo=
difier vos inscriptions aux lettres d'information du Monde.fr, <a
href=3D"ht=
tp://www.lemonde.fr/web/ep/1,27-0,1-0,0.html">cliquez ici</
a>=0D=0A=09 <br=
>Pour ne plus recevoir cette lettre d'information, <a href=3D"http://www.lem=
onde.fr/web/anl/1,28-0,1-0,0.html?nl=3Dalerte">cliquez ici</
a>=0D=0A=09<br><=
/font></td> =0D=0A</tr><tr><td> </td></tr><tr><td><hr
size=3D"1"></td><=
/tr><tr><td> </td></tr><tr>=0D=0A<td style=3D"padding:
3">=0D=0A=09<font=
face=3D"verdana" size=3D"1"><span >Droits de <a href=3D"http://
www.lemonde.=
fr/web/article/0,1-0@2-3388,36-139039,0.html"><u>reproduction</u></a>
et de =
<a href=3D"http://www.lemonde.fr/web/article/
0,1-0@2-3388,36-139039,0.html">=
<u>diffusion</u></a> réservés © <STRONG>Le
Monde.fr</STRO=
NG> 2008=09<br><STRONG>Usage strictement personnel.</STRONG>
L'utilisateur d=
u site reconnaît avoir pris connaissance de la <a href=3D"http://
www.l=
emonde.fr/web/article/0,1-0@2-3388,36-139039,0.html"><u>licence</u></
a> de d=
roits d'usage, en accepter et en respecter les dispositions.=0D=0A
<br><a =
href=3D"http://www.lemonde.fr/web/article/
0,1-0@2-3388,36-139040,0.html"><u>=
Politique</u></a> de confidentialité du site. </span><a
href=3D"http:=
//faq.lemonde.fr"><u>Besoin d'aide ? faq.lemonde.fr</u></a> <a
href=3D"=
http://www.lemonde.fr/web/sequence/0,2-3386,1-0,0.html"><u>Qui sommes-
nous ?=
</u></a></span></font>=0D=0A</td> =0D=0A</tr><tr>=0D=0A<td> <IMG
height=
=3D"1" src=3D"http://logc2.xiti.com/hit.xiti?
s=3D43260&s2=3D3&p=3DNL_alerte&=
" width=3D"1"></td></tr></table>=0D=0A</body>=0D=0A</html>=0D=0A
Kurt Hackenberg
tempest <joelv...@earthlink.net> wrote:
>ok. I saw that my inbox had 591 messages in it and the header x-vm-
>message-order indicated that 560 of them were ordered. the remaining
>31 were not even being tracked in this message-order. starting from
>there i divided the inbox into its first 560 messages and remaining
>31. The first 560 loaded fine. The remaining 31 bombed on the 30th
>message. The 31st is ok.
>
>Here are the contents of the 30th message. I believe it crashes when
>it encounters the string =?utf-8?Q?
...
>X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil]
> ["4578" "Thursday" "4" "December" "2008" "13:54:02" "+0100" "Le
>Monde.fr" "lis...@listes.lemonde.fr" nil "59" =?utf-8?Q?#
>("Derni=C3=A8re?= minute : La BCE abaisse de 0,75 point son principal
>taux directeur =?utf-8?Q?=C3=A0?= 2,50 %" 0 36 (vm-coding iso-latin-9
>vm-charset "ISO-8859-15" vm-string t) 36 37 nil 37 76 (vm-coding iso-
>latin-9 vm-charset "ISO-8859-15" vm-string t) 76 77 nil 77 85 (vm-
>coding iso-latin-9 vm-charset "ISO-8859-15" vm-string t)) "^From:" nil
>nil "12" nil nil (number " " mark " Le Monde.fr Dec 4
>59/4578 " thread-indent "\"=?ISO-8859-15?Q?
>Derni=E8re_minute_:__La_BCE_abaisse_de?= =?ISO-8859-15?Q?
>0,75_point_son_principal_taux_directeur?= =?ISO-8859-15?Q?=E0_2,50 %?=
>\"\n") nil nil nil nil nil nil nil]
> nil)
Ah. I bet that's the same bug somebody else just reported.
X-VM-v5-Data is, I think, combined VM-specific stuff (like message
attributes), and cached, parsed copies of stuff in other headers. Its
value looks like Lisp read syntax. I bet that's a speed hack: VM just
feeds that to the Lisp reader and sucks the whole thing straight in.
That questionable string is probably a copy of non-ASCII, which the
Lisp reader probably can't handle.
You like all that guesswork? I haven't looked at code or anything.
Chris McMahan
behavior for several years now.
- Chris
k...@pnnnnx.kom (Kurt Hackenberg) writes:
--
(. .)
=ooO=(_)=Ooo=====================================
Chris McMahan | first_init...@one.dot.net
=================================================
Aidan Kehoe
Ar an ceathrú lá de mí na Nollaig, scríobh tempest:
My workaround for the moment is to correct the X-VM-v5-Data headers to make
them valid Lisp. So:
X-VM-v5-Data: ([nil nil t nil nil nil nil nil nil]
["113" "Tuesday" "11" "November" "2008" "14:02:18" "+0000" "Homer Simpson" "hsim...@burns-power.com" nil "4" =?iso-8859-1?Q?"D=E9j=E0_Jou=E9"?= "^From:" nil nil "11" nil nil (number " " mark "D Homer Simpson Nov 11 4/113 " thread-indent "\"=?iso-8859-1?Q?D=E9j=E0_Jou=E9?=\"\n") nil nil nil nil nil nil nil]
nil)
becomes
X-VM-v5-Data: ([nil nil t nil nil nil nil nil nil]
["113" "Tuesday" "11" "November" "2008" "14:02:18" "+0000" "Homer Simpson" "hsim...@burns-power.com" nil "4" "=?iso-8859-1?Q?\"D=E9j=E0_Jou=E9\"?=” "^From:" nil nil "11" nil nil (number " " mark "D Homer Simpson Nov 11 4/113 " thread-indent "\"=?iso-8859-1?Q?D=E9j=E0_Jou=E9?=\"\n") nil nil nil nil nil nil nil]
nil)
Hope that helps a little,
Aidan
--
¿Dónde estará ahora mi sobrino Yoghurtu Nghé, que tuvo que huir
precipitadamente de la aldea por culpa de la escasez de rinocerontes?
Skip Montanaro
> them valid Lisp. So:
>
That doesn't help me. I have ripped out all my X-VM-v5-Data headers.
I see no
high-bit characters (\200-\377) either. Still, I get this error. My
only solution
was to stop and restart Aquamacs. According to he output stepping
through the
failing function VM was trying to reference a marker which referred to
a deleted
buffer.
Skip Montanaro