Eudora .TOC format
Robert J. Baker
(My apologies if this has already been posted by others.)
You may remember me asking about this three months ago, under my then alias
of rba...@legend.co.uk; I received two responses, which unfortunately
disagreed with each other on the position or length (in a few cases, even
the type) of some of the fields. So, since I've recently had cause to do a
bit more reasearch on this (Eusless trashed my outbox -- again! -- and it
took three days with a disk sector editor to put it to rights), I've decided
to repost what I think is the correct information.
The sources for the following are indicated as follows: [ML] == Martim
Lyngvig (mar...@acm.org); [RA] == Ram Avrahami (avra...@gramercy.ios.com);
no attribution == both of the former two; [RJB] == myself
(rober...@bigfoot.com).
File header, 104 (0x68) bytes: (absolute addresses)
dec hex type purpose
0 00 dlong? unknown
8 08 ASCIIZ mailbox name
40 28 long? mailbox type: 0=in, 1=out, 2=trash, 3=user [ML]
44 2C long? unknown
48 30 ??? unknown -- even whether 3*int, or int followed by long,
or whatever
54 36 ???? 16*0xff -- why?
70 46 int? unknown
72 48 ???? 30*0x00 -- reserved for future enhancements?
102 66 int number of messages in mailbox
Message header, 218 (0xda) bytes: (addresses relative to start of header)
dec hex type purpose
0 00 long position in .MBX of this message [RA]
4 04 long message length [RA]
8 08 long? message date and time? [RA]
12 0C int? message type: [RJB]
(in the following, .=blank; *=solid round bullet)
Value
| Flag Character
| | Meaning
0 * unread
1 . read
2 R replied
3 F forwarded
4 D redirected
5 . (alternate value for read?)
6 * unsent
7 Q queued
8 S sent
9 - indeterminate; Eudora probably found this message to be
of an invalid type for the type of mailbox, e.g. type 7
messages are converted to type 9 if transferred from the
Out box
14 0E bitstring [0]=signature in use is alternate one (Eudora Pro) [RJB]
[1]=signature used (in Eudora Pro, either one) [ML]
[2]=word wrap [ML]
[3]=tabs in message body [RJB]
[4]=keep copy [ML]
[5]=text attachment in message body [ML]
[6]=include "Return-Receipt-To:" request in sent message
header (Eudora Pro) [RJB]
[7]=Quoted-Printable [ML]
N.B. If [0] is set, so is [1].
15 0F bitstring [0]=MIME [ML]
[1]=UUcode (Eudora Pro) [RJB]
(If neither of these bits are set, BinHex is used.)
[2]..[6] Presumably reserved for future enhancements
[7]=attachment present [ML]
16 10 int? priority: 1=highest, 3=normal
18 12 ASCIIZ date/time sent/received
50 32 ASCIIZ Sender/recipient
114 72 ASCIIZ Subject
178 B2 unknown First few seem always to be 0xFF
186 BA int? Label (0=none) (Eudora Pro 2.1.2 onward) [RA]
188 BC reserved? Always seems to be 30*0x00; probably reserved for future
enhancements
n.b. int=2 bytes, long=4 bytes, dlong=8 bytes, ASCIIZ=string terminated by
0x00 (and rest of space padded with 0x00, in this case), bitstring=up to 8
Boolean variables packed into a byte ([x]=bit x of the string)
By the way, I think that if the above is accurate, it shows slightly sloppy
design on Qualcomm's part, as the message headers are four bytes longer than
they need to be; the 'length' fields are superfluous, as the length of each
message can be calculated by simply subtracting its pointer from that of the
following message. (Assuming that the .TOC entries are in ascending order.)
Of course, this would need a dummy last record pointing to the EOF of the
MBX, but this would consist only of the pointer.
Still, the above provides writers of TOC editors with a check on the message
length, except of course for the last message.
--------------------------------------------------------------------
This posting was from ROBERT.BAK
N.B. Due to ridiculously short article expiry times on my local news server,
plus the fact that I cannot guarantee affording more than one Net session per
week (they cost me #4 per hour -- a lot of money to me), please e-mail me a
copy of any follow-up. Thanks.
Scooper Ong
"Robert J. Baker" <rober...@bigfoot.com> wrote in
comp.mail.eudora.ms-windows:
I got this from Nick Spalding, INTERNET:spal...@iol.ie :
1. Header record is 104 bytes
Offset Content
0-7 Unknown, possibly a version identifier.
8-35 Mailbox Name, padded with hex 00
36 Mailbox type where:
0 In
1 Out
2 Trash
3 User defined mailbox
37-101 Unknown
102-103 Count of messages in the box
2. Message entry is 218 bytes. Layout as follows
Offset Content
0-7 Something with unknown significance which doesn't vary when
status of message changes, eg from queued to sent
8-11 Something with unknown significance which varies when status
changes. I think this has something to do with length
and/or position of message in the .mbx
12 Status of message.
0 Record received, not yet read
1 Record received and read
2 Record received and forwarded
3 ??
4 Record received and redirected
5 ??
6 Record built in out message then saved, not queued
7 Record queued for transmission
8 Record sent
13-17 ??
18-39 Time/date/GMT offset, padded with binary zeroes
40-103 Sender ditto
104-167 Subject ditto
168-175 X'FFFFFFFFFFFFFFFF'
176-217 Binary zeroes
Roger Hill
Robert J. Baker wrote:
> You may remember me asking about this three months ago . . .
> I've decided to repost what I think is the correct information.
> . . . [wholesale snipping here]
Thanks very much for posting your documentation on the Eudora .TOC
format! It is the most complete listing that I have seen yet. Here is
some more information that you might find useful:
(1) Jeramie Hicks <Jerami...@mail.utexas.edu>, who had figured out
much of the TOC, noted a couple of months ago in this newsgroup that
bytes 00-05 in the file header seem to have some ASCII mailbox or Eudora
version information; I believe he indicated that they were "1.0a15" in
version 1.4.4. On the other hand, I found with Eudora Light 1.5.2 and
1.5.4 that those bytes are just 2A 00 00 00 00 00.
(2) Jeff Woolf <jwo...@ix.netcom.com>, who has also been on this
newsgroup, found that the 4-byte long integer starting at location 08 in
the message header does indeed encode the date and time, converted to
GMT, and that this field (rather than the ASCII date field that you see
in the message summary) is what Eudora uses to sort messages by
date/time. The integer is the number of seconds since 01 Jan 1970 (I'm
not sure about the time zone of the latter).
(3) The message length field (the 4 bytes starting at location 04 in the
message header) is not superfluous, because the TOC entries are *not*
always in ascending order. I found that if a mailbox is sorted (by
sender, date/time, subject, etc.), the message summaries are rearranged
in the .TOC file but the .MBX file is untouched. If and when the
mailbox is compacted, however, the messages in the .MBX file will be
rearranged to agree with the order in the .TOC file (and of course
"deleted" messages thrown out), and the message locations in the .TOC
file updated. So in general it would be rather tricky to calculate the
message lengths from just the pointers to the beginnings of the
messages.
(4) According to my experiments, the 8 bytes starting at location 2E
(hex) in the file header are four 2-byte integers which appear to give
the horizontal and vertical coordinates (in pixels) of the upper-left
corner of the mailbox window (when it is not maximized) and the
horizontal and vertical coordinates of the lower-right corner of that
window. These are relative to the upper-left corner of the part of the
main window that is usable for sub-windows.
(5) Similarly, the 8 bytes starting at location B2 (hex) in each message
header are four 2-byte integers which appear to give the horizontal and
vertical coordinates (in pixels) of the upper-left corner of the message
window (when it is not maximized) and the horizontal and vertical
coordinates of the lower-right corner of that window. If the window
hasn't been moved or resized from the default, the bytes are all FF.
It's all an interesting puzzle to be solved!
--- Roger Hill
(P.S. If anyone has a follow-up of this, I would appreciate getting a
copy via E-mail, as I'm going to be traveling during the next week and
don't know whether newsgroup messages be "non-volatile" enough for me to
get them when I am back in contact.)
John Walker
Jeramie Hicks wrote:
>
> Roger Hill <rh...@siue.edu> wrote:
>
> >Robert J. Baker wrote:
> >> You may remember me asking about this three months ago . . .
> >> I've decided to repost what I think is the correct information.
>
> >much of the TOC, noted a couple of months ago in this newsgroup that
> >bytes 00-05 in the file header seem to have some ASCII mailbox or Eudora
> >version information
>
> excellent detective work and record keeping of past discoveries.
>
> Second, I do not believe anybody specifically stated it, but there was
> evidently a significant format change between v1.4.4 and later
> versions. Anything I originally posted works successfully only with
> v1.4.4 files :-(
Am I missing something here: if you eliminate the .toc
file, the program will rebuild it for you.
John
John Walker
Jeramie Hicks wrote:
> >Am I missing something here: if you eliminate the .toc
> >file, the program will rebuild it for you.
>
> TOC files from the MBX file alone. Therefore, at least in my 1.4.4
> version, you'll get one huge message containing the entire MBX
> contents.
Ah. I suggest upgrading to 1.54 at least! I wrote an MS Access
database that sucked up all my mail, splitting the addresses
out into a separate table, and doing some simple indexing for the
various mailing lists I am on. I was constantly having to reload
the mailboxes during testing, but found I didn't need the .toc
files. Eudora would just rebuild them...although I think you might
need an empty file with the right name to begin with.
> Secondly, as I programmer, my personal interest is to use the TOC
> format as a de facto standard for my homemade Email programs. That
> way, I can use Eudora to test parts of my program during development,
> and my program will be compatible with a very popular program already.
Great. But you can still do this with 1.54 and save yourself the
hassle of _having to_ recreate the .toc files.
James Robertson
In article <4s5mep$e...@shell.fore.com>, so...@fore.com says...
>
>"Robert J. Baker" <rober...@bigfoot.com> wrote in
>comp.mail.eudora.ms-windows:
>
>>(My apologies if this has already been posted by others.)
>
>>You may remember me asking about this three months ago, under my then alias
>>of rba...@legend.co.uk; I received two responses, which unfortunately
>>disagreed with each other on the position or length (in a few cases, even
>>the type) of some of the fields. So, since I've recently had cause to do a
>>bit more reasearch on this (Eusless trashed my outbox -- again! -- and it
>>took three days with a disk sector editor to put it to rights), I've decided
>>to repost what I think is the correct information.
>
>>The sources for the following are indicated as follows: [ML] == Martim
>>Lyngvig (mar...@acm.org); [RA] == Ram Avrahami (avra...@gramercy.ios.com);
>>no attribution == both of the former two; [RJB] == myself
>>(rober...@bigfoot.com).
>
I'm not sure I've still got a copy floating about, but I simply asked
Qualcomm. They were happy to send me a complete and accurate description
of the TOC file format.
I've since used this without problems.
J
--
----------------------------------------------------------------------------
James Robertson * jam...@desklaw.com.au
Coding * Design * Layout * Newton * Windows
"Beyond the idea"
Jeramie Hicks
Roger Hill <rh...@siue.edu> wrote:
>Robert J. Baker wrote:
>> You may remember me asking about this three months ago . . .
>> I've decided to repost what I think is the correct information.
>(1) Jeramie Hicks <Jerami...@mail.utexas.edu>, who had figured out
>much of the TOC, noted a couple of months ago in this newsgroup that
>bytes 00-05 in the file header seem to have some ASCII mailbox or Eudora
>version information
First, I'd like to thank Robert J. Baker and Roger Hill for their
excellent detective work and record keeping of past discoveries.
Second, I do not believe anybody specifically stated it, but there was
evidently a significant format change between v1.4.4 and later
versions. Anything I originally posted works successfully only with
v1.4.4 files :-(
Third, keep up the good work, guys.
- Hicks
Jeramie Hicks
John Walker <John_...@mindlink.bc.ca> wrote:
>Am I missing something here: if you eliminate the .toc
>file, the program will rebuild it for you.
Well, first Eudora does not have sufficient information to rebuild the
TOC files from the MBX file alone. Therefore, at least in my 1.4.4
version, you'll get one huge message containing the entire MBX
contents.
Secondly, as I programmer, my personal interest is to use the TOC
format as a de facto standard for my homemade Email programs. That
way, I can use Eudora to test parts of my program during development,
and my program will be compatible with a very popular program already.
- Hicks
Roland Bauer
On 15 Jul 1996 01:03:47 GMT, jam...@magna.com.au (James Robertson)
wrote:
>I simply asked
>Qualcomm. They were happy to send me a complete and accurate description
>of the TOC file format.
Is there a webpage with this information at Qualcom?
Thanks,
Roland
--
Roland...@bigfoot.com (free) >>> http://www.bigfoot.com
work +43(1) 58-801 / 4861
fax +43(1) 58-6-58-47
http://www.ft.tuwien.ac.at/~roland
http://www.geocities.com/BourbonStreet/1877 (free homepage)
--
Scooper Ong
John Walker <John_...@mindlink.bc.ca> wrote in
comp.mail.eudora.ms-windows:
>Jeramie Hicks wrote:
>> >Am I missing something here: if you eliminate the .toc
>> >file, the program will rebuild it for you.
>>
>> Well, first Eudora does not have sufficient information to rebuild the
>> TOC files from the MBX file alone. Therefore, at least in my 1.4.4
>> version, you'll get one huge message containing the entire MBX
>> contents.
>Ah. I suggest upgrading to 1.54 at least! I wrote an MS Access
>database that sucked up all my mail, splitting the addresses
>out into a separate table, and doing some simple indexing for the
>various mailing lists I am on. I was constantly having to reload
>the mailboxes during testing, but found I didn't need the .toc
>files. Eudora would just rebuild them...although I think you might
>need an empty file with the right name to begin with.
Your method is fine only if you don't intend to maintain message
statuses (read, unread, replied, etc) and message label. These are
maintained in the TOC file. These information are lost if you delete
and create TOC files.
>> Secondly, as I programmer, my personal interest is to use the TOC
>> format as a de facto standard for my homemade Email programs. That
>> way, I can use Eudora to test parts of my program during development,
>> and my program will be compatible with a very popular program already.
>Great. But you can still do this with 1.54 and save yourself the