Disappearing emails

[David Thomas]

TB 17.06 on Win XP Pro SP3

All emails in one of my folders have suddenly disappeared. I don't think I
have caused it, but, of course, I can't be sure. All other folders appear
to be OK. The folder in my profile is showing a size of 0 KB. A backup done
on Monday shows a size of 23,716 KB. The .msf file for that folder is also
much smaller now than it was on Monday.

I have tried, using my backup software, to recover both files to their
original locations, but nothing seems to have happened.

Any suggestions please?

[Christian Riechers]

What's your account type - POP or IMAP?
This article has some suggestions for troubleshooting.
http://kb.mozillazine.org/Disappearing_mail

--
Christian

[Eckard]

Le 22/05/13 13:17, David Thomas a écrit :

Open the TB profile folder and quit TB.
Delete the .msf file of the "empty" folder (or is it a file?) showing 0 KB.
Restart TB.
If this is to no avail, quit TB again and manually replace the 0 KB
folder (or file?) by a copy of the equivalent folder (or file) from your
backup done on Monday. Delete the .msf file having the same name as this
folder (or file).
Restart TB.

[David Thomas]

Thanks to both for your suggestions but the problem remains. Nothing in
Mozillazine helped. I followed Eckard's advice, but, when I re-opened TB,
the folder was empty and the folder's file in my profile had reverted to 0 KB.

My account type is POP.

Maybe my folder is so badly corrupted that it is not recoverable?

[Was Greywolf]

See if the folder shows in Trash. You should see Inbox and inbox.msf
file. Try Restoring Inbox (you can ignore *.msf, TB rebuilds those files
if necessary).

However, even if Inbox doesn't show in Trash, it might still be
recoverable. Try a file recovery utility such a Recuva (free) from
Piriform. I've used it, it will find files no longer listed in Trash. NB
that if any of the sectors have been overwritten, you'll have to do a
deep scan, which can take a _very_ long time.

The utility works because deleting doesn't actually delete anything, it
just changes the MFT (Master File Table) entry to show the file as
deleted. the Trash folder lists the files you've deleted. When you
"restore" a file, the MFT is updated. A deep scan can find overwritten
data because overwriting doesn't completely obliterate the old data.
That's the reason you need to "wipe" a drive if you want to ensure date
is unrecoverable. Wiping overwrites the old data several times.

Good luck,
--
Best,
Wolf K.

[Eckard]

Le 23/05/13 04:28, David Thomas a écrit :

Did you manually copy the folder's 23,716 KB file (without the
corresponding .msf file) from your Monday backup into your profile
*after* deleting the existing empty file and its corresponding .msf file
before restarting TB?
Is the empty folder a standard folder or was it created by you?
You also might try to import this folder's file from your backup using
the add-on ImportExportTools
<http://nic-nac-project.de/~kaosmos/mboximport-en.html>

[David Thomas]

Yes, I did a manual copy and paste, but, to be certain, I did it again this
morning. Same result.
The empty folder was created by me and has been working fine for 2-3 years
until now. I have automatic compacting enabled.
I downloaded the add-on and tried to use it, but it seems to accept imports
of .csv and .ics formats only, so no luck.
Thanks for your advice to date.

[Eckard]

You didn't use ImportExportTools the right way.
Right-click on any folder in your account pane (e.g. Inbox) ->
ImportExportTools -> "Import mbox file", in the following popup window
choose "Import directly one or more mbox files", then navigate to the
backup of your profile, select the file of your "empty" folder and click
"Open".
<http://i1274.photobucket.com/albums/y440/mapenzi5/Import_Export_zpsb6c93905.png>
Now the imported mbox file should be displayed as a new sub-folder in
your folder pane. Click on the imported folder to see whether the
messages are displayed.
If you can't import the backed up file, then it might be corrupted:
Also see http://kb.mozillazine.org/Empty_folders

[David Thomas]

OK - did that, but folder still empty.
I think I must accept that the file is corrupted and move on.
I can open the file in a text editor, and the emails seem to be there, but,
of course, finding them and reading them among all the coding, etc is not easy.
Thanks, anyway, for all your efforts to help.

[Eckard]

Le 25/05/13 04:02, David Thomas a écrit :

Even if the file is corrupted there might be a chance to recover the
messages or at least a part of them:
<http://kb.mozillazine.org/Recover_messages_from_a_corrupt_folder>

You can try using the tools "Cut Mbox", "Cut Mbox D" or "Cut to eml"
<http://apveening.a61.nl/software/>

[Was Greywolf]

On 5/24/2013 10:02 PM, David Thomas wrote:
[...]

> I can open the file in a text editor, and the emails seem to be there,
> but, of course, finding them and reading them among all the coding, etc
> is not easy.

OK, that means the emails haven't disappeared. The file that contains
them exists. It may be salvageable. Shut down Tbird first.

Use the text editor's Save As to resave the file, call it Letters-1 or
something like that. Make sure a copy of the re-saved file is in the
correct location in your TBird Profile. Copy it to there if necessary.
Restart Tbird, it should create a Letters-1.msf file, and you should see
the folder Letters-1.

Good luck.

--
Best,
Wolf K.

[David Thomas]

Same result, I'm afraid. The folder showed up but without any emails.

I don't have the time to edit the mbox file (about 24MB) and I suspect I
don't have the required IT smarts or the patience anyway.

I'm calling it quits now. I really appreciate everyone's suggestions and
efforts to help me solve this problem.

[John H Meyers]

On 5/25/2013 9:57 AM, Was Greywolf wrote:

> On 5/24/2013 10:02 PM, David Thomas wrote:
> [...]
>> I can open the file in a text editor, and the emails seem to be there,
>> but, of course, finding them and reading them among all the coding, etc
>> is not easy.

> OK, that means the emails haven't disappeared. The file that contains them exists.
> It may be salvageable. Shut down Tbird first.
>

> Use the text editor's Save As to re-save the file

I would not rely on a text editor to attempt to duplicate any file,
even if you think there is only text in the file, because various
text editors may stumble and fail to exactly duplicate the original file
if it contains certain particular characters (e.g. null bytes),
or lines longer than some particular limit, or unencoded binary data,
or certain unexpected character encodings, etc.

Some text editors might even change some text line endings
(or all of them), which I've seen make a radical difference
to the subsequent interpretation of some mailboxes.

It would not be difficult to use a script to "split"
any file appearing to be in "MBOX" format into a collection
of individual messages, each in its own file within a subfolder,
which then could be explored to see whether most individual messages
are recovered, in case the problem is localized to only
one (or a few) of such parts. Thunderbird-specific flag headers
might also be removed, to see whether it's those which are
"making the messages disappear from view."

I get the feeling that there's more to this story than has yet
been stated, and that there's a missing piece of information,
unknowable to anyone else, which would make it obvious to even lesser
detectives than Sherlock Holmes what the culprit actually is here.

For example, what about account settings which specify
the path to use for storing an account's files?
What if, for example, two accounts now specify the same path?

Good luck, in any case.

--

[Was Greywolf]

On 5/26/2013 6:36 PM, John H Meyers wrote:
[...]

> I would not rely on a text editor to attempt to duplicate any file,
> even if you think there is only text in the file, because various
> text editors may stumble and fail to exactly duplicate the original file
> if it contains certain particular characters (e.g. null bytes),
> or lines longer than some particular limit, or unencoded binary data,
> or certain unexpected character encodings, etc.

[...]

I agree with your caution, there are bound to be files that will be
mangled when displayed in a text editor and resaved.

FWIW, so far I've not had a problem. In fact. I've snipped blocks of
embedded *.jpg code from e-mails, used Save As on the snips, and opened
them without problems. I just did it again, to make sure. ;-)

I believe that's because binary data is a formatted stream of bytes,
which the text editor interprets as characters. When it's resaved, it's
just a stream of bytes again, which the image viewer interprets as image
data. The block of *.jpg data is displayed in lines beginning with 'M',
BTW, which presumably tells the image viewer to read the following bytes
as image data, not text. I mention this because I've also renamed e-mail
files as *.jpg, and the image-viewer displayed the image. Reason:
Thunderbird hiccups with the (text + embedded image) output of a
newsreader known as Turnpike.

"It's all rather confusing, really." (Neddy Seagoon)

--
Best,
Wolf K.

[Eckard]

Le 27/05/13 16:38, Was Greywolf a écrit :

I'm sure your method is working (I tried it myself) but it might be very
time consuming.
For less experienced users I still advocate to use a tool like "Cut
Mbox" or "Cut to eml" to recover the messages from an obviously
corrupted mbox file.
For testing purpose I also used similar tools such as "Mbox Breaker" or
"Split & Concat" on a Mac to split a large mbox file into several
smaller mbox files before importing them manually in TB by simple drag &
drop into the profile folder.
It only took a couple of minutes and all imported mboxes appeared as new
folders displaying all messages on restarting TB.

[John H Meyers]

On 5/27/2013 9:38 AM, Was Greywolf wrote:

> FWIW, so far I've not had a problem. In fact.
> I've snipped blocks of embedded *.jpg code from e-mails,
> used Save As on the snips, and opened them without problems.

Binary files must be encoded to produce text that is allowable
in email; this is usually done using the "bas64" encoding standard,
which represents binary files using only a 64-character subset
of printable ascii characters, which can also be arbitrarily folded
into lines of limited length:
<http://en.wikipedia.org/wiki/Base64>

> I just did it again, to make sure. ;-)
> I believe that's because binary data is a formatted stream of bytes,
> which the text editor interprets as characters. When it's resaved,
> it's just a stream of bytes again,

> which the image a viewer interprets as image data.

Everything I said before still applies to many text editors,
in which there is always a risk (and even a likelihood)
of changing the content if an attempt is made to re-save a binary file.

If you think otherwise, try editing thunderbird.exe with your
text editor (which it would be good to identify),
and let us know how well TB works afterwards.

> The block of *.jpg data is displayed in lines beginning with 'M'

Perhaps you have managed to use "uuencoding" on that file:
<http://en.wikipedia.org/wiki/Uuencoding>

> which presumably tells the image viewer to read the following bytes as image data, not text.

There is no limit to what one can imagine, presume,
or invent as wishful thinking,
but very little of what one assumes without the backing
of knowledge or actual experience turns out to be true.

--

[Was Greywolf]

On 5/29/2013 3:16 AM, John H Meyers wrote:
> On 5/27/2013 9:38 AM, Was Greywolf wrote:
>
>> FWIW, so far I've not had a problem. In fact.
>> I've snipped blocks of embedded *.jpg code from e-mails,
>> used Save As on the snips, and opened them without problems.
>
> Binary files must be encoded to produce text that is allowable
> in email; this is usually done using the "bas64" encoding standard,
> which represents binary files using only a 64-character subset
> of printable ascii characters, which can also be arbitrarily folded
> into lines of limited length:
> <http://en.wikipedia.org/wiki/Base64>

[etc]

Precisely, that's why using a text editor to snip the *.jpg data and
Saving As a *.jpg file works. Which is why I do it. I use Editpad, which
(so far) has not corrupted any image files.

BTW, my surmise was based on knowing that for e-mail/Usenet purposes,
image files must be converted to text, and (unless otherwise indicated)
text means ASCII. (FWIW, I never use Windows ANSI encoding; and I've
noticed that my bro's Mac uses some furshlugginer non-ASCII code for
text. Bah!).

Your suggestion re: messing with an *.exe file is intriguing. May do
that when I have some really idle moments. ;-)

Have a good day,

--
Best,
Wolf K.

[John H Meyers]

On 5/29/2013 9:12 AM, Was Greywolf wrote:

JHM:

>> Binary files must be encoded to produce text that is allowable

>> in email; this is usually done using the "base64" encoding standard,

>> which represents binary files using only a 64-character subset
>> of printable ascii characters, which can also be arbitrarily folded
>> into lines of limited length:
>> <http://en.wikipedia.org/wiki/Base64>

> Precisely, that's why using a text editor to snip the *.jpg data
> and Saving As a *.jpg file works.

Unfortunately it doesn't "work" at all -- a base64-encoded version
of a binary image can not be directly displayed by most image viewers;
rather, it has to first be decoded back to the actual binary image format.

> Which is why I do it. I use Editpad,
> which (so far) has not corrupted any image files.

Which EditPad version, and is it a "Lite" or "Pro" version?

EditPad "Pro" can open and edit a file like a "hex editor,"
which is no longer anything like a "text editor,"
but EditPad "Lite" displays arbitrary images as the kind of
horrible-looking mess that fits an unscientific description
such as "utter garbage," and any adjustment to "wrap" lines
or change line endings mangles that beyond recognition,
as well as beyond interpretation by any image viewer of mine,
as will any attempt to use windows' Notepad or Write
to re-save a simple image (GIF) that just happens to be on my desktop,
even without any attempt to actually perform any editing function.

> Your suggestion re: messing with an *.exe file is intriguing.
> May do that when I have some really idle moments. ;-)

My suggestion to the rest of the community is that re-saving an arbitrary file
using a text editor, rather than copying it using one's operating system's
generalized file copying function, is a bit like playing "Russian Roulette"
with its content, unless the total content of that file has been proved
to be a set of lines of text where every single line is within line length limits,
character set limits, and consistent line endings expected by that text editor,
or that your "text editor" is really a "binary file editor,"
which most of them are not, particularly not what comes with MS Windows.

There will always be someone, however, who regularly carries
spare gas in their car trunk using soda pop bottles rather than
approved gasoline containers, and so long as no member of my family is involved,
and so long as I at least once speak up to offer correct alternative info
to inexperienced people who don't yet know any better,
why should I care any further?

--

[Was Greywolf]

On 5/29/2013 9:02 PM, John H Meyers wrote:
> On 5/29/2013 9:12 AM, Was Greywolf wrote:
>
> JHM:
>>> Binary files must be encoded to produce text that is allowable
>>> in email; this is usually done using the "base64" encoding standard,
>>> which represents binary files using only a 64-character subset
>>> of printable ascii characters, which can also be arbitrarily folded
>>> into lines of limited length:
>>> <http://en.wikipedia.org/wiki/Base64>
>
>> Precisely, that's why using a text editor to snip the *.jpg data
>> and Saving As a *.jpg file works.
>
> Unfortunately it doesn't "work" at all -- a base64-encoded version
> of a binary image can not be directly displayed by most image viewers;

Well, I'll be jiggered, you're right. True for Photo Gallery, Windows
Photo Viewer, Win 8's Photos app, Irfanview, and XnView.

However, no problems with PMView. Fact is, I didn't realise you're right
about "most" image viewers because PMView is my default image viewer.

For text, I use Editpad Lite. For the test that confirms your claim, I
Saved an *.eml with an embedded *.jpg image to the desktop. (The *.eml
was generated by Turnpike, which may be significant: Tbird can't display
that embedded image, so it offers the textfile instead. Here are the
first few lines of the embedded image:

> begin 644 z_Iris_1936.JPG
> M_]C_X``02D9)1@`!`0$`2`!(``#_X25'17AI9@``34T`*@````@`#0$/``(`
> M```&````J@$0``(````.````L`$2``,````!``$```$:``4````!````O@$;
> M``4````!````Q@$H``,````!``(```$Q``(````,````S@$R``(````4````
> MV@$[``(````!``````(3``,````!``(``(*8``(````!`````(=I``0````!
> M```!+)R>``$````^````[@``!#I#86YO;@!#86YO;B!%3U,@-C!$`````$@`
> M```!````2`````%'24U0(#(N-BXQ,0`R,#$S.C`U.C(X(#$X.C,T.C,Q`$D`
> M<@!I`',`.P!'`&\`<@!I`&X`9P`@`%,`=`!E`&4`<`!L`&4`.P!2`&4`80!D
[etc]

I then changed the extension to *.txt, the opened it it Editpad lite,
snipped the message portion of the post, and Saved the shortened file,
which was now a block of what to me looks like *.jpg data. I made a copy
of this file, opened it and Saved As *.jpg. I also changed the extension
on the copy to *.jpg. I now had three *.jpg files derived from the
original *.eml. All three displayed correctly in PMView.

Which suggests that PMView does indeed decode the base-64 code back into
its original binary image format.

I'm sure glad I kept PMView. ;-) It was originally written for OS/2, and
was later ported to Windows (Win 2000 IIRC). Runs without problems on
Win8 64bit. Woohoo! An oldie but real goodie!

[John H Meyers]

On 5/29/2013 9:44 PM, Was Greywolf wrote:

> For text, I use Editpad Lite. For the test that confirms your claim,
> I Saved an *.eml with an embedded *.jpg image to the desktop.
> (The *.eml was generated by Turnpike, which may be significant:
> Tbird can't display that embedded image, so it offers the textfile instead.
> Here are the first few lines of the embedded image:

> begin 644 z_Iris_1936.JPG
> M_]C_X``02D9)1@`!`0$`2`!(``#_X25'17AI9@``34T`*@````@`#0$/``(`
> M```&````J@$0``(````.````L`$2``,````!``$```$:``4````!````O@$;

That's "uuencoding": <http://en.wikipedia.org/wiki/Uuencoding>

A bug about lack of being able to save inline text "attachments":
<https://bugzilla.mozilla.org/show_bug.cgi?id=372696>

TB is supposed to be able, however,
to decode properly identified uuencoded attachments,
but I'm having slight difficulty searching for how to "properly identify" them,
since there is not even a standard name in MIME for "uuencoding"

What has been suggested is that a "properly identified" MIME part
should have headers something like these
(which "view source" should reveal):

Content-Type: application/pdf; name="test.pdf"
Content-Transfer-Encoding: uuencode <== NOT STANDARDIZED
Content-Disposition: attachment; filename="test.pdf"

Additional links below suggest that some objects (e.g. images)
are being included in-line without proper MIME headers,
which any email client should be excused for not handling.

Uuencoded attachments may be rare birds these days;
however, I've set a copy of Eudora to send uuencoded attachments,
and have created one, which I find encoded as shown at the end
of this post, which is the complete attachment.

I found that attachment to be properly decoded by Thunderbird,
as well as by Gmail and by Eudora.

Meanwhile, these may be of interest:

<https://addons.mozilla.org/en-US/thunderbird/addon/show-all-body-parts/>
<https://addons.mozilla.org/en-US/thunderbird/addon/mnenhy/>
<http://forums.mozillazine.org/viewtopic.php?f=39&t=579602>
<http://forums.mozillazine.org/viewtopic.php?t=547904>

Relevant previous threads:
<https://groups.google.com/forum/?fromgroups#!topic/mozilla.support.thunderbird/JXLTRwaWuOQ>
<http://mozilla.6506.n7.nabble.com/Inline-uuencoded-jpeg-images-td185552.html#a185553>
<https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.apps.thunderbird/MK_RXSubdD8>

> I'm sure glad I kept PMView. ;-)

Still available (for US$49.95 after free trial):
<http://www.pmview.com/download.html>

However, you can also use a free decoder and a free viewer, e.g.:
<http://www.webutils.pl/UUencode>
<http://www.irfanview.com/>

To generate a uuencoded object, I had to use a copy of Eudora,
which by default uses only modern MIME-approved encoding,
but which still offers options to use other methods,
in case you still correspond with anyone marooned on an island
ever since much earlier days of internet mail :)

A very small, complete image file follows, sent as a uuencoded
MIME part, then properly received and decoded by Thunderbird,
even if saved as a completely separate text file named buddyicon.eml,
without the third header line, so that the image itself constitutes
the entire message body.

Note that Eudora obeyed MIME specs (RFCs),
naming this non-standard old encoding method as "x-uuencode" (much as
"X-Anything-You-Like:" can be used as a non-standard header name):

Content-Type: image/jpeg; name="buddyicon.jpg"
Content-Transfer-Encoding: x-uuencode
Content-Disposition: attachment; filename="buddyicon.jpg"

[Was Greywolf]

On 6/1/2013 4:46 AM, John H Meyers wrote:
> On 5/29/2013 9:44 PM, Was Greywolf wrote:
>
>> For text, I use Editpad Lite. For the test that confirms your claim,
>> I Saved an *.eml with an embedded *.jpg image to the desktop.
>> (The *.eml was generated by Turnpike, which may be significant:
>> Tbird can't display that embedded image, so it offers the textfile
>> instead.
>> Here are the first few lines of the embedded image:
>
>> begin 644 z_Iris_1936.JPG
>> M_]C_X``02D9)1@`!`0$`2`!(``#_X25'17AI9@``34T`*@````@`#0$/``(`
>> M```&````J@$0``(````.````L`$2``,````!``$```$:``4````!````O@$;
>
> That's "uuencoding": <http://en.wikipedia.org/wiki/Uuencoding>

Thanks, very informative. SFFR.

have a good day,

--
Best,
Wolf K.