Thunderbird hangs with some particular emails (important!)

[extreme...@gmail.com]

-1
down vote
favorite


I hope to find a Thunderbird developer here, or someone who knows them and can kindly notice them this issue, thanks for help.

It is discussed here, but they allerted me no one of developers is watching there, sadly: http://forums.mozillazine.org/viewtopic.php?f=39&t=3030964

I report here: I noticed this inconvenience from years, have never been fixed, but it is so rare that I didn't want to post here. Now I'm using Uber often, and the problem start annoying me.

When Uber mails come (btw not only from Uber) with the results of my taxi ride, Thunderbird hangs, use 100% of CPU costantly and the only way to use it back is to kill the process, access my email via web (webmail) and delete that email, then restart Thunderbird. I notice it downloaded that email many times, maybe each time it self-check the mailboxes (in my case every 10 minutes), so I have several messages of the same email.

Something wrong maybe with mailboxserver, I use POP3, but certainly it's a bug of Thunderbird. I checked with new version nothing change, even if I start it in safe mode (no extensions are loaded!)
The following are screenshot of the problem, it hangs on downloading the first "bugged" email, and it copy it different times, every some minutes.
[img]http://image.ibb.co/c4w2Q5/th1.jpg[/img]

In this moment the main window is "frozen" or really with low reactivity (it may update every 20-30 seconds), and if you draw over another windows this is the result:
[img]http://image.ibb.co/etV7sk/th2.jpg[/img]
It needs to be killed from task manager

If I manually delete the "bugged" email by webmail, thunderbird will open again fully working, and this is the result. The bugged email can now be seen.
[img]http://image.ibb.co/ftvwk5/th3.jpg[/img]

[Wolf K.]

On 2017-06-19 06:11, extreme...@gmail.com wrote:
> -1
> down vote
> favorite
>
>
> I hope to find a Thunderbird developer here, or someone who knows them and can kindly notice them this issue, thanks for help.

[snip story about troubles calling an Uber ride using Thunderbird]

Isn't there an Uber app for this?

--
Wolf K.
https://kirkwood40.blogspot.com
"No keyboard present. Press F1 to continue."

[extreme...@gmail.com]

I suppose you't understand me, but thank you for reply :)
The emails are sent by Uber as invoice after the payment

[WaltS48]

On 6/19/17 6:43 PM, extreme...@gmail.com wrote:
> I suppose you't understand me, but thank you for reply :)
> The emails are sent by Uber as invoice after the payment
>

When is the last time you did a virus scan of your system?

--
Go Bucs!
Coexist <https://www.coexist.org/>
National Popular Vote <http://www.nationalpopularvote.com/>
Ubuntu 16.04LTS - Unity Desktop

[Good Guy]

On 19/06/2017 11:11, extreme...@gmail.com wrote:

Something wrong maybe with mailboxserver, I use POP3, but certainly it's a bug of Thunderbird. I checked with new version nothing change, even if I start it in safe mode (no extensions are loaded!)
The following are screenshot of the problem, it hangs on downloading the first "bugged" email, and it copy it different times, every some minutes. 
[img]http://image.ibb.co/c4w2Q5/th1.jpg[/img]

In this moment the main window is "frozen" or really with low reactivity (it may update every 20-30 seconds), and if you draw over another windows this is the result:
[img]http://image.ibb.co/etV7sk/th2.jpg[/img]
It needs to be killed from task manager

If I manually delete the "bugged" email by webmail, thunderbird will open again fully working, and this is the result. The bugged email can now be seen.
[img]http://image.ibb.co/ftvwk5/th3.jpg[/img]

I suggest disable JavaScript in Thunderbird and then see if it works.  See this picture:



You want to make sure it is set to false as shown in the picture.  After doing that, close Thunderbird before restarting again to see if everything is working as expected.

--

With over 500 million devices now running Windows 10, customer satisfaction is higher than any previous version of windows.

[Wayne]

I worked with a testcase sent by the reporter. There is nothing wrong
with the reporter's folders nor his system. It's just a base64 encoded
email that takes a long time to decode.

[B00ze]

On 2017-06-19 23:10, Wayne <vsee...@lehigh.edu> wrote:

[snip]

> I worked with a testcase sent by the reporter. There is nothing wrong
> with the reporter's folders nor his system. It's just a base64 encoded
> email that takes a long time to decode.

Thanks for looking into it!

--
! _\|/_ Sylvain / B00...@hotmail.com
! (o o) Member:David-Suzuki-Fdn/EFF/Red+Cross/SPCA/Planetary-Society
oO-( )-Oo Save your breath, you'll need it to blow up your date.

[extreme...@gmail.com]

Thanks everyone to help me!
I've disabled javascript but nothing change, this is what I did:
- access webmail, moved the "bad" mail from trash to inbox
- syncronized thunderbird to inbox, and noticed it hangs downloading that message, so I was sure I was ready for test the disable of javascrip
killed process, moved again mail to trash in webmail, opened thinderbird, set the javascript to false, closed, moved again the bad mail to inbox, opened Thunderbird and it hangs again.

U underline, the problem appears **only** during the download, there are no problems to see the email if it is not present more in inbox (after thunderbrd hanged ad then killed, if I start it without that email in the inbox (removed by webmail) I can see the message in the thunderbird on the next run (as you can see clearly in the screenshots, many messages of the same email were downloaded), and there is no problem to open it, it works fluent like other emails, even if it is 64bit, even if it has javascript.

Since this, I don't understand this "There is nothing wrong with the reporter's folders nor his system. It's just a base64 encoded email that takes a long time to decode"
If someone here doesn't believe me, I can make a video with my phone.
Thank you!


Il giorno mercoledì 21 giugno 2017 05:07:20 UTC+3, B00ze ha scritto:

> On 2017-06-19 23:10, Wayne <vseerror@> wrote:
>
> [snip]
>
> > I worked with a testcase sent by the reporter. There is nothing wrong
> > with the reporter's folders nor his system. It's just a base64 encoded
> > email that takes a long time to decode.
>
> Thanks for looking into it!
>
> --

> ! _\|/_ Sylvain / B00ze64@

[ishikawa]

On 2017年06月21日 11:06, B00ze wrote:
> On 2017-06-19 23:10, Wayne <vsee...@lehigh.edu> wrote:
>
> [snip]
>
>> I worked with a testcase sent by the reporter. There is nothing wrong
>> with the reporter's folders nor his system. It's just a base64 encoded
>> email that takes a long time to decode.

^^^^^^^^^^^^^^^^^^^^^^^
But how long?

Does Uber send a VERY LARGE attachment?
Can it be possible that TB does not use BUFFERED WRITE
and slow for this operation. Quite likely.
(I am working on the buffered write issue now.)
Decoding may not take that long, but writing it by a short chunk at a time
calling many |write| system calls may be slow...

[Wayne]

On 6/22/2017 6:28 AM, ishikawa wrote:
> On 2017年06月21日 11:06, B00ze wrote:
>> On 2017-06-19 23:10, Wayne <vsee...@lehigh.edu> wrote:
>>
>> [snip]
>>
>>> I worked with a testcase sent by the reporter. There is nothing wrong
>>> with the reporter's folders nor his system. It's just a base64 encoded
>>> email that takes a long time to decode.
> ^^^^^^^^^^^^^^^^^^^^^^^
> But how long?

45 seconds for the local content - ~9,100 lines of base64 lines.
1 minute 15 seconds for the remote content.

> Does Uber send a VERY LARGE attachment?
> Can it be possible that TB does not use BUFFERED WRITE
> and slow for this operation. Quite likely.
> (I am working on the buffered write issue now.)
> Decoding may not take that long, but writing it by a short chunk at a time
> calling many |write| system calls may be slow...

You might be correct. Because if I send the message to an account and it
is in the offline store, then it displays quickly.

So this might be a good test for your buffering patches :)

[Eric Valette]

On 06/22/2017 03:57 PM, Wayne wrote:

> You might be correct. Because if I send the message to an account and it
> is in the offline store, then it displays quickly.

That is typically why it is actually hardly usable in company
environment where your home is usually on a filer and you need to flush
your imap inbox because of quotas...

Almost 4 years since initial bug...

-- eric

[ISHIKAWA,chiaki]

On 2017/06/22 23:44, Eric Valette wrote:
> On 06/22/2017 03:57 PM, Wayne wrote:
>

>> You might be correct. Because if I send the message to an account and it
>> is in the offline store, then it displays quickly.
>

> That is typically why it is actually hardly usable in company
> environment where your home is usually on a filer and you need to flush
> your imap inbox because of quotas...
>
> Almost 4 years since initial bug...
>
> -- eric
>
>

I hope the buffering patch (and subsequent short read issue patch) will
be in the C-C source tree by the end of this year somehow.


(And I am very serious in getting the buffering patch (and patch to
handle short read) since I found my home NAS based on FreeNAS works very
well and so well in fact that I am thinking of moving my home e-mail
storage to it although I am using Windows TB client at home. I use Linux
TB client at work. )
Fixing this buffering write issue and better error handling (or report)
and patch to handle short read [this is essential for linux client that
needs to talk remote file system. Windows TB does not seem to suffer
since Windows low-level I/O primitives seem to handle the re-trying of
short-read automatically. Linux doesn't. ]


Right now, I am looking into an issue found by the cleanup of the my
buffering patch code.
Quite unexpectedly there is a code structure issue which makes the
"optimization" of using file stream open for Berkeley MBox style file,
which has been the default storage format since the beginning, unusable
with my patch somehow. I have no idea why...
My patch opens/closes file stream quite accidentally and
so did not hit upon this issue of failure to optimize issue during patch
creation.

I don't particularly think that losing this optimization is bad since
with Maildir storage format, which many want for flexible handling of
messages with external programs, can't use this optimization anyway (it
has to open/close a file for writing out each message during download or
move/copy.)

And to be frank, I *think* buffering write will speed things up
considerably for large mail attachment, etc. for people who need to
store their mails on remote file server and incurring one pair of
read/write in real world situation does not hurt much.


Of course, though, there may be a performance issue despite my *GUESS*.
I will be *MEASUREING* the impact of losing the optimization in July so
that we can decide if we can do away with the "optimization" once for
all and simplify the code a bit.
(Turns out I have to modify my local scripts mozconfig files, etc. to
create non-DEBUG version for the comparison. Takes time...)

Preliminary data shows that for empty LOCAL e-message, the optimization
seems to count.
But with real world e-mails,

Chiaki

[ISHIKAWA,chiaki]

Actually I think I figured out the problem why my buffering patch fails
to work with the approach taken by jorgk.
The hash table that stores the file stream for a given header stores a
newly created output file stream the first time an output to a mail
message file is done.
For Berkeley MBox style store used for a "Folder", a file stream is to
be shared between
different messages since they end up in the same "folder" file.

However, my buffering patch creates a buffered file stream from the
original file stream (which is in the hash table) each time the message
write begins.
In doing so a function to do this generates a DIFFERENT file stream from
the original (stored in the hash table.).
This DIFFERENT stream is used during the subsequent output operation
(lots of side effects).
Unfortunately, on the next message write for the same folder, the hash
table is searched and the original unbufferred stream (not used for the
previous writing) is returned.
Since the original stream was not touched upon by the I/O operation in
writing a message, it is not in an expected state by the code ...?
[Oh well, there may be be a flaw with my logic in my error analysis. The
fix for the bug may not be as easy as I thought...]


I filed a bugzilla comment about it.
Maybe once I return from a business trip next month, I can put my hands
on it and hopefully, even if my first cut may not solve the nasty issue,
but when subsequent patch attempts improve, jorgk can clean it up and
put into C-C tree (the famous last words!)

Chiaki

On 2017/06/25 6:20, ISHIKAWA,chiaki wrote:
> On 2017/06/22 23:44, Eric Valette wrote:
>> On 06/22/2017 03:57 PM, Wayne wrote:
>>

>>> You might be correct. Because if I send the message to an account and it
>>> is in the offline store, then it displays quickly.
>>

[extreme...@gmail.com]

Thank you all for your work here. Please consider me a beta tester, I still have on deleted items on the webmail that email, I can easly ove it to inbox and test Thunderbird.

[extreme...@gmail.com]

Please don't forget about this issue!
It probably happens only with POP3 connection. Probably it happens when some images are attached in the body of the email.
Please test this on POP3, not IMAP.

[extreme...@gmail.com]

it looks solved with version 56b3, finally!
Thank you!

[TCW]

On 9/8/2017 8:55 AM, extreme...@gmail.com wrote:
> it looks solved with version 56b3, finally!
> Thank you!
>

Glad to hear.