Are IMAP IDLE and BURL supported?

[VanguardLH]

Does Thunderbird supports IMAP IDLE? I'm pretty sure it does but would
like verification. Can it be turned off in case it is not reliable with
a specific IMAP server (and instead polling be used)?

How about BURL? I haven't found any e-mail clients that support it. In
fact, Google decided to emulate it on their server end because no
clients yet support it. Typically the client uploads an e-mail to the
SMTP server and also saves a copy in its own local Sent folder which
uploads to the IMAP server. Most users have asymmetrical bandwidth:
upstream bandwidth is far slower than downstream. You end up using the
slowest speed to upload a copy to SMTP and upload another copy to IMAP.
BURL has the SMTP server send a copy to the IMAP server; i.e., the
server handles saving a copy into the Sent folder. Then the client gets
a copy into its Sent folder using the much faster downstream bandwidth.
The result is response (as seen by the user) is sped up. Rather than
sending 2 copies upstream, 1 goes upstream and 1 comes downstream. I
don't know of any e-mail clients that support it. That is, in the
keyword string they send to the server, none add the BURL keyword to
notify the server that the client supports BURL. I have found e-mail
providers that support BURL but no e-mail clients that support BURL
although that spec got ratified a decade ago back in 2006.

Message Submission BURL Extension
RFC 4468
http://www.rfc-editor.org/rfc/rfc4468.txt

Because no e-mail clients yet support BURL (that I know of), Gmail
emulates it. Their SMTP server puts a copy in the IMAP's Sent folder.
That's why you need to configure your IMAP e-mail client to *not* save
its own copy in its local Sent folder; else, you end up with 2 copies of
a saved sent message (one for the local copy uploaded by the client,
another for the copy saved up on the server downloaded to the client).
MS Outlook does not support BURL. I haven't found an e-mail client that
supports BURL. I suspect neither does Thunderbird.

[Tanstaafl]

On 4/17/2016 5:05 PM, VanguardLH <V...@nguard.LH> wrote:
> Does Thunderbird supports IMAP IDLE?

Account Settings > Account > Server Settings > Advanced

> How about BURL? I haven't found any e-mail clients that support it.

Me neither, so I guess it is sort of a chicken and egg problem.

I found this email from Mike Abbot in April 2010 announcing Apple's
contribution of a patch for both postfix (BURL) and dovecot (BURL,
CATENATE and URLAUTH):

I also found an email from Wietse back in 2014 saying he'd add support
in postfix if there was a client that he could test with, so I just
asked there about this patch from Apple in 2010. He also said that Apple
didn't support it in any of their clients, which is strange given the
patch they provided in 2010...

http://opensource.apple.com//source/postfix/postfix-229/patches/burl.patch

> I have found e-mail providers that support BURL

Links?

> I haven't found an e-mail client that supports BURL. I suspect
> neither does Thunderbird.

Not yet, although there is an open bug for it:

https://bugzilla.mozilla.org/show_bug.cgi?id=421779

[VanguardLH]

Tanstaafl wrote:

> VanguardLH wrote:
>
>> Does Thunderbird supports IMAP IDLE?
>
> Account Settings > Account > Server Settings > Advanced

I had been there to specify [gmail] as the root folder for Gmail (to
eliminate showing subfolders under a "[Gmail]" folder in Thunderbird).
I don't see anything about IMAP IDLE there.

Back under server settings, and for an IMAP account, is the "Check for
new messages every N minutes". I would expect that for a POP account.
If IMAP IDLE were supported then that would seem a superfluous setting.

Is there a log of the mail session between Thunderbird and the IMAP
server where I can see the commands that were issued by the client and
the statuses issued by the server? As I recall, on connect the server
will announce its capabilities as a list of keywords and the client will
issue some command with keywords that lists it capabilities.

>> How about BURL? I haven't found any e-mail clients that support it.
>
> Me neither, so I guess it is sort of a chicken and egg problem.
>
> I found this email from Mike Abbot in April 2010 announcing Apple's
> contribution of a patch for both postfix (BURL) and dovecot (BURL,
> CATENATE and URLAUTH):
>
> I also found an email from Wietse back in 2014 saying he'd add support
> in postfix if there was a client that he could test with, so I just
> asked there about this patch from Apple in 2010. He also said that Apple
> didn't support it in any of their clients, which is strange given the
> patch they provided in 2010...
>
> http://opensource.apple.com//source/postfix/postfix-229/patches/burl.patch
>
>> I have found e-mail providers that support BURL
>
> Links?

I'd have to look again. Last time I looked into this was probably 2, or
more, years ago when trying to figure out what Gmail was doing to find
they were emulating BURL (they would copy from SMTP to IMAP's Sent
folder so the client would get that copy resulting in users seeing 2
copies of their sent message - 1 copied by the client and another copied
by Google's SMTP server). I found where we had this discussion back in
2014 at:

https://groups.google.com/d/msg/mozilla.support.thunderbird/fTEeSH7bmBw/6eDeYhwUrswJ

I'd have to go back through the SMTP commands and session establish
statuses to remember how I found out an SMTP server supported BURL
(which also meant there had to be a link up between the SMTP and IMAP
servers). My recollection is the client announced BURL in a keyword
string but I don't recall what the server said (in its keyword list upon
establishing a session).

>> I haven't found an e-mail client that supports BURL. I suspect
>> neither does Thunderbird.
>
> Not yet, although there is an open bug for it:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=421779

In the 2-year old discussion, I mentioned that one.

While searching on BURL and postfix and dovecot (I recall reading
somewhere about an IMAP server program that was considering or had BURL
support), I came across:

https://docs.kde.org/trunk4/en/extragear-pim/trojita/using-kapp.html

So there is one client (MUA) that can issue the BURL command.

https://github.com/Exim/exim/wiki/EximIntroduction says Exim does *not*
support BURL.

Gotta go to a doctor appointment so have to cut this short. Don't know
what, if any, IMAP servers support BURL but I would think they would
announce it after handshake in their status keyword list. So I'd have
to hunt around again to find an IMAP server with BURL support.

[Onno Ekker]

Op 17-4-2016 om 23:05 schreef VanguardLH:

> Does Thunderbird supports IMAP IDLE? I'm pretty sure it does but would
> like verification. Can it be turned off in case it is not reliable with
> a specific IMAP server (and instead polling be used)?

IMAP IDLE is supported and can be turned off.

If you're using TB < 45:
Tools | Account Settings | [IMAP Account] | Server Settings |
Advanced | Use IDLE command if the server supports it


In TB45 it's somewhat different:
Tools | Account Settings | [IMAP Account] | Server Settings |
Allow immediate server notifications when new messages arrive

AFAIK BURL isn't supported.

Onno

[Poutnik]

Dne 18/04/2016 v 20:39 VanguardLH napsal(a):

> Back under server settings, and for an IMAP account, is the "Check for
> new messages every N minutes". I would expect that for a POP account.
> If IMAP IDLE were supported then that would seem a superfluous setting.

Not, as IDLE connection need to be refreshed,
as it may be closed on server side.
Given interval is important for both IDLE/PUSH and PULL checks.

I can illustrate it on Android IMAP client K9 that I use.

I use IDLE/PUSH with 16 min just for important emails
from the inbox folder of my personal IMAP accounts,
while other checked IMAP folders for less important emails
and other IMAP accounts are checked by PULL checks.


--
Poutnik

Wise man guards the words he says,
as they may say about him more,
than he says about the subject.

[Tanstaafl]

On 4/18/2016 2:39 PM, VanguardLH <V...@nguard.LH> wrote:
> Tanstaafl wrote:

>> VanguardLH wrote:
>>> Does Thunderbird supports IMAP IDLE?

>> Account Settings > Account > Server Settings > Advanced

> I had been there to specify [gmail] as the root folder for Gmail (to
> eliminate showing subfolders under a "[Gmail]" folder in Thunderbird).
> I don't see anything about IMAP IDLE there.

Apparently you missed the last step - ADVANCED... click the ADVANCED button.

> Back under server settings, and for an IMAP account, is the "Check for
> new messages every N minutes". I would expect that for a POP account.
> If IMAP IDLE were supported then that would seem a superfluous setting.

Expectations are like... something else every one else has...

> Is there a log of the mail session between Thunderbird and the IMAP
> server where I can see the commands that were issued by the client and
> the statuses issued by the server?

Not easily:

https://wiki.mozilla.org/MailNews:Logging

[VanguardLH]

Yep, using TB 45.0. I was used to finding an "IDLE" option somewhere in
TB from a past trial of it, so this option didn't stick out to me as the
IMAP IDLE option. Would've caught it if the option description were
"Allow immediate server notifications (IMAP IDLE) when new messages
arrive." Presumably this option is hidden or disabled for POP accounts.

With that option enabled, seems the "Check for new messages every N
minutes" is superfluous - unless the IMAP server doesn't support IMAP
IDLE which means polling every N minutes is a safety backup to ensure
you get new messages.

[dillinger]

On 04/18/2016 09:16 PM, Tanstaafl wrote:
>> > Is there a log of the mail session between Thunderbird and the IMAP
>> > server where I can see the commands that were issued by the client and
>> > the statuses issued by the server?
> Not easily:
>
> https://wiki.mozilla.org/MailNews:Logging

There is an extension that should make all of that a bit easier:
https://addons.mozilla.org/en-US/seamonkey/addon/tbtracer/

And these days you can restart TB with logging enabled, right at the
bottom of the Thunderbird menu. Much less options of course, but very
handy if you just want to check something fast.

[VanguardLH]

Tanstaafl wrote:

> VanguardLH wrote:
>
>> Tanstaafl wrote:
>>
>>> VanguardLH wrote:
>>>
>>>> Does Thunderbird supports IMAP IDLE?
>>>
>>> Account Settings > Account > Server Settings > Advanced
>

>> I don't see anything about IMAP IDLE there.
>
> Apparently you missed the last step - ADVANCED... click the ADVANCED
> button.

Apparently it changed in TB 45.0. The "IMAP IDLE" option is no longer
in the advanced dialog. It's back in the server setting panel as
""Allow immediate server notifications when new messages arrive" option,
as Ekker noted.

>> Is there a log of the mail session between Thunderbird and the IMAP
>> server where I can see the commands that were issued by the client and
>> the statuses issued by the server?
>
> Not easily:
> https://wiki.mozilla.org/MailNews:Logging

I wrote a batch file with the environment variables, ran TBird, and
looked at the logfile. I'm going to have to figure out how to disable
(not delete) all but one e-mail account so I can see what is going on.
There is a lot of information logged at log levels 4 & 5. With 4
accounts (it can be a lot more), there is too much jumbled together.

Is there a way for me to disable the accounts defined in TBird? I'd
like to see a log of just one account at a time. My old eyes are
missing might be this option. Can accounts be defined but disabled?

[VanguardLH]

dillinger wrote:

> VanguardLH wrote:
>
>> Is there a log of the mail session between Thunderbird and the IMAP
>> server where I can see the commands that were issued by the client
>> and the statuses issued by the server?
>

> There is an extension that should make all of that a bit easier:
> https://addons.mozilla.org/en-US/seamonkey/addon/tbtracer/

That add-on requires the user to set the following environment
variables:

set NSPR_LOG_MODULES=<logtype>:<loglevel>,timestamp
set NSPR_LOG_FILE=<logfile>

as noted in the Mozilla article mentioned by Tanstaafl. Doable but for
now I'll just look at the logfile. I might use TBtracer later if I end
up doing a lot of log analysis to pretty up the output (like I run some
web pages through HTMLtidy to figure out what the hell the author was
trying to code).

> And these days you can restart TB with logging enabled, right at the
> bottom of the Thunderbird menu. Much less options of course, but very
> handy if you just want to check something fast.

Don't see that menu option. Right at the bottom of which menu? Are you
talking about the error console (Tools -> Error Console)? That doesn't
show the communications between client and server.

[Ron K.]

Think there are two options to control which account is logged. First I
would try deselecting auto-polling of accounts I was not concerned about.
If that was still to verbose I would try a testing profile without the
extra accounts being setup.
--
Ron K.
Thunderbird user since May, 2003

[dillinger]

On 04/19/2016 08:06 AM, VanguardLH wrote:
>> > And these days you can restart TB with logging enabled, right at the
>> > bottom of the Thunderbird menu. Much less options of course, but very
>> > handy if you just want to check something fast.
> Don't see that menu option. Right at the bottom of which menu? Are you
> talking about the error console (Tools -> Error Console)? That doesn't
> show the communications between client and server.

Oops, my bad, it comes with with the ProfileSwitcher add-on:
https://addons.mozilla.org/en-US/thunderbird/addon/profileswitcher/

[VanguardLH]

With N accounts, I would have to disable the polling options in N-1
accounts. With 3 options per account enabling mail pull or push, that
would be (N-1)*3 options to disable. With 2 accounts, just 3 options to
disable. With a dozen accounts, that would be 33 options to disable
which is a LOT of navigating between accounts and mouse clicking to
change options - and to do again after log analysis to reenable mail
pull or push in the N-1 accounts.

> If that was still to verbose I would try a testing profile without the
> extra accounts being setup.

That's probably what I would have to do considering there isn't an easy
1-click option in each account to disable or enable it from inclusion in
the mail polls.