Clarifications on imap idle and push email

[SantaClaus]

Hi all,

I was hoping someone could please clarify how imap idle, blackberry
and other systems for push email actually work – hoping I’m not off
topic.

I installed profimail (http://www.lonelycatgames.com/?app=profimail),
an email client which supports imap idle, on my Nokia E71, locked on
the 3 UK (www.three.co.uk) network.
Profimail and imap idle work fine: emails sent to my mailbox (imap
server) arrive on the phone immediately.

However, when I tried Nokia email (email.nokia.com) and ‘email on
3’ (the customized version of Seven’s push email, www.seven.com),
emails were delivered to m phone with considerable delay, sometimes
half a day. Do you have any idea why? The same happened when I
connected Nokia email to a wifi network, rather than to 3’s own
network.

My guess is that, while profimail maintains a constant connection to
the server, which drains the battery, Nokia email and Seven send some
sort of ‘hidden’ sms to my mobile, prompting it to connect to the
server only when needed. This saves battery life but at the same time
is a bottleneck: in my case, the sms must be delayed for some reason I
fail to understand.

Do you think my interpretation is correct? Is there anything I could
do? Out of curiosity, do Blackberry works with this ‘hidden sms’
method, too, or do they use an imap idle connection to the mail
server?

Thanks!

[Mark Crispin]

Blackberry has a proprietary channel, similar to SMS, to wake up the
phone. The mobile phone providers take care to make sure that this
channel is expeditious.

Other services that do Push type email have to use SMS for this purpose.
Since 99% of SMS are kids texting each other, the mobile phone providers
are more lackidasical about getting those delivered promptly.

IMAP is a Pull protocol. IDLE is a mechanism to give the experience of
Push on top of Pull. Here's how it works:

In Pull, the client periodically polls the server to see if there are any
updates. There is no specific command to poll; updates are issued as part
of the response to any IMAP command. So, if the client has nothing
special to do, it just issues a NOOP ("do nothing") command.

This works very well with NAT devices that create fixed mappings based
upon client-originated traffic. Even if the NAT device drops the mapping,
the client poll will recreate it as if it had never been dropped.

IMAP IDLE uses the IMAP command continuation mechanism to put the protocol
in "command in progress" state. The client starts a command (it sends
"IDLE") and completes it (it sends "DONE") as much as 29 minutes later,
but meanwhile is listening for the server to send updates. Thus, the
server can send updates at its whim without having to wait for the client
to poll.

More importantly for mobile phones, once the client sends "IDLE" it can
shut down its transmitter and just listen on the receiver.

There are problems with IDLE.

[1] It doesn't work with NAT. The session goes quiet, and then suddenly
there is server traffic. But the NAT box has dropped the mapping, so the
server gets a connection reset.

[2] IMAP has an autologout inactivity timer of at least 30 minutes. So
the client must exit IDLE and restart it at the 29 minute mark. Outlook
does not do this; hence Outlook does not work well with servers that
support IDLE.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

[SantaClaus]

On Nov 26, 6:13 pm, Mark Crispin <m...@panda.com> wrote:
> Blackberry has a proprietary channel, similar to SMS, to wake up the
> phone.  The mobile phone providers take care to make sure that this
> channel is expeditious.
>
> Other services that do Push type email have to use SMS for this purpose.
> Since 99% of SMS are kids texting each other, the mobile phone providers
> are more lackidasical about getting those delivered promptly.

Thank you very much for the explanations! So, if I understand
correctly:

1) Emails arrive sooner on a Blackberry because whatever proprietary
signal servers send to Blackberry devices are less subject to the
network congestion which can cause sms to arrive with delay
2) The delays I have been experiencing with Nokia email and 'email on
3' depend on a congestion of the network which causes the silent sms
to arrive with delay, not on a problem with my imap server or with the
way Nokia or Seven connect to it. In other words: the imap server
works fine, Nokia and Seven connect to it without problems, send the
silent sms, and that's where problems start...
3) Using imap idle on a mobile device will drain the battery more than
using a system which relies on silent sms

However, if point 2 is true, why do ordinary sms arrive immediately (I
tried a number of times)?

Btw, what's your view on push email solutions for mobile devices?
Which ones do you like / dislike etc?

Thanks a lot!

[Mark Crispin]

On Wed, 26 Nov 2008, SantaClaus posted:

> Btw, what's your view on push email solutions for mobile devices?
> Which ones do you like / dislike etc?

Blackberry undeniably has the best push solution today.

The nascent IETF NOTIFY mechanism may become a satisfactory alternative
(IMAP IDLE, etc. are kludges). But it isn't here yet.

[sar...@gmail.com]

On Nov 26, 6:32 pm, SantaClaus <myemail...@googlemail.com> wrote:
> 3) Using imap idle on a mobile device will drain the battery more than
> using a system which relies on silent sms

Not sure:

"More importantly for mobile phones, once the client sends "IDLE" it
can
shut down its transmitter and just listen on the receiver."

I've been looking all this up today as I've been trying to figure out
what takes up the most power on my E71 (for battery life reasons).
Leaving IMAP in this idle manner seems to draw less than 0.1W however
once every 5 mins it spikes up to about 1.3W for just under a minute.
Not sure how this compares to other Push email solutions however the
energy profiler program suggests that this would lead to the battery
draining in about 30 hours.