Andrew-
Thanks for the reply. I appreciate the time of people here who know so
much about email and whose involvement no doubt would be essential to
making a new Thunderbird concept that really works well. I've started a
week-long sprint to create a proof-of-concept of a Thunderbird
Webapp/Server (which I really should not be doing, but that is another
story). That way I can "show" rather than "tell". After that
proof-of-concept is done, I hope you might be willing to look at a
working (if very primitive) system and then we could expand on these
discussions.
https://github.com/pdfernhout/Twirlip2
In the meanwhile, here are some comments responding to points you
raised, especially the practical difficulty of running a local server on
a desktop when people expect seamless desktop apps.
== What Thunderbird does now
I think I have not explained the Thunderbird Server idea well enough and
so caused some confusion, sorry.
Right now, Thunderbird Desktop has essentially a web browser internally
(95% of its codebase) that talks to an email application (5% of the code
at most) via custom internal function calls in C++. I'm just talking
about essentially the same thing, except the email application runs
under a local webserver and the client part is the standard Mozilla
Firefox, and the protocols in between are JSON.
Those protocols are admittedly, as you point out, likely custom
protocols. However, the traffic across them is at least inspectable and
presumably documented. Those APIs are potentially callable by other
clients -- something presumably in practice the current Thunderbird
internal adhoc protocols defined by C++ functions are not.
So it seems to me like what I propose is still a step forward from the
current situation? Even if it is not perfect? Still, involvement by
people here in designing (or finding as with JMAP, thanks!) good
protocols could help a lot. One issue I'm pondering is how to send
search queries to get subsets of emails. I wonder if that is something
JMAP could help with? So, leveraging that sort of thing might be a big
win to save time, thanks again for pointing it out.
The only proviso I'd say is that I'm interested in integrated messaging
-- so, email, IRC, RSS, whiteboards, note taking, and more, all
together. A standard email protocol, even a better one like JMAP, might
not provide everything needed to support a full feature set. But I guess
I need to learn more about them.
The system will not be perfect at the start. I plan to put in some
arbitrary protocols to do the minimum needed for a proof-of-concept.
But, in an iterative way, I'm very open to someone who knows existing
email protocols well (which is not me), saying, look, you could leverage
that an have one less non-standard part in the application specifically
here. That would be valuable advice.
== Not a regular email server, but it could be
Such a server could of course also support direct IMAP, POP3, or JMAP
access from existing clients perhaps as plugins -- but that is not the
main intent. Actually, until you mentioned it, I had not even thought of
doing that. Thanks! :-)
== Local servers are just another way to modularize software
The confusion may be in my choice of the word "server". Is there a
better one to use here? Maybe I should say "Thunderbird Webapp" instead?
People in general are just not used to running local servers. I've
gotten used to it because I'm using Node.js more and more for all sorts
of things, including just parsing data and transforming it. You are
right to point to the added complexity though.
== Using a server is the only practical way to get Firefox to work well
But the reason I feel that is the way forward anyway (for Thunderbird as
a Mozilla project at least) is because Firefox's support for local data
is a problematical afterthought (as with the ignored Firefox issue I
posted about that 1.5 years ago mentioned at the beginning of the
manifesto). To get Firefox to work predictably as far as same-origin
policy security restrictions, you tend to have to use Firefox to surf to
a live website (even if it is a locally hosted website). I hope that
situation improves through a deep rethinking by Firefox of what same
origin policy means for local applications and local files, and such a
Thunderbird Webapp project might drive such improvements, but that is
the current state as I've experienced it -- using Firefox without a
webserver involved is problematical (which is sad to me).
== Don't look into the Electron beam with other eye :-)
In the long term, hopefully Firefox will provide better support for
embedding apps in the way you can do with Electron/Chromium like
Automattic just did for WordPress/Calypso to run as a desktop app.
Now I could just suggest until then the Thunderbird Server project just
do the same thing as Automattic did and run Thunderbird Server with
Node.js bundled together with Chromium via Electron for easy install as
a standard-looking desktop app -- "but that would be wrong". :-)
So, please don't look here, whatever you do: :-)
http://electron.atom.io/
"Build cross platform desktop apps with web technologies ... Use HTML,
CSS, and JavaScript with Chromium and Node.js to build your app."
== Mozilla ignored a crucial need and now everyone pays the price
More seriously, as I pointed out in a post to Mozilla Governance:
http://pdfernhout.net/thunderbirds-are-grow-manifesto.html
"Beyond the value of such a communications platform itself, if Mozilla
had prioritized this need to improve Thunderbird four years ago, Mozilla
might have improved the Firefox platform in ways that helped keep up its
market share up because Thunderbird was showing a need that became
Electron and Phonegap and other similar software. Then
open-source-friendly companies like Automattic might not be turning to
Electron/Chromium for creating a desktop WordPress/Calypso app instead
of using the Firefox platform. In fact, if Mozilla has done such a
thing, is it possible that much of the time and money and attention that
has gone into WhatsApp, Viber, Line, etc. might have gone into an
enhanced Mozilla Thunderbird and Firefox instead? Maybe then Firefox
market's share and Mozilla's revenues might have even expanded rather
than contracted? Instead, by ignoring the needs of Thunderbird and the
internet users it serves, by not trying to grow in that area, Mozilla
missed a huge business opportunity. ... Frankly, as I see it, if you
look at the adoption trends, and consider the rapid rise of Slack,
between Firefox and Thunderbird, an expanded Thunderbird is actually the
more viable product concept long-term. :-) But I am sorry to have to say
that as a long-time Firefox user. :-( ..."
That post is still in the moderation queue and I wonder if it will see
the light of day. Probably should never post stuff at 5:30am after
staying up all night writing it. I added the text from it to the
manifesto near the end though.
== Benefits of the web server approach
A big benefit to of the webserver approach is the Thunderbird project
could stop maintaining a second copy of Firefox it uses internally
because it can just use regular Mozilla Firefox on the front end
(removing 95% of the Thunderbird codebase and much drudgery, patching,
and security issues). Switching from C++ to JavaScript for the entire
app could also reduce memory bugs and security issues. But that is not
the only benefit.
With the webserver approach, you could potentially also access your
email (and perhaps other communications like IRC chats, RSS feeds,
notes, and more) from any device on your local network (including
phones, tablets, watches). You could setup custom networked devices like
a lamp that lights up when you get a message from someone special.
It would also be possible to access your email outside your network
depending on your network configuration (although that is a risk).
Those are things you can't easily do with Thunderbird Desktop (unless
you do virtual screensharing over the network perhaps, but that is
tedious and impractical for a phone).
For me, I use POP3 and not IMAP. I have Thunderbird delete email from
the server (after a short delay in case of a hard disk crash). I don't
consider myself having an existing "email server" in the sense that I
see our email server just as a relay point, not something that stores
email. I can imagine someone who uses an IMAP (or JMAP) server regularly
might have different expectations though.
Some people would not use an IMAP server at a remote host in the way it
was intended for privacy/legal reasons, because files more than six
months old become easily legally accessible in the USA on servers. For
example:
http://www.mcclatchydc.com/news/politics-government/congress/article24779989.html
"If you’ve been remiss in cleaning out your email in-box, here’s some
incentive: The federal government can read any emails that are more than
six months old without a warrant. Little known to most Americans,
ambiguous language in a communications law passed in 1986 extends Fourth
Amendment protections against unreasonable search and seizure only to
electronic communications sent or received fewer than 180 days ago. The
language, known as the “180-day rule,” allows government officials to
treat any emails, text messages or documents stored on remote servers –
popularly known as the cloud – as “abandoned” and therefore accessible
using administrative subpoena power, a tactic that critics say
circumvents due process."
== Why this should be a Mozilla priority
People at Mozilla believe in web technologies. That part I loved about
Firefox OS and its approach -- as even knowing how to write Android apps
I'd rather work on cross-platform webapps.
Given that belief, making email accessible over the web, but stored
locally and accessed securely to increase privacy, seems to me like a
no-brainer for Mozilla.
Of course, one may then discuss priorities, but given 70% of user
accounts are either email or IM, by one estimate from The Radicati Group
I cite in that latest governance email, it seems supporting such
internet usages of email and IM should potentially have even higher
priority at Mozilla than improving access to social media accounts (the
other 30%). But Mozilla's current funding priorities, especially
defunding Thunderbird development, obvious do not reflect those
statistics. I just don't understand that. :-(
(Anyway, back to coding...)
--Paul Fernhout
http://www.pdfernhout.net/
====
The biggest challenge of the 21st century is the irony of technologies
of abundance in the hands of those still thinking in terms of scarcity.