Future Planning: Thunderbird as a Web App

Showing 1-73 of 73 messages
Future Planning: Thunderbird as a Web App Kent James 9/17/15 3:03 PM
As we are discussing our future, both in relation to radical changes
expected in the Mozilla platform, and our need to express where we are
going to potential partners and donors, we need to discuss and agree on
some big-picture issues. One of those was end-to-end encryption that we
discussed recently. I want to discuss here our future platform, and how
it related to users and their needs.

tl;dr Thunderbird over the next 3 years needs to convert to being a web
app that can run on any browser that supports ES6 Javascript and HTML5.
(web app does not imply cloud-based, only that the underlying platform
is js/html).

There are two independent but both equally critical reasons why we need
to go this direction.

First, the Mozilla platform itself has no long-term commitment to being
available as a general-purpose development environment to run
non-browser software, other than as a web app. At the same time, they
are doing amazing innovations to allow more and more functionality
traditionally associated with native clients to run under the "Web as a
Platform" meaning the HTML5/ES6 stack. Really in the long run we will
either have to fork the Mozilla platform and maintain it ourselves as an
email development environment, or go with the flow and convert. Let's
convert.

Second, more and more users are using a variety of platforms, including
not only our traditional desktop environments but also Android and iOS
(and other mobile OSes hoping to join them). I'm writing this message
using Thunderbird on a Microsoft Surface 3 Tablet, which can now run
traditional Windows applications such as Thunderbird. But we can't
expect most of our users to buy a tablet from a second-tier operating
system, merely to be able to run Thunderbird (as I did). In the last 24
hours, I have accessed my email using 4 different devices (Android phone
and tablet, Windows desktop and tablet). Only 2 of those devices can run
Thunderbird. We need to have a serious mobile story. Starting from the
existing forward-thinking Mozilla web environment, we should be in an
excellent position to convert our existing code to ES6-js/HTML5, and
that is the best long-term way to support our users on the full range of
platforms that they expect to use.

Last year, jcranmer introduced JsMime for some of the mime-related
processing. This year, he is specing a Js database. I am working on
JsAccount that will allow us to create new mailnews accounts and
associated objects (server, folder, URL, protocol, etc.) using js and
interoperate with our existing code. Although initially aimed at new
account types, JsAccount will eventually make it straightforward to
incrementally convert all of those core objects from C++ to Javascript.
We need to plan to convert the entire backend in a similar nature to ES6
javascript.

Looking to a rough long-term schedule, I see future versions of
Thunderbird looking like this:

38 (Avocet), 45 (3/2016 Bunting), 52 (01/2017 Cormorant?): Traditional
XUL/C++ app

59 (09/2017 Dunlin?) Last and traditional XUL/C++ release. By this date,
a reasonable possibility is that the Mozilla platform will no longer be
usable for non-browser XUL-based development. This version of
Thunderbird, in that case, would need to ship on a fork of Mozilla from
the point where XUL-based development becomes untenable. This will also
be our last major XUL-based release, and we will not attempt to keep
current with the new non-XUL Mozilla platform. It would also be useful
to ship a stripped-down version of Thunderbird as a web app with this
release.

07/2018 Egret? (no point in matching release numbers to Gecko anymore):
Thunderbird ships a full-version web app.

I don't see what choice we reasonably have, given the announced changes
in the Mozilla platform. We should view this as an opportunity to both
get out from the current Mozilla regression-driven development, to being
able to focus on our own issues. Hoping we can adapt to changes in the
Mozilla platform, with no commitment from Mozilla that they will try to
make that easy, is an enormous risk with little upside for us.

Comments?

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Future Planning: Thunderbird as a Web App Sean M. Pappalardo 9/17/15 3:29 PM
Hello.

The direction you outline makes sense to me and opens up some
interesting possibilities. I just have a few questions.

As a "web app," will Thunderbird still look and feel like a desktop one
on desktop OSs? Will there need to be a hidden Web server process in the
background? How much of a speed penalty will there be with JS vs C++ code?

Will it be easier (even trivial, dare I say) to build Thunderbird as a
native iOS and Android app once it's a Web app?

Would it be possible to run Thunderbird as a cloud app too? I.e. the
back-end on a Web server and front end usable by any modern browser?
(Much like SOGo's Web interface: http://www.sogo.nu/tour/screenshots.html)

If this is the direction things go, how will that affect my desire to
add full LDAP editing? Should I wait until the application core is
changed over first? Or will rewriting the Address Book module be part of
the code conversion?

Thanks for your time.

Sincerely,
Sean M. Pappalardo
Sr. Networks Engineer
Renegade Technologies
spapp...@renegadetech.com
Office: (630) 631-6188
http://www.renegadetech.com

--
This communication, along with any documents, files or attachments, is
intended only for the use of the addressee and may contain confidential
information. If you are not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of any
information contained in or attached to this communication is strictly
prohibited.
If you have received this message in error, please notify the sender
immediately and destroy the original communication and its attachments
without reading, printing or saving in any manner.

Re: Future Planning: Thunderbird as a Web App Axel Grude 9/17/15 4:06 PM

Subject: Future Planning: Thunderbird as a Web App
To: Tb-planning
From: Kent James
Sent: Thursday, 17/09/2015 23:03:46 23:03 GMT ST +0100 [Week 37]
As we are discussing our future, both in relation to radical changes expected in the Mozilla platform, and our need to express where we are going to potential partners and donors, we need to discuss and agree on some big-picture issues. One of those was end-to-end encryption that we discussed recently. I want to discuss here our future platform, and how it related to users and their needs.

tl;dr Thunderbird over the next 3 years needs to convert to being a web app that can run on any browser that supports ES6 Javascript and HTML5. (web app does not imply cloud-based, only that the underlying platform is js/html).

There are two independent but both equally critical reasons why we need to go this direction.

First, the Mozilla platform itself has no long-term commitment to being available as a general-purpose development environment to run non-browser software, other than as a web app. At the same time, they are doing amazing innovations to allow more and more functionality traditionally associated with native clients to run under the "Web as a Platform" meaning the HTML5/ES6 stack. Really in the long run we will either have to fork the Mozilla platform and maintain it ourselves as an email development environment, or go with the flow and convert. Let's convert.
We will have to convince Addon authors to convert away from XUL, so there needs to be documentation or workshops helping to get away from this. And obviously we need a transition time where "both" technologies can be used concurrently to find out about performance / compatibility issues. According to the timeline you give below, there is not much wiggle room there.


Second, more and more users are using a variety of platforms, including not only our traditional desktop environments but also Android and iOS (and other mobile OSes hoping to join them). I'm writing this message using Thunderbird on a Microsoft Surface 3 Tablet, which can now run traditional Windows applications such as Thunderbird. But we can't expect most of our users to buy a tablet from a second-tier operating system, merely to be able to run Thunderbird (as I did). In the last 24 hours, I have accessed my email using 4 different devices (Android phone and tablet, Windows desktop and tablet). Only 2 of those devices can run Thunderbird. We need to have a serious mobile story. Starting from the existing forward-thinking Mozilla web environment, we should be in an excellent position to convert our existing code to ES6-js/HTML5, and that is the best long-term way to support our users on the full range of platforms that they expect to use.
I am using Android Gmail App to access my email in parallel with the more comfortable Thenderbird on my desktop when I actually try to get some work done. Most email providers already allow web access to their emails. But a unified Thunderbird frontend on Android and Firefox OS would sure be nice.


Last year, jcranmer introduced JsMime for some of the mime-related processing. This year, he is specing a Js database. I am working on JsAccount that will allow us to create new mailnews accounts and associated objects (server, folder, URL, protocol, etc.) using js and interoperate with our existing code. Although initially aimed at new account types, JsAccount will eventually make it straightforward to incrementally convert all of those core objects from C++ to Javascript. We need to plan to convert the entire backend in a similar nature to ES6 javascript.

Looking to a rough long-term schedule, I see future versions of Thunderbird looking like this:

38 (Avocet), 45 (3/2016 Bunting), 52 (01/2017 Cormorant?): Traditional XUL/C++ app

59 (09/2017 Dunlin?) Last and traditional XUL/C++ release. By this date, a reasonable possibility is that the Mozilla platform will no longer be usable for non-browser XUL-based development. This version of Thunderbird, in that case, would need to ship on a fork of Mozilla from the point where XUL-based development becomes untenable.
can you elaborate what you mean? Forking would make XUL-based development impossible? Or XUL based development necessitates forking?

This will also be our last major XUL-based release, and we will not attempt to keep current with the new non-XUL Mozilla platform. It would also be useful to ship a stripped-down version of Thunderbird as a web app with this release.
for the web app, will there be local storage? a "gloda-like" repository that can be stored on the hard drive? will the data be cross compatible with any other desktop email clients?

07/2018 Egret? (no point in matching release numbers to Gecko anymore): Thunderbird ships a full-version web app.

I don't see what choice we reasonably have, given the announced changes in the Mozilla platform. We should view this as an opportunity to both get out from the current Mozilla regression-driven development, to being able to focus on our own issues. Hoping we can adapt to changes in the Mozilla platform, with no commitment from Mozilla that they will try to make that easy, is an enormous risk with little upside for us.

Comments?
Any plans on who from the Thunderbird team could / would lead development. Are we going to get some help from the Firefox team?



Re: Future Planning: Thunderbird as a Web App Joshua Cranmer �� 9/17/15 5:56 PM
CC'ing asuth, who I think absolutely has some information worth conveying.

On 9/17/2015 5:03 PM, Kent James wrote:
> As we are discussing our future, both in relation to radical changes
> expected in the Mozilla platform, and our need to express where we are
> going to potential partners and donors, we need to discuss and agree
> on some big-picture issues. One of those was end-to-end encryption
> that we discussed recently. I want to discuss here our future
> platform, and how it related to users and their needs.
>
> tl;dr Thunderbird over the next 3 years needs to convert to being a
> web app that can run on any browser that supports ES6 Javascript and
> HTML5. (web app does not imply cloud-based, only that the underlying
> platform is js/html).

There is one point of caution I want to make. The JS ecosystem is still
rather fragmented. An example of all of the different execution
environments:
* Node.js
* Mozilla XPCOM/Chrome JS
* B2G-style JS apps
* Chrome-style JS apps (/ web extensions?)
* JS apps in places like Windows RT (I don't follow mobile enough to
know all the fragmentation)
* Pure HTML5 webapps (including aspirational specifications)

Even basic APIs can be radically different between them--for example,
every single environment I've listed has a different API for reading
from a socket. There is a slow convergence between these different APIs,
but it's on the order of a few years. And, unfortunately, a part of the
missing API is rather fundamental concepts: node.js only started
embracing Promises with version 4, released this past month (!), and
Streams are still nowhere in sight.

There is also some good news in that JS tooling infrastructure is now
racing to properly support es6. I did grab code coverage of my latest
es6 codebase (an UTS #46 mapping tool, so I can feed that into JSMime),
and I'll probably poke at making it work with JSMime when I get time for
that.

I actually have more JS projects I've been working on:
1. JSMime (<https://github.com/mozilla-comm/jsmime>)
2. Emailsasl (a SASL client library)
(<https://github.com/jcranmer/TEMP-emailsasl>)
3. Email-socket (abstracting away the combination text control/binary
data nature of NNTP/SMTP/POP/IMAP) (not public yet, since I haven't
gotten very far with it).
4. UTS #46 support (<https://github.com/jcranmer/idna-uts46>)

Something that you'll notice is that I've been also making sure, at
least for the low level code, that I can work simultaneously with
node.js tests. I didn't do it for JSMime because node.js was in a really
horribly place several years ago, and I haven't had the time to go back
and do the fixes necessary to get it working.


>
> Looking to a rough long-term schedule, I see future versions of
> Thunderbird looking like this:
>
> 38 (Avocet), 45 (3/2016 Bunting), 52 (01/2017 Cormorant?): Traditional
> XUL/C++ app
>
> 59 (09/2017 Dunlin?) Last and traditional XUL/C++ release. By this
> date, a reasonable possibility is that the Mozilla platform will no
> longer be usable for non-browser XUL-based development. This version
> of Thunderbird, in that case, would need to ship on a fork of Mozilla
> from the point where XUL-based development becomes untenable. This
> will also be our last major XUL-based release, and we will not attempt
> to keep current with the new non-XUL Mozilla platform. It would also
> be useful to ship a stripped-down version of Thunderbird as a web app
> with this release.
>
> 07/2018 Egret? (no point in matching release numbers to Gecko
> anymore): Thunderbird ships a full-version web app.
>
> I don't see what choice we reasonably have, given the announced
> changes in the Mozilla platform. We should view this as an opportunity
> to both get out from the current Mozilla regression-driven
> development, to being able to focus on our own issues. Hoping we can
> adapt to changes in the Mozilla platform, with no commitment from
> Mozilla that they will try to make that easy, is an enormous risk with
> little upside for us.
>
> Comments?

I think it is absolutely vital to share maintenance work of "low-level"
protocol code with Gaia email or other components, by which I mean:
* IMAP, POP, SMTP, NNTP (not that Gaia is likely to do anything with
NNTP, but they all ought to share the same email-socket layer).
* MIME (+ TNEF decoding, S/MIME, PGP, I expect)
* vCard / CardDAV support
* iCal / CalDAV support
* Chat protocols
* JMAP
* Sieve / managesieve

I do want to counter that I'm not entirely sure the current state of
affairs is ready for us to go whole-hog on JS conversion. ES6 modules,
in particular, are a big change that is going to have lots of
repercussions, but no engines have fully implemented it yet (look for Q4
2015 or Q1 2016?), and some aspects of the standardization process are
in a holding pattern (e.g., HTML imports) until after people get more
experienced with them.

That's not to say we shouldn't move forward: rewriting some libraries in
JS (e.g., JSMime usage) is probably still worthwhile, and I think we can
get agreement now that new API design should be focused on ease of use
for JS consumers, with the XPIDL interface for C++ use being a
second-class citizen.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

Re: Future Planning: Thunderbird as a Web App Matthew Sotoudeh 9/17/15 7:06 PM
Hello! I don't think I've replied to this list before, but I have
enjoyed keeping up with interesting threads whenever I can :)

As a hobbyist web developer and Thunderbird user this sounds really
interesting, though I would like to clarify a few things about what
exactly this would mean & throw in my own opinion:

On 9/17/2015 3:03 PM, Kent James wrote:
> As we are discussing our future, both in relation to radical changes
> expected in the Mozilla platform, and our need to express where we are
> going to potential partners and donors, we need to discuss and agree
> on some big-picture issues. One of those was end-to-end encryption
> that we discussed recently. I want to discuss here our future
> platform, and how it related to users and their needs.
>
> tl;dr Thunderbird over the next 3 years needs to convert to being a
> web app that can run on any browser that supports ES6 Javascript and
> HTML5. (web app does not imply cloud-based, only that the underlying
> platform is js/html).

I'd like to clarify what exactly the end state of becoming a "web app"
would be.

Does this literally mean that I should be able to go to (for example)
"https://thunderbird.com/MyEmail" and use a fully-featured version of
Thunderbird inside of Firefox, Chrome, and Edge (possibly using
something along the lines of https://github.com/hiddentao/browsermail)?
Or would this simply mean a push to move existing XUL and C++ components
to HTML and Javascript, but still using the Gecko "shell" (not sure what
the official term is, app runner? bootstrapper?) to make it a real
desktop application now that Mozilla is pulling back on XUL?

Or would the short-term idea be to push existing XUL/C++ to HTML/JS with
the long-term goal being to open up the possibility of moving everything
to a web app/HTML-based mobile app at a later time?

If the former (a real "website" web-app), and especially if this were
done completely in HTML/CSS/Javascript (without a backend component) I
would like to point out that this would open up Thunderbird to a whole
new world of contributors who know the "web platform" and would like to
contribute to Thunderbird, but who may not have the skills, drive, or
patience to learn mercurial or the Mozilla build tools (or wait multiple
hours for the codebase to compile O.O).

This would also make it 10x easier to recommend Thunderbird to people
("just go to this website and log in with GMail" vs. "install this
program, find your account credentials, oh you have two-factor auth? go
make an app access token...").

In addition, wouldn't treating Thunderbird as essentially a website
(even if there was a desktop/mobile version of it) align more with
Mozilla's current focus on the web and the web platform? Not sure if you
were getting at that in the original email, but from what little
understanding I have of Mozilla's goals it seems like this could
possibly help put Thunderbird back on their radar.

While I don't know enough about the C++ portions of Thunderbird to guess
at how huge of a project it would be to port it to a literal,
Internet-accessible web app, I would throw whatever my opinion is worth
in and say I would be *very* excited to see such a project become the
future of Thunderbird.

Thanks for reading,
Matthew Sotoudeh
Re: Future Planning: Thunderbird as a Web App Andrew Sutherland 9/17/15 9:21 PM
On Thu, Sep 17, 2015, at 08:56 PM, Joshua Cranmer ūüźß wrote:
> CC'ing asuth, who I think absolutely has some information worth
> conveying.

The gaia mail back-end and any Thunderbird-evolved-to-webapp should
absolutely share protocol level code.  (And as you've noted, gaia email
is already in the process of doing this in order to get streaming MIME
parsing via the JSMime library you authored for Thunderbird.)

It would also be great if we could reuse code at a higher abstraction
level too, but there's no clear evolutionary path for Thunderbird there
since the semantics don't line-up and it's hard to draw a clean line
through the Thunderbird implementation surface where things could be
easily shimmed/converted without throwing away large swathes of existing
Thunderbird functionality.  (The refactored gaia mail backend, being
conversation-centric, is most akin to the gloda representation, but only
the faceted search UI and popular Thunderbird Conversations extension
consume that representation.)

My hope right now is that the gaia mail backend makes enough progress
that at some point in a few months I can make a pitch for people to
consider trying to help me create a desktop mail UI backed by the gaia
mail backend.  I know, it's a very tall order for any app to provide the
feeling of immediacy that Thunderbird has; the thread-pane provides an
almost tactile experience.  It's snappy after folder load, you know all
your messages are in there and with some combination of filtering,
sorting, and scrolling, you can usually find what you're looking for if
you look hard enough.  (Compare with search-based mechanisms like Gmail
search or gloda global search where it can feel like sticking your hand
in a haystack and grabbing a bunch of stuff.  Maybe the needle is in
that handful, maybe it's not.  The sensation of being at a remove from
your messages can be frustrating.)

If anyone's interested in the current state, well, I'll send a different
message now with some pointers to tb-planning to avoid cluttering up
this thread.

Andrew


_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Future Planning: Thunderbird as a Web App Eric Moore 9/17/15 10:23 PM
> From: Kent James<ke...@caspia.com>

> tl;dr Thunderbird over the next 3 years needs to convert to being a web
> app that can run on any browser that supports ES6 Javascript and HTML5.
> (web app does not imply cloud-based, only that the underlying platform
> is js/html).

I understand the need for Thunderbird to become a web app. However, its
not clear to me why that web app needs to be restricted to just
Javascript and HTML5. That seems to be a serious handicap, and likely to
cause a lot of problems with widgets. Think of the classic arguments why
somebody might prefer Thunderbird over a good webmail implementation
that supports accounts with multiple email providers. It seems to boil
down to more customization, expandability (via add-ons), better widgets
and being able to support a platform specific look and feel. I'm
concerned we'd lose most of that.

Mailpile uses Python, JS, HTML5 and will let you "Host your install of
mailpile on your laptop, desktop, Raspberry PI, server in the cloud, or
put it on a USB stick and carry it in your pocket." They plan on
eventually supporting apps for Android and iPhone/iPad. "The Plugin
architecture has being expanded to allow self-contained plugins which
can contain a mixture of python, HTML (jinja2) and javascript code, but
this work is incomplete."

Please explain the tradeoffs between MailPile's definition of a web app
and one that only supports js/html. I'm not trying to argue that we
should mimic MailPile, they're just a convenient example of a different
approach.

One benefit of limiting Thunderbird to just js/html appears to be that
we could share "maintenance work of "low-level" protocol code with Gaia
email or other components". However, that is a two edged sword as in
practice it could also mean that we'd have to constantly adapt to
changes in the Mozilla platform.

Another would be that Thunderbird would be available on many more
platforms, including ones that we don't even know about. I agree that we
need Mobile support, but there are diminishing returns at some point in
trying to support everything.

How does supporting only js/HTML effect other possible long term goals
such as end-to-end encryption?
Re: Future Planning: Thunderbird as a Web App Alan Lord 9/18/15 12:18 AM
On 17/09/15 23:03, Kent James wrote:
>
> I don't see what choice we reasonably have, given the announced changes
> in the Mozilla platform. We should view this as an opportunity to both
> get out from the current Mozilla regression-driven development, to being
> able to focus on our own issues. Hoping we can adapt to changes in the
> Mozilla platform, with no commitment from Mozilla that they will try to
> make that easy, is an enormous risk with little upside for us.
>
> Comments?

I like the idea and the plan - this is a good way to
invigorate/stimulate TB development and support.

Things that "concern" me ever so slightly:

1. Offline support; local storage
2. Extensions

Alan
Re: Future Planning: Thunderbird as a Web App Mark Banner 9/18/15 1:03 AM
On 18/09/2015 00:03, Kent James wrote:
>  (web app does not imply cloud-based, only that the underlying
> platform is js/html).
How about calling it something like "Thunderbird post-XUL" or
"Thunderbird on web tech"? The issue with "Web App" is that this is
going to pre-load people's thoughts about what you mean with something
completely different.

> Second, more and more users are using a variety of platforms,
> including not only our traditional desktop environments but also
> Android and iOS (and other mobile OSes hoping to join them).
I would recommend talking to people from the Android/iOS teams. For
example, they switched a lot of the Android front-end implementation
from XUL to native android a few years back because of performance
issues. Obviously all the backend is still the existing gecko & I
believe some js base, but just switching isn't necessarily going to give
you this for free.

Going with the mobile theme for a bit, I think consideration needs to be
given about what a mobile Thunderbird would look like. Something that's
as fully featured as desktop is likely to be very difficult to do a good
UX for and is likely to be a lot slower.

Hence if you're looking at rewriting a lot of the back-end, then it
would be worth giving a thought to how it would operate/be constructed
in a mobile world where you don't necessarily want all the features. For
example, lets assume filters aren't required on mobile (I don't know of
a mobile app that'd do that) - it be good to have that part of the
back-end "plug-in" when required, potentially with hooks like today's
add-ons can use, rather than for it to be built-in permanently even
though there's no filters included.

Mark
Re: Future Planning: Thunderbird as a Web App Volker Birk 9/18/15 4:56 AM
Hi,

I'm disappointed about how the discussion is going. In my new:


1) Thunderbird has to continue for sure

For p‚Č°p Thunderbird is a strategic platform. We offered ‚Äď and are still
offering ‚Äď the following:

a) We finance the project. We were asking Thunderbird council for their
   view, and offered the budget we were told is needed. We are still
   willing to do that.

b) We offer the needed infrastructure. We offer a complete replacement
   here if needed.

c) We offer missing development resources. If it is needed, we will fork
   XULRunner, and put the needed resources on it.


2) Moving Thunderbird to a web app will shut down Thunderbird project

a) There is already gaia. I cannot see why a second web app will help
   here. It would be way more clever to port gaia to Firefox than to
   make Thunderbird a web app.

b) Web apps may be an idea for mobile, but on the desktop for the user
   an email web app is indistinguishable from a web mailer. This way
   Thunderbird users will probably be converted to web mailer users, and
   Thunderbird will fade out until it's gone.


3) Why Thunderbird is vital and inevitable for crypto community

Thunderbird is the No. 1 email client for OpenPGP users. Credit goes
here to Enigmail project.

Crypto implementations ONLY can go from locally running software to
locally running software. Crypto implementations must not be server
based in any way, but have to be peer-to-peer only. Only then we have
end-to-end cryptography, only then we have security offered by crypto at
all, and not a simulation of security instead.


Because of these reasons, p‚Č°p is offering to carry Thunderbird project
if Mozilla does not want to do so any more.  Because Thunderbird is
vital to crypto community, p‚Č°p will keep the Thunderbird application up
and running either way.

Yours,
VB.
--
Volker Birk, p‚Č°p project
mailto:v...@pep-project.org  http://www.pep-project.org
Re: Future Planning: Thunderbird as a Web App Gervase Markham 9/18/15 5:04 AM
On 18/09/15 09:46, Volker Birk wrote:
> Because Thunderbird is
> vital to crypto community, p‚Č°p will keep the Thunderbird application up
> and running either way.

On the question of "forking the platform": can someone name some recent
innovations written by non-Thunderbird engineers for the shared codebase
which were beneficial to Thunderbird?

To put it another way: Thunderbird seems to spend a lot of time and
energy keeping up with the changes in mozilla-central. What value does
it get in exchange for that?

Gerv


Re: Future Planning: Thunderbird as a Web App Mark Banner 9/18/15 5:55 AM
On 18/09/2015 13:04, Gervase Markham wrote:
To put it another way: Thunderbird seems to spend a lot of time and energy keeping up with the changes in mozilla-central. What value does it get in exchange for that?
Here's my opinion, in no particular order, probably very incomplete:
  • A reasonable HTML parser & display
    • which also provides the options to integrate more things into Thunderbird, like web pages e.g. what's new, or other things that add-on gives
  • Lower level protocol handling, certificates
  • Security with the content vs chrome display of emails
  • File handling routines (including download)
  • Database handling (for the places we use sqlite)
  • Content type handling
  • General layout of the UI
  • Crash reporting
  • Telemetry
  • Metrics
  • Installers (windows)
  • Update handling
  • Connection handling, e.g. proxies, offline etc
  • Add-ons
  • Accessibility harnesses
  • L10n
  • Internationalization
  • spell checking
  • Graphics handling
  • Test harnesses
  • Developer tools
  • Profile management
  • Various widgets & things such as autocomplete
  • Regular security updates and fixes to all of the above.

I'll grant that there's nothing that couldn't be replaced, but there's quite a lot of core functionality there that you need in an application before you can start building out what it does.

Mark.

Re: Future Planning: Thunderbird as a Web App Gervase Markham 9/18/15 6:33 AM
On 18/09/15 13:55, Mark Banner wrote:
> Here's my opinion, in no particular order, probably very incomplete:

I think perhaps I wasn't clear - sorry about that. I wasn't asking "what
have the mozilla-centrals ever done for us" :-), I was asking "In, say,
the last two years, what improvements to mozilla-central have been
useful to Thunderbird, as compensation for the last two years of breakage"?

To put it another way: "what would we have lost if we had forked m-c two
years ago"?

I don't expect the list to be an empty set, but its size and the
magnitude of the features on it would be useful in determining the
feasibility of such a step in the future.

Gerv
Re: Future Planning: Thunderbird as a Web App Benjamin Kerensa 9/18/15 6:34 AM
Could always re-write Thunderbird from the ground up in React and have a Desktop, Android and iOS app
all with a HTML5/JS front end. 

:) That solves the issue with a desktop mail client becoming less and less relevant as people continue
to migrate to mobile. This is a important issue as desktop is indeed dying and continues on a downward spiral.


I do not think however a hosted web app Thunderbird sounds like a great idea.

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning




--
Benjamin Kerensa

Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 8:52 AM
On 9/18/2015 1:46 AM, Volker Birk wrote:
> 2) Moving Thunderbird to a web app will shut down Thunderbird project
>
> a) There is already gaia. I cannot see why a second web app will help
>     here. It would be way more clever to port gaia to Firefox than to
>     make Thunderbird a web app.
>
> b) Web apps may be an idea for mobile, but on the desktop for the user
>     an email web app is indistinguishable from a web mailer. This way
>     Thunderbird users will probably be converted to web mailer users, and
>     Thunderbird will fade out until it's gone.
Mark Banner suggested that "web app" was probably not the best term to
describe the proposed direction, because of confusion of what that means.

I've often tried to describe to friends why Thunderbird is not "the web"
in Mozilla speak. "The Web" as I understand it, and as Mozilla promotes
it, is really a programming stack based on JS and HTML.

Thunderbird, when it uses standardized TCP/IP-based protocols for
communication, is not "The Web" in Mozilla-speak, it is a traditional
desktop client. Mozilla-the-organization is currently dedicated to
making such applications obsolete, and replaced by "The Web". Hence the
strategic misfit of Thunderbird within Mozilla.

If I were to program a game, running entirely locally and offline, but
using the JS/HTML programming stack with Firefox as the underlying
operating system, that IS "The Web". If that program is entirely
indistinguishable from its previous C++/native version, then Mozilla can
claim success in its mission.

On mobile, the equivalent battle is to replace the native Objective-C
iOS or Java Android apps with a JS/HTML equivalent. Really the closest
analogy was the dream of Java to be a write-once, run-anywhere
programming environment.

Undoubtedly many at Mozilla will object to this gross simplification of
their Mission. But I have developed this analogy mostly to present as
benign a view as possible of why Mozilla seems to reject Thunderbird.
There could be another perspective, and I think it is the perspective
taken by most Thunderers-as-Mozillians, that the INTERNET is what is
important, and some of the biggest battles in the near-future to the
INTERNET concern communications privacy and siloing, and Thunderbird is
really the strongest product in the Mozilla family to be the vanguard of
that battle. But we are constantly told that Thunderbird is not
strategic, because we are not "The Web".

In my original proposal, I made one comment that this proposal is not
turning Thunderbird into a "cloud-based" app. But there has been some
confusion here, and Mark Banner is correct that we need to use a
different term than "Web App" which is confusing.

So from the perspective of end-to-end encryption, which as I've
mentioned before is something that Thunderbird agrees is important and
needs to promote, I don't think this discussion affects that directly.
The issues more concern performance and practicality of replacing C++
components and XUL elements with their equivalent in JS and HTML. But
that is an issue that the whole of Mozilla is trying to solve, and has
made enormous progress. I'm just saying we should embrace that direction
- which incidentally leads us down a path of being less dependent on
Mozilla when accomplished.

:rkent
Re: Future Planning: Thunderbird as a Web App Joshua Cranmer �� 9/18/15 9:29 AM
On 9/17/2015 10:23 PM, Eric Moore wrote:
> One benefit of limiting Thunderbird to just js/html appears to be that
> we could share "maintenance work of "low-level" protocol code with
> Gaia email or other components". However, that is a two edged sword as
> in practice it could also mean that we'd have to constantly adapt to
> changes in the Mozilla platform.

In the time since I started working on JSMime in 2011, I have had to
make exactly one change to JSMime to adapt for something Mozilla did,
and that wasn't even a core aspect of the code (it was my use of XHR
sync for use in non-TB test environments). There's a few things I'd like
to change (namely using typed arrays instead of binary strings, since
typed arrays didn't exist when I started), but not because I have to. In
contrast, there have been dozens of small changes and fixes that we need
to handle using the XPCOM infrastructure. Moving to a more pure JS/web
tech-based approach would actually reduce that level of minor change
adaptation, since there is far more conservatism about breaking
web-exposed APIs than add-on-exposed/internal APIs.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________


tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Future Planning: Thunderbird as a Web App Volker Birk 9/18/15 10:04 AM
On Fri, Sep 18, 2015 at 08:52:04AM -0700, Kent James wrote:
> So from the perspective of end-to-end encryption, which as I've mentioned
> before is something that Thunderbird agrees is important and needs to
> promote, I don't think this discussion affects that directly. The issues
> more concern performance and practicality of replacing C++ components and
> XUL elements with their equivalent in JS and HTML.

Hello, Kent,

thank you very much for making that clear! I must admit I'm one of the
people who were confused from your previous statement. The way you
discribe it here completely solves that.

Edouard Tisserant and me were working on a web technology based IDE
project (which now stalled because of p‚Č°p). Therefore you can see that
we have no problem with web based technology at all, but the opposite is
true.
Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 10:39 AM
On 9/17/2015 3:20 PM, Sean M. Pappalardo wrote:
> Hello.
>
> The direction you outline makes sense to me and opens up some
> interesting possibilities. I just have a few questions.
>
> As a "web app," will Thunderbird still look and feel like a desktop
> one on desktop OSs? Will there need to be a hidden Web server process
> in the background? How much of a speed penalty will there be with JS
> vs C++ code?
>
> Will it be easier (even trivial, dare I say) to build Thunderbird as a
> native iOS and Android app once it's a Web app?
>
> Would it be possible to run Thunderbird as a cloud app too? I.e. the
> back-end on a Web server and front end usable by any modern browser?
> (Much like SOGo's Web interface:
> http://www.sogo.nu/tour/screenshots.html)

Performance: is a major concern as others who have gone down this path
have encountered, and we see with JsMime and ical.js  I think it will be
important that we not simple assume that the performance issues will be
solved by faster js, but we need to identify performance-sensitive areas
and a mitigation strategy. That might include using asm.js more
deliberately, or even proposing needed WebIDL components that are needed
to be developed and exposed to the web that solve critical performance
constraints.

native iOS or Android: No this is not a goal or easier. If it were, we
would follow the path of Fennec, but that is not the plan here.

Thunderbird as a cloud app: This would not be a short-term goal. But we
might need to be open to this as a possibility when we design basic
protocols. For example, we will need to consider the path that transmits
information about messages to the front-end code. At the very least we
will be moving to make this async. At the same time, we might consider
if that same interface could be adapted to a cloud-based back end in the
future. You might, for example, use a cloud-friendly protocol like JMAP
to interface to your local message storage, so that a future cloud-based
app could be converted easily.

>
> If this is the direction things go, how will that affect my desire to
> add full LDAP editing? Should I wait until the application core is
> changed over first? Or will rewriting the Address Book module be part
> of the code conversion?

LDAP is a great example of how thinking might change. I think we have
discussed for years that LDAP code needs to convert to a standard,
non-binary library in the long run. I think that we should resist the
urge to try to adapt our existing C++-based library to support editing.
But if you are interested in LDAP, we would really, really appreciate
someone who would research available, license-compatible LDAP js
libraries, and adapt our code to use that. Editing should be added in
the context of that new library.

As for user interface, if you do a new UI you should consider using HTML
instead of XUL. This is already done in a few places in Thunderbird
code. See for example accountProvisioner.xhtml The js support for that
brings in many existing XPCOM-based Thunderbird js scripts, so you can
see that it is actually not as difficult as you might imagine to use
HTML instead of XUL.

:rkent
Re: Future Planning: Thunderbird as a Web App Andrew Sutherland 9/18/15 10:41 AM
On 09/18/2015 04:46 AM, Volker Birk wrote:
> Crypto implementations ONLY can go from locally running software to
> locally running software. Crypto implementations must not be server
> based in any way, but have to be peer-to-peer only. Only then we have
> end-to-end cryptography, only then we have security offered by crypto at
> all, and not a simulation of security instead.

So it's clear, the Firefox OS Gaia email app is currently a packaged and
signed application that runs locally only.  The only servers contacted
are the user's mail servers and those contacted in the course of running
autoconfiguration/autodiscovery.  A content-security policy prevents
code from being run from remote locations.

The direction Gecko is going with APIs like TCPSocket that are hard to
explain to users as permission prompts and for which standardization
really isn't happening is for them to be add-on-only APIs.  So a
pure-HTML/CSS/JS Thunderbird would effectively be a cryptographically
signed Firefox add-on.

This is already the strategy used by whiteout.io, a PGP-focused mail
client built on HTML/CSS/JS running as a Chrome extension, and which
likely will also run as a Firefox extension once the platform is further
built out.

Andrew
Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 11:12 AM
On 9/17/2015 4:06 PM, Axel Grude wrote:

We will have to convince Addon authors to convert away from XUL, so there needs to be documentation or workshops helping to get away from this. And obviously we need a transition time where "both" technologies can be used concurrently to find out about performance / compatibility issues. According to the timeline you give below, there is not much wiggle room there.

Compare the schedule I have proposed to Firefox. They care claiming "12 - 18 months" as their timetable for getting rid of XUL (which schedule I think is crazy talk, but that is another discussion.) If you look at the rough schedule that I have proposed, it is almost three years before we ship Thunderbird in a non-XUL version, including a full cycle of forked m-c if needed to keep XUL going. That is 2-3 times the wiggle that Firefox has proposed.

Part of the hope here is that Thunderbird addon developers will have a lot of time to see how Firefox addon developers reacts to the whole deXULification of Firefox, and develop mitigation strategies. I wish that Firefox had a better mitigation strategy for addons in place before they announced these changes as a done deal, but we have to adapt. It is really, really hard for us to say much more until the Firefox situation gets clearer. But as you know several of the core Thunderbird team (myself and Fallen) come from the addon world, so we are quite committed to maintaining powerful addons as a possibility for Thunderbird.

I'm using Android Gmail App to access my email in parallel with the more comfortable Thenderbird on my desktop when I actually try to get some work done. Most email providers already allow web access to their emails. But a unified Thunderbird frontend on Android and Firefox OS would sure be nice.

Right. The problem with the current status quo is that you are forced to minimize your use of advanced Thunderbird features (including filters, tags, etc.) because they do not adapt to Android or webmail apps. If we are going to provide advanced strategies for managing and categorizing email, which I think we should, those strategies also need to work in at least an Android environment, and preferable also in a browser or iOS app.


59 (09/2017 Dunlin?) Last and traditional XUL/C++ release. By this date, a reasonable possibility is that the Mozilla platform will no longer be usable for non-browser XUL-based development. This version of Thunderbird, in that case, would need to ship on a fork of Mozilla from the point where XUL-based development becomes untenable.
can you elaborate what you mean? Forking would make XUL-based development impossible? Or XUL based development necessitates forking?

Forking for Dunlin means that, per the announced Firefox timing, in that time period Firefox will have converted to a non-XUL environment. What I am proposing is that, if needed, Thunderbird continue availability of a XUL-based environment, even if it means forking mozilla-central as a means of extending that time period. (Personally I doubt if Firefox can abandon XUL in their announced timeframe, but we can't plan on that basis.)


This will also be our last major XUL-based release, and we will not attempt to keep current with the new non-XUL Mozilla platform. It would also be useful to ship a stripped-down version of Thunderbird as a web app with this release.
for the web app, will there be local storage? a "gloda-like" repository that can be stored on the hard drive? will the data be cross compatible with any other desktop email clients?

We need to maintain at least the choice of keeping a full local copy of email, as we do now. That includes a local database that summarizes the data in some way. Cross-compatible data is not a current goal, nor is it likely to be in the future.


Any plans on who from the Thunderbird team could / would lead development. Are we going to get some help from the Firefox team?

Who: I would hope that people currently involved in Thunderbird development would contribute toward this new strategy. But if we manage to hit the funding targets that I have discussed, we will also clearly be looking to add some additional people. If you have people in mind, please point them out to me privately.

:rkent

Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 11:27 AM
On 9/17/2015 5:44 PM, Matthew Sotoudeh wrote:
> I'd like to clarify what exactly the end state of becoming a "web app"
> would be.
>
> Does this literally mean that I should be able to go to (for example)
> "https://thunderbird.com/MyEmail" and use a fully-featured version of
> Thunderbird inside of Firefox, Chrome, and Edge (possibly using
> something along the lines of https://github.com/hiddentao/browsermail)?
> Or would this simply mean a push to move existing XUL and C++ components
> to HTML and Javascript, but still using the Gecko "shell" (not sure what
> the official term is, app runner? bootstrapper?) to make it a real
> desktop application now that Mozilla is pulling back on XUL?

As has been said in a few other responses, not the former but the latter.

> Or would the short-term idea be to push existing XUL/C++ to HTML/JS with
> the long-term goal being to open up the possibility of moving everything
> to a web app/HTML-based mobile app at a later time?

At the very least, the goal should be to allow running Thunderbird on a
powerful Android tablet under a browser (but as a stand-alone
application rather than an http:// link in a browser.) This is important
because I think that we need to clearly provide a valuable user benefit
to at least partially justify this effort. Refactoring is hard to sell.


> If the former (a real "website" web-app), and especially if this were
> done completely in HTML/CSS/Javascript (without a backend component) I
> would like to point out that this would open up Thunderbird to a whole
> new world of contributors who know the "web platform" and would like to
> contribute to Thunderbird, but who may not have the skills, drive, or
> patience to learn mercurial or the Mozilla build tools (or wait multiple
> hours for the codebase to compile O.O).

Actually this is also true of the latter. It is not the http:// starting
point that provides access to the world of js/html development, it is
the framework. That's why we start talking about react.js as possible
basis for future development.

> In addition, wouldn't treating Thunderbird as essentially a website
> (even if there was a desktop/mobile version of it) align more with
> Mozilla's current focus on the web and the web platform? Not sure if you
> were getting at that in the original email, but from what little
> understanding I have of Mozilla's goals it seems like this could
> possibly help put Thunderbird back on their radar.

This is true, but it is not a primary motivation. I think that we need
to do what we think is in the best interest of our users, subject to the
ethical direction of the Mozilla Manifesto, without feeling some need to
align with a web and Firefox-focused Mozilla Mission. If that ends up
being in alignment with the Mozilla Mission, then perhaps that validates
the Mission as a Good Idea, but it is not what is driving us.

:rkent
Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 11:41 AM
On 9/18/2015 1:03 AM, Mark Banner wrote:
> How about calling it something like "Thunderbird post-XUL" or
> "Thunderbird on web tech"? The issue with "Web App" is that this is
> going to pre-load people's thoughts about what you mean with something
> completely different.

Right - as responses to this thread have already established. I'll avoid
"Web App" in the future!

> I would recommend talking to people from the Android/iOS teams. For
> example, they switched a lot of the Android front-end implementation
> from XUL to native android a few years back because of performance
> issues. Obviously all the backend is still the existing gecko & I
> believe some js base, but just switching isn't necessarily going to
> give you this for free.

It seems to me that we are operating at a level above the Fennec user
interface. But we'll need all of the advice we can get.

>
> Going with the mobile theme for a bit, I think consideration needs to
> be given about what a mobile Thunderbird would look like. Something
> that's as fully featured as desktop is likely to be very difficult to
> do a good UX for and is likely to be a lot slower.

Agreed that a phone app is a bit of a stretch, and likely would require
some radical rethinking. But an Android tablet is quite feasible, and I
think should be a goal. Maybe I'm just more tablet focused than most,
but I spend hours per day on my tablets.

>
> Hence if you're looking at rewriting a lot of the back-end, then it
> would be worth giving a thought to how it would operate/be constructed
> in a mobile world where you don't necessarily want all the features.
> For example, lets assume filters aren't required on mobile (I don't
> know of a mobile app that'd do that) - it be good to have that part of
> the back-end "plug-in" when required, potentially with hooks like
> today's add-ons can use, rather than for it to be built-in permanently
> even though there's no filters included.

I'm all in favor of addons as a way to manage complexity. It's great for
me to hear that Hello wants to ship as an addon, as to me that means
that Firefox will continue to think carefully of issues associated with
using complex addons. But it is a good time to remind ourselves of that
strategy. There are many pieces of Thunderbird that I would prefer to
break out as shipped addons (such as junk mail processing, or LDAP, or
account migration) only because they are major pieces of code only used
by a minority that can complicate the UI and code for the majority.

:rkent
Re: Future Planning: Thunderbird as a Web App R Kent James 9/18/15 11:58 AM
On 9/18/2015 6:32 AM, Gervase Markham wrote:
> On 18/09/15 13:55, Mark Banner wrote:
>> Here's my opinion, in no particular order, probably very incomplete:
> I think perhaps I wasn't clear - sorry about that. I wasn't asking "what
> have the mozilla-centrals ever done for us" :-), I was asking "In, say,
> the last two years, what improvements to mozilla-central have been
> useful to Thunderbird, as compensation for the last two years of breakage"?
>
> To put it another way: "what would we have lost if we had forked m-c two
> years ago"?
>
> I don't expect the list to be an empty set, but its size and the
> magnitude of the features on it would be useful in determining the
> feasibility of such a step in the future.

For Exquilla, I maintain compatibility with code that is sometimes two
years old (having only recently dropped support of Thunderbird 24) so I
have a lot of personal experience with this, both using js and C++. For
about 2 years it is possible with little workarounds, beyond that it
gets increasingly difficult. If the code is written to current m-c, it
is much harder - as even for recent landing of patches to esr38 for code
Thunderbird code, I had to work around issues of Array.includes
compatibility.

Examples of workarounds from my code, relative to 2-year old Thunderbird:

1)    Promises.all was not available in Thunderbird 24.
2)    Thunderbird 31 includes critical patches to NTLM management which
I added to m-c, and without this you can only support a single account
per server.

Overall these are pretty minor. But as you stretch much past 2 years,
say to Thunderbird 17, things start to get much, much harder. Also there
are all of the security fixes, which we (for better or for worse) have
some exposure to given that we allow full HTML rendering. So forking is
possible for short periods of time, but long-term is is not really viable.

Also, in my discussion with an experienced outsider (Michael Meeks of
The Document Foundation), he had this to say after observing our
situation, which is quite accurate:

"AFAICS your fundamental problem is an engineering / community decision
that doesn't match your resources whereby you run up this steep
travellator trying to keep up with Mozilla changes which ( in theory )
can provide a long-term benefit to make it easier to develop T-bird, but
in doing so you sacrifice actually doing the development on T-bird, so
that future ~never comes =)    Breaking that seems to me to be your top
priority."

I fully agree with that, but I only see two choices: fork and freeze the
Mozilla platform that we use, or migrate away from it. I am proposing
the latter.

:rkent
Re: Future Planning: Thunderbird as a Web App Joshua Cranmer �� 9/18/15 1:05 PM
On 9/18/2015 12:39 PM, Kent James wrote:
> LDAP is a great example of how thinking might change. I think we have
> discussed for years that LDAP code needs to convert to a standard,
> non-binary library in the long run. I think that we should resist the
> urge to try to adapt our existing C++-based library to support
> editing. But if you are interested in LDAP, we would really, really
> appreciate someone who would research available, license-compatible
> LDAP js libraries, and adapt our code to use that. Editing should be
> added in the context of that new library.

I've done this research before, and the results aren't great. Our best
hope for LDAP is using OpenLDAP, possibly via js-ctypes. ldap.js is
based on node's networking libraries (see my last message about the
variability of networking APIs), and it is missing some features that I
think we would be loath to regress (e.g., SASL GSSAPI support).

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________


tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Future Planning: Thunderbird as a Web App Volker Birk 9/18/15 1:24 PM
On Fri, Sep 18, 2015 at 01:41:18PM -0400, Andrew Sutherland wrote:
> On 09/18/2015 04:46 AM, Volker Birk wrote:
> >Crypto implementations ONLY can go from locally running software to
> >locally running software. Crypto implementations must not be server
> >based in any way, but have to be peer-to-peer only. Only then we have
> >end-to-end cryptography, only then we have security offered by crypto at
> >all, and not a simulation of security instead.
> So it's clear, the Firefox OS Gaia email app is currently a packaged and
> signed application that runs locally only.  The only servers contacted are
> the user's mail servers and those contacted in the course of running
> autoconfiguration/autodiscovery.  A content-security policy prevents code
> from being run from remote locations.

Cool. If you want, we port p‚Č°p on it ;-)

> This is already the strategy used by whiteout.io, a PGP-focused mail client
> built on HTML/CSS/JS running as a Chrome extension, and which likely will
> also run as a Firefox extension once the platform is further built out.

That makes the following issues:

1) PGP is a needed step in-between, but as it has lists of privacy
   issues itself, it's not a solution. The web of trust on keyservers
   means you're leaking your complete contact network including the
   information whom you trust. There is no way to hide meta-data,
   instead PGP creates additional meta data with key signatures.

2) GnuPG has real advantages when it comes to feature completeness and
   hardening against i.e. side channel attacks, which all other
   implementations still lack
   https://www.cs.tau.ac.il/~tromer/papers/radioexp.pdf
Re: Future Planning: Thunderbird as a Web App Robert Kaiser 9/18/15 1:51 PM
Joshua Cranmer ūüźß schrieb:
> On 9/18/2015 12:39 PM, Kent James wrote:
>> LDAP is a great example of how thinking might change. I think we have
>> discussed for years that LDAP code needs to convert to a standard,
>> non-binary library in the long run. I think that we should resist the
>> urge to try to adapt our existing C++-based library to support
>> editing. But if you are interested in LDAP, we would really, really
>> appreciate someone who would research available, license-compatible
>> LDAP js libraries, and adapt our code to use that. Editing should be
>> added in the context of that new library.
>
> I've done this research before, and the results aren't great. Our best
> hope for LDAP is using OpenLDAP, possibly via js-ctypes. ldap.js is
> based on node's networking libraries (see my last message about the
> variability of networking APIs), and it is missing some features that I
> think we would be loath to regress (e.g., SASL GSSAPI support).

Might there be a decent way to compile OpenLDAP into asm.js/WebAssembly
via EmScripten? That might be a way to get away from using actual native
code but still profit from that work - if it works decently.

KaiRo

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

Re: Future Planning: Thunderbird as a Web App Philipp Kewisch 9/18/15 3:45 PM
On 9/18/15 7:39 PM, Kent James wrote:
>>
>> If this is the direction things go, how will that affect my desire to
>> add full LDAP editing? Should I wait until the application core is
>> changed over first? Or will rewriting the Address Book module be part
>> of the code conversion?
>
> LDAP is a great example of how thinking might change. I think we have
> discussed for years that LDAP code needs to convert to a standard,
> non-binary library in the long run. I think that we should resist the
> urge to try to adapt our existing C++-based library to support
> editing. But if you are interested in LDAP, we would really, really
> appreciate someone who would research available, license-compatible
> LDAP js libraries, and adapt our code to use that. Editing should be
> added in the context of that new library.
I don't want to derail the discussion, but I also wanted to throw this
in: http://ldapjs.org/
Further discussion on LDAP should happen in another thread.

Philipp
Re: Future Planning: Thunderbird as a Web App Philipp Kewisch 9/18/15 3:47 PM
On 9/19/15 12:45 AM, Philipp Kewisch wrote:
> I don't want to derail the discussion, but I also wanted to throw this
> in: http://ldapjs.org/
> Further discussion on LDAP should happen in another thread.
>
Please ignore my message, I see Joshua has already commented on this :)
Re: Future Planning: Thunderbird as a Web App Patrick Brunschwig 9/19/15 8:49 AM
On 18.09.15 20:58, Kent James wrote:
[...]
> Also, in my discussion with an experienced outsider (Michael Meeks of
> The Document Foundation), he had this to say after observing our
> situation, which is quite accurate:
>
> "AFAICS your fundamental problem is an engineering / community decision
> that doesn't match your resources whereby you run up this steep
> travellator trying to keep up with Mozilla changes which ( in theory )
> can provide a long-term benefit to make it easier to develop T-bird, but
> in doing so you sacrifice actually doing the development on T-bird, so
> that future ~never comes =)    Breaking that seems to me to be your top
> priority."
>
> I fully agree with that, but I only see two choices: fork and freeze the
> Mozilla platform that we use, or migrate away from it. I am proposing
> the latter.

In my eyes the plans of Mozilla severely endanger the future of Thunderbird.

- If we migrate away from the Mozilla platform, we'll have to rewrite
everything that's currently in XPCOM, all GUIs, and a substantial part
of the JS-Code. That's an awful lot; my guess is 70 to 80% of the code.
This means nothing else than developing a completely new application(!).
It's obvious that requires a lot of manpower and a lot of time - both of
which we currently don't seem to have.

- OTOH, forking and freezing the platform does not seem like a long-term
solution. It would require Thunderbird to maintain the platform on it's
own. That's a huge challenge which would require quite some manpower
(which we again don't have). I honestly doubt that this will be feasible
in the long term.

I therefore think that if we want Thunderbird survive for more than just
a few years, we will have to migrate away from the Mozilla platform.
Given that the Mozilla plans are so short-term, the idea to fork and
freeze the platform for a limited time period seems to be the only way
to mitigate the risk of being too slow.

However, it should be clearly said as well: I'm certain that we will
loose a lot of the current addons (which I believe is a very important
reason for many of our users to have chosen Thunderbird). I for myself
cannot promise today that I will be able to keep up with the changes
that will be coming...

-Patrick
Re: Future Planning: Thunderbird as a Web App Matt Harris 9/19/15 5:26 PM
On 19/09/2015 4:11 AM, Kent James wrote:

I'm all in favor of addons as a way to manage complexity. It's great for me to hear that Hello wants to ship as an addon, as to me that means that Firefox will continue to think carefully of issues associated with using complex addons. But it is a good time to remind ourselves of that strategy. There are many pieces of Thunderbird that I would prefer to break out as shipped addons (such as junk mail processing, or LDAP, or account migration) only because they are major pieces of code only used by a minority that can complicate the UI and code for the majority.



I am only in favor of Add-0ns if we develop a better configuration process.  I am thinking here of something like the V3 migration wizard that can be run over and over, suggesting perhaps the 10 most used features and ending in AMO searches. Our users are not skilled.  They often lack basic skills like being able to right click without instruction. We still see people who do not 'get' tabs.   KISS applies more now than ever. The general populace see no reason they need to learn anything to use everything.

Matt

--
‚ÄúAgainst stupidity the gods themselves contend in vain.‚ÄĚ ‚Äē Friedrich von Schiller, Die Jungfrau von Orleans
Why we need Gecko updates (was: Future Planning: Thunderbird as a Web App) Ben Bucksch 12/8/15 4:38 PM
Gervase Markham wrote on 18.09.2015 15:32:
> To put it another way: "what would we have lost if we had forked m-c two
> years ago"?

Hey Gerv,

That's simple to answer: Security patches.

As you know from being at the Mozilla Security Group and the public
advisories, the Gecko rendering and JS engine has security holes fairly
regularly. Every month (or every few months), there's a critical
security hole, whereby "critical" means that any random ad published via
an ad server on a website you visit can read all your files on your
computer, install random malware, impersonate as you, and generally can
do whatever you can do on your own computer.

In Thunderbird, the risk is even larger than in Firefox. In Firefox, you
need to actively go to a website, and that website needs to attack you
(possibly via an ad server). We still assume that an attacker will
manage to get you to his website somehow, and we consider such a
critical bug the end of the computing world. In Thunderbird, the
attacker has it even easier: He just needs to send you an HTML email.
You view it, and you're done. Dead.

The default in Thunderbird is the HTML viewer.

Mitigation:
* JS is disabled by default
* "View | Message body as | Simple HTML", which tries to prevent most
security holes with HTML.
Neither is bullet-proof and some classes of bugs, e.g. in the parser, or
image decoders or - worse - in the complex native video codecs, are
still going to hit you with their full force.

Unfortunately, Mozilla gave up on supporting old Gecko versions with
security patches. Time's over once the ESR release is unsupported, which
is currently 6-8 months. Anything else was considered not feasible for
Firefox security team. There's no chance that the Thunderbird team can
keep up.

So, as much as I'd like that personally, forking Gecko is not an option.

(These dreaded security holes and the lack of patches have crossed many
plans, in many different areas and products and companies.)
Re: Why we need Gecko updates Axel Grude 12/8/15 5:26 PM
I wonder how Postbox manages, AFAIK they still use Gecko 9.0

Did Mozilla ever consider splitting Gecko from M-C? I would believe if it was in a separate branch it would be much easier to stay in sync.

Axel
 
--
Axel Grude
Software Developer
Thunderbird Add-ons Developer (QuickFolders, quickFilters, QuickPasswords, Zombie Keys, SmartTemplate4)
AMO Editor Get
          Thunderbird!

Subject: Why we need Gecko updates (was: Future Planning: Thunderbird as a Web App)
To: Tb-planning, Gervase Markham
From: Ben Bucksch
Sent: Wednesday, 09/12/2015 00:40:09 00:40 GMT ST +0000 [Week 49]
Re: Why we need Gecko updates R Kent James 12/8/15 5:46 PM
On 12/8/2015 5:25 PM, Axel Grude wrote:
> I wonder how Postbox manages, AFAIK they still use Gecko 9.0
>
> Did Mozilla ever consider splitting Gecko from M-C? I would believe if
> it was in a separate branch it would be much easier to stay in sync.
>
> Axel
I think that Postbox has basically just said that users don't care that
much about security updates, so neither do they. I don't think that they
do them, or at least if they do only a few.  Security updates are like a
religion at Mozilla, and yet the lack of pushback to Postbox on this
issue shows that users are not that concerned. Not that we should drop
our religion, just saying that if we want to be open to all options, we
should at least ask "Maybe security updates are not as important as we
think that  they are?" BenB, I only said as the question, I am not
implying that I know the answer.

I still hold to my estimates from a few months ago. That is, TB 45 and
52 will ship in sync with the Mozilla esr versions. After that (59?) we
will be on a fork of m-c, and probably can only maintain that with
security updates for a year at best. But that is of course a wild estimate.

:rkent
Re: Why we need Gecko updates Mihovil Stanić 12/9/15 12:30 AM
If remote servers are disabled by default and java script disabled in email, how big threat are those vurnabilities?
That doesn't sound like a big threat to me, but I might be mistaken.

Additionaly, I really don't see point in surfing webpages with TB.
Re: Why we need Gecko updates Ben Bucksch 12/9/15 12:33 AM
Axel Grude wrote on 09.12.2015 02:25:
> I wonder how Postbox manages, AFAIK they still use Gecko 9.0

They don't manage.
If they still use Gecko 9.0, I presume they have lots of security holes.

Ben
Re: Why we need Gecko updates Ben Bucksch 12/9/15 12:42 AM
R Kent James wrote on 09.12.2015 02:46:
> I think that Postbox has basically just said that users don't care
> that much about security updates, so neither do they. I don't think
> that they do them, or at least if they do only a few. Security updates
> are like a religion at Mozilla, and yet the lack of pushback to
> Postbox on this issue shows that users are not that concerned. Not
> that we should drop our religion, just saying that if we want to be
> open to all options, we should at least ask "Maybe security updates
> are not as important as we think that they are?" BenB, I only said as
> the question, I am not implying that I know the answer.

I think that security updates are mandatory.

If you *know* that there are security holes, and you do know at latest
when Firefox published security advisories, and you don't fix them, you
might be liable by law for all implications, and you might not be able
to exclude that in any contract, because it might be considered willful
negligence. The fact that you know that risk (or should have known)
makes a big difference in law. Just as when you don't put fences in a
place where you ought to, and somebody falls and dies, you're on the hook.

Law aside, at the very least, it's highly irresponsible to expose your
users to that risk. Saying that users don't care when their most private
data is open for grabs, and identity theft is easy, and hacking them is
easy, is just closing eyes.

Frankly, I can't believe you even put this up for discussion. That
"users don't care about security" theory is so Y2K. Even Microsoft
learned the hard way that it does matter, even if users are not aware of
it. They turned the ship with WinXP SP2 in 2004, but too late. Nobody
trusts them anymore.

Ben
Re: Why we need Gecko updates Ben Bucksch 12/9/15 12:43 AM
Mihovil Stanińá wrote on 09.12.2015 09:04:
> If remote servers are disabled by default and java script disabled in
> email, how big threat are those vurnabilities?

No JavaScript stops 90% of the holes. But not all of them. Some are in
lower level libraries.

Ben


_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates Jim 12/9/15 12:57 AM
On Wed, Dec 9, 2015 at 2:44 AM, Ben Bucksch <ben.b...@beonex.com> wrote:
Mihovil Stanińá wrote on 09.12.2015 09:04:
If remote servers are disabled by default and java script disabled in email, how big threat are those vurnabilities?

No JavaScript stops 90% of the holes. But not all of them. Some are in lower level libraries.

Right. If you eliminate 90% of the holes, it's probably possible to handle the remaining 10% on your own (through a combination of porting any relevant security fixes from Gecko, plus handling any security bugs found in Postbox itself). Of course, this means that you lose the benefit of having Firefox play the role of a giant target for hackers that you can use to stress-test all the code. However, I can't say for sure if that benefit outweighs the regular introduction of new vulnerabilities due to Gecko patches constantly landing; I'd have to guess that many of the new vulnerabilities are in new code that simply hasn't had as much time to get all the bugs removed.

- Jim
Re: Why we need Gecko updates Ben Bucksch 12/9/15 1:14 AM
Jim wrote on 09.12.2015 09:57:
> it's probably possible to handle the remaining 10% on your own

Go ahead and try it.

I've tried.
Re: Why we need Gecko updates Gervase Markham 12/9/15 3:02 AM
On 08/12/15 19:40, Ben Bucksch wrote:
> * JS is disabled by default

It would be interesting to catalogue the security bugs discovered over
the past 2 years to see which required JS (or video or audio playback,
both of which I hope TB disables as well) and which did not.

> Unfortunately, Mozilla gave up on supporting old Gecko versions with
> security patches. Time's over once the ESR release is unsupported, which
> is currently 6-8 months. Anything else was considered not feasible for
> Firefox security team. There's no chance that the Thunderbird team can
> keep up.

I think more analysis is required to say that it would be impossible for
the TB team to adapt or backport the number of security fixes over the
past two years which did not require JS to be enabled.

Re: Postbox:
> If they still use Gecko 9.0, I presume they have lots of security
> holes.

If this is true, why not spend a bit of time proving your point by
finding one? The MFSA list would be a good place to start. :-)

Gerv
Re: Why we need Gecko updates Robert Kaiser 12/9/15 6:53 AM
Note that Thunderbird's feed reader has all those bells and whistles
enabled and is vulnerable to almost any browser-style attack.

KaiRo

Gervase Markham schrieb:
Re: Why we need Gecko updates Magnus Melin 12/9/15 11:37 AM
On 09.12.2015 03:46, R Kent James wrote:
> I think that Postbox has basically just said that users don't care
> that much about security updates, so neither do they. I don't think
> that they do them, or at least if they do only a few. Security updates
> are like a religion at Mozilla, and yet the lack of pushback to
> Postbox on this issue shows that users are not that concerned. Not
> that we should drop our religion, just saying that if we want to be
> open to all options, we should at least ask "Maybe security updates
> are not as important as we think that they are?" BenB, I only said as
> the question, I am not implying that I know the answer.
>
> I still hold to my estimates from a few months ago. That is, TB 45 and
> 52 will ship in sync with the Mozilla esr versions. After that (59?)
> we will be on a fork of m-c, and probably can only maintain that with
> security updates for a year at best. But that is of course a wild
> estimate.
>

I'd like to add to the above that forking m-c isn't likely as long as we
could still build Thunderbird from it. If building gets impossible then
you'd have to go from there.

Besides lacking security updates you'd also build on dead-end technology
and nobody really wants to work with that a few years down the line.

  -Magnus
Re: Why we need Gecko updates Magnus Melin 12/9/15 11:38 AM
On 09.12.2015 13:02, Gervase Markham wrote:
> Re: Postbox:
> If they still use Gecko 9.0, I presume they have lots of security
> holes.
>
> If this is true, why not spend a bit of time proving your point by
> finding one? The MFSA list would be a good place to start.

Well one doesn't have to look pretty far on
https://www.mozilla.org/en-US/security/known-vulnerabilities/thunderbird/
There have been quite many fixes for various buffer overflows etc, and
most things related to images should easily affect us very much. That a
vulnerability might require certain expertise to understand how to
exploit is beside the point.

  -Magnus
Re: Why we need Gecko updates R Kent James 12/9/15 1:14 PM
On 12/9/2015 11:37 AM, Magnus Melin wrote:
> I'd like to add to the above that forking m-c isn't likely as long as
> we could still build Thunderbird from it. If building gets impossible
> then you'd have to go from there.
>
> Besides lacking security updates you'd also build on dead-end
> technology and nobody really wants to work with that a few years down
> the line.
>

I fully agree with this - up to a point. But the issue we face is this:
MoCo is telling us that they do not want to spend any time on
Thunderbird-related issues. If we take them at their word, then all of
these requests we make to m-c for changes to support Thunderbird will no
longer be accepted. In that case, m-c changes WILL break us. The option
you are presenting is one that MoCo is telling us is not possible.

As I see it, you are asking that we assume that MoCo will continue with
the status quo of the last few years, that is that they will give
minimum attention to Thunderbird, but they will still keep us building.
What I keep hearing them say is that they want to stop doing this,
because the tax on Firefox is too great. Then there is elephant in the
room, that they are really hoping to convert large parts of Firefox away
from C++, XUL, and XPCOM and toward the servo-based browser using Rust.
I cannot see any reasonable chance of converting Thunderbird to Rust, or
even wanting to.

Looking forward, my crystal ball tells me this schedule:

Thunderbird 46 - 52 (TB 52 ships around 2017-01): Largely continue the
status quo (though I think it is time to have our own branch of m-c, or
of a merged m-c/c-c, so that we are not broken all of the time, and we
can merge m-c to our build enviroment in a controlled manner including a
few of our own fixes). (Thunderbird 52 builds from mozilla-esr52,  or
the equivalent combined repo using merged m-c/c-c, from 2017-01 through
2017-12.)

Thunderbird 53 - 59: (TB 59 ships around 2017-09) Forked m-c, still
using XUL and XPCOM, merging in security patches. No longer merging
complete mozilla-central but only selected patches. Thunderbird 59
builds using security patches merged in from mozilla-esr59, from
2017-09  - 2018-08)

Thunderbird 60+ Around 2018-08, mozilla-esr59 will EOL, and we will no
long have any stable Mozilla repo to use as a basis for security
patches. mozilla-esr66 will be too far from buildable with Thunderbird
to be a usable base.

Here is the problem: to have any reasonable chance of switching to an
alternate platform, the time required is probably 2-3 years. If we
follow the current status quo, but otherwise keeping to the assumptions
above, somewhere in mid 2017, the pain to try to continue using m-c will
become too great, but by then we will only have about one year left
before any hope of keeping up with security patches is lost. If the
project dies without security patches, then we are dead in 3 years.

At some point we are going to have to make assumptions about the future,
and start acting on it. The current assumption is "Mozilla is bluffing".
I can't buy that. You are welcome to challenge any of my assumptions,
but just hoping that this issue is going to go away seems to me to be
wishful thinking.

:rkent
Re: Why we need Gecko updates al...@instantbird.org 12/9/15 1:19 PM
> Gervase Markham wrote on 18.09.2015 15:32:
>> To put it another way: "what would we have lost if we had forked m-c two
>> years ago"?

Two examples beside security bugs:

* Ongoing improvements to the JS engine and APIs: Within the past two
years, this includes (off the top of my head) ES6, Promises, Tasks,
OS.File, shutdown blockers. These have already been very useful, and are
key for any attempt to move more of the backend code from C++ to JS.

* Support for new OS versions: for example, support for UI theming that
fits with Windows 10 and OS X Yosemite/El Capitan (for the latter, e.g.
vibrancy and system font handling changes). GTK3 is likely a similar
story.

Answering this question may be a bit like pointing at the tip of an
iceberg - there are a lot of improvements and fixes that TB has been
getting for free, and many of them will be bugs that were never filed
for TB because the platform took care of them already.

--
http://www.fastmail.com - Email service worth paying for. Try it for free
Re: Why we need Gecko updates Axel Grude 12/9/15 1:35 PM

Subject: Re: Why we need Gecko updates
To: Tb-planning
From: R Kent James
Sent: Wednesday, 09/12/2015 21:14:32 21:14 GMT ST +0000 [Week 49]
On 12/9/2015 11:37 AM, Magnus Melin wrote:
I'd like to add to the above that forking m-c isn't likely as long as we could still build Thunderbird from it. If building gets impossible then you'd have to go from there.

Besides lacking security updates you'd also build on dead-end technology and nobody really wants to work with that a few years down the line.


I fully agree with this - up to a point. But the issue we face is this: MoCo is telling us that they do not want to spend any time on Thunderbird-related issues. If we take them at their word, then all of these requests we make to m-c for changes to support Thunderbird will no longer be accepted. In that case, m-c changes WILL break us. The option you are presenting is one that MoCo is telling us is not possible.

As I see it, you are asking that we assume that MoCo will continue with the status quo of the last few years, that is that they will give minimum attention to Thunderbird, but they will still keep us building. What I keep hearing them say is that they want to stop doing this, because the tax on Firefox is too great. Then there is elephant in the room, that they are really hoping to convert large parts of Firefox away from C++, XUL, and XPCOM and toward the servo-based browser using Rust. I cannot see any reasonable chance of converting Thunderbird to Rust, or even wanting to.
So here is the thing, if we fork M-C at some stage before it gets unmaintainable, is there a reasonable way of shrinking it to make it maintainable for the Thunderbird team? Or can it simply be frozen?



Looking forward, my crystal ball tells me this schedule:

Thunderbird 46 - 52 (TB 52 ships around 2017-01): Largely continue the status quo (though I think it is time to have our own branch of m-c, or of a merged m-c/c-c, so that we are not broken all of the time, and we can merge m-c to our build enviroment in a controlled manner including a few of our own fixes).
sounds good to me

(Thunderbird 52 builds from mozilla-esr52,  or the equivalent combined repo using merged m-c/c-c, from 2017-01 through 2017-12.)

Thunderbird 53 - 59: (TB 59 ships around 2017-09) Forked m-c, still using XUL and XPCOM, merging in security patches. No longer merging complete mozilla-central but only selected patches.
SO this is where it may get more(?) or less (?) work to keep in Sync? Obviously the big difference here is that the merging is completely done from our side without any support from the Fx dev team.

Thunderbird 59 builds using security patches merged in from mozilla-esr59, from 2017-09  - 2018-08)

Thunderbird 60+ Around 2018-08, mozilla-esr59 will EOL, and we will no long have any stable Mozilla repo to use as a basis for security patches. mozilla-esr66 will be too far from buildable with Thunderbird to be a usable base.
Which is fine if we fork M-C at an earlier stage and then start to own it ourselves. Hence my question about splitting Gecko. Gecko could potentially be kept in sync for longer (assuming Fx continues using that as browser engine) and the rest of (forked) M-C could be frozen so that we can continue to push C-C forward. Completely ripping out M-C is probably a no-go from all the work that would be necessary.



Here is the problem: to have any reasonable chance of switching to an alternate platform, the time required is probably 2-3 years. If we follow the current status quo, but otherwise keeping to the assumptions above, somewhere in mid 2017, the pain to try to continue using m-c will become too great,
even assuming it is forked? If we no longer have to keep up with the whole of M-C and own our own version? I think that's pretty much what Postbox does at the moment.

but by then we will only have about one year left before any hope of keeping up with security patches is lost. If the project dies without security patches, then we are dead in 3 years.

At some point we are going to have to make assumptions about the future, and start acting on it. The current assumption is "Mozilla is bluffing". I can't buy that. You are welcome to challenge any of my assumptions, but just hoping that this issue is going to go away seems to me to be wishful thinking.
How do we remove reliance on Mozilla's m-c, if not forking? A complete re-write?

Axel
Re: Why we need Gecko updates Axel Grude 12/9/15 1:39 PM

> Gervase Markham wrote on 18.09.2015 15:32:
>> To put it another way: "what would we have lost if we had forked m-c two
>> years ago"? 

Two examples beside security bugs:

* Ongoing improvements to the JS engine and APIs: Within the past two
years, this includes (off the top of my head) ES6, Promises, Tasks,
OS.File, shutdown blockers. These have already been very useful, and are
key for any attempt to move more of the backend code from C++ to JS.
true. I used three out of these five and had to write wrappers to keep my Addons Postbox compatible. So there is an argument for staying "in sync" as long as possible to benefit from more improvements. The question is whether (in an ideal world) we should prepare a parallel / alternative platform for an exit strategy. I have very little hope that there will be funds / willingness for a "rewrite" to completely move away from m-c...

Axel


Re: Why we need Gecko updates Magnus Melin 12/9/15 2:22 PM
On 09.12.2015 23:14, R Kent James wrote:
> On 12/9/2015 11:37 AM, Magnus Melin wrote:
>> I'd like to add to the above that forking m-c isn't likely as long as
>> we could still build Thunderbird from it. If building gets impossible
>> then you'd have to go from there.
>>
>> Besides lacking security updates you'd also build on dead-end
>> technology and nobody really wants to work with that a few years down
>> the line.
>>
>
> I fully agree with this - up to a point. But the issue we face is
> this: MoCo is telling us that they do not want to spend any time on
> Thunderbird-related issues. If we take them at their word, then all of
> these requests we make to m-c for changes to support Thunderbird will
> no longer be accepted. In that case, m-c changes WILL break us. The
> option you are presenting is one that MoCo is telling us is not possible.
>
> As I see it, you are asking that we assume that MoCo will continue
> with the status quo of the last few years, that is that they will give
> minimum attention to Thunderbird, but they will still keep us
> building. What I keep hearing them

There's a difference between stopping to give the kind of support
currently provided and outright refusing to take patches that would keep
us building. A vast majority of these bugs requiring m-c changes has
been due to the m-c/c-c split and not "code" changes. With the split
gone, they should be quite few.

> say is that they want to stop doing this, because the tax on Firefox
> is too great. Then there is elephant in the room, that they are really
> hoping to convert large parts of Firefox away from C++, XUL, and XPCOM
> and toward the servo-based browser using Rust. I cannot see any
> reasonable chance of converting Thunderbird to Rust, or even wanting to.

Whatever happens behind the scenes, Firefox has the desire to do the
front-end using HTML/JavaScript. That would be a fine solution for
Thunderbird too, and likely it's the only good long-term solution for us.

>
> Here is the problem: to have any reasonable chance of switching to an
> alternate platform, the time required is probably 2-3 years. If we
> follow the current status quo, but otherwise keeping to the
> assumptions above,

As I see it the only platform reasonable switchable to is doing things
in HTML/JavaScript. Now, this should be inline with Firefox's desires,
so in that regard there's a reasonable expectation we could run in the
same kind of app shell eventually.

> somewhere in mid 2017, the pain to try to continue using m-c will
> become too great, but by then we will only have about one year left
> before any hope of keeping up with security patches is lost. If the
> project dies without security patches, then we are dead in 3 years.
>
> At some point we are going to have to make assumptions about the
> future, and start acting on it. The current assumption is "Mozilla is
> bluffing". I can't buy that. You are welcome to challenge any of my
> assumptions, but just hoping that this issue is going to go away seems
> to me to be wishful thinking.
>
It's probably a good assumption that moving code to the web platform
would be more robust. At the time things have been moved over the
question of what kind of app shell to run it on should be decided based
on what's available then. At the moment we can't really tell.

I don't think it's a question of "Mozilla is bluffing" or not, it's just
a lack of resources to do the transition which is years of work.

  -Magnus
Re: Why we need Gecko updates Jim 12/9/15 3:00 PM
On Wed, Dec 9, 2015 at 3:16 AM, Ben Bucksch <ben.b...@beonex.com> wrote:
Jim wrote on 09.12.2015 09:57:
it's probably possible to handle the remaining 10% on your own

Go ahead and try it.

I've tried.

Unlike Postbox, I'm not a corporation with (I assume?) multiple developers who can work full-time on supporting the product. Without contacting the Postbox folks, we can't know how they handle this, but I'm not going to just assume they're doing things poorly without any evidence supporting that theory. From the outside, all we can say is that security issues don't *appear* to be a major issue for Postbox.

Of course, this doesn't mean that we should necessarily follow their lead, but I'd be interested in seeing what an actual Postbox developer has to say about this issue.

- Jim
Re: Why we need Gecko updates Tanstaafl 12/9/15 3:16 PM
On 12/8/2015 7:40 PM, Ben Bucksch <ben.b...@beonex.com> wrote:
> As you know from being at the Mozilla Security Group and the public
> advisories, the Gecko rendering and JS engine has security holes fairly
> regularly. Every month (or every few months), there's a critical
> security hole, whereby "critical" means that any random ad published via
> an ad server on a website you visit can read all your files on your
> computer, install random malware, impersonate as you, and generally can
> do whatever you can do on your own computer.

So...

Has anyone ever looked into how much effort would be involved in making
the HTML rendering engine somehow modular? But I guess that would
introduce other build issues of its own...
Re: Why we need Gecko updates Tanstaafl 12/9/15 3:17 PM
On 12/9/2015 3:04 AM, Mihovil Stanińá <mih...@miho.im> wrote:
> If remote servers are disabled by default and java script disabled in
> email, how big threat are those vurnabilities?
> That doesn't sound like a big threat to me, but I might be mistaken.
>
> Additionaly, I really don't see point in surfing webpages with TB.

I agree, and I neglected to follow-up, but this - 'surfing web pages
with TB' - is what I meant when I said the 'browser should be ripped out
at its roots'...

I understand that we need an HTML rendering engine for rendering HTML
emails, but that should be a very stripped down engine, no need for a
full blown browser engine. Personally, I don't see a problem with
completely disabling JS in the rendering of the email itself (separate
from magic done by Addons using JS)...


_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates Axel Grude 12/9/15 4:35 PM
Get
        Thunderbird!

Subject: Re: Why we need Gecko updates
To: Tb-planning
From: Tanstaafl
Sent: Wednesday, 09/12/2015 15:12:12 15:12 GMT ST +0000 [Week 49]
On 12/9/2015 3:04 AM, Mihovil Stanińá <mih...@miho.im> wrote:
> If remote servers are disabled by default and java script disabled in
> email, how big threat are those vurnabilities?
> That doesn't sound like a big threat to me, but I might be mistaken.
> 
> Additionaly, I really don't see point in surfing webpages with TB.

I agree, and I neglected to follow-up, but this - 'surfing web pages
with TB' - is what I meant when I said the 'browser should be ripped out
at its roots'...

I understand that we need an HTML rendering engine for rendering HTML
emails, but that should be a very stripped down engine, no need for a
full blown browser engine. Personally, I don't see a problem with
completely disabling JS in the rendering of the email itself (separate
from magic done by Addons using JS)...
+1

JS is for the chrome and Addons :-)
Re: Why we need Gecko updates Aceman 12/10/15 3:02 AM
What is that tax really?

Is there any noticeable work by FF devs to code stuff for TB?
They even called it "demands" from TB in the official message. Is there anything in m-c that was added just by TB requiring it? Is there code in m-c that they want to remove but can't because of TB? I am not aware of these cases, but maybe there are some. That is why I ask.

Or is it just that TB building/tests run on their servers? But then that does not affect the future direction of Firefox in any way.
______________________________________________________________
> Od: R Kent James <ke...@caspia.com>
> Komu: <tb-pl...@mozilla.org>
> D√°tum: 09.12.2015 22:14
> Predmet: Re: Why we need Gecko updates


>
>On 12/9/2015 11:37 AM, Magnus Melin wrote:
>> I'd like to add to the above that forking m-c isn't likely as long as
>> we could still build Thunderbird from it. If building gets impossible
>> then you'd have to go from there.
>>
>> Besides lacking security updates you'd also build on dead-end
>> technology and nobody really wants to work with that a few years down
>> the line.
>>
>
>I fully agree with this - up to a point. But the issue we face is this:
>MoCo is telling us that they do not want to spend any time on
>Thunderbird-related issues. If we take them at their word, then all of
>these requests we make to m-c for changes to support Thunderbird will no
>longer be accepted. In that case, m-c changes WILL break us. The option
>you are presenting is one that MoCo is telling us is not possible.
>
>As I see it, you are asking that we assume that MoCo will continue with
>the status quo of the last few years, that is that they will give
>minimum attention to Thunderbird, but they will still keep us building.
>What I keep hearing them say is that they want to stop doing this,
>because the tax on Firefox is too great.
_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates R Kent James 12/10/15 9:20 AM
Although "demand" can have have the meaning in English of "an insistent
and peremptory request, made as if by right" it can also mean "to call
for or require as just, proper, or necessary"  I don't think that
Mitchell intended to say that Thunderbird has been unreasonably
insistent in their interactions with Mozilla.

Specifically in response to "Is there code in m-c that they want to
remove but can't because of TB? " one good example is the cache
directory, which has been superseded by cache2 in FF but is kept for
Thunderbird. Another is the "deprecated" interface to the proxy code.
There have certainly been examples that I am aware of in character sets
as well, though they have been more aggressive there about removing
unneeded things.

There are also countless little ways throughout infrastructure in which
Thunderbird has code that is maintained, and places some burden on
Firefox. Think of AMO, BMO, Dataviz (that measures ADI), SUMO, Nucleas
(where release notes are written), and practically any other
infrastructure item, and there are little Thunderbird-specific features
that demand small amounts of attention from MoCo staff. Frequently
Thunderbird does not keep up with the current state of things, so the
versions that they maintain for us are older and therefore harder to
work with (think the old SVN website for example, where Kohei has
single-handedly brought us out of the old world into the new
Django/Bedrock world).

I also suggest that you listen to some of Chris Beard's speeches on Air
Mozilla about planning for 2016. One of the points he makes is that the
attempt of Mozilla to refocus on desktop Firefox requires that they
identify things that they used to do that are not adding value to
Firefox, and try to eliminate them. At the highest management levels,
they are looking around for what can be cut to allow better focus.
Thunderbird is an obvious target of this. (Apparently, so is Firefox OS
phones.)

I also do not think that we have the full story, and probably never
will. In my talks with Doug Turner about our infrastructure migration
(in which he was extremely cooperative and encouraging, by the way),
there were a couple of subtle hints about issues with Thunderbird. For
example, he thought we probably were not looking at crash stats (but
then I said we of course do, and our crash rate is lower than Firefox),
and he made comments about mostly unmaintained fundamental network code,
like for SMTP security. I don't think he is well informed here, and
probably has no need or desire to be well informed, but I took that as a
hint that Mozilla management may be concerned that Thunderbird is not
able to maintain the quality that they think should be associated with
the Mozilla brand.

Although there are lots of little points of intersection between
Thunderbird and Mozilla that could be the source of our tax, the area
they have asked us to focus on is release and build. I have not seen
anyone step forward and way, "Yes! I want to work on that". We really
need to find ways to solve this and move forward. Mozilla needs to see
that the Thunderbird team is willing to take on areas that cause high
levels of taxation in the Firefox world. How can we reorganize to do that?

:rkent

Re: Why we need Gecko updates Jim 12/10/15 12:09 PM
On Thu, Dec 10, 2015 at 11:20 AM, R Kent James <ke...@caspia.com> wrote:
I also suggest that you listen to some of Chris Beard's speeches on Air Mozilla about planning for 2016. One of the points he makes is that the attempt of Mozilla to refocus on desktop Firefox requires that they identify things that they used to do that are not adding value to Firefox, and try to eliminate them. At the highest management levels, they are looking around for what can be cut to allow better focus. Thunderbird is an obvious target of this. (Apparently, so is Firefox OS phones.)

The issues with Firefox OS provide a good contrast. There are a *lot* of Firefox OS developers (at least 100, I think). Compare that to Thunderbird, where we have just a handful. Given that we're just about dropping support for Firefox OS phones (I'm sure we'll still let people use B2GDroid and such, but that's not where we'll be focusing a lot of our time), the requirements for Firefox OS are now quite a bit smaller. If we focus on the one product that seems to be doing ok (smart TVs), about half the apps we've written become superfluous. Do you really want to check your calendar on a TV? (To say nothing of making phone calls.) The media apps are the important ones; all of the others can start winding down.

It's not clear to me what will happen to people working on core apps that are less important now. Will they get moved to desktop? I'd argue that at least a couple could be moved to Thunderbird. 2-3 full-time developers on Thunderbird could make a huge difference, and combining our efforts with the Firefox OS productivity team would help both parties out. We might be able to use code from Firefox OS to help modernize Thunderbird.

One good example would be enhancing <gaia-fast-list> to replace our usage of XUL trees. XUL trees are a mess, and also the main reason why we don't have 2-line items in the thread pane. Using an enhanced version of <gaia-fast-list> would resolve this while also helping us to move away from XUL. We could even pitch this to the rest of Mozilla as a way the Thunderbird developers could "give back" to Firefox. If we turn <gaia-fast-list> into a XUL tree replacement, then Firefox could also use this in places like the bookmarks/history window.

- Jim
Re: Why we need Gecko updates R Kent James 12/10/15 12:44 PM
On 12/10/2015 12:09 PM, Jim wrote:
> I'd argue that at least a couple could be moved to Thunderbird. 2-3
> full-time developers on Thunderbird could make a huge difference, and
> combining our efforts with the Firefox OS productivity team would help
> both parties out.
As far as I am concerned, it was a huge strategic error of Mozilla to
keep these teams separate in the first place. At Toronto a couple of
months ago we heard some anecdotes about how firm the resolve was of
Mozilla to prevent Thunderbird from infecting the Gaia email team.

I have been very reluctant personally though to push any attempt to
revive Thunderbird funding onto Mozilla. The reason for that is that I
assume that Mozilla management believes that interactions we have with
them are subtle attempts to ask for money. I don't want to reinforce
that, and I still think that we have no right to Firefox money. But I
welcome attempts from other people to make the case.

The fact that Thunderbird continues to grow in spite of official
neglect, while many other initiatives of Firefox have failed to live up
to expectations, shows that there is something missing in the standard
Mozilla description of what Thunderbird is, and how it is doomed to
failure and oblivion. Pay attention to your customers! They might know
something that we don't know.

On a related note, I have a Google hangout scheduled tomorrow with the
CEO of Nylas, who make the N1 email client. That is an innovative
desktop client using a lot of the technologies we have been talking
about considering. I don't have any brilliant ideas of how we could work
together, but I do believe fundamentally that we could be more effective
together than apart (they are GPL with a corporate, venture-funded plan
for monetization). Pity that I am talking to them and not Mozilla Gaia
email.

:rkent
Re: Why we need Gecko updates Andrew Sutherland 12/10/15 2:02 PM
On Thu, Dec 10, 2015, at 03:44 PM, R Kent James wrote:
> At Toronto a couple of
> months ago we heard some anecdotes about how firm the resolve was of
> Mozilla to prevent Thunderbird from infecting the Gaia email team.

I'm not quite sure I totally understand what you mean when you say
"infect", but this makes things sound like there was a nameless cabal
with a grudge against Thunderbird.  As the only person initially working
on the email app at the time the "how do we do email on firefox os"
decisions were made, my memory of the situation is this:
- Brendan Eich thought Thunderbird was best-in-breed and we should use
it as the basis for the mail app.
- I think Andreas Gal was of the mind that we did not want to add
unnecessary C++ components and web APIs.
- Having worked on Thunderbird for multiple years, I was definitely of
the mind that the right call was a pure JS solution for many reasons.

If there's some other anecdote you're thinking about, I'd like to help
clear the air there too.

> Pity that I am talking to them and not Mozilla Gaia
> email.

Can you elaborate on what you mean by this?  The gaia email team, of
which I am a part, is already represented here on this public mailing
list (and IRC), but I think you know that.  If you'd like to discuss
anything, I'm very happy to do that here, or on IRC (I'm still :asuth),
or in a Google hangout or wherever.

Andrew
Re: Why we need Gecko updates R Kent James 12/10/15 3:06 PM
I'm sorry, my posting came out much more negative than I had intended. I
was not an insider in the days when Gaia email was initiated, so I don't
know the full story of the how or why. But still, I think it is a pity
that we have not figured out ways to work together better. I certainly
hope that we can achieve more convergence in the future.

In my dreams there is a grand coalition of several key players in the
open-source email client world, that would all work together to make
something that could be truly competitive to the mega players in this
space. Thunderbird is part of that dream team, as well as Gaia email,
Postbox, and perhaps other newer players like N1 or others.

I don't believe I am the right person to lead that team, but if we could
at least start talking, maybe the right leadership would emerge. I do
not think that lots of small teams is going to ever compete seriously
with Microsoft and Google, but a larger coalition might, so let's talk.

I am pushing very hard to get the JsAccount work landed in TB 45, so
that JS-based backends could hook into TB. The Gaia Active Sync
implementation would be a great demo of that, perhaps I'll look into
that. If you have other ideas of ways we could try to work together, I
would love to hear it. I know you have been thinking about expanding the
Gaia Email work into a desktop client. If you do, how can we work
together instead of apart?

:rkent
Re: Why we need Gecko updates Andrew Sutherland 12/10/15 10:44 PM
On Thu, Dec 10, 2015, at 06:06 PM, R Kent James wrote:
> In my dreams there is a grand coalition of several key players in the
> open-source email client world, that would all work together to make
> something that could be truly competitive to the mega players in this
> space. Thunderbird is part of that dream team, as well as Gaia email,
> Postbox, and perhaps other newer players like N1 or others.

The major difficulty with banding together is that many of the existing
open source clients have all made and are all committed to mutually
incompatible architectural and implementation decisions.  Mailpile and
Nylas use Python based intermediary servers (like the Mozilla Messaging
"raindrop" experiment).  Thunderbird, Postbox, and Gaia Email are client
side without intermediary servers.  Much of Thunderbird and Postbox are
C++, while Gaia email is pure JS, etc.

There needs to be overlap to find synergies/efficiencies/etc., and in
many of these cases there is little overlap.

Thunderbird and Gaia email have already found some overlap in terms of
low level details.  On gaia mail's conversations development branch,
we're using Joshua Cranmer's excellent work on JSMime for MIME parsing.
Gaia email has also been able to make use of and contribute to the many
excellent http://emailjs.org/ libraries authored and maintained by
Andris Reinman, Felix Hammerl, and Tankred Hase of the (regrettably now
defunct) whiteout.io email client.

> I am pushing very hard to get the JsAccount work landed in TB 45, so
> that JS-based backends could hook into TB. The Gaia Active Sync
> implementation would be a great demo of that, perhaps I'll look into
> that.

ActiveSync is a frustrating, limited protocol in its normal usage-mode
that we implemented to support hotmail.com/outlook.com prior to their
(buggy but improving in quality) IMAP support.  To a lesser extent, we
also wanted to support Exchange servers, but it's hard to know if we
succeeded in that because we have no telemetry and have no test accounts
on old exchange servers, only new ones.  I would advise against
supporting it!  But if you really want to support ActiveSync, the gaia
libs at https://github.com/mozilla-b2g/jsas and
https://github.com/mozilla-b2g/jswbxml/ and perhaps our higher level use
of them at
https://github.com/mozilla-b2g/gaia-email-libs-and-more/tree/convoy/js/activesync/smotocol
would probably useful to build on.

I think potentially a more interesting account type would be support for
"backfilling" mailing lists by ingesting their archives.  Much of this
would likely resemble heuristics/rules for figuring out where the
archives are for the various mailing list managers informed by list-*
headers and then using an appropriate mbox-like parser.  This is again
somewhat more low level, but I think makes a perfect area of overlap
between Thunderbird between gaia mail and other JS mail clients.  I
would love to help contribute to something like this, and hopefully
others would too.

> If you have other ideas of ways we could try to work together, I
> would love to hear it. I know you have been thinking about expanding the
> Gaia Email work into a desktop client. If you do, how can we work
> together instead of apart?

In the immediate term, I think our area of overlap continues to be JS
implementations of protocols.

Longer term and at a higher level, it's tricky.

My vision in all of my work on Thunderbird was to use gloda as a
normalizing abstraction layer on top of the existing folder/message
representations.  It added an understanding of cross-folder
conversations and other nice things like the potential to handle people
having multiple email addresses, etc.

The idea was to build new UI stuff on this, and once we had the UI
sitting on top of that and decoupled from the existing highly-coupled
folder/message storage mechanism, we could ideally overhaul the
underlying message stores, etc.  (The big problem with all of this is
captured in the phrase "new UI".  New UI of such a magnitude arguably
ends up being a new product and not the existing Thunderbird people know
and love.)

My underlying goal for the gaia email app backend has always been to
produce a back-end capable enough to meet the needs of a powerful
desktop mail app as well as its immediate goal of being a powerful phone
mail app.  And to do so informed by what I learned from my efforts on
posterity, core Thunderbird, gloda, junius/raindrop, and deuxdrop.

It is my hope that sometime, not so long from now, I can make a post
here where I suggest that the gaia mail backend has reached that
threshold and that we can all now try and create an HTML-based UI that
is able to re-create the essence of what Thunderbird users love.

I haven't been shouting this goal from the rooftops because it has been
and continues to be a hope and not a guarantee.  We've all seen repeated
claims of "reinventing email"/etc. and I'm very sensitive to creating
harmful stop energy.  Thunderbird has been and continues to be a viable
product despite its technical debt issues, and I wouldn't want to
discourage people from working on it or coming up with solutions I was
not wise enough to see.  Additionally, I knew this would be a long slog
(though it's been much longer than I envisioned to get to where things
are now) and could not make guarantees about many things: continued
funding, that I wouldn't burn out and become a mime, etc.

If you want to shoot for maximum synergy with gaia mail and do it
starting now, here are some good options I see:

- I can use help on the existing gaia email conversations refactor.  The
pre-reqs on this based on current velocity and complexity are probably
someone with 10 hours/week to spend and a fairly good understanding of
the email problem domain, basic database transaction semantics, and a
reasonable understanding of ES6 (destructuring, generators).  A good
litmus test is probably for a potential contributor to look at the gloda
code and see if they understand what it's generally doing and can resist
cursing too much while reading the code.  (You made some very
significant contributions to the batching mechanism and I don't think
there was any cursing in the bugzilla comments and so are pre-qualified!
;)

- Assume the gaia email backend will get there and try and help distill
down what makes Thunderbird what it is so we can move faster when it
gets there (which it may not).  Create UX specs, mock-ups, prototypes,
add telemetry to Thunderbird to understand better how people use it,
etc.  For example, it's my (arbitrary, not-based-on-data) theory that
people like the "tactile" feel of the Thunderbird thread-pane that lets
you scrub through messages.  I think that its many columns, support for
sorting on all columns, and high performance at scrolling and the fact
that it shows you *all* the messages helps give you confidence that your
messages are there and that you can find what you're looking for, even
if you have to scroll for a long time.  In contrast, paginated and
search-biased interfaces like gmail may leave you feeling like your mail
is a haystack and all you can do is walk around the periphery and plunge
your hand in randomly and hope you find that needle.

The problem is that most of this is not useful to Thunderbird itself
right now.  And similarly, while the JsAccounts you implement could be
useful in a hypothetical gaia-mail-backend future, the JsAccount effort
itself is probably not since I would assume it uses existing Thunderbird
folder/message semantics.  Indeed, most efforts on Thunderbird that are
not pure JS/HTML and involve XUL or anything starting with nsI* are
probably not.  And writing that, that seems 1) jerky, 2) stop energy, 3)
I kinda really want to delete this paragraph but based on your comments
it seems like I have been under-communicating my own personal vision so
I won't.  But since I'm not deleting this, I absolutely want to convey
how tremendous your efforts on Thunderbird technically and
organizationally have been from my perspective.  While you have
different technical opinions than I do, I absolutely have confidence in
your technical skills and am confident your decisions are based on
bad-ass expertise.  (And with the rest of Thunderbird assumed, JsAccount
does seem like a very wise implementation choice.)

Hopefully this message is somewhat helpful in clarifying the state of
things re: gaia mail even if it doesn't provide immediate hope for the
collaboration that we all want.  Unfortunately, I should probably also
call out that Firefox OS is currently in a state of priority flux and
while I personally hope current MoCo engineering resources on the gaia
mail app continue at current levels of effort or greater, I can't
guarantee that they will.  What I can say with high confidence is that I
personally will continue to hack on the gaia mail app backend in my
spare time even if I am no longer paid to, fwiw.

Andrew
Re: Why we need Gecko updates Joshua Cranmer 12/11/15 6:11 AM
On 12/8/2015 6:40 PM, Ben Bucksch wrote:
> In Thunderbird, the risk is even larger than in Firefox. In Firefox,
> you need to actively go to a website, and that website needs to attack
> you (possibly via an ad server). We still assume that an attacker will
> manage to get you to his website somehow, and we consider such a
> critical bug the end of the computing world. In Thunderbird, the
> attacker has it even easier: He just needs to send you an HTML email.
> You view it, and you're done. Dead.

I think you're wrong here. It is probably about two to three orders of
magnitude harder to deliver an exploit to an email client than it is a
web browser. Much web traffic remains in HTTP, which permits MITM
attacks, and ad servers are a great way to feed malware to users
(particularly pernicious are ads on freeware download sites--just mimic
a download button, you're virtually guaranteed a good click-through
rate). Also, web browsers willingly send their identities to the
servers, making it trivial to target malware specific to the user's
machine. By contrast, email requires guessing a user's email address,
guessing their client, and passing anti-spam and anti-virus. It's easier
to phish the user into opening a malware attachment than it is to
correctly guess which client is being used to target it specifically,
and it's certainly a better use of limited email throughput (high-volume
email outflow tends to get quickly binned as spam).

That's not to say that email is immune to risk--if I wanted to
specifically target someone, I'd probably try via email instead of a web
browser.
> Mitigation:
> * JS is disabled by default
> * "View | Message body as | Simple HTML", which tries to prevent most
> security holes with HTML.
> Neither is bullet-proof and some classes of bugs, e.g. in the parser,
> or image decoders or - worse - in the complex native video codecs, are
> still going to hit you with their full force.

Mozilla actively fuzzes JS and all of its input formats (HTTP, HTML,
CSS, multimedia codecs), and I've gotten the sense that the security
team badly wants the codecs in particular moved to Rust, which would
drastically reduce the scope and likelihood of exploitable security
bugs. So I reckon the riskiest and most vulnerable code in Thunderbird
is not in mozilla-central, but comm-central. Our protocol client
implementations are almost certainly insecure against hostile server
implementations, and some of our MIME and mbox code is rife with mixed
NUL-terminated/non-NUL-terminated strings, which is a recipe for
security bugs. Unlike Firefox, we're not mitigating these risks by
fuzzing them to any degree. I would not be surprised if there were an
exploitable security bug that could be used against every version of
Thunderbird from the first working public CVS build to the current
tip-of-trunk; the same I would not say for Firefox.

--
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth
Re: Why we need Gecko updates Joshua Cranmer �� 12/11/15 8:08 AM
On 12/10/2015 2:44 PM, R Kent James wrote:
> On 12/10/2015 12:09 PM, Jim wrote:
>> I'd argue that at least a couple could be moved to Thunderbird. 2-3
>> full-time developers on Thunderbird could make a huge difference, and
>> combining our efforts with the Firefox OS productivity team would
>> help both parties out.
> As far as I am concerned, it was a huge strategic error of Mozilla to
> keep these teams separate in the first place. At Toronto a couple of
> months ago we heard some anecdotes about how firm the resolve was of
> Mozilla to prevent Thunderbird from infecting the Gaia email team.

I have talked to asuth a fair amount over the past few years about the
interaction of Gaia email and Thunderbird; unfortunately the relevant
IRC channels aren't logged, so I can't point you at those records.

The basic summary, though, that we can both agree on is that there is
very little that Thunderbird had to offer Gaia email when it was started
up. The state of JS libraries was extremely poor back then (and is still
somewhat shoddy), so trying to contribute shared JS code from Gaia to
Thunderbird is difficult, particularly because Thunderbird is extremely
feature rich, and I don't think any part of the relevant JS libraries is
at feature-parity with our code yet (!). For example, SMTP, the simplest
of the core protocols, needs to support GSSAPI, CRAM-MD5, NTLM SASL
auth, non-UTF-8 8-bit-messages (thanks, Japan), proxies, and message
delivery notifications, none of which Gaia's library (emailjs.org's
smtpclient) supports.

Having talked with both the Gaia email and the emailjs.org people, I've
more or less gotten people to agree on some of the changes to be made,
but I've lacked any time to actually develop those changes. If people
can spare some time, fleshing out the email-socket library and hooking
up smtpclient and email-sasl to that would open the doors to being able
to share some more code between various email projects.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

Re: Future Planning: Thunderbird as a Web App Sean M. Pappalardo 12/11/15 9:41 AM
Hello everyone.

On 09/17/2015 04:06 PM, Axel Grude wrote:
>> tl;dr Thunderbird over the next 3 years needs to convert to being a
>> web app that can run on any browser that supports ES6 Javascript and
>> HTML5. (web app does not imply cloud-based, only that the underlying
>> platform is js/html).

I would like to bring up SOGo, an open-source product by Inverse that
uses a Thunderbird-lookalike HTML/CSS/JS Web interface for its v2
product. http://sogo.nu/ (The also supply XUL/JS extensions to use the
real Thunderbird+Lightning with its server components, FWIW.)

To me, this Web interface seems an excellent starting point since the UI
is already done and has been stable for years. The issues I see are any
license incompatibilities (SOGo is dual-licensed under GPLv2 and
LGPLv2.1) and obviously replacing the back-end with something that can
run in a desktop context.

Thoughts?

Sincerely,
Sean M. Pappalardo
Sr. Networks Engineer
Renegade Technologies
spapp...@renegadetech.com
Office: (630) 631-6188
http://www.renegadetech.com

--
This communication, along with any documents, files or attachments, is
intended only for the use of the addressee and may contain confidential
information. If you are not the intended recipient, you are hereby
notified that any dissemination, distribution or copying of any
information contained in or attached to this communication is strictly
prohibited.
If you have received this message in error, please notify the sender
immediately and destroy the original communication and its attachments
without reading, printing or saving in any manner.

Re: Why we need Gecko updates Joshua Cranmer �� 12/11/15 10:16 AM
On 12/11/2015 12:44 AM, Andrew Sutherland wrote:
> I think potentially a more interesting account type would be support for
> "backfilling" mailing lists by ingesting their archives.  Much of this
> would likely resemble heuristics/rules for figuring out where the
> archives are for the various mailing list managers informed by list-*
> headers and then using an appropriate mbox-like parser.  This is again
> somewhat more low level, but I think makes a perfect area of overlap
> between Thunderbird between gaia mail and other JS mail clients.  I
> would love to help contribute to something like this, and hopefully
> others would too.

Back when I was originally exploring adding new account types in 2007
(!), the two account types I was looking at implementing were mailing
list archives and web forums. It's worth pointing out that Mailman 3 has
talked about (implemented? I'm not entirely sure) exposing their archive
history via NNTP, since it's sort of the exact kind of simple protocol
that's useful for indicating archive histories.


> In the immediate term, I think our area of overlap continues to be JS
> implementations of protocols.

Would it be possible to draft you or someone else on the Gaia email team
for structuring JS library imports into comm-central? As we look into
moving stuff into JS, it would be great to share the protocol
implementations more widely, and that really means that we need to make
it easier to port between smaller package repositories (like JSMime) and
comm-central. Thunderbird has a sufficiently large userbase that it's
arguably the best chance for email-relevant libraries to actually make
sure they work on real world data by confirming that they can be used by
TB's multi-million-large userbase.

For what it's worth, the rough list of stuff that is sanely shareable
IMHO: POP, IMAP, SMTP, NNTP, chat protocols (IRC, Oscar, Yahoo, MSN,
???), LDIF, vCard, CardDAV, iCal, CalDAV, MIME (+ mbox, S/MIME, PGP,
TNEF, uuencode, yEnc), SASL, Bayesian message classification,
SIEVE/MANAGESIEVE. Probably a few utility libraries as well.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________


tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates Wayne Mery (Thunderbird QA) 12/11/15 11:20 AM
On 12/10/2015 12:20 PM, R Kent James wrote:
> Although "demand" can have have the meaning in English of "an insistent
> and peremptory request, made as if by right" it can also mean "to call
> for or require as just, proper, or necessary"  I don't think that
> Mitchell intended to say that Thunderbird has been unreasonably
> insistent in their interactions with Mozilla.
>
> Specifically in response to "Is there code in m-c that they want to
> remove but can't because of TB? " one good example is the cache
> directory, which has been superseded by cache2 in FF but is kept for
> Thunderbird.

Indeed. I seem to recall talking to someone who said cache2 should not
be difficult to implement.

I should think that Seamonkey would also want this.

Great to hear Doug is so engaged. And yes both crash-stats (mostly major
crashes only) and bugzilla reports are well looked after - although I
think it's mainly just me and Magnus (and standard8 prior to that), so
we need others would step forward to share the load [1].

And yes our crash rate is far lower than Firefox's, about 1/3 [2].
Although I suspect our users data is a little more at risk from
corruption and loss from crashes.

[1] crash bugs filed in past year: http://mzl.la/1J0suhi
[2] crash rates ...
Graph: http://imgur.com/iQ5MDn1
TB query: http://bit.ly/1O0m6hD
FF query: http://bit.ly/1NgxAtO


> and he made comments about mostly unmaintained fundamental network code,
> like for SMTP security. I don't think he is well informed here, and
> probably has no need or desire to be well informed, but I took that as a
> hint that Mozilla management may be concerned that Thunderbird is not
> able to maintain the quality that they think should be associated with
> the Mozilla brand.

Given the fact that there is no active module owner, sure, there is
reason for concern. But then, over the last 10 or so years, we've rarely
had security issues in these areas.

Re: Why we need Gecko updates Andrew Sutherland 12/11/15 11:37 AM
On Fri, Dec 11, 2015, at 01:16 PM, Joshua Cranmer ūüźß wrote:
> Would it be possible to draft you or someone else on the Gaia email team
> for structuring JS library imports into comm-central? As we look into
> moving stuff into JS, it would be great to share the protocol
> implementations more widely, and that really means that we need to make
> it easier to port between smaller package repositories (like JSMime) and
> comm-central.

Can you clarify...  Do you mean making sure protocol libraries are
available as npm-packaged modules?  Making comm-central be able to use
npm-packaged modules?  Other?

Andrew


_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates Joshua Cranmer �� 12/11/15 7:16 PM
On 12/11/2015 1:36 PM, Andrew Sutherland wrote:
> On Fri, Dec 11, 2015, at 01:16 PM, Joshua Cranmer ūüźß wrote:
>> Would it be possible to draft you or someone else on the Gaia email team
>> for structuring JS library imports into comm-central? As we look into
>> moving stuff into JS, it would be great to share the protocol
>> implementations more widely, and that really means that we need to make
>> it easier to port between smaller package repositories (like JSMime) and
>> comm-central.
> Can you clarify...  Do you mean making sure protocol libraries are
> available as npm-packaged modules?  Making comm-central be able to use
> npm-packaged modules?  Other?

I'm imagining that we want each protocol to be available as a separate
npm package, and my thought is that they would leave in separate
repositories that are kept more-or-less in-sync with comm-central. How I
packaged JSMime was intended to follow that model, but since it's built
into a single file for comm-central, it makes moving changes between the
two versions painful. It's possible that my thought is a distinctly
worse development proposal, though.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________


tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates R Kent James 12/12/15 12:58 PM
On 12/11/2015 7:16 PM, Joshua Cranmer ūüźß wrote:
> I'm imagining that we want each protocol to be available as a separate
> npm package, and my thought is that they would leave in separate
> repositories that are kept more-or-less in-sync with comm-central. How
> I packaged JSMime was intended to follow that model, but since it's
> built into a single file for comm-central, it makes moving changes
> between the two versions painful. It's possible that my thought is a
> distinctly worse development proposal, though.

If we picture a world where we both receive upstream code regularly
which we incorporate into our product, as well as maintain modules that
we hope others will share from us, I think it would be better if we flip
the model of where the canonical repository is. That is, we should keep
things like jsmime as separately downloadable packages on github, and
then have code in client.py (or something similar) that pulls specific
version of upstream repositories into our code tree. We still need some
mechanism to allow us to land Thunderbird-specific changes locally, if
only for urgent changes, perhaps using a CANONICAL_UPSTREAM branch for
updates, then merge into our default branch.

If someone wants to use jsmime (or other shared packages) outside of
Thunderbird, they need some convenient way of specifying the changes
that they need without having to understand all of the complexities of
interacting with the Mozilla world, and any required changes in
Thunderbird to support that. Github is the most common way this is done
these days, and you see much Mozilla code migrating there already. Yes
it makes our life like a little more difficult, but I think it would be
great if we would learn to be more cooperative with other groups, and
making jsmime and similar packages follow typical open source comment
and pull-request patterns would help with that.

Plus if we ever start using packages like react in our own code, we are
going to be forced to support this approach anyway.

:rkent


_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning
Re: Why we need Gecko updates Tony Mechelynck 12/13/15 3:18 PM
On Sat, Dec 12, 2015 at 9:58 PM, R Kent James <ke...@caspia.com> wrote:
>
> On 12/11/2015 7:16 PM, Joshua Cranmer wrote:
>>
>> I'm imagining that we want each protocol to be available as a separate npm package, and my thought is that they would leave in separate repositories that are kept more-or-less in-sync with comm-central. How I packaged JSMime was intended to follow that model, but since it's built into a single file for comm-central, it makes moving changes between the two versions painful. It's possible that my thought is a distinctly worse development proposal, though.
>
>
> If we picture a world where we both receive upstream code regularly which we incorporate into our product, as well as maintain modules that we hope others will share from us, I think it would be better if we flip the model of where the canonical repository is. That is, we should keep things like jsmime as separately downloadable packages on github, and then have code in client.py (or something similar) that pulls specific version of upstream repositories into our code tree. We still need some mechanism to allow us to land Thunderbird-specific changes locally, if only for urgent changes, perhaps using a CANONICAL_UPSTREAM branch for updates, then merge into our default branch.

In fact, we (Thunderbird and SeaMonkey) are already doing this for
part of our code: specific mail, mailnews and suite code lives in
comm-central, onto whose clones client.py grafts clones of
mozilla-central, the LDAP SDK, ChatZilla and DOMi, each of which has a
separate Mercurial repository with its own specific code. To keep
things simple for client.py, I would recommend Mercurial repositories
in preference to git repositories for anything we want to outsource
the same way, and that would mean "not github", but we could put
anything not already found on hg.m.o. on e.g. BitBucket, your concept
wouldn't need to change.
>
> If someone wants to use jsmime (or other shared packages) outside of Thunderbird, they need some convenient way of specifying the changes that they need without having to understand all of the complexities of interacting with the Mozilla world, and any required changes in Thunderbird to support that. Github is the most common way this is done these days, and you see much Mozilla code migrating there already. Yes it makes our life like a little more difficult, but I think it would be great if we would learn to be more cooperative with other groups, and making jsmime and similar packages follow typical open source comment and pull-request patterns would help with that.

(in view of my comment to the previous para.) s/Github is the most
common way/BitBucket is a common way/. IMHO adding appropriate
Mercurial tags for anyone (including comm-aurora, comm-beta and
comm-release) not willing to use the repository tip wouldn't be a big
hassle.
>
> Plus if we ever start using packages like react in our own code, we are going to be forced to support this approach anyway.
>
> :rkent

Best regards,
Tony.
Re: Why we need Gecko updates Ben Bucksch 12/15/15 11:24 PM
Magnus Melin wrote on 09.12.2015 23:22:
> A vast majority of these bugs requiring m-c changes has been due to
> the m-c/c-c split and not "code" changes. With the split gone, they
> should be quite few.

So, wait a second. Why does the split pose all that problem? If m-c and
c-c would check out into the same directory, mailnews/ and mail/ it
would be on the same level as browser/, no? Why can't you do that?
Should be just a matter of git or hg tricks, a one-time work.

(The old layout was a good idea as long as there still has a XULRunner
around.)
Re: Why we need Gecko updates Ben Bucksch 12/15/15 11:32 PM
R Kent James wrote on 10.12.2015 18:20:
> Frequently Thunderbird does not keep up with the current state of
> things, so the versions that they maintain for us are older and
> therefore harder to work with (think the old SVN website for example,
> where Kohei has single-handedly brought us out of the old world into
> the new Django/Bedrock world).

Now, that part would really piss me off, if I had to maintain system,
and make me want to discontinue support.

It also seems something that TB can reasonably improve.
Re: Why we need Gecko updates Ben Bucksch 12/16/15 1:09 AM
Joshua Cranmer wrote on 10.12.2015 19:49:
> I think you're wrong here. It is probably about two to three orders of
> magnitude harder to deliver an exploit to an email client than it is a
> web browser.

Reality proved me right. Google and several other big companies got
hacked by China a few years ago. This hack was bad enough to make the
CEOs so upset that they went public about it and Google even closed down
google.cn , citing this hack as reason / last straw. (Which may not be
the full story, as the public story rarely is. Either way, that's just
one case in point that...)

Email is being actively used as attack vector to hacking on the highest
levels, and even companies who really ought to know better fell for it.
And the cases we know about it probably are just 1% of what's actually
happening.

More technical rebuttal:
> ad servers are a great way to feed malware to users

And then, there's spam...

> web browsers willingly send their identities to the servers, making it
> trivial to target malware specific to the user's machine.

I think the email address is more tied to a person than a web browser.
If you want to hack someone specific, HTML email is the easiest way. No
need to redirect Internet traffic, pick the right target out of
hundreds. Just send a well-crafted email.

> if I wanted to specifically target someone, I'd probably try via email
> instead of a web browser.

Right. OK, we agree, then. That's what I was talking about. Most
high-value hacks require to target someone specific. Mass-hacks are boring.

Ben
Re: Future Planning: Thunderbird as a Web App Ben Bucksch 12/17/15 1:43 PM
Sean M. Pappalardo wrote on 11.12.2015 16:03:
> I would like to bring up SOGo, an open-source product by Inverse that
> uses a Thunderbird-lookalike HTML/CSS/JS Web interface for its v2
> product. http://sogo.nu/ (The also supply XUL/JS extensions to use the
> real Thunderbird+Lightning with its server components, FWIW.)

http://www.sogo.nu/tour/screenshots.html

LOL. Hello "Thunderbird as a web app" :)
Re: Why we need Gecko updates Magnus Melin 12/18/15 1:21 AM
On 16.12.2015 09:24, Ben Bucksch wrote:
> So, wait a second. Why does the split pose all that problem? If m-c and c-c
> would check out into the same directory, mailnews/ and mail/ it would be on
> the same level as browser/, no? Why can't you do that? Should be just a
> matter of git or hg tricks, a one-time work

Just checking out into the same directory would cause a lot of pain as
mentioned in the other thread.  We need to have an easy way to "just get the
code without tinkering", and to be able to hunt down an exact regressing
changeset which is really hard atm since you have two repos that interact and
it may be impossible to get a building version within a regression range.

  -Magnus
More topics »