platform Email and forked versions

273 views
Skip to first unread message

Andrew Stadler (Google)

unread,
Feb 12, 2009, 11:26:05 AM2/12/09
to android-...@googlegroups.com
There has been some discussion about K9 merging back into the platform
Email sources. I'd like to address some of the things that have come
up and try to shed some light on the situation.

Who am I? I have been a member of the Android software team since
late 2007, but a couple of months ago I shifted over and became the
lead maintainer for the Email client. I have written or supervised
all of the post-1.0 changes. Some of these you can see as reliability
& compatibility improvements (the primary focus of the 1.1 effort, for
Email). Others changes are in the form of improved localizability,
now that Android devices are starting to ship in non-US locales.
Finally, if you look in the cupcake sources, you'll find continued
improvements in all of those areas, as well as a unit testing
framework (which was not present in the original 1.0 open source
release).

The basic question at hand is: When will K9 be bulk-merged back into
the mainline sources?

The short answer is: It will not.

The long answer is:

We are very excited about the existence of forks like K9 (and others
which might be out there). This is one of the highlights of open
source work, seeing a different group take your work and extend it to
the next level. In contrast to a comment made in the other thread, we
are quite happy to see forks, because they provide opportunities for
other developers to explore new ideas, and they contribute to a robust
market for Android applications.

But somewhere along the line there has been an impression that somehow
K9 would become the "new" platform email application, via a
large-scale merge. I'm not sure how this happened. If it was said by
Googlers, please consider this email both an apology and a correction.
If it was wishful thinking, well, I'm here to let you down a bit :(
But in any case, I want to clearly state that this is not the plan.

While I don't mean to dwell on the negative, here are a few of the
reasons why not.

1. K9 is great - I know Googlers who are quite happy using it. But
K9 is not somehow earmarked to become the new platform Email app.
What would happen if another group of coders showed up 3 months later
with another client fork? Should we merge that next? Where does it
stop?

2. K9 was developed under a very different set of constraints than we
have in the platform Email client. I must work within user interface
guidelines and remain consistent with the rest of the platform, as
well as user testing, and the requirements of carrier partners. The
K9 team has a lot more flexibility to experiment with changes to the
user interface than I do, and they have the benefit of much more
immediate feedback from their download customers.

3. When I work on the platform Email client, I get to deal with
localization, multi-carrier requirements, and a lot of testing. And
most of all, I have to work within platform release schedules, which
limits the windows in which I can add new features. Third party
application teams can work on a much more flexible release schedule.

But I'd like to cover some positive territory too.

First, the email client in cupcake is in better shape than 1.1, and
that's in better shape than 1.0. I encourage the K9 team, as well as
any other Email forks, to take as much from cupcake as possible.
You'll find a more reliable POP3 implementation, improved
interoperability (welcome back, comcast.net users!), and overall
cleanup of some of the rough edges in the client. It has a long way
to go, but it's a lot better than it was. You'll find a couple of new
features, too. And we're not done yet.

Second, I encourage & welcome patches from anybody interested in
making Email better (including the K9 team). Pushing patches upstream
is good for everybody: It improves the platform app, and it makes
life easier for everyone who is forking from it.

Here are some things that will make for successful patches.
* Good patches will make the client better for users, by fixing bugs,
improve efficiency or increasing interoperability. Or, they make the
client better for developers, by making it more extensible,
debuggable, or testable.
* Patches that make direct changes to the UI are much less likely to
be accepted. But patches that make it easier to change the UI in a
downstream fork, are fine.
* As David Turner said in the previous email, smaller really is
better. Big patches take far, far longer to approve (or may simply be
sent back.)
* Test, test, test, test. Please plan to include unit tests for your
changes. I know that some parts of the code are more testable than
others. If you can't test a piece of Email, that's a bug and we can
work together to correct that. I've been finding that fairly small
refactorization is all that's necessary to make the code testable.

I'm sure there's more to cover but this is already longer than I had
intended to write.

I look forward to discussing patches & bugs!

Andy Stadler
Google

Disconnect

unread,
Feb 18, 2009, 11:55:13 AM2/18/09
to android-...@googlegroups.com
(Disclaimer: I have nothing to do with k9, nothing to do with google, etc.)

Let me open by saying I'm -very- disappointed in your position on this.  According to google, AOSP is the "prime" Android product, and the various closed-source product forks are merely that - secondary forks. So far the behavior has been the exact opposite, and this is a glaring example of that.

I'll address the individual points inline.. (Or you can skip to the very end :) ..)

On Thu, Feb 12, 2009 at 11:26 AM, Andrew Stadler (Google) <sta...@android.com> wrote:

But somewhere along the line there has been an impression that somehow
K9 would become the "new" platform email application, via a
large-scale merge.  I'm not sure how this happened.  If it was said by
Googlers, please consider this email both an apology and a correction.
 If it was wishful thinking, well, I'm here to let you down a bit :(
But in any case, I want to clearly state that this is not the plan.

I can find the references if you really want, but basically there were statements to the list (by googs and k9 devs) indicating that K9 would merge with DAmail (iirc) to gain imap-idle support, and then begin merging with mainline. 
 
While I don't mean to dwell on the negative, here are a few of the
reasons why not.

1.  K9 is great - I know Googlers who are quite happy using it.  But
K9 is not somehow earmarked to become the new platform Email app.
What would happen if another group of coders showed up 3 months later
with another client fork?  Should we merge that next?  Where does it
stop?

The reason it was so far from upstream is because the upstream tree was closed. You were more than welcome (and capable) of pulling patches out of the k9 tree, but the rest of us could not even submit typo fixes to the upstream tree until much later. (At which point we were told "don't bother, there is a new tree coming someday, submit patches when that happens.")
 
2.  K9 was developed under a very different set of constraints than we
have in the platform Email client.  I must work within user interface
guidelines and remain consistent with the rest of the platform, as
well as user testing, and the requirements of carrier partners.  The
K9 team has a lot more flexibility to experiment with changes to the
user interface than I do, and they have the benefit of much more
immediate feedback from their download customers.

"I have secret docs". Thats fine. If the are part of AOSP, they can't be secret. If they are not part of AOSP, they have no bearing AT ALL on the email client that goes into AOSP, and perhaps you should find a maintainer that is not bound by them and shift over to maintaining the closed source client. (For the record, I'm not saying there are no UI problems.)

3.  When I work on the platform Email client, I get to deal with
localization, multi-carrier requirements, and a lot of testing.  And
most of all, I have to work within platform release schedules, which
limits the windows in which I can add new features.  Third party
application teams can work on a much more flexible release schedule.

"I have more secret docs." ....sigh. 
 

This is an excellent time for google to decide if the AOSP is or is not Android. So far, AOSP is the afterthought, if even that. There were deadlines, surprises, etc. Fine. Whatever, the past is past. But moving forward, if AOSP is really the main product, its time to make it the priority and let the closed source carrier forks take a back seat.

Or not, but either way the actions of you and the other maintainers, admins, committers, reviewers and members right now are going to have long term, lasting consequences.

Bradley Young

unread,
Feb 18, 2009, 1:15:07 PM2/18/09
to android-...@googlegroups.com
Andy,

I apologize if I created the impression that bulk-merging all of K-9 was expected.  Put succinctly, there are things in K-9 that would have wide applicability, and fix issues with the default email client, but they cannot be merged in "small patches".  E.g.: Microsoft Exchange support could not be merged piecemeal.


> Patches that make direct changes to the UI are much less likely to
> be accepted.  But patches that make it easier to change the UI in a
> downstream fork, are fine.

So, let's take, for example, the "self-signed certificates" functionality:  this requires a change to the user interface, since the user has to accept the certificate.  According to the above, this makes the change much less likely to be accepted, correct?  I realize that there are some UI changes in K-9 that are "questionable", and would not be accepted, but it seems odd that all UI changes are inversely correlated to patch acceptance.  Can you clarify?

The K-9 team wants to contribute to the mainline, but we're asking for some flexibility on your part.  Your email implies that Google's position is that forks are preferable to outside contributors (apparently none of us have "constraints", "internationalization", or "testing" experience).

I do find it disheartening that Google takes this position: the current email client is in desperate need of improvement.

Bradley

On Thu, Feb 12, 2009 at 8:26 AM, Andrew Stadler (Google) <sta...@android.com> wrote:

There has been some discussion about K9 merging back into the platform
Email sources.  I'd like to address some of the things that have come
up and try to shed some light on the situation.

<snip>

Jesse Vincent

unread,
Feb 18, 2009, 11:17:54 PM2/18/09
to android-platform, sta...@android.com
Hi folks. I'm Jesse, the guy originally responsible for K9 being
forked from the core mail client. I originally created K9 to "scratch
an itch" - I wanted IMAP deletes to work right on my brand new G1.
Since then, K9 has taken on a a bit of a life of its own.

> The basic question at hand is:  When will K9 be bulk-merged back into
> the mainline sources?
>
> The short answer is:  It will not.

For my part, I never intended to suggest that I expected a bulk merge
of K9 into the core Mail client. Early on, I thought it might be
possible that K9 would end up as a playground for experimental
features that might eventually end up in the core client. I had high
hopes and grand plans for making sure we submitted all our fixes and
improvements upstream, but never expected that those patches would
all be accepted.

I was never under the illusion that what makes sense for K9 makes
sense ending up in the core tree and on every Android handset.

> But somewhere along the line there has been an impression that somehow
> K9 would become the "new" platform email application, via a
> large-scale merge.  I'm not sure how this happened.  If it was said by
> Googlers, please consider this email both an apology and a correction.

I don't believe you have ANYTHING to apologize for. Nobody ever led
me to believe that K9 was going to be 'adopted' and we've been making
changes that would be very hard to justify bringing into core.


> First, the email client in cupcake is in better shape than 1.1, and
> that's in better shape than 1.0.  I encourage the K9 team, as well as
> any other Email forks, to take as much from cupcake as possible.
> You'll find a more reliable POP3 implementation, improved
> interoperability (welcome back, comcast.net users!), and overall
> cleanup of some of the rough edges in the client.  It has a long way
> to go, but it's a lot better than it was.  You'll find a couple of new
> features, too.  And we're not done yet.

I'm absolutely thrilled to see the bug fixes, features and new test
framework in Cupcake's mail client. Things are headed in the right
direction.

>
> Second, I encourage & welcome patches from anybody interested in
> making Email better (including the K9 team).  Pushing patches upstream
> is good for everybody:  It improves the platform app, and it makes
> life easier for everyone who is forking from it.

Andy, I owe you some mail about the first patch I'd submitted a while
back. The sad truth is that K9 became "good enough" that I use it
every day and rarely run into things that frustrate me and my
attention has drifted back to other projects that are more directly my
'responsibility'. Other K9 hackers have been improving things in my
absence, but my attention drifted before I extracted the "most
important" bug fixes and performance improvements for submission
upstream. I'm really sorry about that :/

I've tried to encourage K9 contributors to sign android contributor
agreements to make sure that we can push patches upstream.

At some point, I'd love to have a discussion with folks about how to
make it easier for downstream projects to branch from core
applications in a way that will make it easier to share code. Bit this
isn't the right time and place.

I'm hopeful for K9's future and for the core Email client's future and
I'm thrilled that this discussion is happening.

Android's openness let me go from an initial git checkout to a google
code project and an improved (at least by one definition) of Mail on
the market in less than an hour.

Andy, again, I'm sorry if our early enthusiasm caused mistaken
impressions on the part of other users or contributors.

Best,
Jesse Vincent
K9 dog walker

Andrew Stadler (Google)

unread,
Feb 23, 2009, 2:31:52 AM2/23/09
to Jesse Vincent, dc.dis...@gmail.com, young....@gmail.com, android-platform
Hello all-

Sorry I haven't rejoined this thread more quickly - I was out of town
for most of last week, and just now catching up.

There's a lot to discuss and I'm going to take it in small pieces
(like a good patch). I'm quite sure I won't answer every question
that's been posed, or in enough depth to satisfy all. But it's good
to keep the conversation going.

Upfront: I appreciate your thoughts on the overall AOSP, but I am
only here to discuss the Email client.

Regarding statements about merging clients. I'm really not aware of
any plans between DAMail and K9, so I really don't have much to
contribute there. Regarding any statements of a complete merge of any
forked client back into the main truck, as I tried to say clearly:


"If it was said by Googlers, please consider this email both an

apology and a correction." I'm not sure if it's worth a lot of
time/effort digging through old emails, but feel free if you wish to.

Regarding pulling patches from K9: You're right, I could. I'd like
to do more of that. I'm hoping that we can do more open discussion of
bugs on this board, and move patches in either direction when we can.
But it starts with bugs filed and discussions (like a good one going
on right now about IMAP folder subscriptions).

Regarding "secret docs": Well, I didn't say that, but I'll be happy
to elaborate on the things I did say.

Regarding UI changes in patches: Let's use your example of better
handling of self-signed certificates. I actually see this one as two
or three patches. The first one or two would be the underlying
plumbing. I say two because there's work to be done in the transport
areas that will need to be implemented & tested, and there's also the
need to reliably and securely store accepted certs in the per-account
preferences. Assuming good code quality, internal documentation,
review, and sufficient unit testing, I'm sure these patches can make
it (and I am looking forward to seeing them). Obviously the final
patch would be the UI to present certificate errors, describe them to
the user, and provide options for handling them. Would it be
accepted? I hope so! But I think the important issue here is not
plumbing vs. UI, but rather that these patches would be reviewed
independently, on their own merits, and would each contribute to the
platform.

Regarding forks vs. outside contributors: I don't think I said that
forks are preferable to outside contributions. Quite simply I don't
see it as an either-or situation. I believe that forks are good
because they create choice for end users. I believe that outside
contributors are good because they improve the platform. I think it's
great that members of the K9 team are in the position to do both.

Regarding being closed to contributions: My inbox has a lot more
messages about the idea of accepting patches, than actual patches.
Hint hint :)

Andy Stadler
Google

p.s. Jesse, you have no reason to apologize for enthusiasm!

Disconnect

unread,
Feb 23, 2009, 10:04:55 AM2/23/09
to android-platform
On Mon, Feb 23, 2009 at 2:31 AM, Andrew Stadler (Google) <sta...@android.com> wrote:
Hello all-

Sorry I haven't rejoined this thread more quickly - I was out of town
for most of last week, and just now catching up.

How dare you have a life :) 
 
There's a lot to discuss and I'm going to take it in small pieces
(like a good patch).  I'm quite sure I won't answer every question
that's been posed, or in enough depth to satisfy all.  But it's good
to keep the conversation going.
 
Upfront:  I appreciate your thoughts on the overall AOSP, but I am
only here to discuss the Email client.

Which is part of the AOSP. My only point is that your actions here set a precedent for the other maintainers and the outside contributors. But doing the right thing for one is likely the right thing for both, so..

 
Regarding pulling patches from K9:  You're right, I could.  I'd like
to do more of that.

You did a lot of work in secret, and while you were in a position to pull patches from K9 and company (esp at the earliest days of the fork, where - presumably - the code bases were similar) nobody out here was in a position to do the same from you. (Moving forward, that has mostly changed and is still getting better. So..)
 
Regarding "secret docs":  Well, I didn't say that, but I'll be happy
to elaborate on the things I did say.

From earlier:

I must work within user interface guidelines and remain consistent with the rest of the platform, as well as user testing, and the requirements of carrier partners.

Are those guidelines and requirements published somewhere? If not, they are "secret docs" that outside contributors must follow without being able to read. (And #3 in your email is again a list of documents and schedules that are not provided to outside contributors.)  Regardless of the scope you are responsible for, this is a very important point. You seem to be saying, in so many words, that the closed-source branches of email are more important and dictate the requirements, schedules and plans of the open source "community" project. This is, at least from where I am sitting, a very bad thing to be doing. (I'm sure the carriers would disagree, however.)

Regarding forks vs. outside contributors:  I don't think I said that
forks are preferable to outside contributions.  Quite simply I don't
see it as an either-or situation.  I believe that forks are good
because they create choice for end users.  I believe that outside
contributors are good because they improve the platform.  I think it's
great that members of the K9 team are in the position to do both.

Contributions are great so long as they are lucky enough to adhere to your unpublished schedules, requirements and guidelines?

Regarding being closed to contributions:  My inbox has a lot more
messages about the idea of accepting patches, than actual patches.
Hint hint :)

With the missing google apps (and sync), most people are not in a position to do serious development/testing on master. I am not going to go on about that, its (probably) not your fault or problem :) but it is the most common problem I hear with my master builds, and the biggest barrier to anyone outside google who wants to help with development.

Matt

unread,
Feb 23, 2009, 3:53:57 PM2/23/09
to android-platform
Andy,
I initially wrote the WebDav support so I could have Exchange on my
phone (IMAP wasn't a realistic option with literally thousands of
public folders that showed up). When I was still trying to figure out
the format to submit patches, I ran across K9 and kind of took off
running with the opportunity to have other users running/testing it as
well. One of the things hinted at in your response here has been a
large reason why I haven't had any patches to submit yet for the Email
app.
On Feb 23, 1:31 am, "Andrew Stadler (Google)" <stad...@android.com>
wrote:
> Regarding UI changes in patches: Let's use your example of better
> handling of self-signed certificates. I actually see this one as two
> or three patches. The first one or two would be the underlying
> plumbing. I say two because there's work to be done in the transport
> areas that will need to be implemented & tested, and there's also the
> need to reliably and securely store accepted certs in the per-account
> preferences.

For the first month or two I was actively looking to try and find the
preferred format and process for submitting patches. I would love to
see my WebDav support included in the default client, but it's not a
small patch by any means (WebDavStore.java is 2020 lines and is sparse
on comments). In addition, there's the UI changes for account setup
and so on. The main reason I haven't submitted anything yet is that I
don't want to spend more time massaging the patches than I did when
writing the code when realistically it should be possible to know all
the requirements ahead of time. Should it be submitted as the
framework for transport implementing only enough to not break, then a
patch for each level of functionality for it? Or should it be done as
a large patch so that architectural decisions can readily be seen and
adjusted/refactored as necessary? Or should it be something in the
middle?

I think the fact that we have a dialog going on here is a good start,
but with the volume of changes that (I personally think) the Email app
needs, there needs to be a good way to discuss new features and bugs.
I've tried looking at the bugtracker at http://code.google.com/p/android/issues/,
but finding email application related issues requires searching for
"email" or "e-mail" or "mail" or other variants and the end result is
a lot of noise (I'll admit, this may be my failure at using the
tool). Back on point, the reason I think there needs to be a way to
discuss these things (dedicated bug tracker, tags for specific
applications in the platform bugtracker, or even application specific
email alias/mailing list) is that I still don't even know if Google
would even consider a patch for WebDav support in the email client.
So far, the only answer I have is a friend talked to a friend who
works at Google who talked to a friend who works on the Android team
who talked to a friend who works on the Email team, and then the
relayed response all the way back was "Google won't develop it
themselves due to licensing." Now, there's potential that there was
miscommunication in there somewhere (ie, at some point thought the
question had to do with ActiveSync), but it still doesn't indicate
whether it will get a blanket denial just because of what the patch
is.

Once we have the process for submitting the patches (not just use
repo, but how you would prefer them to be broken down and set up to
make them easier to integrate and test) and have a good method for
feedback on what should reasonably be submitted (based on a bug
tracker or discussions), it would be easier on both sides to
drastically improve the email client (and this goes for any of the
Google provided applications...the Calendar is another one that some
day I would like to gut and do in a more modular fashion).

On an only slightly related note...K-9 is in a unique position. It's
developers that are associated only by the application. There's lots
of changes that have become semi-blurred as to the original author (or
multiple author's created the final piece) that could be submitted as
a patch. At this point in time, what would Google (or you personally)
accept to be the best method to resolve the potential issue of
multiple contributors on a single submission, or a different
contributor than author?

Thanks,
Matt Brace

ajg23

unread,
Jan 24, 2011, 8:00:03 PM1/24/11
to android-...@googlegroups.com
[I know this is an old thread, but the Ars article here led me to it]

The weakness of the built-in email app is a great threat to the long-term survival of Android (especially with Verizon getting the iPhone, to say nothing of the next, LTE-enabled, iPhone!).

As someone who many people in a university/hospital system look to for mobile device recommendations, there are multiple, EASY-TO-FIX obstacles to me being able to recommend Android devices to non-geeks. 

Perhaps the biggest one is the default Android email program. Not only does it (in most incarnations) not allow the user to enter an IMAP root folder path, so no one in my entire hospital/university system (thousands of students, residents, nurses, docs, etc, etc!) can access email via it. It's also (at least on our IMAP system) slow to the point that I thought something was wrong with the network. I installed K-9 on another faculty member's phone recently, and it was immediately usable (vs intolerably slow). Further, I could map that sent items go to the sent-mail IMAP folder, deleted items go to the appropriate IMAP folder, etc. 

Yes, I could tell everyone who I might recommend an Android device  to get the K-9 app and help them set it up, but that's another hurdle that will make people think "I'll just get an iPhone"! They should be able just to use the default email program and enter the mail server info and get their mail.

I am also profoundly annoyed by gingerbread's new requirement that selecting a word requires tap-and-hold, then a menu selection--a step down from froyo, and a deviation from all GUI interfaces since the '80s. In addition to this being slow, it makes me fear more for Android's demise. Using an iPhone recently I was reminded how nice it was to double-tap on a word and just type something else (or extend the selection). 

These kinds of things are why I'm moving more towards telling colleagues to get iPhones these days  ... It kills me how successful Verizon's iPhone is going to be, even though the network it will use is slower than AT&T's...!

Reply all
Reply to author
Forward
0 new messages