Plan: K-9 4.6 will be the last release that supports Gingerbread

[Jesse Vincent]

We've talked about this for a long time, but I think it's finally time to make the jump. 

We spend a LOT of our time working to make K-9 work well on older devices. We do a pretty good job. But it's soul-deadening work. And there aren't a lot of us contributing to K-9.

If any developers would like to step up to do the work to make sure that K-9 runs well on Froyo and Gingerbread, I'd be willing to consider it. But you have to commit to actually _doing_ the work. 

Failing that, I propose that as of October 1, we bump our minSdkVersion to 15 and be willing to take patches (or commit changes) that remove backward compatibility shims -- Everything from the workarounds to support old versions of Gallery or the Contacts API to ActionBarSherlock.

I DO NOT suggest that we spend a bunch of time just going on a deletion frenzy, but if you get excited to clean the code, I look forward to the pull requests.

We will continue to make older versions of K-9 available on the Play Store and on our code hosting site.

Thoughts?

Jesse

[Andrew Chen]

Without seeing the data on what percentage of our users are still on Froyo and GB, it's hard to make an informed decision, but my gut feeling is that they're so old that it's not worth spending the effort to continue maintaining it.

[cketti]

On 17.09.2013 19:15, Andrew Chen wrote:

Without seeing the data on what percentage of our users are still on Froyo and GB, it's hard to make an informed decision, but my gut feeling is that they're so old that it's not worth spending the effort to continue maintaining it.

Android 2.3.3 - 2.3.7: 25.26%
Android 2.2: 4.93%
Android 2.1: 1.26%
Android 3.2: 0.99%
Android 3.1: 0.31%
Others: 0.46% (includes Android 4.0-4.0.2)

[cketti]

+1

I propose a large increase of the version code to leave lots of room for
bugfix releases of K-9 Mail 4.6.
I also volunteer to rip out the backward compatibility code.

[Hans-Christoph Steiner]

Would it be possible to make a 4.6 release that also includes the OpenPGP
Keychain API? I can probably contribute some to that effort. That would
allow those 30% of users on old devices to have flexibility for which crypto
provider they can use for PGP email.

.hc

--
PGP fingerprint: 5E61 C878 0F86 295C E17D 8677 9F0F E587 374B BE81

[Jesse Vincent]

I'd be willing to have us backport that and either release it as a 4.6.x or a 4.8 (if we were to, say, jump to 5.0 for the next major release.)

At this point, how far off is a second implementation that speaks that API? I don't want us to rush out a less-than-well tested version of the API and get stuck supporting it in K-9 and the various pgp implementations.

--
--
You received this message because you are subscribed to the K9 Mail Dev List.
To post to this group, send email to k-9...@googlegroups.com
To post an issue, go to http://code.google.com/p/k9mail/issues/list
To unsubscribe, email k-9-dev+u...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/k-9-dev

---
You received this message because you are subscribed to the Google Groups "K9 Mail Developers List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to k-9-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dominik Schürmann]

On 25.09.2013 22:44, Jesse Vincent wrote:
> I'd be willing to have us backport that and either release it as a 4.6.x or
> a 4.8 (if we were to, say, jump to 5.0 for the next major release.)
>
> At this point, how far off is a second implementation that speaks that API?
> I don't want us to rush out a less-than-well tested version of the API and
> get stuck supporting it in K-9 and the various pgp implementations.

I am not sure if I understand your point. Why do you need to support
various pgp implementations? They will hopefully all implement the same
API as a provider.
Besides this, to improve/test the API I need comments :) on
https://github.com/k9mail/k-9/pull/385

Regards
Dominik

[Jesse Vincent]

On Thu, Sep 26, 2013 at 9:54 AM, Dominik Schürmann <dom...@dominikschuermann.de> wrote:

> At this point, how far off is a second implementation that speaks that API?
> I don't want us to rush out a less-than-well tested version of the API and
> get stuck supporting it in K-9 and the various pgp implementations.

I am not sure if I understand your point. Why do you need to support
various pgp implementations? They will hopefully all implement the same
API as a provider.

The key word there is 'hopefully' -- Generally, a lot of assumptions get flushed out the second time someone implements an API.

 

Besides this, to improve/test the API I need comments :) on
https://github.com/k9mail/k-9/pull/385

Yeah. I know that I'm just not good enough at this part of the stack to give useful feedback. Hans - have you taken a look at Dominik's work?

 

Regards
Dominik

[Bernhard Redl]

For me this looks like alot of percent are missing? Shouldn't they add
up to 100% in total? With this data it looks like most of our users use
2.3 (Gingerbread)

[cketti]

Those are just the Android versions that would be left behind. The rest
(66.79%) was SDK 15+

I just checked again and now the percentage of devices supporting SDK
15+ is 67.26

[Hans-Christoph Steiner]

On 09/26/2013 12:21 PM, Jesse Vincent wrote:
> On Thu, Sep 26, 2013 at 9:54 AM, Dominik Sch�rmann <

Yes, I've reviewed the API and the demo app over the past few revisions. This
most recent version is looking good from my point of view, as long as it
provides everything that K-9 needs.
https://github.com/dschuermann/openpgp-keychain/blob/master/OpenPGP-Keychain-API-Demo/src/org/openintents/openpgp/IOpenPgpService.aidl

The open questions are:

* how should the calling app get info from the OpenPGP provider about whether
a contact has a key, that key ID, etc? As far as I can tell, such a thing
will be required to make K-9 able to show which contacts have PGP keys available.


* do calling apps need to be able to encrypt using key IDs (e.g.
f123eb458239289d) or is using email address style identifiers sufficient? For
an email program, I can see only using the email address and having the
OpenPGP provider handle the rest, like prompting the user if there are
multiple keys that match a given email address. But these functions should
also work for other standard uses of OpenPGP, like signing software. So for
example, a OpenPGP key for a Launchpad PPA apt repo has a signing key
associated with that. Those keys often do not have email-style user IDs
associated with the key. But an Android should be able to still work with
keys like that using this API. For an example, check out:

gpg --recv-keys 97BB3B58
gpg2 --fingerprint --with-colons 97BB3B58
apt-key list (on Debian-deriv systems)


* what exact behaviors the API functions trigger? One thing that is clear to
me is that the OpenPGP provider should handle all stuff related to the
password and prompting for it. The calling apps should be happily ignorant of
the state of the password caching and UI prompting. I think that the OpenPGP
provider should also handle choosing the right key when there are multiple
matches for a given user ID.


* maybe the allowUserInteraction option should be a global settable flag
rather than in the calling functions. I think most apps will either choose
one state or the other and stick with it rather than occasionally use it or
not. This is how gpgme works, it has a whole bunch of global options.

* @param allowUserInteraction
* Allows the OpenPGP Provider to handle missing keys by
showing activities


* one thing I haven't figured out how to map cleanly to an API is all the
variations of binary/ASCII input and output. Maybe there could be a set of
simple methods that always output the same format as is input. Then keep the
current functions for people who need to go from binary to ASCII or vice
versa. So the methods would be like:

encrypt(in String inputAscii, in String[] encryptionUserIds, in
IOpenPgpCallback callback);
encrypt(in byte[] inputBytes, in String[] encryptionUserIds, in
IOpenPgpCallback callback);
encrypt(in byte[] inputBytes, in String[] encryptionUserIds,
in boolean asciiArmor, in IOpenPgpCallback callback);

But then the output is returned via IOpenPgpCallback, so adding two sets of
callback types might be annoying. If we're marking the input with an
asciiArmor boolean, then the output should also be marked. So perhaps its
easiest to have separate bytes[] and String IOpenPgpCallback methods. Or
maybe its enough to just have this defined in IOpenPgpCallback:

oneway void onSuccess(in String outputAscii, in OpenPgpSignatureResult
signatureResult);
oneway void onSuccess(in byte[] outputBytes, in OpenPgpSignatureResult
signatureResult);
oneway void onError(in OpenPgpError error);


* Also, Dominik, one detail about this comment: "Encode for ASCII (Radix-64,
33 percent overhead compared to binary)". I've never seen anyone refer to
ASCII-armor as "Radix-64". It is called "base64". If you're on UNIX, try
`man gpg` or `man base64` for docs on that.


.hc


>
>
>
>> Regards
>> Dominik

[Dominik Schürmann]

Hi Hans,

thanks for your feedback.

Before I answer the specific points, let me make some arguments why I
designed the API like it is:
1. The API should be as easy as possible to use, even for newbies in
Android development. This means that as much functionality as possible
should be implemented on the side of the OpenPGP Provider. This lead to
my decision to never expose key ids via the API.
- So on first use the private key used for an app is choosen by the
user -> The app never gets the key id of the private key
- To encrypt something the app only needs to know the user id of the
recipient. In most cases this is the email (could also be jabber ids in
future applications).
- This also introduces more security and control for users , e.g. the
app can't choose on itself which key to use for signing

2. To make the API easy, I am also trying to keep the interface as small
as possible, but I already had some problems due to this self-imposed
limitations :(

3. Last main point: As much as I like to have a perfect API that solves
every possible corner case, I would really like to get forward and get
this accepted as the API version 1. As I said in previous mails, I think
the need for a new OpenPGP app that works with k9mail is urgent (APG has
many big bugs: selection of multiple encryption keys is broken, PRNG
fixes are not applied). I for short release times: release early and
ofter :D

On 27.09.2013 20:18, Hans-Christoph Steiner wrote:
> Yes, I've reviewed the API and the demo app over the past few revisions. This
> most recent version is looking good from my point of view, as long as it
> provides everything that K-9 needs.
> https://github.com/dschuermann/openpgp-keychain/blob/master/OpenPGP-Keychain-API-Demo/src/org/openintents/openpgp/IOpenPgpService.aidl
>
> The open questions are:
>
> * how should the calling app get info from the OpenPGP provider about whether
> a contact has a key, that key ID, etc? As far as I can tell, such a thing
> will be required to make K-9 able to show which contacts have PGP keys available.

Yes this is indeed a problem I faced already while implementing the
k9mail integration. The option to automatically encrypt is currently not
available with the new API (and is properly greyed out in the account
preferences)

To prevent exposing key ids we could introduce a simple function:

boolean isKeyAvailable(in userId)

which is not asynchronous as the other functions, thus no callback.

> * do calling apps need to be able to encrypt using key IDs (e.g.
> f123eb458239289d) or is using email address style identifiers sufficient? For
> an email program, I can see only using the email address and having the
> OpenPGP provider handle the rest, like prompting the user if there are
> multiple keys that match a given email address. But these functions should
> also work for other standard uses of OpenPGP, like signing software. So for
> example, a OpenPGP key for a Launchpad PPA apt repo has a signing key
> associated with that. Those keys often do not have email-style user IDs
> associated with the key. But an Android should be able to still work with
> keys like that using this API. For an example, check out:
>
> gpg --recv-keys 97BB3B58
> gpg2 --fingerprint --with-colons 97BB3B58
> apt-key list (on Debian-deriv systems)

This is indeed a limitation of the API when not exposing key ids. If
there is really need for something like this, I would propose
implementing a new interface exactly for this special case.
Additionally I don't like this approach of using untrusted keys from a
keyserver...

> * what exact behaviors the API functions trigger? One thing that is clear to
> me is that the OpenPGP provider should handle all stuff related to the
> password and prompting for it. The calling apps should be happily ignorant of
> the state of the password caching and UI prompting. I think that the OpenPGP
> provider should also handle choosing the right key when there are multiple
> matches for a given user ID.

I think the behavior is the freedom of the OpenPGP Provider ;)
Keychain handles the selection of the private key on first use, password
promt, selection of encryption keys on duplicate keys for one user id,
querying missing keys (not implemented right now).

> * maybe the allowUserInteraction option should be a global settable flag
> rather than in the calling functions. I think most apps will either choose
> one state or the other and stick with it rather than occasionally use it or
> not. This is how gpgme works, it has a whole bunch of global options.
>
> * @param allowUserInteraction
> * Allows the OpenPGP Provider to handle missing keys by
> showing activities

I also thought to make it a global option, but my current use case is as
follows:
On opening a signed mail, it gets automatically verified and then the
verification status is displayed. When the key is missing it could be
annoying to always get a dialog asking to download the key. So I thought
to make the first function call with allowUserInteraction=false. And
when signature check fails a button gets visible which does the same
call but with allowUserInteraction=true to allow the download of missing
keys.

Note: In the current k9mail pull requests this is not implemented, but I
would like to do it in a new pull request when I got time.

>
>
> * one thing I haven't figured out how to map cleanly to an API is all the
> variations of binary/ASCII input and output. Maybe there could be a set of
> simple methods that always output the same format as is input. Then keep the
> current functions for people who need to go from binary to ASCII or vice
> versa. So the methods would be like:
>
> encrypt(in String inputAscii, in String[] encryptionUserIds, in
> IOpenPgpCallback callback);
> encrypt(in byte[] inputBytes, in String[] encryptionUserIds, in
> IOpenPgpCallback callback);
> encrypt(in byte[] inputBytes, in String[] encryptionUserIds,
> in boolean asciiArmor, in IOpenPgpCallback callback);
>
> But then the output is returned via IOpenPgpCallback, so adding two sets of
> callback types might be annoying. If we're marking the input with an
> asciiArmor boolean, then the output should also be marked. So perhaps its
> easiest to have separate bytes[] and String IOpenPgpCallback methods. Or
> maybe its enough to just have this defined in IOpenPgpCallback:
>
> oneway void onSuccess(in String outputAscii, in OpenPgpSignatureResult
> signatureResult);
> oneway void onSuccess(in byte[] outputBytes, in OpenPgpSignatureResult
> signatureResult);
> oneway void onError(in OpenPgpError error);

If the input is an ascii armored string or just a byte array should be a
check on OpenPGP Keychain's side, so no need to specify the format of
the input.
asciiArmror option allows to request for a specific output format. So I
am not sure if it's always ASCII to ASCII and binary to binary (consider
attachments).
But that said, I also don't like the option, because it's another
parameter in the long function declaration ;)

> * Also, Dominik, one detail about this comment: "Encode for ASCII (Radix-64,
> 33 percent overhead compared to binary)". I've never seen anyone refer to
> ASCII-armor as "Radix-64". It is called "base64". If you're on UNIX, try
> `man gpg` or `man base64` for docs on that.

Radix-64 is base64 with a checksum. See
http://tools.ietf.org/html/rfc4880#page-53. Actually
This should also be the ecnoding used in gpg and bouncy castle when
using armoring.


The next 3 days I am on vacation, and the following week I need to do
some other work, so don't expect fast answers ;)

Regards
Dominik

[Bernhard Redl]

hmm still sounds like many people would be cut off from updates. Maybe
we should announce this on the general maillinglist and ask for users to
participate to test/develop for pre 4.0.4? I fear normal users only
monitor the normal maillinglist (if any) and would react kindof upset if
we just "announce" the change.

For me it seems quiet harsh to leave 1/3 of the users behind with no
updates. Maybe we should wait until we have 80:20? But i have to admit
that i have no idea how fast these numbers are gonna change in the
future toward 80:20.

At least it would be good if we can get the new GPG api working also
with pre 4.0.4 devices.

just my 2cents

[Dominik]

Hey,
I thought about the API and its limitations again, and I like to propose the following:
- leaving the API mostly as it is now
- only removing the boolean allowUserInteraction

Then, defining an extended API that allows for more fine grained control and automatization. This could also allow for automatic import from kezservers (receiv-keys) by key id and several other methods without user interaction.

I think most apps that already require user interaction, like all mail apps, will then use the simple API.
All complicated use cases like in your example can be done with an extended api. I am not sure right now what I like to expose in this API, but this could be discussed at a later time.

What do you think?

Regards
Dominik

"Dominik Schürmann" <dom...@dominikschuermann.de> schrieb:

--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[Hans of Guardian]

So I think we all agree that the API should be as simple as possible. What makes that hard is that OpenPGP is complicated. So we could really screw some devs if we over-simplify the API. This API doesn't need to support all corner cases, but it should do the fundamentals well: email, file encryption, file verification.

Some more comments below (plus we had a long conversion on im which I plan on posting somewhere...)

Those commands are there so people can download a widely used, reputable key that does not have an email address style ID in it. I'm not suggesting it as a behavior.

>> * what exact behaviors the API functions trigger? One thing that is clear to
>> me is that the OpenPGP provider should handle all stuff related to the
>> password and prompting for it. The calling apps should be happily ignorant of
>> the state of the password caching and UI prompting. I think that the OpenPGP
>> provider should also handle choosing the right key when there are multiple
>> matches for a given user ID.
>
> I think the behavior is the freedom of the OpenPGP Provider ;)
> Keychain handles the selection of the private key on first use, password
> promt, selection of encryption keys on duplicate keys for one user id,
> querying missing keys (not implemented right now).

This is a recipe for bad user experience. If the app developer doesn't know what's going to happen when they call this API, that means its hard to make a UI that handles all possible cases. I don't think we need to nail down every single detail but the big things do need to be nailed down, like:

* password handling is entirely in the service, and the client is
totally ignorant of it
* there should be a method of selecting keys from the local keyring
programmitically that does not trigger any UI
* there should be a method of selecting keys from the local keyring
that prompts the user only if there is more than one match. If there
is a single match, it transparently uses that key.

Oops, there would need to be one more function to cover all cases:

# ascii -> ascii

encrypt(in String inputAscii, in String[] encryptionUserIds, in
IOpenPgpCallback callback);

# ascii -> binary

encrypt(in String inputAscii, in String[] encryptionUserIds,

in boolean asciiArmor, in IOpenPgpCallback callback);

# binary -> binary

encrypt(in byte[] inputBytes, in String[] encryptionUserIds, in
IOpenPgpCallback callback);

# binary -> ascii

encrypt(in byte[] inputBytes, in String[] encryptionUserIds,
in boolean asciiArmor, in IOpenPgpCallback callback);

>> * Also, Dominik, one detail about this comment: "Encode for ASCII (Radix-64,
>> 33 percent overhead compared to binary)". I've never seen anyone refer to
>> ASCII-armor as "Radix-64". It is called "base64". If you're on UNIX, try
>> `man gpg` or `man base64` for docs on that.
>
> Radix-64 is base64 with a checksum. See
> http://tools.ietf.org/html/rfc4880#page-53. Actually
> This should also be the ecnoding used in gpg and bouncy castle when
> using armoring.

Doh.. of course the IETF procedings would come up with more confusing, overloaded jargon ;) I'd describe it then as "OpenPGP Radix-64" and include that link.

.hc