Replaceable lock screen

[qvark]

Hi,

Currently the phone "lock screen" is built into the platform and
cannot be replaced by third party applications, breaking a bit the
"all applications are equal" principle. I know it wasn't a preference
for 1.0 and I understand it is not a big deal for most of the
applications, but I wanted to know if you (I mean Android team) have
any plans to allow that feature.

If you don't have plans for it in a short term, but you find it an
interesting feature, we would like to offer our modest help to develop
it and contribute to the platform. Our idea is that an application
with the right permission (REPLACE_LOCK_SCREEN) could be invoked (via
intent) when the phone is locked or the user wants to unlock it. So,
instead of the pattern based lock screen, the user could choose one
based on a password, or a biometric method, or something so cool like
this: http://www.youtube.com/watch?v=QhwHIOW5dYE

I'm now studying the current implementation (which, if I'm not wrong
is mainly in this package:
http://android.git.kernel.org/?p=platform/frameworks/policies/base.git;a=tree;f=phone/com/android/internal/policy/impl;hb=master),
but I wanted to know your opinion about the idea in general and
possible challenges, security risks, design tips, etc.

Thanks in advance,

Jose Luis.

[Dianne Hackborn]

This would be nice to have, but there are no near-term plans to add the functionality.  Note that doing this is going to be way more complicated than just adding a permission and running some other app.  There are at least two things to consider:

- The lock screen has deep interaction with the low-level power management of the device and wakeup when pressing keys, complicated interaction with the in-call screen, specific APIs, etc.

- The current implementation runs the lock screen code in the core system process, so it can be assured that it will be there.  A third party lock screen would need to run in some other process, which the system couldn't count on being there all the time.  For example, it is third party code, so if it crashes this can't crash the entire system, yet the phone still must remain locked.  And someone providing their own lock screen would either need to have their process launched as needed and the system dealing with all of that asynchronicity or just remain running all of the time...  the former is fairly complicated to write, the latter would have the unfortunately consequence of using the resources of a new process all of the time.

We would certainly be interested in contributed patches for adding to the base platform...  but if you embark on this, you need to be aware that coming up with something that is acceptable and covers the myriad of issues involved is likely to be a long, grueling process. :)  This is why it wasn't something we could do for 1.0 and have prioritized below many other things.

Fwiw, one of the first things that probably needs to be done to implement this feature is to rework how applications interact with the lock screen to determine when it is okay to remove.  This is actually stuff we may get in to cupcake, though that's not certain.  If not cupcake, we'll probably have that work done in the following version.

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support.  All such questions should be posted on public forums, where I and others can see and answer them.

[Dianne Hackborn]

One other thing -- adding a permission to allow apps to be used as a lock screen is probably in my opinion the wrong approach.  You might want to look at how the new input method stuff works, where an app just publishes a service that says it can serve as an IME, but the system just ensure that the user needs to be involved in picking it rather than relying on a permission.  (There is, however, a permission used to ensure that only the system can talk with an IME.)

[Jose Luis Huertas Fernández]

Hi Dianne! That's a great feedback, thanks!

>>>
We would certainly be interested in contributed patches for adding to the base platform...  but if you embark on this, you need to be aware that coming up with something that is acceptable and covers the myriad of issues involved is likely to be a long, grueling process. :)  This is why it wasn't something we could do for 1.0 and have prioritized below many other things.
>>>

I was pretty sure it wasn't trivial or otherwise it would have been done before :) The important thing is we should come up with a solution that you are comfortable with, otherwise is very unlikely you are going to accept it and it would be a waste of time.

>>>
- The lock screen has deep interaction with the low-level power management of the device and wakeup when pressing keys, complicated interaction with the in-call screen, specific APIs, etc.
>>>

Now I'm aware of it, but I think all those details can be kept in the system code. The third party code only would have to deal with showing a lock screen and an unlock one (and return a pass/fail result) when the system ask for it. Only the system will know under what conditions these screens should be invoked (power off/on, keys pressed, keyboard opened/closed, etc.).

>>>
A third party lock screen would need to run in some other process, which the system couldn't count on being there all the time.
>>>

I agree the system cannot rely on a third party lock screen always been present. The current implementation of the lock screen should be kept and fallback to it if no other lock screen implementation is found (or if the user still prefers the default one).

>>>
For example, it is third party code, so if it crashes this can't crash the entire system, yet the phone still must remain locked.
>>>

I agree again... we could always start the system lock screen behind the "user defined" and if this crashes the system one will always be there to keep the device locked. It is a bit ugly, but it is just an idea...

>>>
the latter would have the unfortunately consequence of using the resources of a new process all of the time.
>>>

In your experience... a process started but doing nothing is too much overhead? how is the IME stuff going to work?

>>
one of the first things that probably needs to be done to implement this feature is to rework how applications interact with the lock screen to determine when it is okay to remove.
>>

Do you mean the interaction provided by this (http://code.google.com/intl/es-ES/android/reference/android/app/KeyguardManager.html)? how is it going to be reworked? I cannot see any changes in the cupcake branch yet...  should this API be aware somehow of the actual implementation of the lock screen? (I don't think so)

>>>
adding a permission to allow apps to be used as a lock screen is probably in my opinion the wrong approach.  You might want to look at how the new input method stuff works, where an app just publishes a service that says it can serve as an IME, but the system just ensure that the user needs to be involved in picking it rather than relying on a permission.  (There is, however, a permission used to ensure that only the system can talk with an IME.)
>>>

Mmmm... interesting approach, but... would it mean the user has to be involved in picking it every time the device has to be unlocked? or the system is going to be able to remember the decision? Also, I find a bit dangerous that every application could register as a lock screen provider without any special permission (even if the user is going to make the final decision about the screen that is going to be used), don't you think?

Also, I have finished my first review to the currently implemented code and I have found it a very modular and extensible solution. Most of the code is independent of the current pattern-based implementation and I have found only little fragments that are tied to it (like in KeyguardViewMediator where an instance of LockPatternKeyguardViewProperties is created).

I have drawn a very basic diagram (only the main relationships are shown) with the classes I have understood so far (http://img187.imageshack.us/img187/7071/keyguardclassesbg7.png). Only the green ones are specific of the pattern-based implementation while the others are mostly generic code that could be kept with little changes. I think that "all" we need is a new kind of KeyguardViewBase that delegates the lock and unlock screens to third party code.

Well, until now there are many more questions than answers, but I'm pretty sure we can get it with some support on your side ;)

Thanks for your help,

Jose Luis.

[Dianne Hackborn]

On Tue, Dec 30, 2008 at 10:18 AM, Jose Luis Huertas Fernández <joseluishuer...@gmail.com> wrote:

Now I'm aware of it, but I think all those details can be kept in the system code. The third party code only would have to deal with showing a lock screen and an unlock one (and return a pass/fail result) when the system ask for it. Only the system will know under what conditions these screens should be invoked (power off/on, keys pressed, keyboard opened/closed, etc.).

Yes, something like this would be reasonable, though if you look at the code it is actually pretty complicated to separate out.  For example, there is special key dispatching that happens for the lock screen, so you have whoever is doing the UI plus the window manager plus the policy all having to do a well-coordinated dance.

I think the first thing that would need to be done is some refactoring of the code to make the UI more separate from the locking logic (and the thing I mentioned about dealing with hiding the lock screen that may get in cupcake is one part of this, not but not the only).

Fwiw, before the final 1.0 we did some work to run the lock screen UI as a separate piece of code in another process, but backed that out because it introduced many more problems than we could deal with for 1.0.  And that was still a ways off from allowing third party lock screens.

Also I am not the primary engineer on this particular code, so if you are seriously wanting to start working on it we will need to have you talking with the people who know it best.  Most people are on vacation this week (I am too though it may not look like it) so it's probably best to wait until next week before discussing it more.
 

I agree again... we could always start the system lock screen behind the "user defined" and if this crashes the system one will always be there to keep the device locked. It is a bit ugly, but it is just an idea...

Having two whole UIs running would be pretty unfortunate, both in resource usage and start-up time.  This is another area where I think it makes sense for the window manager to have more built-in knowledge about the lock-screen to be able to make some basic assurances about it, such as long as we are in a state where it should be displayed then the user will not be able to see any windows behind it whether or not its window is capable of being shown.
 

In your experience... a process started but doing nothing is too much overhead? how is the IME stuff going to work?

Currently an empty process is about 2MB of overhead, so it is not something to be thrown around lightly.  The currently running IME does need a process to run it, but there is really no way around that since it is actually in use the entire time any application is being used.  This is not the case with the lock screen, so it would be more unfortunate to blow another 2MB on that during times the user is actually doing things that could make use of that memory.
 

Do you mean the interaction provided by this (http://code.google.com/intl/es-ES/android/reference/android/app/KeyguardManager.html)? how is it going to be reworked? I cannot see any changes in the cupcake branch yet...  should this API be aware somehow of the actual implementation of the lock screen? (I don't think so)

As I said, this is something we want to do, but may not get in to cupcake.  The API you quote will need to remain, but we would like to add a window flag to make that the lock screen does not need to be displayed while that window is shown, and deprecate the existing api to hide/show the lock screen.  The window manager would then take responsibility for knowing when the lock screen should be shown, and the existing api would be implemented more directly in the window manager.
 

>>>
adding a permission to allow apps to be used as a lock screen is probably in my opinion the wrong approach.  You might want to look at how the new input method stuff works, where an app just publishes a service that says it can serve as an IME, but the system just ensure that the user needs to be involved in picking it rather than relying on a permission.  (There is, however, a permission used to ensure that only the system can talk with an IME.)
>>>
Mmmm... interesting approach, but... would it mean the user has to be involved in picking it every time the device has to be unlocked? or the system is going to be able to remember the decision?

No, look at how the input method stuff works.  The user selects which input methods can be used, and the system will only allow the ones the user has selected to ever be used.
 

Also, I find a bit dangerous that every application could register as a lock screen provider without any special permission (even if the user is going to make the final decision about the screen that is going to be used), don't you think?

Not if the user gets to explicitly select the lock screen.  There is no point in checking with the user twice: both when they install and app, and when they select the lock screen.  As a general rule, normal and dangerous permissions (the ones the user sees) should only be used as a last resort; it is far better to design things so that the user shows they are okay with something through natural traversal of the UI rather than being questioned up-front (or IMHO worse in an "this guy wants to do that, is it okay?" dialog).
 

Also, I have finished my first review to the currently implemented code and I have found it a very modular and extensible solution. Most of the code is independent of the current pattern-based implementation and I have found only little fragments that are tied to it (like in KeyguardViewMediator where an instance of LockPatternKeyguardViewProperties is created).

It is not horrible code, but there are lots of very subtle interactions. :)  I am not saying this can't be done, but getting it really correct is going to be hard.
 

I have drawn a very basic diagram (only the main relationships are shown) with the classes I have understood so far (http://img187.imageshack.us/img187/7071/keyguardclassesbg7.png). Only the green ones are specific of the pattern-based implementation while the others are mostly generic code that could be kept with little changes. I think that "all" we need is a new kind of KeyguardViewBase that delegates the lock and unlock screens to third party code.

Nice job. :)  Again I am not the expert on this, so I'll see about getting the responsible engineer into the discussion.  There are definitely some basic things that need to be dealt with, such as the general repercussions of running that part of the code in another process.  Also there needs to be more flexible control over who the windo manager allows to be added as a lock screen, again very much like what is currently being done for input methods where the client is given a special token it passes back to the window manager to show it is the one that gets to be shown.

[Joe Onorato]

I'm not sure if I'll be working on keyguard in the future, but I was one of the original authors of it.

I think the way I would allow apps to have custom keygards would be that the system still puts up the big, full-screen view (let's call it the "blocking window").  What the permission would give you would be two things.  It would give you the ability to access an interface that allowed you to tell the system part of the keyguard that it's time to dismiss it, and it would give you the ability to put up windows on top of the system's keyguard window (call these the "visible keyguard window").  The window manager would probably have to be changed to forcibly hide both windows.  If the 3rd party one crashes, the blocking window would still be there to protect everything until the other one comes back up.

Changing KeyguardViewBase probably isn't going to work.  I believe that WindowManagerPolicy has intimate knowledge of that, and it has to run in the system process.  I really don't like the way that the event pre-dispatching stuff and the pokeWakeLock stuff turned out with the keyguard.  It's just way too complicated.  I wouldn't be opposed to overhauling it; it's just that because of its intersection with the power management code, there are a lot of subtle edge cases that crop up, the result of which is that the device gets into a state where it won't turn off or won't turn on.

-joe

[qvark]

Wow! Thanks Dianne and Joe, there is a lot of information to
assimilate ;)

As you said, it is probably better if we wait until next week when
most people will have come back from their vacations (I don't want to
take more of your free time now).

In the meantime I will take a closer look at the keyguard code
(including Window Manager) and how the IME stuff has been implemented.

Thanks for your support,

Jose Luis.

[Karl Rosaen]

Just to follow up with what Joe and Dianne mentioned, I don't think
fiddling with classes inside of com.android.internal.policy.impl is
really a way to support third party keyguards.

You could replace LockPatternKeyguardView with a new implementation /
subclass of KeyguardViewBase, and you would in fact have a different
keyguard, but it is still a far cry from supporting third party lock
screens in the robust we way do for say, the home screen:
- LockPatternKeygaurdView runs in the system process, and therefor any
crash or memory leak has serious implications (runtime restart and
general system slowness respectively)
- because it is just a View, and not and Activity, it has to go
through a lot of extra hoops to transition between screens, handle
orientation changes and the like, the whole KeyguardScreen interface
was created just to manage the complexity of the implementation.
expecting third party developers to do something similar each time is
a lot to ask
- it is trusted to behave with power management and input handling; a
poor implementation could prevent the screen from ever turning on
- etc etc

The bottom line is that the existing implementation is tightly coupled
into the window manager and power management code; it was done with
efficiency and user responsiveness in mind, and there simply wasn't
enough time before launch to get something more safe for third party
extension in place. For instance, you might notice that sometimes the
home screen takes a few seconds to come up when other applications,
such as the browser, have run and caused the home application to be
killed. Would users tolerate this delay for a keyguard? That's what
makes having the usual application responding to intent approach work
well for the keyguard.

Here are some hurdles / requirements I see needing to be met to get to
support third party keyguards:
- the code runs in a different process
- the implementation must respond within a certain amount of time,
otherwise a default keygaurd is brought up
- power management is not put in the hands of the keyguard (whether
the screen turns on, and for how long)

All that said, I commend your ability to dive into the policy.* code
and make sense of it. If you just want to play around with your own
keyguard implementation, implementing your own KeyguardViewbase and
building / flashing your own device would be a good way to start.

Let us know with any ideas for this you come up with, I'm happy to
provide any assitance I can in terms of understanding the existing
code, and / or evaluating ideas.

Karl

On Dec 31 2008, 3:57 am, qvark <joseluishuertasfernan...@gmail.com>
wrote: