Re: Screen orientation API proposal

[Mounir Lamouri]

On 01/26/2012 10:33 PM, Chris Jones wrote:
> ----- Original Message -----
>> From: "Mounir Lamouri" <mou...@lamouri.fr>
>> To: "Chris Jones" <cjo...@mozilla.com>
>> Cc: pba...@zynga.com, "Chris Pearce" <cpe...@mozilla.com>, "Boris Zbarsky" <bzba...@MIT.EDU>, di...@apple.com,
>> dev-w...@lists.mozilla.org
>> Sent: Thursday, January 26, 2012 6:08:11 AM
>> Subject: Re: Screen orientation API proposal
>>
>> On 01/26/2012 08:30 AM, Chris Jones wrote:
>>> ----- Original Message -----
>>>> From: "Mounir Lamouri" <mou...@lamouri.fr>
>>>> To: dev-w...@lists.mozilla.org
>>>> Cc: pba...@zynga.com, "Chris Pearce" <cpe...@mozilla.com>,
>>>> "Boris Zbarsky" <bzba...@MIT.EDU>, di...@apple.com
>>>> Sent: Wednesday, January 25, 2012 3:56:35 PM
>>>> Subject: Re: Screen orientation API proposal
>>>>
>>>> interface ScreenLockRequest {
>>>> readonly attribute DOMString readyState; // "processing" or
>>>> "done"
>>>> readonly attribute DOMError? error;
>>>> attribute EventListener onsuccess;
>>>> attribute EventListener onerror;
>>>> attribute ScreenLock? result;
>>>> }
>>>>
>>>> interface ScreenLock {
>>>> void unlock();
>>>> }
>>>>
>>>
>>> Is there any reason for these to be separate interfaces? I would
>>> find it more convenient to have unlock() on the request object,
>>> and just ignore the unlock() if the request failed or the lock had
>>> already been released.
>>
>> That would break the purpose of Request objects and result attribute.
>
> What's the purpose of the result attribute? I think you're begging the question.

DOMRequest.result is set when the request is done. When a result appear
and an action can finally be done with it.

> When discussing examples of this pattern with jlebar, I argued that it's not more code than global lock()/unlock(). With these separate interfaces, it is.
>
> window.screen.lockOrientation("portrait");
> //...
> window.screen.unlockOrientation();
>
> vs.
>
> var lock;
> var req = window.screen.requestOrientationLock("portrait");
> req.onsuccess = function () { lock = req.result; };
> //...
> if (lock)
> lock.unlock();
>
> So I withdraw my argument. Sorry jlebar!
>
> I'm not seeing concrete reasons for separating the two interfaces. Would like to see them.

It's indeed shorter to right the first code, however, the interface
design is worse because you can try to unlock something that is not
locked. For example:
window.screen.lockOrientation("portrait");
window.screen.unlockOrientation();

Will not work because lockOrientation() is async and unlockOrientation
is going to unlock nothing. The author might thing it could work. The
author might even run into race conditions if there is something between
lock and unlock and locking is sometimes done before unlock being called.

However, this design seems cleaner:
var req = window.screen.lockOrientation("portrait");
req.onsuccess = function() { req.result.unlock(); };

Because whatever happens, the author just can't call unlock if a lock
have not happen yet. It's less error-prone, thus clean.

>>> Another case is most relevant to pages launched within browser
>>> chrome. For the reasons Dean brought up earlier, a browser might
>>> disallow orientation lock for in-chrome pages. But they would
>>> likely want to allow orientation lock for pages that go to
>>> full-screen (up to the UA still). However, when transitioning
>>> from full-screen to not-full-screen, the page transitions from an
>>> orientation-lock-allowed state to -disallowed state. On that
>>> transition, the UA might want to drop all orientation locks. This
>>> may or may not need to specified.
>>>
>>> The spec *should* mention that any/all orientation locks can be
>>> dropped, at any time. I don't know if a notification of this is
>>> absolutely needed, but if we find a use case it would be pretty
>>> easy to fire an "onerror" callback to a lock object, or something
>>> else.
>>
>> Sending an error event to the Request object wouldn't be doable.
>> Request
>> objects receive one event (success or error) and at that point, their
>> state shouldn't change. At least, not being reverted.
>> I think we might need something to let the author know the lock has
>> been
>> removed mostly because removing the lock is going to be UA specific
>> neither specs nor authors can predict when UA are going to do that.
>>
>
> I don't have a use case in mind yet for authors reacting to revoked locks. What would they do? But your proposal sounds OK, whichever interface it goes on. We can also add that in a later spec.

In addition, we could imagine that with unlock() being in ScreenLock
object, we could always send a unlock event when unlock happens like:
req.result.onunlock = function() { alert("unlocked"); };
req.result.unlock();

This design seems nicer because if we unlock with unlock() or if the
unlock happens because of the UA forcing it, the same event handler can
be used.
In addition, we could easily throw an exception if unlock is called
after the unlock event being sent.

Sorry for the late reply and I might be repeating arguments because of
that. You have to blame the DOM Bindings work week ;)

Cheers,
--
Mounir

[Chris Jones]

That makes sense for a request where data is returned. In this case though, there's no data returned.

If you thought of unlock() as a cancel() interface, for canceling a request in any state, whether locked or not, could we agree on unifying the two interfaces?

> However, this design seems cleaner:
> var req = window.screen.lockOrientation("portrait");
> req.onsuccess = function() { req.result.unlock(); };
>
> Because whatever happens, the author just can't call unlock if a lock
> have not happen yet. It's less error-prone, thus clean.
>

What's to stop an author calling

req.result.unlock(); req.result.unlock();

in your example above? We have to make unlock() bullet proof anyway. I'm just requesting that we expand the window of bullet-proofedness a bit longer.

If we went so far as to rename unlock() to cancel(), could we agree on unifying the interfaces?

Cheers,
Chris

[Mounir Lamouri]

On 02/03/2012 01:08 PM, Chris Jones wrote:
>> However, this design seems cleaner:
>> var req = window.screen.lockOrientation("portrait");
>> req.onsuccess = function() { req.result.unlock(); };
>>
>> Because whatever happens, the author just can't call unlock if a lock
>> have not happen yet. It's less error-prone, thus clean.
>
> What's to stop an author calling
>
> req.result.unlock(); req.result.unlock();

Before the success event, req.result is null. In the success event
handler, nothing prevents the caller to do that. Though, I think we
could throw.

> in your example above? We have to make unlock() bullet proof anyway. I'm just requesting that we expand the window of bullet-proofedness a bit longer.
>
> If we went so far as to rename unlock() to cancel(), could we agree on unifying the interfaces?

I don't think it's useful to be able to cancel a lock request: this is
async but except that, it should be fairly immediate. Canceling seems
pretty useless then.

I'm still not convinced with your arguments for unlock() being in the
request object and you are obviously not convinced with mine. Let's have
Jonas break the tie.

Cheers,
--
Mounir

[Chris Jones]

----- Original Message -----
> From: "Mounir Lamouri" <mou...@lamouri.fr>
> To: dev-w...@lists.mozilla.org
> Cc: "Jonas Sicking" <jo...@sicking.cc>
> Sent: Monday, February 6, 2012 4:16:31 AM
> Subject: Re: Screen orientation API proposal
>

> I don't think it's useful to be able to cancel a lock request: this
> is
> async but except that, it should be fairly immediate. Canceling seems
> pretty useless then.
>
> I'm still not convinced with your arguments for unlock() being in the
> request object and you are obviously not convinced with mine. Let's
> have
> Jonas break the tie.
>

This is a developer hygiene issue. I appreciate the goal of having APIs use common patterns. But in this case, the pattern seems to be derived from other scenarios in which there's nothing useful that can be done until the onsuccess event is fired, like querying a database. The .result is data returned from a query, e.g.

In this case, the .result seems to exist only to follow the common pattern. I think that putting the unlock() method on an object returned by .result just increases the amount of boilerplate that authors need to use this API (example in previous post). The benefit for this overhead is adhering to the common pattern.

So I think the trade-off is between an easier-to-use API and an API that follows a pattern used elsewhere. My vote is for the easier-to-use API.

Cheers,
Chris

[Jonas Sicking]

Sorry guys, I really should have jumped into this thread earlier.
Here's my opinion on various things:

Conflicting requests within tab:
I think we should completely ignore any requests for orientation locks
from iframes. It should be in the control of the top-level frame. If
I'm looking at a pdf and pdf.js has requested portrait mode, and then
a small iframe opens in the page and starts playing a video, I think
I'd prefer to not have the whole screen rotate on me.

The only exception I can think of is if an element in an iframe goes
fullscreen. Then I think we could pay attention to any orientation
locks that the document of the full-screened element has.

This reduces conflicts to conflicts within a single page. I think
that's the page's problem. I can't think of any cases where a page
would want to have multiple active locks at the same time and have
that make sense. There is only one screen so having multiple actors
try to control it at the same time doesn't really make sense.

This is different from resource locks for example where I think the
page doing some IO operation and thus wanting to prevent CPU from
going to sleep, at the same time as playing a video and thus wanting
to prevent screen dimming, makes perfect sense.


Conflicting requests between tabs:
Could we simply ignore any requests from background tabs? And switch
to that tabs orientation once the user switches to that tab. If we
want to get really fancy-pants we could even fire
orientationchange/success and start reflowing the background page to
the size it would have under the new orientation, but not change the
UI until the user actually switches to that tab.


Security:
I think the security aspects here are very low. There isn't a whole
lot that a page can do with screen orientation that it couldn't do
with CSS-transforms. So I like the approach which I think Chris
suggested earlier in the thread:

* Requests from user gestures are always granted
* Requests outside of gestures are only granted X seconds after the
last orientation change (or possibly after the last orientation change
that was due to a lock rather than the user tilting the device)
* Requests from installed apps are always granted


API:
The currently proposed API seems more complex than needed. As stated
above, I don't think we need to deal with multiple conflicting
requests coming from one page. This would remove the need for
returning any type of request objects.

I think something as simple as:

partial interface DOMScreen {
// Returns the current screen orientation.
// Values can be "portrait-primary", "portrait-secondary",
// "landscape-primary", "landscape-secondary".
readonly attribute DOMString orientation;

// Returns whether the orientation in argument is allowed for the
// system. Returns false if the orientation isn't known at all.
bool orientationAllowed(DOMString orientation);

// If this changes the screen orientation, an 'orientationchange'
// event will be fired. Returns same thing as orientationAllowed.
bool lockOrientation(in DOMString orientation);
}

I'm not sure what the value is of locking to current orientation
without knowing what that orientation is. At worst a page could simply
grab the current screen.orientation value and pass that in. Seems rare
enough that that would be an acceptable solution.

It also seems better to not throw if the orientation isn't known by
the device. Throwing should only be used in exceptional cases, like if
the page has a bad bug. I don't think pages should be forced to call
orientationAllowed before safely calling lockOrientation.

And given the simple security model, if a page requests a given
orientation we can immediately tell it if we'll end up rotating to
that orientation. If the orientation isn't supported we'll immediately
know. If the orientation is supported we'll do it eventually, possibly
after X seconds if the call is "spammy", at which point a
orientationchanged event will be fired.


However I also think that being able to use markup to choose a
orientation is a good thing. Consider the iPhone youtube app for
example. It always displays videos in landscape mode, but the rest of
the app is in portrait mode. It would be very nice if it could do this
by simply sticking a <meta> tag in the video-playing-page and not
worry about sticking a script tag high up enough in the document
before any rendering has happened.

But I do agree that being able to choose an orientation in the
manifest is still a good idea. That way you don't have to make sure to
stick a <meta> tag in each and every page if they all use the same
orientation. But being able to override using markup in a single page
seems useful.


So if we do that, that leaves the question, is it common enough to
want to change the orientation while staying on a page, that we should
have a dedicated lockOrientation API. If manifest+<meta> covers the
vast majority of cases, then simply making the <meta> live, like Dean
suggests, might remove the need for the lockOrientation function.

/ Jonas

[Justin Lebar]

> Conflicting requests within tab:
> I think we should completely ignore any requests for orientation locks
> from iframes. It should be in the control of the top-level frame.

Well, the top-level frame inside an <iframe mozbrowser> should be able
to request an orientation lock, right?

So unfortunately we don't get to punt on the question of what happens
when an iframe requests the lock. I don't think I even want to think
about what happens when two <iframe mozbrowser>s request opposing
orientation locks...

> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi

[Jonas Sicking]

On Fri, Feb 10, 2012 at 12:41 PM, Justin Lebar <justin...@gmail.com> wrote:
>> Conflicting requests within tab:
>> I think we should completely ignore any requests for orientation locks
>> from iframes. It should be in the control of the top-level frame.
>
> Well, the top-level frame inside an <iframe mozbrowser> should be able
> to request an orientation lock, right?
>
> So unfortunately we don't get to punt on the question of what happens
> when an iframe requests the lock.  I don't think I even want to think
> about what happens when two <iframe mozbrowser>s request opposing
> orientation locks...

Could we handle this by making <iframe mozbrowser> fire events when a
containing page requests a particular lock? That way the browser app
could decide when to orient itself which way (using the lock API)
depending on which tab it's rendering.

So gecko would take care of figuring out the locking rules, i.e. when
to ignore iframes, when to delay orientation-locking due to "spam"
requests etc, and expose the result of those rules by firing events on
browser-iframe.

/ Jonas

[Justin Lebar]

> Could we handle this by making <iframe mozbrowser> fire events when a
> containing page requests a particular lock? That way the browser app
> could decide when to orient itself which way (using the lock API)
> depending on which tab it's rendering.

We certainly could do this, and maybe it's the right thing to do.

But one concern about the browser API -- maybe *the* concern -- is
that it's going to become huge, encompassing every conceivable browser
feature and therefore be in a constant state of flux. It will
therefore be the Mozilla browser API and never the Web browser API.

So my instinct is, if we can solve a problem without adding to the
browser API, perhaps we should.

[Jonas Sicking]

On Fri, Feb 10, 2012 at 1:22 PM, Justin Lebar <justin...@gmail.com> wrote:
>> Could we handle this by making <iframe mozbrowser> fire events when a
>> containing page requests a particular lock? That way the browser app
>> could decide when to orient itself which way (using the lock API)
>> depending on which tab it's rendering.
>
> We certainly could do this, and maybe it's the right thing to do.
>
> But one concern about the browser API -- maybe *the* concern -- is
> that it's going to become huge, encompassing every conceivable browser
> feature and therefore be in a constant state of flux.  It will
> therefore be the Mozilla browser API and never the Web browser API.
>
> So my instinct is, if we can solve a problem without adding to the
> browser API, perhaps we should.

I definitely understand your concern (though in some sense it seems
inevitable in the short term that we'll need moz-specific API in
addition to any standardized browser API).

I think we can sort-of avoid having orientation-locking specific API
on the <iframe mozbrowser> element. We can do this by having an API
for the browser-app to indicate which of the <iframe mozbrowser>
elements is the currently active "tab" and then letting gecko figure
out the rest. I.e. it would apply the rules described in my proposal
but pretending that the active <iframe mozbrowser> element is the
top-level element. It would then orient the app according to those
rules and fire orientationchange on both the toplevel app <iframe>, as
well as the active <iframe mozbrowser>.

However this has two downsides:
1. We hardcode the orientation behavior into gecko. I.e. another
browser can't use a different behavior for how to deal with pages
requesting orientation locking. This is to some extent true even if we
expose the API I proposed on <iframe mozbrowser> since for example how
to deal with "spamming" is still hard-coded into gecko. But it's more
flexible.

2. The browser UI can't depend on pages requesting locks. For example
the tab-switching UI can't show such pages rotated.

/ Jonas

[Justin Lebar]

> 1. We hardcode the orientation behavior into gecko. I.e. another
> browser can't use a different behavior for how to deal with pages
> requesting orientation locking.

I'm OK with this inasmuch as we can always change it later if we want
to. YAGNI.

> 2. The browser UI can't depend on pages requesting locks. For example
> the tab-switching UI can't show such pages rotated.

But this is kind of a bummer. :-/

> We can do this by having an API
> for the browser-app to indicate which of the <iframe mozbrowser>
> elements is the currently active "tab" and then letting gecko figure
> out the rest.

Would we want to know which mozbrowser is active for any other
reasons? Things like timeout throttling are probably going to be
based on visibility rather than active-ness.

[Jonas Sicking]

On Sat, Feb 11, 2012 at 5:50 PM, Justin Lebar <justin...@gmail.com> wrote:
>> 2. The browser UI can't depend on pages requesting locks. For example
>> the tab-switching UI can't show such pages rotated.
>
> But this is kind of a bummer.  :-/

Yeah, I'm more worried about this one too.

>> We can do this by having an API
>> for the browser-app to indicate which of the <iframe mozbrowser>
>> elements is the currently active "tab" and then letting gecko figure
>> out the rest.
>
> Would we want to know which mozbrowser is active for any other
> reasons?  Things like timeout throttling are probably going to be
> based on visibility rather than active-ness.

Isn't "visibility" and "active-ness" the same thing here? I guess in
theory you could have a UI which displays several <iframe>s at the
same time, in which case only one of them would be active, but so far
mobile browsers don't do that. Until someone wants to create such a UI
we can probably keep visible == active.

/ Jonas

[Justin Lebar]

>>> We can do this by having an API
>>> for the browser-app to indicate which of the <iframe mozbrowser>
>>> elements is the currently active "tab" and then letting gecko figure
>>> out the rest.
>>
>> Would we want to know which mozbrowser is active for any other
>> reasons? Things like timeout throttling are probably going to be
>> based on visibility rather than active-ness.
>
> Isn't "visibility" and "active-ness" the same thing here? I guess in
> theory you could have a UI which displays several <iframe>s at the
> same time, in which case only one of them would be active, but so far
> mobile browsers don't do that. Until someone wants to create such a UI
> we can probably keep visible == active.

Sure, sounds good to me. Then we don't need to introduce any new APIs
at all, at least for now.

-Justin