Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

"App rendered" event for better screenshots/ux & perf

59 views
Skip to first unread message

James Burke

unread,
Apr 12, 2013, 2:45:18 PM4/12/13
to dev-...@lists.mozilla.org
Some data points:

* I notice the screenshot service is triggered in the email app but
before the app has finally rendered our first card. The app at this
point is basically a white page, and it looks like it takes around
50ms with a reflow for the screenshot. It looks like this delays the
email logic from doing its final first card determination.

* The datazilla startup graph is measuring cold startup uses
mozbrowserloadend, but this is not an accurate measure of when the app
is first usable, as many apps need to do async DB work to determine
their state, and may defer some JS loading before really knowing the
final state.

* Platform paint events seem to be used to know when to hide an
existing screenshot for the app and show the real app. However, since
the app has not fully rendered, this leads to a Flash of White (FoW)
before the app actually appears. This seems undesirable from an
experience point of view.

Possible solution:

What I would like to have instead is a custom "mozapprendered" event.
Something I could trigger in app logic, since the app knows when its
UI is actually usable/visible, the system cannot guess it.

With this event:

* Do not trigger the screenshot service until that event is received.
This would give the email app hopefully around 50ms more to get to
their "what card should I display" logic faster, and then therefore
faster cold boot. I expect this could help other apps too.

* More accurate graphs for datazilla: track this new event since it
matches closer to the user experience of being able to use the app.

* Better user experience since it could avoid the flash of white we see now.

James

Julien Wajsberg

unread,
Apr 14, 2013, 7:20:31 AM4/14/13
to James Burke, dev-...@lists.mozilla.org
Hey James,

Regarding the screenshot, I think Vivien had basically the same idea of
you. You should think together during this week ;)

Regarding the performance tests, we already have a framework to add some
other measure points than the default one. For example, see the
directory test/performance/ in the contacts app and
https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Platform/Testing/Gaia_performance_tests.
These are not perfect yet, but still they exist.

Regards,
--
Julien
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia


signature.asc

Gabriele Svelto

unread,
Apr 14, 2013, 8:49:43 AM4/14/13
to James Burke, dev-...@lists.mozilla.org
Hi James,

> * I notice the screenshot service is triggered in the email app but
> before the app has finally rendered our first card. The app at this
> point is basically a white page, and it looks like it takes around
> 50ms with a reflow for the screenshot. It looks like this delays the
> email logic from doing its final first card determination.

The first screen-shot is now taken when the app goes idle for the first
time or 2s after launch, whichever comes first. The reason why we
enforce the 2s deadline is that we must make sure that a snapshot will
be taken eventually; we cannot rely on the app to go idle (a real-time
app such as a game might never have idle time).

> * Platform paint events seem to be used to know when to hide an
> existing screenshot for the app and show the real app. However, since
> the app has not fully rendered, this leads to a Flash of White (FoW)
> before the app actually appears. This seems undesirable from an
> experience point of view.

Yes, it's very undesirable and it breaks the illusion that the app has
already started that we're trying to convey by showing the screen-shot
in the first place.

> * Do not trigger the screenshot service until that event is received.
> This would give the email app hopefully around 50ms more to get to
> their "what card should I display" logic faster, and then therefore
> faster cold boot. I expect this could help other apps too.

That would be nice but this should something that well-behaved
applications should be able to opt-in. In general I'd keep the existing
screen-shot mechanism in place to ensure that we get some kind of
screen-shot even in the presence of a badly behaved application that
never goes idle and never tells us when it's a good time to capture one.

> * Better user experience since it could avoid the flash of white we see now.

I think this is far more important than the 50ms savings.

Gabriele

James Burke

unread,
Apr 14, 2013, 11:57:22 PM4/14/13
to Julien Wajsberg, dev-...@lists.mozilla.org
On Sun, Apr 14, 2013 at 4:20 AM, Julien Wajsberg <jwaj...@mozilla.com> wrote:
> Hey James,
>
> Regarding the screenshot, I think Vivien had basically the same idea of
> you. You should think together during this week ;)

My first thought was allowing some new custom event to be dispatched
by app code that would be allowed to pass through mozbrowser elements
as a new the mozbrowser event:

https://developer.mozilla.org/en-US/docs/WebAPI/Browser#Events

so that the gaia system can listen for it. However, I am very ignorant
of platform concerns, and Vivien will have a much better handle on how
to solve it. I am not at that work week, but Andrew Sutherland and
James Lal know the details from the email app side.

> Regarding the performance tests, we already have a framework to add some
> other measure points than the default one. For example, see the
> directory test/performance/ in the contacts app and
> https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Platform/Testing/Gaia_performance_tests.
> These are not perfect yet, but still they exist.

I tried that route, but it appears the definition of
mozPerfHasListener occurs too late, at least for what I was trying in
the email app. My branch:

https://github.com/jrburke/gaia/tree/bug837677-measure-email

specific commit:

https://github.com/jrburke/gaia/commit/997812edd376ca97b9f0554bf409526cf0647ea1

I also notice the contacts test drops the report of an earlier event
for a similar reason, but mozPerfHasListener does get defined before
the 'startup-path-done' dispatch, and since that event marks the end
of the test, it finishes out. In my attempt for the email app,
mozPerfHasListener is still not defined when I dispatch
'startup-path-done', so the test hangs. I could be doing something
very silly though, pointers are appreciated.

James

James Burke

unread,
Apr 15, 2013, 12:25:55 AM4/15/13
to Gabriele Svelto, dev-...@lists.mozilla.org
On Sun, Apr 14, 2013 at 5:49 AM, Gabriele Svelto <gsv...@mozilla.com> wrote:
> The first screen-shot is now taken when the app goes idle for the first time
> or 2s after launch, whichever comes first. The reason why we enforce the 2s
> deadline is that we must make sure that a snapshot will be taken eventually;
> we cannot rely on the app to go idle (a real-time app such as a game might
> never have idle time).

When hacking around, I ended up disabling the screenshot for email
(just in a local branch for hacking) around here:
https://github.com/mozilla-b2g/gaia/blob/master/apps/system/js/window_manager.js#L524

If "idle" is one of those events, at least in the email case, that was
not an accurate reflection of the app, and the screenshot at that
moment took time away the app could use to start up faster. The app
was still waiting on some async operations to complete before it was
ready for use.

> That would be nice but this should something that well-behaved applications
> should be able to opt-in. In general I'd keep the existing screen-shot
> mechanism in place to ensure that we get some kind of screen-shot even in
> the presence of a badly behaved application that never goes idle and never
> tells us when it's a good time to capture one.

Having fallbacks make sense. My main concern (for the screenshot data
point) was avoiding a screenshot by the platform guessing that the app
was ready when it was not, and in a way that prolonged app startup
time.

If that means the email app needs to opt-in to avoiding a screenshot
during the "idle" determination, I am happy to do the work to set that
up if someone gives guidance on the preferred mechanism.

A blacklist in window_manager may be enough, but not sure if an "app
rendered" events gets set up if there is a better mechanism. For
instance, maybe apps that trigger "app rendered" register that in
their manifest, and then the window_manager could use that knowledge
to listen to that event instead of the idle ones.

James

Tim Chien

unread,
Apr 15, 2013, 7:52:41 AM4/15/13
to James Burke, dev-...@lists.mozilla.org
James,

What you proposed sounds like a good idea to fix the screenshot timing
issue on the FxOS side, but I failed to understand what's the difference
between the event you are proposing and the "mozbrowserfirstpaint" event we
already got on the System app.

The problem about the e-mail lies on the fact first rendering happens
before the first card is ready, and the app has really no control about
that. People have been calling on abandoning the screenshot approach
altogether -- but I personally don't like that because the alternative will
be adding |{ splashImages: ... }| to Open Web App manifest, which further
diverge the web platform v.s. open web app ecosystem, technical-wise.



On Sat, Apr 13, 2013 at 2:46 AM, James Burke <jrb...@gmail.com> wrote:

> Some data points:
>
> * I notice the screenshot service is triggered in the email app but
> before the app has finally rendered our first card. The app at this
> point is basically a white page, and it looks like it takes around
> 50ms with a reflow for the screenshot. It looks like this delays the
> email logic from doing its final first card determination.
>
> * The datazilla startup graph is measuring cold startup uses
> mozbrowserloadend, but this is not an accurate measure of when the app
> is first usable, as many apps need to do async DB work to determine
> their state, and may defer some JS loading before really knowing the
> final state.
>
> * Platform paint events seem to be used to know when to hide an
> existing screenshot for the app and show the real app. However, since
> the app has not fully rendered, this leads to a Flash of White (FoW)
> before the app actually appears. This seems undesirable from an
> experience point of view.
>
> Possible solution:
>
> What I would like to have instead is a custom "mozapprendered" event.
> Something I could trigger in app logic, since the app knows when its
> UI is actually usable/visible, the system cannot guess it.
>
> With this event:
>
> * Do not trigger the screenshot service until that event is received.
> This would give the email app hopefully around 50ms more to get to
> their "what card should I display" logic faster, and then therefore
> faster cold boot. I expect this could help other apps too.
>
> * More accurate graphs for datazilla: track this new event since it
> matches closer to the user experience of being able to use the app.
>
> * Better user experience since it could avoid the flash of white we see
> now.
>
> James
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>



--
Tim Guan-tin Chien, Senior Front-end Dev., Mozilla Corp. (Taiwan)

Vivien

unread,
Apr 15, 2013, 8:12:50 AM4/15/13
to Tim Chien, James Burke, dev-...@lists.mozilla.org
On 15/04/2013 13:52, Tim Chien wrote:
> James,
>
> What you proposed sounds like a good idea to fix the screenshot timing
> issue on the FxOS side, but I failed to understand what's the difference
> between the event you are proposing and the "mozbrowserfirstpaint" event we
> already got on the System app.

mozbrowserfirstpaint does not means the app is fully rendered. Someone
(i think Etienne) said one day this is like taking a picture of someone
when he/her wake up after a short night.
>
> The problem about the e-mail lies on the fact first rendering happens
> before the first card is ready, and the app has really no control about
> that. People have been calling on abandoning the screenshot approach
> altogether -- but I personally don't like that because the alternative will
> be adding |{ splashImages: ... }| to Open Web App manifest, which further
> diverge the web platform v.s. open web app ecosystem, technical-wise.

I don't like it neither but I spend quite some time playing with the
dynamic screenshoting approach and I didn't find a good way for the app
to define a correct screenshot without using some setTimeout in the
starup path (which is awful).
So right now I feel like this is the only viable approach. I already
have some patches for it I did in the system app with Etienne, combined
with some app's patches to eliminate some flashes that are generated by
the background-color of the body.

I'm sitting on the other corner if you want to have a look at it :)

I also quickly spoke to Fabrice, Jonas and Mounir about it already and
it seems like there is not much concerns about it. So globally I feel
like people don't like it but the other solutions does not really fit
(firing a custom event is not a good solution neither since this is not
part of any specs - and the system app can't listen them anyway).

Vivien.

Andreas Gal

unread,
Apr 15, 2013, 8:30:50 AM4/15/13
to Vivien, James Burke, dev-...@lists.mozilla.org, Tim Chien
We could disable early reflow for packed apps. Dbaron is at the work
week. Lets chat with him.

Sent from Mobile.

Fabrice Desre

unread,
Apr 15, 2013, 9:24:15 AM4/15/13
to Vivien, James Burke, dev-...@lists.mozilla.org, Tim Chien
On 04/15/2013 05:12 AM, Vivien wrote:
>
> I also quickly spoke to Fabrice, Jonas and Mounir about it already and
> it seems like there is not much concerns about it. So globally I feel
> like people don't like it but the other solutions does not really fit
> (firing a custom event is not a good solution neither since this is not
> part of any specs - and the system app can't listen them anyway).

One issue with the manifest approach is to deal with multiple screen
resolutions. We could also try something similar to what the android
runtime is doing: display a "wait screen" with the app icon, app name
and a background color extracted from the icon.

Fabrice
--
Fabrice Desr�
b2g team
Mozilla Corporation

Julien Wajsberg

unread,
Apr 15, 2013, 9:58:48 AM4/15/13
to Fabrice Desre, Vivien, James Burke, dev-...@lists.mozilla.org, Tim Chien
We would still need to know when to remove this wait screen ;)

So basically that time would be a perfect time to take a screenshot. And
then we don't need the wait screen at all (except maybe the first time
we launch the app) ;)

--
Julien

signature.asc

James Burke

unread,
Apr 15, 2013, 12:00:27 PM4/15/13
to Vivien, dev-...@lists.mozilla.org, Tim Chien
On Mon, Apr 15, 2013 at 5:12 AM, Vivien <vnic...@mozilla.com> wrote:
> I also quickly spoke to Fabrice, Jonas and Mounir about it already and it
> seems like there is not much concerns about it. So globally I feel like
> people don't like it but the other solutions does not really fit (firing a
> custom event is not a good solution neither since this is not part of any
> specs - and the system app can't listen them anyway).

It would be good to hear more about exploring a custom event, as this
seems to be one of those things the web may need to grow in general.
Maybe custom event is the wrong word here, but some sort of Event that
the app can trigger, and the platform lets bubble out of mozbrowser so
the gaia system app can listen and use. It can be "moz" prefixed for
now if the concern is spec stuff. It seems like a generally useful
event to have though.

In addition to better handling screenshot capture, it means a more
precise signal for the system to remove any screenshot that is shown
while the app is loading, so startup will feel smoother to the user
without a white flash. And we have a reliable event we can use for
plotting startup performance.

James

Tim Chien

unread,
Apr 15, 2013, 12:33:21 PM4/15/13
to James Burke, Vivien, dev-...@lists.mozilla.org
We could definitely hook back the postMessage() API back, in mozbrowser,
w/o exposing the parent window reference. I agree it will be good for
experimenting. Unfortunately, I don't trust ourselves to use it responsibly
-- we could easily ended up binding system app with other apps to deep, and
in turn break the system with apps -- there are architectural reasons to
isolate apps, not just for security nor securing uniformity of WebAPI.

(Sorry about for hijacking the thread to said things less related)

Vivien

unread,
Apr 15, 2013, 1:01:28 PM4/15/13
to Julien Wajsberg, Fabrice Desre, James Burke, dev-...@lists.mozilla.org, Tim Chien
On 15/04/2013 15:58, Julien Wajsberg wrote:
> Le 15/04/2013 15:24, Fabrice Desre a �crit :
>> On 04/15/2013 05:12 AM, Vivien wrote:
>>> I also quickly spoke to Fabrice, Jonas and Mounir about it already and
>>> it seems like there is not much concerns about it. So globally I feel
>>> like people don't like it but the other solutions does not really fit
>>> (firing a custom event is not a good solution neither since this is not
>>> part of any specs - and the system app can't listen them anyway).
>> One issue with the manifest approach is to deal with multiple screen
>> resolutions. We could also try something similar to what the android
>> runtime is doing: display a "wait screen" with the app icon, app name
>> and a background color extracted from the icon.
>>
>> Fabrice
> We would still need to know when to remove this wait screen ;)
Remove it on loadend. After that this is the responsibility of the app
to looks pretty and to have a nice start sequence.

> So basically that time would be a perfect time to take a screenshot. And
> then we don't need the wait screen at all (except maybe the first time
> we launch the app) ;)
This is not so easy. Taking a screenshot is expensive and nobody wants
to block app load for taking this screenshot. So the current approach is
to wait for the event loop to be idle. But app load is usually a very
busy time and app it sounds bad to have to delay the 'usable' state of
the app.



James Burke

unread,
Apr 15, 2013, 3:03:56 PM4/15/13
to Tim Chien, Vivien, dev-...@lists.mozilla.org
On Mon, Apr 15, 2013 at 9:33 AM, Tim Chien <timd...@mozilla.com> wrote:
> We could definitely hook back the postMessage() API back, in mozbrowser, w/o
> exposing the parent window reference. I agree it will be good for
> experimenting. Unfortunately, I don't trust ourselves to use it responsibly
> -- we could easily ended up binding system app with other apps to deep, and
> in turn break the system with apps -- there are architectural reasons to
> isolate apps, not just for security nor securing uniformity of WebAPI.

postMessage seems like a hacky way to do it, but I would be happy to
use it over nothing. I still favor an event though as it is more
targeted, harder to abuse. It also seems to fit with the other events
that indicate levels of "loaded".

DOMContentLoaded/window.onload are about measuring asset loading and
HTML readiness, but not code logic completion. With the move to more
async app startup and all of the assembly done in JS vs. HTML, the
platform does not have enough info to know really when the app is
"ready". A signal that can be triggered from the app's JS is more
precise.

James

Mounir Lamouri

unread,
Apr 17, 2013, 6:02:24 AM4/17/13
to dev-...@lists.mozilla.org
Why not using a splash screen as Android Web Runtime is doing? It is
using the application icon with a throbber and the background is the
main icon colour. I understand that this might be a bit weird in some
use cases but it would definitely be better for performances. Another
solution could be to use the application icon, put it in the middle and
have a simple black background. That would be fairly unexpensive and
would work as long as UX is considered okay.

All of those solutions seems better than what we are currently doing
actually, given that UX wise, you might end up with terrible screenshots
currently and this is quite heavily using the CPU.

--
Mounir

Tim Chien

unread,
Apr 17, 2013, 6:24:34 AM4/17/13
to Mounir Lamouri, dev-...@lists.mozilla.org, Jaime Chen, Josh Carpenter
Sounds like a good idea to me.

Let's CC'ing some UX people for them to give some responses.

Andreas Gal

unread,
Apr 17, 2013, 7:45:38 AM4/17/13
to Tim Chien, Jaime Chen, dev-...@lists.mozilla.org, Mounir Lamouri, Josh Carpenter

For packaged apps this should be a non-issue. We should never ever need a screenshot. A packaged app can easily launch significantly below 1s on very low-end hardware. In fact, most apps launch around 600ms now, which is about the time we take to animate the opening animation. If your app is slower, you are doing something wrong. Lets focus on making apps always start up fast. Loading splash screens are absolutely aggravating as a user. Its a different story for apps that load from the cloud and don't use app cache (facebook et al), but I would focus any changes on that subset.

Andreas

Andrew Sutherland

unread,
Apr 17, 2013, 9:31:49 AM4/17/13
to Andreas Gal, Mounir Lamouri, dev-...@lists.mozilla.org, Jaime Chen, Josh Carpenter, Tim Chien
https://bugzilla.mozilla.org/show_bug.cgi?id=837338 covers making
screenshots and animation effects configurable and setting screenshots
to disabled by default. It has a patch from Vivien on it that cjones
posted. Can we get it tef+'d?

Andrew

James Burke

unread,
Apr 17, 2013, 12:03:59 PM4/17/13
to Andrew Sutherland, Jaime Chen, Josh Carpenter, Tim Chien, Andreas Gal, dev-...@lists.mozilla.org, Mounir Lamouri
On Wed, Apr 17, 2013 at 6:31 AM, Andrew Sutherland
<asuth...@asutherland.org> wrote:
> https://bugzilla.mozilla.org/show_bug.cgi?id=837338 covers making
> screenshots and animation effects configurable and setting screenshots to
> disabled by default. It has a patch from Vivien on it that cjones posted.
> Can we get it tef+'d?

This would be great to have in general for apps that need specialized
needs. Something like a password manager that would not want
screenshots taken when sensitive info is shown. Or apps that want the
more traditional splash screen thing.

However, in the case of email, I would like the system to just do the
screenshots of the app for me. If I could just ask the system to do it
when the app is rendered, and not when it is in the middle of starting
up, that would give a better overall user experience.

It gives a better reflection of current app state and when the
screenshot is turned off, the transition to live app state is
smoother. No white page flash, the typical giveaway of a web page, and
it is done in a way that requires the app developer to just send a
signal to the system, and not code up their own screenshot logic. This
should help perceived performance, as well as real perf by not
interfering in app logic startup time.

So I would still like to see some way to signal to the system app that
the app is rendered, and see the system use that for swapping between
screenshot and live app. Perhaps it is better to ask on dev-b2g
instead. Will sort that out in a bit.

James

James Burke

unread,
May 16, 2013, 11:57:23 PM5/16/13
to dev-...@lists.mozilla.org
On Fri, Apr 12, 2013 at 11:45 AM, James Burke <jrb...@gmail.com> wrote:
> Possible solution:
>
> What I would like to have instead is a custom "mozapprendered" event.
> Something I could trigger in app logic, since the app knows when its
> UI is actually usable/visible, the system cannot guess it.

Some progress on this, and I would like to figure out if we could land
some of it:

This discussion ended up triggering this ticket:

https://bugzilla.mozilla.org/show_bug.cgi?id=863499

In that ticket and subsequent threads on the whatwg, it seemed like
the front runner was a paired JS API to delay and cancel the delay of
the document load event. I hacked something up for a
document.mozDelayLoadEvent() and document.mozStopDelayingLoadEvent().
A patch for that hack is attached to the above ticket. I will need
some help completing it out though as I am new to platform hacking,
and I want to be sure the error path is correct (see ticket).

But the patch works well enough that I applied to the mozilla-b2g
branch and latest gaia and installed it on an unagi device.

Previously, I noticed some non-app work clogging up the event loop:

https://bugzilla.mozilla.org/show_bug.cgi?id=863870

With the patch applied, and changing the gaia window_manager.js to:

1) hide the app iframe (visibility: hidden) until mozbrowserloadend fires.
2) Also wait for mozbrowserloadend to do screenshot work.

then app code placed on the event loop no longer seems blocked by
non-app work on startup (fixes #863870).

You can confirm this yourself if you apply the 863499 patch to gecko
and use this branch of gaia:

https://github.com/jrburke/gaia/tree/eventloop-screenshot

Run the "Event Loop" and "Event Loop New" test apps and watch the
logcat. The "Event Loop New" app version uses
document.mozDelayLoadEvent where the other app does not. There is
about a 100ms savings in the "new" one for when items placed on the
event loop get to firing, but both apps seem to operate better than
when that ticket was originally filed, closer to a 200ms savings,
comparing original file with the "New" app. I believe the full 200ms
diff is a result of hiding the app iframe until the document load
event fires, holding the document load event until app work is done,
but probably also changes in the platform code since the original
filing.

In that eventloop-screenshot branch, the email app is modified to use
document.mozDelayLoadEvent, and it feels more like an app to have the
screenshot hold for a moment then transition to the live app -- no
white flash, and no partial UI render as more data comes in.

So I would like to figure out:
------------

1) How the 863499 patch could land for gecko, maybe for v1-train?

2) Get more info on the switch to app icon/splashscreen instead of a
screenshot, done in this ticket:
https://bugzilla.mozilla.org/show_bug.cgi?id=866174

3) Any pitfalls if I prep a change in window_manager.js to use
mozbrowserloadend for the screenshot and iframe visibility change?
Perhaps it can be combined with the splash changes: if no screenshot
use the splash image.

On the splash changes in 866174: I expect the motivations were to
reduce load times and non-app work on app startup.

I believe the above changes would give a similar effect, and since the
iframe is hidden, some unnecessary paints could be avoided. It may
also open the window for other optimizations in the platform if we
know the frame is not visible until the mozbrowserloadend event fires.
For the email app, with the event loop cleared of the paints and
screenshot work during app load, I have some other experiments I want
to run to reduce email cold startup.

To give you a feel for how the above changes work with splash screens
and screenshotting+onload changes:

Email, Calendar and Settings with gaia master (has splash screen work
from 866174):

Email: https://vimeo.com/66368325
Calendar: https://vimeo.com/66368324
Settings: https://vimeo.com/66368326

Email, Calendar and Settings with screenshots taken on load, email app
using document.mozDelayLoadEvent:

Email: https://vimeo.com/66368471
Calendar: https://vimeo.com/66368472
Settings: https://vimeo.com/66368470

I believe the Settings app is doing work after onload so there is no
screenshot? So more investigation is needed on that one.

End result though, I believe these changes could avoid white flashes
and "a web page is drawing" feel for apps on startup, and delay
screenshotting until an appropriate time.

James

Tim Chien

unread,
May 17, 2013, 4:53:33 AM5/17/13
to James Burke, dev-...@lists.mozilla.org
Hi James,

Great progress! As I am not familiar with stuff other than Gaia itself
(Vivien might), I could only answer #3:

My concerning on "mozbrowserloadend" event would be, for
non-appcache'd hosted app that gets transmitted over the wire, we
would be waiting for too long before showing the page. What if the
page is just waiting for a big <img> to be downloaded, and other than
that it have essentially, "ready"? Do we really want to stop the
entire page to show up just because an image is not entirely visible?

To push it a bit further, this is essentially a divergence of
browser/runtime behavior between web v.s. web apps. I don't know if
it's bad or not in this case, but if no one is sure of it, we should
be conservative and have this API for packaged app only.
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia



--
Tim Guan-tin Chien, Engineering Manager and Front-end Lead, Firefox
OS, Mozilla Corp. (Taiwan)

James Burke

unread,
May 17, 2013, 4:14:36 PM5/17/13
to Tim Chien, dev-...@lists.mozilla.org
On Fri, May 17, 2013 at 1:53 AM, Tim Chien <timd...@mozilla.com> wrote:
> My concerning on "mozbrowserloadend" event would be, for
> non-appcache'd hosted app that gets transmitted over the wire, we
> would be waiting for too long before showing the page. What if the
> page is just waiting for a big <img> to be downloaded, and other than
> that it have essentially, "ready"? Do we really want to stop the
> entire page to show up just because an image is not entirely visible?

This is a great point. Thinking about this, and how apps may want to
provided their own launch image (or just use the icon), it leads to
the following questions:

* What kind of launch image does the app want? (icon, url, screenshot)
* When should the launch image be removed? (onpaint or onload)

This points something new in the webapp manifest. We should do this
anyway, because gaia master is using the largest "icons" image, but
for some of the apps (like Calendar and Settings), the image is not
really an icon, but a launch image. This will be confusing/wrong if
there are large displays that would want to use that image as really
an icon and not an launch_image.

Here is a cut at that new webapp manifest section:

"launch_image": {
"type": "icon" | "url" | "screenshot-onload" | "screenshot-onunload"
"urls": {
"/style/launch/320.png": {
"width": 320,
"height": 480,
"resolution": 2
}
}
"removed": "onpaint" | "onload"
}

Default for "type" is "icon". Default for "removed" is "onpaint".

"screenshot-onload" means "take a screenshot on the load event and use
that for launch_image".

"screenshot_onunload" means "take a screenshot when the app is about
to be quit". This would be useful for apps that maintain their state
between kills. "screenshot-onunload" may be tricky to nail down on
when it is taken, so for now I would not implement it and could leave
it out for now. But I mention it to explain why the other value is
"screenshot-onload" instead of just "screenshot".

"type" of "url" would use the "urls" section. The "urls" structure is
different from "icons" since the images are not square, and this
allows the platform to choose what image best fits the screen. I
considered a format like this:

"urls": {
"320x480@2": "/style/launch/320.png"
}

but it seems to lead to a bunch of string deconstruction where the
data could just be broken out by type in the JSON.

So I will work up a patch that morphs the current gaia master code's
icon use to use this new structure, as it will fix the misuse of
"icons" and it can land separate from the the mozDelayLoadEvent work
and still be useful.

This will also be enough to avoid at least some paint work if
"removed": "onload" is used, and for the email app, would code that
section of the manifest to be:

"launch_image": {
"type": "screenshot-onload",
"removed": "onload"
}

and with mozDelayLoadEvent, it will get even better, as far as
transitions and avoidance of paint/screenshot work in the middle of
the app startup logic. It will also give a better screenshot.

With the defaults of "icon" and "onpaint", it will work well for apps
that are loaded from the network, and maintains the behavior that is
in gaia master now without requiring existing apps to include a
"launch_image" section.

James
0 new messages