Alpha criteria for Fennec
Christian Sejersen
important that we all are aware of the criteria for reaching alpha.
Stuart will post the notes from the meeting at the summit, where we
agreed on the criteria, but there is one criteria I would like to
bring up for discussion: performance.
AFAIR the criteria we agreed upon was that it shouldn't "feel slow",
or "performance would be acceptable" or similar wordings, but I think
we need to get to a more objective criteria for performance. For one,
because we never now if we ever hit our target, and secondly so we
also can verify we are improving and raising the bar for follow-up
releases.
I don't have a good solution to what that criteria for performance
should be, but I will suggest two:
1) performance at least on par with MicroB on comparable features
2) x fps during animations
What do you think?
Regards,
Christian Sejersen
Mozilla Corp.
m: +45 2993 7003
Johnathan Nightingale
to say that I agree with the emphasis here.
As I've said before, I think that having a pleasing, responsive *feel*
is the sine qua non for a mobile browser. I think it is not what will
differentiate us, necessarily, other phone-based browsers are getting
good at visceral, kinetic appeal. I think we'll differentiate
ourselves the way we always do - extensibility, openness, overall
performance, standards compliance, ui innovation, security, &c. Those
are what will make it a mozilla browser, but I think that none of
those things will matter very much at all if the core browsing
operations aren't responsive and pleasant to use.
This is also why I've been so belligerently resistant to the
suggestion that the platform puts a ceiling on our abilities here.
I'm sure that's true at some level, but I really don't want us to give
ourselves permission there. As you point out, MicroB is more
responsive on the same hardware, so that, at least, should be a
performance floor, but I'm not sure I'd consider that fast enough
either.
Still, it's arguably a good alpha target, and I know that before I
went on vacation in August, Stuart was saying that we had things in
the pipeline that should get us clear of that mark, at least. I feel
like Vlad and Jeff had thoughts about making us gogogo too. What is
the state of those things?
Cheers,
Johnathan
> _______________________________________________
> dev-platforms-mobile mailing list
> dev-platfo...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platforms-mobile
---
Johnathan Nightingale
Human Shield
joh...@mozilla.com
Mike Beltzner
- if I touch a page with my finger and move my finger, the page
should move immediately
- if I swipe my finger for some UI gesture, the response should come
up in under 200ms
- zooming should show some change (even if it's a pixely change)
within 100ms
Madhava probably has better guidance.
cheers,
mike
Madhava Enros
What Mike lays out here are the right end goals for responsiveness.
These things, like the 100ms and 200ms timeframes for immediacy and
sense-of-causality, respectively, are the sort of non-negotiable "way
the brain works" determinants of whether the UI will feel responsive
and natural (physical) to people.
That said, whether these are *alpha* goals is a question of scheduling
vs. how natural we need it to be at this point, compared to abilities
of the browser. Do we have a sense of how close we are to being able
to do these kind of things?
Madhava
--
Madhava Enros
User Experience Designer, Mozilla Corporation
Stuart Parmenter
> We are getting close to the alpha release for Fennec and I think it is
> important that we all are aware of the criteria for reaching alpha.
> Stuart will post the notes from the meeting at the summit, where we
> agreed on the criteria, but there is one criteria I would like to
> bring up for discussion: performance.
>
The list we discussed at the summit included:
- stable
- usable/"fast enough"
- responsive "enough"
- our ui looking how we have it speced out
- getting our features in and working
- downloads
- prefs
- easily installable
> AFAIR the criteria we agreed upon was that it shouldn't "feel slow",
> or "performance would be acceptable" or similar wordings, but I think
> we need to get to a more objective criteria for performance. For one,
> because we never now if we ever hit our target, and secondly so we
> also can verify we are improving and raising the bar for follow-up
> releases.
>
I agree that the wording is vague, however I believe our starting
point for measuring should be what we consider good enough. We have
several patches from folks that should be landing in the next few days
that should put us at that point, however we should all feel
comfortable that that is the case.
> I don't have a good solution to what that criteria for performance
> should be, but I will suggest two:
>
> 1) performance at least on par with MicroB on comparable features
I'm not sure it is fair to anyone to say that an alpha's performance
needs to be as fast as a shipping product with years of optimization
behind it. Perhaps this is a better beta goal. We should set the bar
high for ourselves for our final release and probably have something
comparing ourselves to other applications out there.
> 2) x fps during animations
>
As I mentioned above, this might be a fine way to measure, but I think
our starting point should be based on what we feel is usable and then
continuously set the bar higher as we move forward. I don't think we
gain much by trying to pick a random number for alpha that may or may
not be usable.
stuart
Stuart Parmenter
> Hey -
>
> What Mike lays out here are the right end goals for responsiveness.
> These things, like the 100ms and 200ms timeframes for immediacy and
> sense-of-causality, respectively, are the sort of non-negotiable "way
> the brain works" determinants of whether the UI will feel responsive
> and natural (physical) to people.
>
> That said, whether these are *alpha* goals is a question of scheduling
> vs. how natural we need it to be at this point, compared to abilities
> of the browser. Do we have a sense of how close we are to being able
> to do these kind of things?
>
> Madhava
>
Hey Madhava, Mike,
Having this kind of insight is great. If we could put together a list
of numbers like those tied to a specific tasks we can measure I think
that would go a long way towards us shipping a great final product.
For alpha specifically, I don't know that we have to hit those exact
numbers. If we're already there, fantastic, but I believe there is a
number a bit higher than those where things are still totally usable
and people will be able to use it and give feedback without feeling it
is "too slow." I'm not sure how to determine those numbers besides us
just using it and saying "this feels like something we can have for
alpha."
My feeling is that we're pretty close (once we get a few critical
performance fixes in) to where we need to be for alpha, but still have
a ways to go before we can ship a final release.
stuart
Christian Sejersen
I don't mind the starting point being what we consider "good enough",
but I think it has to be tied to some objective numbers, e.g. ms as
suggested by Beltzner.
>
>> I don't have a good solution to what that criteria for performance
>> should be, but I will suggest two:
>>
>> 1) performance at least on par with MicroB on comparable features
>
> I'm not sure it is fair to anyone to say that an alpha's performance
> needs to be as fast as a shipping product with years of optimization
> behind it. Perhaps this is a better beta goal. We should set the bar
> high for ourselves for our final release and probably have something
> comparing ourselves to other applications out there.
We should set the bar high, and I think one goal should be: "We are at
least on par with MicroB on SunSpider and Dromaeo".
>
>> 2) x fps during animations
>>
>
> As I mentioned above, this might be a fine way to measure, but I think
> our starting point should be based on what we feel is usable and then
> continuously set the bar higher as we move forward. I don't think we
> gain much by trying to pick a random number for alpha that may or may
> not be usable.
Again, I don't think we can have a serious discussion about what you
feel is usable compared to what I feel is usable, but I think you also
agree with me, since you are talking about set the bar higher as we
move forward, that is has to be measurable in some way.
Of course I am not arguing for picking random numbers for the alpha, I
am just suggesting that one approach could be to instrument the code
(e.g. measuring ms for zoom, panning, sliding in/out) and then set the
targets based on whether the feature feels sluggish or not. We can
then continuously measure the performance and raise the bar as we move
forward.