On Wed 01 May 2013 03:29:42 PM PDT, Cheng Wang wrote:
>
>
> Steve Fink wrote:
> <snip>
>>
>>> * Add-ons don't work reliably.
>>
>> There must be some way to improve this, but I don't know where to best
>> apply the effort. What are the typical causes of unreliable/unavailable
>> addons on Aurora? Obviously, some addons need to be updated for Aurora
>> and won't be before the relevant change is in Aurora, but is this the
>> bulk of the problems? Are we now too cavalier about addon-incompatible
>> changes?
>
> I don't think any binary add-ons work, for example. But there's also
> the lag. Let's say it takes an add-on developer a week (out of six) to
> update an add-on and once every few releases breaks something and
> fixes it next day. That's pretty fast response. Except for a user the
> experience is the add-on works, works, works, doesn't work, works,
> doesn't work, doesn't work, works, works, everything is broken, works
> again. It just feels random which is more annoying than just being
> broken.
Make add-on breakage block updates?
Who comprises the Aurora community? I'd guess some early adopters who
want to see what's coming or just feel on the edge, some people who hit
a bug in release and switched to Aurora because it was fixed there,
some web developers who want to use the new tools without suffering the
instability of Nightly, and some people who want to help. I'm guessing
some of those piles are far larger than others.
Maybe break them down into active testers, passive testers, and
unwilling/unknowing testers. Making the Aurora-ness more obvious helps
the size of the active pool and hurts the size of the unknowing pool.
The effect on the passive pool depends on how cool/fun/scary/... the
differences are, I guess. Having extra communication channels available
doesn't seem like it would hurt, as long as you're not trying to shove
them down anyone's throat. Whether they'd get critical mass to help is
an open question.
We shouldn't assume that someone is super core community just because
they use Aurora, but the onramp to the community ought to be clearly
visible and a gentle slope. Right now, switching to Aurora is something
you can do once and then forget about until it causes issues. The only
path leads out. Then again, more frequent reminders will make more of
the unwilling leave, which sucks for the passive pool, but is perhaps
worth the risk if the active/semi-active users are higher value. (And I
don't know that they are. For problems that are just numbers games, it
seems like having a huge passive pool is way more helpful.)
> Honestly, we should answer every feedback request (see lack of
> staffing below) with something. Either, we're trying to fix it but we
> need more info, or it's a result of X. Or their problem isn't
> Aurora/Firefox specific, here's help.
But we can't, so isn't the next best thing to clearly document that and
provide a default path? Something like "No response after 3 days?
Sorry. Here's why, here's what you would do to make your report
actionable if you chose to do so, here's how to install multiple
versions so you can run the release build when you need to. And here's
a forum you can go to to complain to other people who may not be able
to do anything more than commiserate." It sucks that we can't help
everyone or even give them a personalized response, but many users
understand that. They still deserve to have *some* way to proceed, even
if it requires enough effort that they will choose not to.
>> I think the benefits of Aurora specialization far outweigh the benefits
>> of looking as much like the Beta-to-be as possible.
>
> This has the result of biasing our feedback and meaning that someone
> has to chase down/test/try to reproduce all n=1 pieces of feedback in
> case it turns out to be a problem that disproportionately affects
> release over this specialized group.
Huh? I'm just talking about some additional menu entries and help text.
Well, and access so they can more easily tweak their own installations
knowingly.
>>> Right now, we don't have the staff to handle the feedback we do get, I
>>> have no idea how to handle 10 times more. Everything we do is reading
>>> tea leaves and educated guesswork. We read 1000 pieces of unstructured
>>> feedback and 2 people mention something about tabs lagging a little:
>>> this could be a blow-up issue on release or it could be nothing. There
>>
>> That's an example where an Aurora-specific forum might help, if it
>> resulted in the community doing more of the triage. (Easier said than
>> done, I know.)
>>
>
> I'd love to get community involved in feedback gathering in general.
> We've tried, it sometimes works but it's not reliable. We can't
> guarantee that someone has read ALL of the feedback to make sure the
> one report that we need isn't overlooked. We've also had it turn into
> someone only advocating for their pet bug and only finding examples
> that support it.
I was handwaving about something less structured. Right now, there's a
certain chance that any given bit of feedback is important. If someone
with a real problem not only submits feedback but also goes to the
forums, finds other people who can confirm/reproduce it, and one of
them files a new bug for it, then the chance that any given feedback is
important *by itself* goes down, so the cost of ignoring (not being
able to keep up with) feedback is less. People scanning through
feedback can adjust their filters to ignore more questionable cases and
Mozilla still gets the same value out of the incoming flood.
But that's totally theoretical; I have no personal experience with
wading through the feedback storm, so I don't really know what it's
like. I am grateful for those of you brave enough to do it.