Process survey results

[Jeffrey Yasskin]

Hello Blink-dev!

We took a survey of your impressions of our process and tooling back in February, and I wanted to share the results. This survey was an iteration of a survey conducted in early 2021; where interesting, we’ll compare responses. We only got 22 responses this year, down from 62 last year, so the results are more suggestive than statistically significant.

Background of respondents

• 81% of respondents work on Chromium full-time, down from 95% last year.

• Of the 20 people who work on Chromium as part of their jobs, 4 people identified themselves as working for non-Google companies, about the same as the 30% last year. Unlike last year, nobody identified themself as from Microsoft, only Opera and Intel.

• 81% develop features.

• 59% participate in a standards body.

• 40% are API owners or decide what features to enable in a Chromium-based browser, which overrepresents high-level decision-makers in the survey results.

Key summary points

The people who replied feel better about their understanding of the process than last year:

Most of the more-detailed questions about understanding follow this pattern, with some remaining confusion about how to gather Web Developer Signals, using feedback from Origin Trials, and how the Chromium process interacts with processes within standards bodies. This is better than the understanding last year, although this result could be skewed by the higher proportion of API owners and decision-makers in the responses.

Respondents reported that it's hard to maintain features in Chrome Status, partly because the UI is confusing and not tailored to all of the types of features that people need to create and maintain. This contributes to the information in Chrome Status being out of date. Developers also complained that issues they file don't get fixed promptly.

Many of the respondents are discouraged about TAG review and review by other browser engines. TAG review is considered slow and unhelpful, and one developer said they sometimes feel like a punching bag for other vendors.

Googlers appreciated fast build speeds and Goma but folks complained about flaky tests and CQ reliability. There were some complaints that the infrastructure teams have launched new versions of some tools without addressing the bugs in them.

I've also collected more details and people's text responses in this document.

Jeffrey

[Thomas Steiner]

Thanks, Jeffrey, for sharing.

On "Filling in 'signals' in Chromestatus is the most serious pain point, particularly developer signals. Only 27% of contributors know how to fill in Web Developer Signals.": As the author of goo.gle/developer-signals, I wonder what could be made clearer? Do people read this document? Looking at click stats suggests they don't. 

Now people may access the doc via the direct URL or have it bookmarked, but the hypothesis is that it's not as widely read as it should be. Does it need to be featured more prominently in the process? Or is it one of already too many documents engineers have to go through?

Do people realize DevRel even offers "Developer Signals as a Service" (DSaaS)? Quoting from the doc:

———————————

Obtaining developer feedback

Chromium/Chrome Developer Relations (DevRel) can—no surprise—help with reaching developers. The team can be contacted via chromiu...@chromium.org. For external-facing developer outreach, DevRel has access to the official @ChromiumDev Twitter account—well noting that this may introduce follower bias. DevRel can signal-boost or create polls, and ask people to chime in on conversations happening on GitHub, Discourse, and elsewhere. For more in-depth surveys, the Chrome team is working  with C SPACE to build a dedicated community of about 600 Web developers from a range of backgrounds, experiences, skill sets, and industries who will provide direct feedback to questions that you come up with. The Chrome team also works with a group of volunteer Google Developer Experts (filter the map for “Web Technologies”) who may be able to provide expert feedback. Contact DevRel via the alias above if any of this is of interest and we will set you up. In all cases, a reasonable amount of time should be allowed for responses to this process.

———————————

Really interested in learning how we could improve here.

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANh-dXkk%2Bcm%3Dz940TAOAau%2BpJ0GpXRLFvmHP5HWWBjr4ghEJoQ%40mail.gmail.com.

--

Thomas Steiner, PhD—Developer Relations Engineer (https://blog.tomayac.com, https://twitter.com/tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.3.4 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck0fjumBl3DCharaCTersAttH3b0ttom.hTtPs://xKcd.cOm/1181/

----- END PGP SIGNATURE -----

[Jeffrey Yasskin]

Note that your quote is from last year's summary, while this year 40% said they understood Web Developer signals at least "well enough", and only 10% said "not at all". https://goo.gle/developer-signals is now linked from its field in Chrome Status... 38 clicks in 90 days does seem low, given that we had ~50 intents to ship over that time. Picking 2 arbitrarily, neither hidden=until-found nor MediaCapabilities API has developer signals, but the API owners only complained about it (and pointed to goo.gle/developer-signals) for the latter.

I suspect other people on this list will have better ideas about what would help them understand developer signals better.

Jeffrey

[Thomas Steiner]

On Tue, May 17, 2022 at 6:44 PM Jeffrey Yasskin <jyas...@google.com> wrote:

Note that your quote is from last year's summary

Yes, well understood, just thought I would highlight this specifically…

 

while this year 40% said they understood Web Developer signals at least "well enough", and only 10% said "not at all"

…since even with those newer numbers we're not where we should be. I could have been clearer about that, sorry.

 

https://goo.gle/developer-signals is now linked from its field in Chrome Status...

Which is great.

 

38 clicks in 90 days does seem low, given that we had ~50 intents to ship over that time. Picking 2 arbitrarily, neither hidden=until-found nor MediaCapabilities API has developer signals, but the API owners only complained about it (and pointed to goo.gle/developer-signals) for the latter.

For hidden=until-found, it would have been easy: https://twitter.com/addyosmani/status/1520459804656824320 was retweeted 111 times and liked 487 times. This is evidence. Or https://twitter.com/stefanjudis/status/1519890352785838080, with even higher numbers. Literal search time: <1min: https://twitter.com/search?q=%22hidden%3Duntil-found%22&src=typed_query.

 

I suspect other people on this list will have better ideas about what would help them understand developer signals better.

Finding evidence for developer love gets hairier when a Twitter search doesn't reveal strong signals like in the example above. The document clearly defines what alternatives to Twitter feedback are. My suspicion still holds that the document is simply not as well-known as it could be.

Cheers,

Tom