Formalizing signals from web and framework developers in an intent-to-ship

[Thomas Steiner]

Esteemed API Owners,

Similar to the Signals from other Implementers in an Intent-to-Ship, I wanted to formalize the Signals from Web / Framework Developers (🆕.) The core contribution of this doc is to define:

• ① what exactly counts as support, with a special focus on…
• ② libraries that (potentially partly) already do what the feature proposes as a very strong positive signal.

The rationale is to make the relevant section of ChromeStatus (screenshot below) more actionable, and to help DevRel with tangible interaction points to grow feature adoption.

To make this last point ② clearer, here is an example: the Screen Wake Lock API allows the screen to stay awake. The same functionality was implemented as a hack in NoSleep.js by tricking the browser into requesting a media wake lock for an invisible video that the library plays in the background. DevRel has PR'ed native Screen Wake Lock support into NoSleep.js, so upon the next update of the library its users will magically just use the new feature.

For feature authors, identifying and finding such libraries as part of the process can thus—with the help of DevRel—be a good recipe for potentially gaining developer uptake by replicating the example above. It is well noted that not all features will be a good candidate for ②, but in such cases the added clarity of ① is hopefully still helpful in providing evidence of developer support.

Looking forward to hearing your opinions.

Cheers,

Tom

[Thomas Steiner]

(Friendly ping, since this was originally sent on US election day and may have been lost…)

[Yoav Weiss]

Apologies for the delay responding to this... Busy couple of weeks :/

Thanks for putting this doc together! I left a couple of comments, but I think my main issue with it is the following.

I think that more often than not, Chromium engineers are not sure how to get a signal from developers and opt for neutral/no-signal without actually doing the outreach.

That may change if we send them to chromiu...@chromium.org for help, but I'd then expect most of them to not do much more than to reach out, and expect a response that includes the signals acquired.

In the past year we've seen ~190 related intents (I2S+I2E).

I think we need to ask ourselves what would happen if this process would prove to be successful:

• Cost
• Web DevRel folks would then need to triage and provide feedback on O(200) intents a year, roughly 1 per working day
• If Twitter polls are used for that purpose, won't that result in crowd fatigue? ("ugh, another poll from @chromiumdev...")
•  Benefit
• We would get significantly higher-quality signals regarding developers and their needs
• We may be able to avoid shipping some unnecessary APIs and put them through more scrutiny (through OTs, etc)

Maybe there's a middle-ground path with some more guidance on how Chromium engineers can do some of that work themselves, to make the above more sustainable.

As is, I'm concerned about the "cost" part (even if excited about the "benefits" part, if we can pull this off).

Thoughts?

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrLnBcJ8-qGv1r61Cj9zqGVpttX2ij%2BPVrTwH7DSpb5GHNg%40mail.gmail.com.

[Thomas Steiner]

Hi Yoav, all,

These are very good thoughts! Let me respond inline…

On Wed, Nov 18, 2020 at 10:57 AM Yoav Weiss <yoav...@google.com> wrote:

Apologies for the delay responding to this... Busy couple of weeks :/

Thanks for putting this doc together! I left a couple of comments, but I think my main issue with it is the following.

I think that more often than not, Chromium engineers are not sure how to get a signal from developers and opt for neutral/no-signal without actually doing the outreach.

That may change if we send them to chromiu...@chromium.org for help, but I'd then expect most of them to not do much more than to reach out, and expect a response that includes the signals acquired.

In the past year we've seen ~190 related intents (I2S+I2E).

I think we need to ask ourselves what would happen if this process would prove to be successful:

• Cost
• Web DevRel folks would then need to triage and provide feedback on O(200) intents a year, roughly 1 per working day

Part of the exercise is to understand at what stage of the process this is needed and for what kind of features to ask for it. Not all I2{S,E} are the same, some are about deprecating features, some about introducing new features, and some about modifications to existing features. 

• If Twitter polls are used for that purpose, won't that result in crowd fatigue? ("ugh, another poll from @chromiumdev...")

This can absolutely happen if we overdo it. I want to note that Twitter polls would be one tool of several, not everything can be dealt with that way. 

 

•  Benefit
• We would get significantly higher-quality signals regarding developers and their needs

This would be one of the aims, yes. Publicly justifying that there is a recorded need. 

• We may be able to avoid shipping some unnecessary APIs and put them through more scrutiny (through OTs, etc)

This, too. Although some features might be only relevant to a handful of partners at times.

 

Maybe there's a middle-ground path with some more guidance on how Chromium engineers can do some of that work themselves, to make the above more sustainable.

There are definitely examples where teams have identified artifacts like polyfills that I would formally want to include in the list of developer signals. I don't think the whole work would be offloaded to DevRel. 

 

As is, I'm concerned about the "cost" part (even if excited about the "benefits" part, if we can pull this off).

Thoughts?

I'm very much interested in hearing more API owner's thoughts. Thanks in advance!

Cheers,

Tom

On Wed, Nov 11, 2020 at 12:34 PM 'Thomas Steiner' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

(Friendly ping, since this was originally sent on US election day and may have been lost…)

On Tue, Nov 3, 2020 at 10:57 AM Thomas Steiner <to...@google.com> wrote:

Esteemed API Owners,

Similar to the Signals from other Implementers in an Intent-to-Ship, I wanted to formalize the Signals from Web / Framework Developers (🆕.) The core contribution of this doc is to define:

• ① what exactly counts as support, with a special focus on…
• ② libraries that (potentially partly) already do what the feature proposes as a very strong positive signal.

The rationale is to make the relevant section of ChromeStatus (screenshot below) more actionable, and to help DevRel with tangible interaction points to grow feature adoption.

To make this last point ② clearer, here is an example: the Screen Wake Lock API allows the screen to stay awake. The same functionality was implemented as a hack in NoSleep.js by tricking the browser into requesting a media wake lock for an invisible video that the library plays in the background. DevRel has PR'ed native Screen Wake Lock support into NoSleep.js, so upon the next update of the library its users will magically just use the new feature.

For feature authors, identifying and finding such libraries as part of the process can thus—with the help of DevRel—be a good recipe for potentially gaining developer uptake by replicating the example above. It is well noted that not all features will be a good candidate for ②, but in such cases the added clarity of ① is hopefully still helpful in providing evidence of developer support.

Looking forward to hearing your opinions.

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrLnBcJ8-qGv1r61Cj9zqGVpttX2ij%2BPVrTwH7DSpb5GHNg%40mail.gmail.com.

--

Thomas Steiner, PhD—Developer Advocate (https://blog.tomayac.com, https://twitter.com/tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.1.23 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck0fjumBl3DCharaCTersAttH3b0ttom.hTtPs://xKcd.cOm/1181/
-----END PGP SIGNATURE-----

[Thomas Steiner]

[Adding in Paul and Robert following a discussion with the former.]

On Tue, Nov 3, 2020 at 10:57 AM Thomas Steiner <to...@google.com> wrote:

[Yoav Weiss]

Thanks!

I'd particularly love Paul's opinions on https://groups.google.com/a/chromium.org/g/blink-api-owners-discuss/c/tVzdDwXNy_0/m/XDXghKrNAwAJ and the cost/benefit calculus question.

--

You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrL%3Djfyd%2B2HUkByS1_WN-gPJ3b_prvyVviRCQr%2B7NduG72A%40mail.gmail.com.

[Robert Nyman]

+Philip Jägenstedt +Chris Harrelson who I am sure will have more thoughts. 

Thanks for adding me, Tom!

This is a really interesting area, and with DevSAT work we have been discussing this a lot in terms of survey results and data (MDN surveys and our quarterly ones) together with starring bugs in crbug. One more challen which has come up is Stack Overflow, which might be worth adding to the doc.

Tangentially, we (Chrome DevRel + Engineering) also met with Mozilla Engineering last week to discuss developer pain point signals – meeting notes.

The main takeaways were:

• Bugs filed by another implementer are high quality, are important and should be prioritized

• Best ways to detect developer pain:

• Duplicate bugs

• Survey results

• Stack overflow results

• Starring for crbug

• We should collaborate on fixing main areas at the same time. Suggesting CSS Flexbox and CSS Grid, and communicating publicly about this, both to get good input from developers, and also make them aware this area is being prioritized

• We should collaborate on new major features. Container queries seem to be a good one, and based on Mozilla resource constraints, Google can do some of the initial heavy lifting work

On a final note, I also started work on DevSAT and survey options for 2021 and thinking there might be good overlap here in terms of detecting developer signals.

Would be happy to talk more about it!

– Robert

[Paul Kinlan]

For me, I think the piece is DevRel should be involved in the critical initiatives and DevRel should be able to help get actionable feedback back to the engineering teams to assess the needs and perspectives of developers throughout the lifecycle of an API's development. For the API's that are not in the WP "critical initiatives" focus, then I would expect that Blink should have a consistent process that DevRel should help run (we have an OKR for that) that any team can use to gather feedback either at scale, or from a representative set of developers to help us justify the decisions that are being made in the design of the platform.

[Thomas Steiner]

Responses to both Robert's and Paul's mails inline…

On Mon, Nov 23, 2020 at 1:18 PM Paul Kinlan <paulk...@google.com> wrote:

For me, I think the piece is DevRel should be involved in the critical initiatives and DevRel should be able to help get actionable feedback back to the engineering teams to assess the needs and perspectives of developers throughout the lifecycle of an API's development. For the API's that are not in the WP "critical initiatives" focus, then I would expect that Blink should have a consistent process that DevRel should help run (we have an OKR for that) that any team can use to gather feedback either at scale, or from a representative set of developers to help us justify the decisions that are being made in the design of the platform.

I fully agree on what you said, my only question is your definition of "critical" in the above paragraph? Who would decide whether a certain I2S touches upon critical Web Platform features? 

 

On Mon, Nov 23, 2020 at 11:26 AM Robert Nyman <ny...@google.com> wrote:

+Philip Jägenstedt +Chris Harrelson who I am sure will have more thoughts. 

Thanks for adding me, Tom!

This is a really interesting area, and with DevSAT work we have been discussing this a lot in terms of survey results and data (MDN surveys and our quarterly ones) together with starring bugs in crbug. One more challen which has come up is Stack Overflow, which might be worth adding to the doc.

Added StackOverflow. Thanks! 

 

Tangentially, we (Chrome DevRel + Engineering) also met with Mozilla Engineering last week to discuss developer pain point signals – meeting notes.

The main takeaways were:

• Bugs filed by another implementer are high quality, are important and should be prioritized

This wasn't fully clear to me. Does it refer to the Edge comment that their "everyone can report bugs" feature results in low quality bugs?

• Best ways to detect developer pain:

• Duplicate bugs

Which is a signal also used by Apple.

 

• Survey results

Absolutely, thanks for the hard work you all are doing in this area! 

 

• Stack overflow results

Agreed.

 

• Starring for crbug

It's a good signal, but also an easily "gameable" one. @Kent Tamura probably has more insights in how far this is actually a problem.

 

• We should collaborate on fixing main areas at the same time. Suggesting CSS Flexbox and CSS Grid, and communicating publicly about this, both to get good input from developers, and also make them aware this area is being prioritized

Yes.

 

• We should collaborate on new major features. Container queries seem to be a good one, and based on Mozilla resource constraints, Google can do some of the initial heavy lifting work

Would love to see us collabore as early as possible. And I think we do.

[Robert Nyman]

• Bugs filed by another implementer are high quality, are important and should be prioritized

This wasn't fully clear to me. Does it refer to the Edge comment that their "everyone can report bugs" feature results in low quality bugs?

Basically, both Chrome and Firefox Engineers agreed that bugs from other implementers were high quality.

The Edge comment was a separate thing, namely if we lower the bar for reporting bugs, we also get lower overall quality and too much work triaging.

• Best ways to detect developer pain:

• Duplicate bugs

Which is a signal also used by Apple.

Good point!

It would be nice to have a list with commonalities across bug reports for engines for what are strong and good signals.

• Starring for crbug

It's a good signal, but also an easily "gameable" one.  @Kent Tamura probably has more insights in how far this is actually a problem.

Yeah, that is a risk. For things raised in the MDN surveys as the biggest issues, we saw a strong correlation with star count. 

That said, itäs definitely worth exploring more, and also how we combine signals. E.g. dupes + starts + …

 

• We should collaborate on new major features. Container queries seem to be a good one, and based on Mozilla resource constraints, Google can do some of the initial heavy lifting work

Would love to see us collabore as early as possible. And I think we do.

Yeah, there's some collaboration, for sure. This was also about identifying which features to prioritize and possibly work closer there.

Specific example: Firefox are working on masonry layouts where we think container queries should probably be the highest importance (and we are planning on how to properly raise this to them).

– Robert

 

 

[TAMURA, Kent]

On Tue, Nov 24, 2020 at 7:44 PM Robert Nyman <ny...@google.com> wrote:

• Bugs filed by another implementer are high quality, are important and should be prioritized

This wasn't fully clear to me. Does it refer to the Edge comment that their "everyone can report bugs" feature results in low quality bugs?

Basically, both Chrome and Firefox Engineers agreed that bugs from other implementers were high quality.

The Edge comment was a separate thing, namely if we lower the bar for reporting bugs, we also get lower overall quality and too much work triaging.

• Best ways to detect developer pain:

• Duplicate bugs

Which is a signal also used by Apple.

Good point!

It would be nice to have a list with commonalities across bug reports for engines for what are strong and good signals.

• Starring for crbug

It's a good signal, but also an easily "gameable" one.  @Kent Tamura probably has more insights in how far this is actually a problem.

Yeah, that is a risk. For things raised in the MDN surveys as the biggest issues, we saw a strong correlation with star count. 

That said, itäs definitely worth exploring more, and also how we combine signals. E.g. dupes + starts + …

If an issue in crbug.com is merged to another issue, the number of stars of the former issue is added to the number of stars of the latter issue. So we might not need to take a look at the number of dupes.

The number of stars can be gameable, however I have never seen issues which have a huge number of stars but look unimportant.

My rough impression is that:

  - 10+ stars: strong signal

  - 5+ stars: positive

--

TAMURA, Kent
Software Engineer, Google

[Thomas Steiner]

On Thu, Nov 26, 2020 at 5:20 AM TAMURA, Kent <tk...@google.com> wrote:

On Tue, Nov 24, 2020 at 7:44 PM Robert Nyman <ny...@google.com> wrote:

• Bugs filed by another implementer are high quality, are important and should be prioritized

This wasn't fully clear to me. Does it refer to the Edge comment that their "everyone can report bugs" feature results in low quality bugs?

Basically, both Chrome and Firefox Engineers agreed that bugs from other implementers were high quality.

The Edge comment was a separate thing, namely if we lower the bar for reporting bugs, we also get lower overall quality and too much work triaging.

• Best ways to detect developer pain:

• Duplicate bugs

Which is a signal also used by Apple.

Good point!

It would be nice to have a list with commonalities across bug reports for engines for what are strong and good signals.

• Starring for crbug

It's a good signal, but also an easily "gameable" one.  @Kent Tamura probably has more insights in how far this is actually a problem.

Yeah, that is a risk. For things raised in the MDN surveys as the biggest issues, we saw a strong correlation with star count. 

That said, itäs definitely worth exploring more, and also how we combine signals. E.g. dupes + starts + …

If an issue in crbug.com is merged to another issue, the number of stars of the former issue is added to the number of stars of the latter issue. So we might not need to take a look at the number of dupes.

The number of stars can be gameable, however I have never seen issues which have a huge number of stars but look unimportant.

My rough impression is that:

  - 10+ stars: strong signal

  - 5+ stars: positive

Thank you, this is very helpful! I have added the "star" signal explicitly in the doc and left a question there for you.

[Robert Nyman]

The number of stars can be gameable, however I have never seen issues which have a huge number of stars but look unimportant.
My rough impression is that:
  - 10+ stars: strong signal
  - 5+ stars: positive

Great, thanks for clarifying! 

[Chris Harrelson]

I'm supportive of asking intent owners to find signals from developers, and enumerating the kinds of signals we think are useful.

I also think it's reasonable to consider requiring such signals as part of shipping an API - i.e. show evidence that the API matters to developers. If we ship an API, we should be able to show that it has customers.

--

You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAA1RLOF0HORXLBQ3rwyKo%2BCPc5%3DKLAcyQc5pVajhfqKq%3D739tw%40mail.gmail.com.

[Thomas Steiner]

Thanks for your comments, Chris. I have incorporated them in the doc.

On Thu, Dec 17, 2020 at 3:00 AM Chris Harrelson <chri...@chromium.org> wrote:

I'm supportive of asking intent owners to find signals from developers, and enumerating the kinds of signals we think are useful.

Happy to hear that! I have just invited you to be an editor, so you can remove the "non-public draft" banner from the doc and do whatever needs to be done to make this an official step of the process. Probably needs to be re-published under a Chromium account to allow for public sharing?! Please go ahead and do so.

 

I also think it's reasonable to consider requiring such signals as part of shipping an API - i.e. show evidence that the API matters to developers. If we ship an API, we should be able to show that it has customers.

I obviously agree, the more proof points we can show, the better.

[Ben Kelly]

For URLPattern I have been thinking about creating a short survey when the API is entering final dev trials and/or close to ship ready.  This could give us a metric like "67% positive out of 542 responses" or something.  It would of course suffer from selection bias of who chooses to fill out the survey, but might be a reasonable signal.

--

You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrLkCjTLZLo1kA9JaS3XtECW-_%2Byc%2BBKoubFqqfJeseFb2g%40mail.gmail.com.

[Thomas Steiner]

On Thu, Dec 17, 2020 at 4:00 PM Ben Kelly <wande...@chromium.org> wrote:

For URLPattern I have been thinking about creating a short survey when the API is entering final dev trials and/or close to ship ready.  This could give us a metric like "67% positive out of 542 responses" or something.  It would of course suffer from selection bias of who chooses to fill out the survey, but might be a reasonable signal.

Absolutely, I love this idea. Acknowledged bias is fine, the problem is when one isn't aware of bias. Thanks for considering this!

[Thomas Steiner]

Hi API owners,

Since we seem to have an agreement I have now moved this document into the Chromium org, made it visible to anyone in the organization, and gave it an easy to remember short link (goo.gle/developer-signals). What now remains would be to link to this document somewhere in ChromeStatus I think. Who could do this?

I'm happy that the recent I2S for permformance.measureMemory() has a brilliant developers signal section (insert "coincidence, I think not" meme here… 🤣). 

Cheers,

Tom   

[Manuel Rego Casasnovas]

On 08/01/2021 12:35, 'Thomas Steiner' via blink-api-owners-discuss wrote:
> Since we seem to have an agreement I have now moved this document into
> the Chromium org

> <https://docs.google.com/document/d/11NlEHOnQ9yZwwMPOU5oupqKmOsRBQDHRNttxNl6Apzk/edit?usp=sharing>,

> made it visible to anyone in the organization, and gave it an easy to
> remember short link (goo.gle/developer-signals

> <https://goo.gle/developer-signals>). What now remains would be to link

> to this document somewhere in ChromeStatus I think. Who could do this?

I'd make the document public, so anyone (even folks without a
@chromium.org) could read it.

In theory you could fill a bug at
https://github.com/GoogleChrome/chromium-dashboard/issues/ to get it
added into chromestatus UI.
Thought we haven't been lucky yet to get https://bit.ly/blink-signals
linked from there (see
https://github.com/GoogleChrome/chromium-dashboard/issues/1051). O:-)

And probably this new document should be linked from
https://bit.ly/blink-signals too, as there's a mention to web developers
signals.

Bye,
Rego

[Thomas Steiner]

I'd make the document public, so anyone (even folks without a
@chromium.org) could read it.

I did that now, but it seems like with my Chromium account I can only "publish to the web", which is different from making a document visible in document mode. It did the job, though, I have updated the short link to point to the public version now.

 

In theory you could fill a bug at
https://github.com/GoogleChrome/chromium-dashboard/issues/ to get it
added into chromestatus UI.
Thought we haven't been lucky yet to get https://bit.ly/blink-signals
linked from there (see
https://github.com/GoogleChrome/chromium-dashboard/issues/1051). O:-)

I piled yet another comment onto the existing issue. They can address both in the same flow.

 

And probably this new document should be linked from
https://bit.ly/blink-signals too, as there's a mention to web developers
signals.

@Chris Harrelson, can you do this? Thanks!

Cheers,

Tom

[Yoav Weiss]

I'm still not 100% clear on what non-Google Chromium contributors are to make of the Obtaining developer signals section. How are non-Google contributors supposed to get help from the Chrome Web DevRel team? Is there a mailing list they can use for that? Seems like it would be good to clarify that part.

[Thomas Steiner]

On Sat, Jan 9, 2021 at 7:51 PM Yoav Weiss <yoav...@google.com> wrote:

I'm still not 100% clear on what non-Google Chromium contributors are to make of the Obtaining developer signals section. How are non-Google contributors supposed to get help from the Chrome Web DevRel team? Is there a mailing list they can use for that? Seems like it would be good to clarify that part.

It's been added at the bottom of the document: "The team can be contacted via chromiu...@chromium.org". For the time being, I'm monitoring this alias.

[Yoav Weiss]

Oh cool! I missed that! :)

[Chris Harrelson]

Re adding to the official process: I'd like to discuss this at our weekly meeting this Thursday at noon PST. I think it's a good idea to add this to the process, but would like to have a brief in-person discussion first. Thomas, if you are able to and would like to attend, I can add you to this instance.

Assuming we make the decision on Thursday, an API owner should announce the result on blink-dev and a date after which this will be enforced, and also to collect any remaining feedback from the community.

[Thomas Steiner]

Hi Chris,

Re adding to the official process: I'd like to discuss this at our weekly meeting this Thursday at noon PST. I think it's a good idea to add this to the process, but would like to have a brief in-person discussion first. Thomas, if you are able to and would like to attend, I can add you to this instance.

I'm happy to attend. Since this is happening late-ish in my local time zone, I'd appreciate if it could be one of the early agenda items. Could you share the calendar invite with me, please?

 

Assuming we make the decision on Thursday, an API owner should announce the result on blink-dev and a date after which this will be enforced, and also to collect any remaining feedback from the community.

Sounds great, thanks!

Cheers,

Tom

[Chris Harrelson]

Done. Your agenda item will be first in line.

[Thomas Steiner]

On Tue 12. Jan 2021 at 17:56 'Chris Harrelson' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

Done. Your agenda item will be first in line.

Perfect, thanks very much for having me in advance!

[Shubhie Panicker]

This is a great addition to the process, thank you!

Wondering if it's worth calling out what a high quality signal looks like as opposed to a drive by twitter response, or ad hoc twitter polls which could have arbitrary signal-to-noise ratio. 

I'd say a high quality signal would be interest from a developer in using the proposed feature in a production website or links to libraries showing existing usage in userland and high star count for bug fixes.

Also I assume "framework" is a proxy for wide adoption, if so considering the popularity of the framework would help.

Also I recognize this is an addition to an already tedious process for web platform engineers, and I'd love to offer help -- my team works on a regular basis with a number of production sites as well as popular frameworks.

I'll follow up to see how we could concretely help at least Google engineers who are implementing standards.

--

You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CALgRrL%3DVxO2wZ5CQYteJAj3GmJQY10z6qVqiH1eSb6-chnW-rg%40mail.gmail.com.

[Thomas Steiner]

Hi Shubie, all,

On Mon, Feb 8, 2021 at 7:05 AM Shubhie Panicker <pani...@google.com> wrote:

This is a great addition to the process, thank you!

Happy to hear that :-D 

Wondering if it's worth calling out what a high quality signal looks like as opposed to a drive by twitter response, or ad hoc twitter polls which could have arbitrary signal-to-noise ratio.

I definitely agree that not all signals are created equal, and formalizing what even counts as a signal was the first step. Read on…

 

I'd say a high quality signal would be interest from a developer in using the proposed feature in a production website or links to libraries showing existing usage in userland and high star count for bug fixes.

I agree with you for the former two (albeit questions remain: "How [bindingly] is the interest expressed?", "How close to the to-be-built API is the userland implementation?"), but the latter needs to be taken with a grain of salt. Someone with a high profile Twitter account asking people to star a given bug can easily skew the bug singal. None of these signals are bad per se, we just need to read each of them factoring in its inherent bias (and this bias is not necessarily direct, as my example with the crbug star signal triggered by a high profile Twitter user showed).

 

Also I assume "framework" is a proxy for wide adoption, if so considering the popularity of the framework would help.

Agreed. Albeit I mostly mentioned "framework" because this is how the signal is called. If it were for me, I'd have called it "Web developer views".

 

Also I recognize this is an addition to an already tedious process for web platform engineers, and I'd love to offer help -- my team works on a regular basis with a number of production sites as well as popular frameworks.

I'll follow up to see how we could concretely help at least Google engineers who are implementing standards.

Greatly appreciated, and yes, the burden on engineering is high! I'm hopeful with the joint effort from your team and DevRel we can smooth the process a bit. 

So concluding, I think for the time being we should not try to "rank" (I know this isn't the word you used, I'm simplifying a bit) signals by quality, but rather try to factor in the aspects I mention above with a common sense reading of those signals. 

Cheers,

Tom