Thoughts on reviewing specs before I2S?

[Jeffrey Yasskin]

Hi spec mentors,

In https://groups.google.com/a/chromium.org/g/blink-api-owners-discuss/c/uFU2j8WSXNg/, I suggested that we should put someone in charge of checking that our specifications are high-quality before we ship features based on them. Rick Byers suggested "How about for intents referencing a new (or substantially overhauled) spec, we ask for a one-line spec maturity summary from someone who has done a review? We could link to our spec mentorship program as one such venue for getting that, and can internally say it's required for Googlers if we want."

Before signing this group up to provide these reviews, I wanted to check with this group. Up to now, the amount of review we've done as mentors has varied depending on what the feature team asked for: if a team didn't think they needed a lot of review, we might not be very involved at all. This change would mean that anyone who signs up to mentor a feature with a new specification should also expect to review and eventually say something about the quality of the specification. Is that ok with people?

I also want to make sure that everyone gets credit for this extra work. For Googlers, this means I'll email the managers of active spec mentors at performance review time, and we could look into making this count as a sheriffing rotation or similar. If non-Googlers start volunteering, I'll have to see how they want to get credit.

Let me know what you think,

Jeffrey

[Reilly Grant]

When I was mentoring for the WebHID specification we accidentally shipped before all the specification work was done, which got interpreted as evidence we were acting in bad faith on standards position threads. I'm supportive of having API OWNERS ask us to check our work to prevent this from happening in the future.  

Reilly Grant | Software Engineer | rei...@chromium.org | Google Chrome

--
You received this message because you are subscribed to the Google Groups "spec-mentors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spec-mentors...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/spec-mentors/CANh-dXkHBmpdtf8NUhaxEsG5O0aVOfBr%2BMYSR%2BNWDSTyaUF-yQ%40mail.gmail.com.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[Joshua Bell]

SGTM

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/spec-mentors/CAEmk%3DMZxEUADKgDmytqrGSLE4szhkmn2J2sNBh6vFGxyK3%2BkGg%40mail.gmail.com.

[Johann Hofmann]

I like the idea of setting clearer expectations around what spec mentors should (or should not) do, so that idea SGTM overall. As mentioned elsewhere before, I want to generally caution against using voluntary spec mentors to compensate (or be a canary) for systemic issues (e.g. tight deadlines that cause teams to push out incomplete work). We should make sure that all participants in the process (teams, mentors, API Owners) have a common understanding of the expected spec quality from the start to avoid pushing the difficult conversations from blink-dev to individual mentors.

Cheers,

Johann

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/spec-mentors/CAD649j6GTGE6Xv7YPj-tRy0DWh%3DAZ%2B24Cf6-Gu0bHy-si79zjw%40mail.gmail.com.

[Yoav Weiss]

It might be good to run a survey and see how much bandwidth spec mentors can dedicate to this, and/or maybe run a smaller scale experiment before switching the process over.

I think it can be extremely useful for our spec quality, but it's the spec mentors who'd have (at least in some cases) to significantly increase their time investment.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/spec-mentors/CAD_OO4hQdhztKwmtFkR3SGje%2BZOcda7xPdUc17q2R1RhTr_Mkw%40mail.gmail.com.

[Dominic Farolino]

I'm definitely supportive, and +1 to Yoav's comments. Separately, I'm trying to understand @Jeffrey Yasskin's comment about a potential "rotation" program. I think there are two cases here:

1. Features that have a spec mentor: In these cases, the mentor would be involved in spec quality reviews as we approach ship time, even if they otherwise had very low involvement throughout its development
2. Features that don't have a spec mentor: This case is probably more rare since the intersection of Chromies and Googlers is quite large, and all Googlers shipping features have to have a spec mentor I believe. This small corner seems to be the only case where a "rotation" seems necessary, as there is not already a mentor assigned.

Is my analysis correct? It seems the proposal is basically asking to secure a more complete quality review for the average case (1), and perhaps a new rotation-like thing for (2)?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/spec-mentors/CAL5BFfXA5Vm6-CfG7Ue08A1RxxntWTgTfBjhyfVNzL3eLLVPLg%40mail.gmail.com.

[Jeffrey Yasskin]

I don't mean to suggest that we should define a formal spec-mentor rotation, but rather that the benefit to the team from an active mentor is similar to the benefit from a sheriffing rotation, so people should get similar credit for the work. It's possible the (Google-internal) sheriffing wg will disagree, or want us to define a formal rotation in order for it to count, but we won't know unless we have that conversation.

[Jeffrey Yasskin]

I started putting a survey together, at https://docs.google.com/forms/d/1biSjbPv6rq3Rd2lcbPGSbhnCANrMWBHiKMmMhUj5nCw/edit. Does that capture the information you think we should gather?

I also wrote up the change to the mentorship instructions that I think would go with this, at https://chromium-review.googlesource.com/c/website/+/4453886 / https://chromium-website--cl4453886-ps1-xm8zil2g.web.app/blink/spec-mentors/#reviewing-the-specification. Do you think the guidance there is clear enough? This doesn't include the matching change to the launch process, but I'm leaning toward including both in the same CL.

I don't think we'd be using volunteers to compensate for systemic issues: the systemic issue here is that we haven't documented all of the spec-quality expectations or given anyone the responsibility for checking those expectations, not that teams are generally too rushed to meet a bar they know about. I think part of rolling this out will be fleshing out the documentation like in the above CL, and I'll need to make that documentation clear that it's the API owners who will have any difficult conversations that are necessary because a spec mentor discovered that a specification wasn't good enough.

On the "volunteer" aspect, all of our active spec mentors work on the web platform full time, so work to help other teams ship their features is within our job descriptions, just like keeping the Chromium tree green is part of our core responsibilities. What I'm trying to ask in this thread is how we can make sure that everyone is actually recognized for this work. I gave some guesses in my initial post, but I want to make sure that everyone's comfortable those guesses will actually work.