Intent to Continue Experimenting: Serial API

[Reilly Grant]

Design docs

https://github.com/WICG/serial/blob/gh-pages/EXPLAINER.md

TAG review (in progress)

https://github.com/w3ctag/design-reviews/issues/431

Summary

Adds an API for communicating with hardware devices over a physical or virtual serial port.

Link to “Intent to Prototype” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/GcqEnSW5yHs/r7G3iMmDCQAJ

Risks

Interoperability and Compatibility

Gecko: Negative

WebKit: Negative

Web developers: Strongly positive

Developers have provided strongly positive feedback on the Origin Trial and have helped identify bugs and missing features which will be available in this extended trial. As an example application, Vaadin has published a data plotter component using this API. 

Goals for experimentation

The goal of continuing this experiment is to collect developer feedback on changes that are landing in Chrome 86, including:

• Discarding buffered data is now possible by calling cancel() and abort() on the readable and writable attributes of the SerialPort object. (crbug/989653)

• Waiting for buffered data to be written is now possible by calling close() on the writable attribute of the SerialPort object. (crbug/989656)

• The spelling of many dictionary fields has been changed to standardize on lowerCamelCase for multi-word names and to avoid acronyms. (crrev/c/2347375)

Experimental timeline

This experiment will begin with the stable-channel release of Chrome 86 and end shortly before the stable-channel release of Chrome 89.

Reason this experiment is being extended

Experimentation with this API began in Chrome 80 and was extended in Chrome 83 due to the cancelation of Chrome 82 because of COVID-19 and to allow developers to experiment with new device filtering and connection event interfaces. Without extension the current Origin Trial will expire on September 29, 2020, shortly before the stable release of Chrome 86. Further changes the APIs (described above) are landing in Chrome 86 for which we would like to collect developer feedback and experience.

Ongoing technical constraints

This feature will be available on all desktop platforms (Windows, macOS, Linux and Chrome OS). It will not be available on Android or Android WebView because the Android platform SDK does not include support for serial ports. Android support for USB-based serial ports is possible using the WebUSB API and a polyfill. The permissions UX will be essentially the same for either API. Work is also in-progress to add support for connecting to Bluetooth Classic Serial Port Profile (SPP) devices using this API. This will enable an implementation of this API for Android which only provides access to Bluetooth devices.

Debuggability

Developers are able to debug script using this feature using the existing DevTools console and debugger. For debugging device communication issues it is often convenient to use the readable.tee() method to split the streams going to or from the device and print them to the console for inspection.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No, see above.

Is this feature fully tested by web-platform-tests?

No, but tentative tests for the feature are available in the wpt_internal/serial directory and can be exported to the Web Platform Tests repository as they mature.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6577673212002304

This intent message was generated by Chrome Platform Status and cleaned up by hand.

Reilly Grant | Software Engineer | rei...@chromium.org | Google Chrome

[Yoav Weiss]

On Wed, Aug 12, 2020 at 11:30 PM Reilly Grant <rei...@chromium.org> wrote:

Design docs

https://github.com/WICG/serial/blob/gh-pages/EXPLAINER.md

TAG review (in progress)

https://github.com/w3ctag/design-reviews/issues/431

Summary

Adds an API for communicating with hardware devices over a physical or virtual serial port.

Link to “Intent to Prototype” blink-dev discussion

https://groups.google.com/a/chromium.org/d/msg/blink-dev/GcqEnSW5yHs/r7G3iMmDCQAJ

Risks

Interoperability and Compatibility

Gecko: Negative

WebKit: Negative

Web developers: Strongly positive

Developers have provided strongly positive feedback on the Origin Trial and have helped identify bugs and missing features which will be available in this extended trial. As an example application, Vaadin has published a data plotter component using this API. 

Goals for experimentation

The goal of continuing this experiment is to collect developer feedback on changes that are landing in Chrome 86, including:

• Discarding buffered data is now possible by calling cancel() and abort() on the readable and writable attributes of the SerialPort object. (crbug/989653)

• Waiting for buffered data to be written is now possible by calling close() on the writable attribute of the SerialPort object. (crbug/989656)

• The spelling of many dictionary fields has been changed to standardize on lowerCamelCase for multi-word names and to avoid acronyms. (crrev/c/2347375)

Experimental timeline

This experiment will begin with the stable-channel release of Chrome 86 and end shortly before the stable-channel release of Chrome 89.

Reason this experiment is being extended

Experimentation with this API began in Chrome 80 and was extended in Chrome 83 due to the cancelation of Chrome 82 because of COVID-19 and to allow developers to experiment with new device filtering and connection event interfaces. Without extension the current Origin Trial will expire on September 29, 2020, shortly before the stable release of Chrome 86. Further changes the APIs (described above) are landing in Chrome 86 for which we would like to collect developer feedback and experience.

The length of the requested OT (M80-M89, even if we discount M82 that never happened) starts to be concerning.

At the same time, trying out changes that resulted from OT feedback is definitely something we want to encourage.

What's the level of risk for this API to be baked in? Did the above changes require changes from developers participating in the trial? (the last point seems to suggest so, but I don't know how used those fields are)

How sad would participating properties be if the API became unavailable for some period of time?

Otherwise, can you share developer feedback from the OT so far?

 

Ongoing technical constraints

This feature will be available on all desktop platforms (Windows, macOS, Linux and Chrome OS). It will not be available on Android or Android WebView because the Android platform SDK does not include support for serial ports. Android support for USB-based serial ports is possible using the WebUSB API and a polyfill. The permissions UX will be essentially the same for either API. Work is also in-progress to add support for connecting to Bluetooth Classic Serial Port Profile (SPP) devices using this API. This will enable an implementation of this API for Android which only provides access to Bluetooth devices.

Debuggability

Developers are able to debug script using this feature using the existing DevTools console and debugger. For debugging device communication issues it is often convenient to use the readable.tee() method to split the streams going to or from the device and print them to the console for inspection.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No, see above.

Is this feature fully tested by web-platform-tests?

No, but tentative tests for the feature are available in the wpt_internal/serial directory and can be exported to the Web Platform Tests repository as they mature.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6577673212002304

This intent message was generated by Chrome Platform Status and cleaned up by hand.

Reilly Grant | Software Engineer | rei...@chromium.org | Google Chrome

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEmk%3DMbpi-DgCtYn4Zvz9iaK6NqmEyHpmxNmPqpE4HdR5pGmAg%40mail.gmail.com.

[Thomas Steiner]

How sad would participating properties be if the API became unavailable for some period of time?

Very sad (also check the surrounding tweets). It is to be noted that this particular case was paired with a misunderstanding of how origin trials worked (which we have addressed in the PR), but they were not alone with this perception. One solution would be to reward good origin trialers. 

[Yoav Weiss]

Hey Henrik!! :)

On Thu, Aug 13, 2020 at 6:05 PM <hjor...@gmail.com> wrote:

Hey Yoav/Thomas/Reilly, long time no see! :)

Just to add some more to what Thomas said... yeah. Would I would be very sad if it went away.

Specifically, we have a couple of customers on AnesthesiaCharting.com using web serial API to be able to communicate with patient vitals monitors for charting purposes.

That's a great use-case!

 

My main points:

1. This was absolutely my fault for not understanding that there *was* a planned gap between OT and shipping.

To be clear, I don't think a *mandatory* gap between OT and shipping is always helpful, and have supported cases where such a gap was not enforced.

What I'm trying to evaluate here is the risk of the API being baked in if the OT were to continue for 9 consecutive milestones.

A gap between OTs would be one way to break such bake-in, but I also understand that it will cause a lot of user pain, so I'm trying to balance those concerns.

 

If I had understood this, I would not have shipped it to any real users. (which incidentally, would have decreased volume/value of OT feedback).
2. I took that gamble, because I knew I had a fallback... and I made disclaimers to customers that it *could* break or go away. 
3. The fallback I have sucks: they have to install our electron app instead (that uses node-serialport).
4. It sucks to have to contact customers that are using it directly in chrome and tell them they have to download our electron app for a bit, only to tell them a few weeks later that they don't need it anymore. I would love to avoid that... changing user habits is hard, and incurs a cost in their goodwill toward the product. For reasons i outlined here: https://joreteg.com/blog/project-fugu-a-new-hope I'd much rather keep them in the PWA instead of Electron.

Anyway, thanks for considering all this stuff. I know it's hard to balance interests and really appreciate the work you all are doing to add these badass APIs to the web <3

Thanks for the detailed explanation, your feedback on the OT and for pushing PWAs to the limits of the platform (and beyond)!! :)

  

- Henrik

[Domenic Denicola]

It's great to see this discussion happening, and thanks Henrik for that valuable web developer feedback!

My main concern about losing the breakage period is that it would prevent us from making changes---in particular, small changes. I think everyone agrees that if the API is fatally flawed, or needs a major overhaul, then it'll get fixed before graduating from origin trial to shipping. But another important purpose of origin trial is to help us refine API sharp edges. For example, as the serial API has progressed through spec review, Reilly has agreed to change a variety of option and property names to be less cryptic and more aligned with web camelCase conventions.

Are these changes super-important? No. Would we break customers of a stable API in this way, for such small benefits? Absolutely not. But I think it's valuable to make these sorts of changes during origin trial, and I'm worried that if we want to avoid the mandatory total-breakage period, we'll also bias toward avoiding these sorts of small-breakages in the API surface, and the final shipping API will be worse for it.

Reilly's plan is for the new origin trial to include such a small breakage, which is encouraging. He was not biased away from making such changes. I'm curious to hear from Henrik how he sees such small changes. Are they something you can adapt to easily? Are they as bad as the mandatory breakage period? Probably somewhere in between?

There's probably also a question as to how to communicate such small breakages to current origin trial users. Would it have been clear to Henrik and other customers that Chrome 83 and Chrome 84's APIs will be slightly different? This might necessitate more directed outreach to all the email addresses who have signed up for (and renewed) origin trial tokens.

[hjor...@gmail.com]

Hey Yoav/Thomas/Reilly, long time no see! :)

Just to add some more to what Thomas said... yeah. Would I would be very sad if it went away.

Specifically, we have a couple of customers on AnesthesiaCharting.com using web serial API to be able to communicate with patient vitals monitors for charting purposes.

My main points:

1. This was absolutely my fault for not understanding that there *was* a planned gap between OT and shipping. If I had understood this, I would not have shipped it to any real users. (which incidentally, would have decreased volume/value of OT feedback).

2. I took that gamble, because I knew I had a fallback... and I made disclaimers to customers that it *could* break or go away. 
3. The fallback I have sucks: they have to install our electron app instead (that uses node-serialport).
4. It sucks to have to contact customers that are using it directly in chrome and tell them they have to download our electron app for a bit, only to tell them a few weeks later that they don't need it anymore. I would love to avoid that... changing user habits is hard, and incurs a cost in their goodwill toward the product. For reasons i outlined here: https://joreteg.com/blog/project-fugu-a-new-hope I'd much rather keep them in the PWA instead of Electron.

Anyway, thanks for considering all this stuff. I know it's hard to balance interests and really appreciate the work you all are doing to add these badass APIs to the web <3

- Henrik

On Thursday, August 13, 2020 at 1:19:17 AM UTC-7, Thomas Steiner wrote:

[Alex Russell]

Hey All,

This was discussed at today's API OWNERS meeting and the consensus was:

• We're happy to see this new OT run given that the API changes would likely break existing users; that is, per Domenic's points, we don't want to get backed into long-running unbreakable "experiments" that aren't. To the extent that this isn't that, LGTM
• There aren't yet clear criteria for a no-breakage launch when this leaves OT. The OWNERS have discussed various bars that could be applied (testimonials like Henrik's, etc.) in addition to published OT results, but without explicit criteria we don't wan't to commit to a no-breakage launch here. Hopefully we can use the time created by this next OT period to sort that out and create a process for evaluating appeals of this form.

Regards

[Reilly Grant]

Thank you Alex.

I think Henrik has articulated the challenges of using the Origin Trial framework when developing an API like this. As I highlighted in the "risks" section of the original Intent to Experiment (which seems to have disappeared from the current template) this API is enabling a new category of applications to be built for the web platform and so there aren't good options for fallback if the Origin Trial ends without shipping.

The alternative to running an Origin Trial is to encourage developers to build against the API while it is behind a flag. To a large extent I think this is actually reasonable for an API like this. Looking at the project I've seen using this API many haven't bothered signing up for Origin Trial keys because they don't consider asking users to flip a flag to be a blocker. To me, taking this feature into the Origin Trial phase is an extra signal to developers that now is the time to try it out and provide feedback. It also provides us with a better communication channel with developers though as Henrik mentions we could use this better to keep them up to date with breaking changes.

There are two aspects to "burn in". One is that we would settle for a suboptimal design because there were too many developers relying on a previous version. I think that this update proves that this isn't the case. I've seen other Origin Trials attempting to smooth over the transition for participating developers but I don't think we've ever forced ourselves to ship something we didn't like just because of burn-in due to an Origin Trial.

The other aspect is the question of whether we would choose not to ship this API at all at the conclusion of the trial. Giving developers a chance to see what they could do with a capability makes this a more difficult decision which clearly gets more difficult the longer the feature is allowed to remain in the experimental state. We have a similar challenge with flags throughout the Chrome codebase, as developers and users alike tend to begin to rely on them despite our protestations that they are unsupported.

The fact that this Origin Trial has gone on so long is mostly a reflection of my ability to devote time to development of the API. There have been many other demands on my time over the last 6 months. I was perhaps overeager to start the Origin Trial in order to get something into developers' hands and so launched before we were close enough to being in a ship-ready state.

Reilly Grant | Software Engineer | rei...@chromium.org | Google Chrome

[Thomas Steiner]

Incompatible API changes are always going to consume developer time, even during the OT phase. A good way to deal with this from an OT point of view was IMHO demonstrated by the Native File System API, where the spec authors published a changes document. Documents like this could be surfaced at the OT token renewal process. 

For highly attractive OTs like Native File System, there may be some value in easing the time from OT to final API by means of pony- or polyfill libraries (if possible). We have seen this work with browser-nativefs (used by OpenChakra, Excalidraw [full disclosure: due to my PR], pwa-inking, Paint, and others). From a DevRel perspective, we see a lot of demand for cross-platform compatibility, i.e., the need for fallback solutions for non-adopting browsers.

For cases like Web Serial, where no fallback is possible due to the API enabling completely new use cases, we probably could incentivize more OT signups by establishing a policy where developer tokens remain active even during the breakage gap, especially since these developers have been on the (potentially API surface changing) OT for a while, so would probably not care much to make some final adjustments to their code to match the shipping API. 

[Henrik Joreteg]

Hey Dominic!

> I'm curious to hear from Henrik how he sees such small changes. Are they something you can adapt to easily? Are they as bad as the mandatory breakage period? Probably somewhere in between?

Changes like this are precisely what I thought I was signing up for with the OT, they are expected and totally OK. Even if they were not easy to adapt to that is OK too because it's a bit like running a preview version of the APIs... it's part of the package!

Ideally, there'd be some communication about the changes to the OT APIs to those who have active origin trials. Even if it was just a: "hey, you're part of the OT for Web Serial there are changes shipping in Chromium version X, beware these changes could break your app: {{ link to changeset }}"

I think the approach of turning and API "off" before shipping is much less problematic when it's not something like web serial, which completely changes the types of applications you can build. It's hard to gracefully fall back from that. If there's a period where the API is disabled, I will have to go tell my customers to download our Electron app.  That's the only viable fallback (I haven't been able to use WebUSB as a reliable fallback because various USB/Serial adapter cables have different chips).

And... if I tell them to go install the app, very likely they won't go back to PWA later.

I think it's safe to say we all want them on PWA.