Finch trials and webexposed tests

[Yoav Weiss]

Hey folks,

The Viewport Height trial is using the Finch infrastructure in order to experiment with this feature, because the feature is aiming to test Client Hints on the navigation request, which is sent before the OT token arrives at the browser and can be validated.

As part of that, they need to enable the feature in the field trial configs, so that the bots would test it as part of the CQ. So, +Max Curran sent me a CL to do that, but a side effect of adding features to the field trial configs is that they are also required to be part of the webexposed expectation files.

I'm hence not sure how to move forward here - it seems bad to enable the not-yet-web-exposed feature as part of the webexposed expectations, but at the same time I'm not sure how to enable the field trial otherwise.

Y'all's thoughts would be appreciated.

Cheers :)

Yoav 

[Ian Clelland]

My understanding here is that for those clients who are in the experiment, these attributes *are* going to be webexposed, so maybe it's not bad to need API_OWNERS review of those changes before they're visible to some users on stable.

However, if that's really not the case, and this is just a finch config for something that is not going to stable (and you just need the field trial config so that it can be enabled in beta) then my read of https://chromium.googlesource.com/chromium/src/+/master/testing/variations/README.md#:~:text=Note%3A%20While%20experiments%20is%20defined%20as%20an%20array%2C%20currently%20only%20the%20first%20entry%20is%20used

suggests that adding another (maybe blank) entry at the start of the "experiments" list would cause that one to be used on the CQ bots. You could still test on the bots with a virtual test suite, but the default for test builds would run without the experiment.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfVFGjb0X1ELA2rN-YfgV2u%2Bfx5BDPCYoUkNWJhbbwFwuw%40mail.gmail.com.

[Yoav Weiss]

On Thu, Jun 2, 2022 at 10:33 PM Ian Clelland <icle...@chromium.org> wrote:

My understanding here is that for those clients who are in the experiment, these attributes *are* going to be webexposed, so maybe it's not bad to need API_OWNERS review of those changes before they're visible to some users on stable.

On the one hand, I agree it's good that an API owner sign off is good for this small %age web exposure. On the other hand, this means that such a sign off will not be required for full web-exposure.

So it's not ideal from my perspective.

 

However, if that's really not the case, and this is just a finch config for something that is not going to stable (and you just need the field trial config so that it can be enabled in beta) then my read of https://chromium.googlesource.com/chromium/src/+/master/testing/variations/README.md#:~:text=Note%3A%20While%20experiments%20is%20defined%20as%20an%20array%2C%20currently%20only%20the%20first%20entry%20is%20used

suggests that adding another (maybe blank) entry at the start of the "experiments" list would cause that one to be used on the CQ bots. You could still test on the bots with a virtual test suite, but the default for test builds would run without the experiment.

On Thu, Jun 2, 2022 at 3:27 PM Yoav Weiss <yoav...@chromium.org> wrote:

Hey folks,

The Viewport Height trial is using the Finch infrastructure in order to experiment with this feature, because the feature is aiming to test Client Hints on the navigation request, which is sent before the OT token arrives at the browser and can be validated.

As part of that, they need to enable the feature in the field trial configs, so that the bots would test it as part of the CQ. So, +Max Curran sent me a CL to do that, but a side effect of adding features to the field trial configs is that they are also required to be part of the webexposed expectation files.

I'm hence not sure how to move forward here - it seems bad to enable the not-yet-web-exposed feature as part of the webexposed expectations, but at the same time I'm not sure how to enable the field trial otherwise.

Y'all's thoughts would be appreciated.

Cheers :)

Yoav 

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfVFGjb0X1ELA2rN-YfgV2u%2Bfx5BDPCYoUkNWJhbbwFwuw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAK_TSXJLoJ9vQ2OiF%2BtPHLj_5ckLDph8q70ZD_i0J0AZY6J2ug%40mail.gmail.com.

[Mike West]

I agree that this isn’t ideal, but it also seems unlikely to be common. Folks generally are able to use the OT infrastructure to do their experiments without relying on default-enabled flags. For those that need it, we can probably do the extra due diligence to ensure that we don’t accidentally ship something, and rely on the team to do the right thing with Finch.

I wouldn’t consider building additional infrastructure to deal with this until it becomes a problem. :)

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfVR9wSR3H3GdjBJMLaDWnJo%2Bex8PLAE%2BzpiM-p1e2sp9w%40mail.gmail.com.

--

-mike

[Yoav Weiss]

OK, if y'all think it's not awful, I'll LGTM while asking the team to be extra careful here! Thanks! :)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAKXHy%3Df_c-%2B3mXAF0NxOCsxfzdQTpqrYMOkzQmva9hx0ah0rrg%40mail.gmail.com.