RFC: Removing the `--load-extension` flag in branded Chrome builds

[Oliver Dunk]

Hi all,

We're looking for feedback on how you use the `--load-extension` flag in Chrome. This allows you to start Chrome with an unpacked extension from the command line.

We know this is used for development, and some popular tooling like web-ext, but it is unfortunately commonly abused to load malicious and unwanted software. As a result, we are evaluating additional restrictions on its usage. We need to do more than restricting this to developer mode as that is unfortunately too easy for software to bypass.

One approach we are considering is restricting this to non-Chrome branded builds like Chromium and Chrome for Testing. Tooling could automatically download these builds to continue supporting specific workflows. We have no plans to remove the "Load Unpacked" button on the chrome://extensions page.

We'd love to know:

• How do you currently use the `--load-extension` flag?
• Would you be able to use Chrome for Testing or Chromium for your use cases?
• Would this change add significant friction to your development, and if so, in what ways?

Even a brief comment is helpful. We'd really like to understand usage in the ecosystem more before we decide on a path forward.

Thanks,
Oliver on behalf of Chrome Extensions DevRel

[Moe Bazzi]

Thanks! Here is my response:

• How do you currently use the `--load-extension` flag?

• with Playwright, using Chromium and Chrome For Testing for automated testing around my chrome extension

• Would you be able to use Chrome for Testing or Chromium for your use cases?

• yes

• Would this change add significant friction to your development, and if so, in what ways?

• no

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/7f8fd3e5-3fd3-4186-bf97-5c96bfff6c52n%40chromium.org.

[Alexei Miagkov]

Privacy Badger uses it in Selenium tests [*] and for scanning the web with Badger Sett, Privacy Badger's training project.

It would be unnacceptable to restrict this ability to a special test-only version of Chromium. We need to test and crawl the web using the same versions of Chrome used by regular Chrome users.

[*] https://github.com/EFForg/privacybadger/blob/9af0fd29ea54dfe31a5e2f11745237c46a02a3e5/tests/selenium/pbtest.py#L166

[Peter Bloomfield]

In principle, we'd be very happy to see more built-in safeguards around the command line flags. We've had to write software to let our customers block some flags which can override enterprise policies, such as "--disable-extensions".

However, disabling the --load-extension flag unilaterally on the standard Chrome release would probably cause us some challenges, and increase risk for our release process. Perhaps that flag could be allowed if Dev Tools are explicitly enabled by enterprise policy?

During development and testing, we only use the normal version of Chrome, not Chrome for Testing. The reason is that we want to be as certain as possible that we're getting the same results as customers would get, in particular around performance. Since migrating to mv3, we've had major problems with inconsistent performance of extension code, so I'd be very wary of switching to a different browser build in case we miss something. (Performance is a critical concern because we're using the web request blocking API -- if our extension runs slowly then it degrades the entire browsing experience for the end user.)

Part of our development and testing workflow involves using our own scripts to load our extension into Chrome via the command line. The scripts do various other things, like allow-listing the extension so it can access enterprise-only APIs, and setting up sandboxed user data directories so we can test different builds side by side. Having to add a manual step to load the extension would certainly cause some friction.

[Vlas Bashynskyi]

Hello, I currently use the --load-extension flag in automated end to end tests for several chrome extensions.
I didn't know Chrome for Testing existed, but it looks like it should be able to cover my use cases.
To reduce friction when doing automated testing - it would be great if Chrome DevTools Protocol had more methods related to extensions, like clicking the action icon, activating a context menu item, or changing allowed websites.
Thank you.

[xmcp]

We used `--load-extension` in a Selenium script to test behaviors related to extensions. Although using Chromium seems okay for this use case, it's unknown whether the Chromium will have the exact same behavior as Chrome stable, so it may cause troubles in the future.

I will suggest keeping this flag while showing a banner when this flag is present (similar to the current "Chrome is being controlled by automated test software" banner).

[Justin Wernick]

Hello,

Like several others, we're using `--load-extension` in Selenium to run automated testing of our extensions.

We use this not only for Chrome itself, but also other Chromium forks like Edge. I don't know how this change would affect Edge, but of course variants like Chrome for Testing and Chromium wouldn't be sufficient to test how Edge behaves.

Kind regards,

Justin Wernick

[Ursa Kacar]

I believe that as long as Chromium keeps the  `--load-extension` flag we should be OK- if the feature is only removed on Chrome, I don't see how that would affect Chromium based Edge, unless Edge browser later decides to also remove the option. Does that sound right or am I misunderstanding something?

For tests we run with selenium webdriver, we use Chromium, as Chrome binaries can be tricky to get.... We weren't aware of the Chrome for Testing, but it looks cool, and we'll try to see if we can use it instead of Chromium going forward. 

[Oliver Dunk]

Thanks everyone for the feedback so far.

A common theme seems to be the differences between normal Chrome builds and Chrome for Testing. I think that's a really fair question and honestly something I am still learning about. These are tagged for each release (Chrome Stable, Chrome Beta, Chrome Canary) and shouldn't be impacted by server side experiments, which actually makes them more deterministic and in some regards better to test against. As far as what flags are available / which channel they mirror, it is closest to a build of Chromium. The goal of Chrome for Testing is that it is the best fit for testing - it is used by default in tools like Puppeteer and if over time there are differences that are causing a lot of pain I think the team would be very open to looking into those.

With that said, I'll do my best to answer some of the specific questions...

However, disabling the --load-extension flag unilaterally on the standard Chrome release would probably cause us some challenges, and increase risk for our release process. Perhaps that flag could be allowed if Dev Tools are explicitly enabled by enterprise policy?

Thanks for sharing this. I believe there are cases where enterprise policy is set maliciously, so this would still leave things open to too much to abuse. I did mention it to the team though and we had a discussion about it, so I appreciate you bringing it up.

To reduce friction when doing automated testing - it would be great if Chrome DevTools Protocol had more methods related to extensions, like clicking the action icon, activating a context menu item, or changing allowed websites.

Yes, absolutely! We're actively working on this as part of the WebDriver BiDi project. We recently added support for loading unpacked extensions and in the future hope to look at some of the other things you've mentioned.

I believe that as long as Chromium keeps the  `--load-extension` flag we should be OK- if the feature is only removed on Chrome, I don't see how that would affect Chromium based Edge, unless Edge browser later decides to also remove the option. Does that sound right or am I misunderstanding something?

That sounds right - we're only making this change for branded Chrome builds :)

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

[Alex]

Having had a look at Chrome for Testing, one major difference I discovered is that Chrome for Testing blocks third-party cookies.

This doesn't seem to be the "block third-party cookies" preference (controllable by enterprise policy), but the Privacy Sandbox third-party cookie phaseout thing. Can we turn this off via flags or policies? If we can, it's not clear how. This seems like a good time to mention that Google already backpedaled out of phasing out third-party cookies.

I also noticed that the new version of the extensions menu, the one that makes it really easy to break your extensions by clicking the permissions toggle while hiding the "pin" icon, the one thing that users will be interested in finding in this menu, is also enabled by default in Chrome for Testing.

I assume there are other differences, but we're already off to a bad start.

Rather than closely tracking the matching branded Chrome channel, Chrome for Testing appears to be treated as an experimental "let's enable all the flags that users may never seen in practice" build, regardless of channel/version number.

[Jennifer Shehane]

Hi, Cypress team member here. 

We expose an API to our users to load extensions within the browser they're testing. Under the hood, we're passing that extension path to a --load-extension arg that we pass to Chromium browsers (including branded Chrome if that's what the user chose to run the tests within). 

We could handle if this is no longer supported in Chrome branded browsers and throw an error for our users informing them to use Chromium or Chrome for Testing. 

Ideally there would be no change required for our users though - like there would be an option for us to load an extension by the path in some other way (like WebDriver Bidi) in ALL Chrome browsers.

We have many, many users who just do not feel comfortable testing on a browser that is not the exact browser that their customers are using. 

[Oliver Dunk]

Hi Jennifer,

Thanks so much for reaching out.

While we haven't made any final decisions yet, we would likely support the new Extensions.loadUnpacked CDP command in all builds. This is also available in WebDriver BiDi. In both cases, the command is only available if:

• The `--enable-unsafe-extension-debugging` flag is passed to Chrome.
• The connection to Chrome uses `--remote-debugging-pipe`, not `--remote-debugging-port`.

Is it possible this would work for Cypress?

Please also feel free to ping me privately and we can discuss more if that would be helpful.

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

[ghucz...@gmail.com]

Dear Oliver

FYI I use '--load-extension' in combination with '--enable-logging=stderr' for testing from a unix bash script. The script loads the extension, runs the tests, and then filters the chrome log to output progress and report success/fail as a unix error code.

This meshes nicely with other infrastructure scripts running from the unix command line. Chrome DevTools protocol isn't used at all.

Greg