Proposal for content settings extension API

[Jochen Eisinger]

bcc: chromium-extensions

Hey all,

I've put together a proposal for a content settings extension API at

https://docs.google.com/a/google.com/document/d/1IIQLPtUAuLBHjUHGVsGw6w84BMcvmwq5Ef8Esd3mC74/edit?hl=en&authkey=CKnZrZIE#

the purpose of this API is to expose Chromium's content setting
sub-system to extension, so they can implement different usage models
than what we've implemented in Chromium.

Please comment!

thanks
-jochen

[Aaron Boodman]

Cool, I really like the general idea. I will try and review it in
detail tonight.

- a

> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
>    http://groups.google.com/a/chromium.org/group/chromium-dev
>

[Nico Weber]

An enterprising extension author could use this as a general data
store (all the different content settings give you a couple bits per
host name, and there can be infinitely many host names like
www.myextensionN.com), since the storage for this isn't limited.
Should this "attack" be addressed somehow?

Nico

On Tue, Nov 30, 2010 at 11:07 AM, Jochen Eisinger <joc...@chromium.org> wrote:

[Jochen Eisinger]

On Tue, Nov 30, 2010 at 8:49 PM, Nico Weber <tha...@chromium.org> wrote:
> An enterprising extension author could use this as a general data
> store (all the different content settings give you a couple bits per
> host name, and there can be infinitely many host names like
> www.myextensionN.com), since the storage for this isn't limited.
> Should this "attack" be addressed somehow?

We could introduce a maximal number of content setting rules that can
exist at any time (similar to cookies). Otoh, why would an extension
do this instead of encoding the data in a single history entry?

[Aaron Boodman]

On Tue, Nov 30, 2010 at 11:51 AM, Jochen Eisinger <joc...@chromium.org> wrote:
> On Tue, Nov 30, 2010 at 8:49 PM, Nico Weber <tha...@chromium.org> wrote:
>> An enterprising extension author could use this as a general data
>> store (all the different content settings give you a couple bits per
>> host name, and there can be infinitely many host names like
>> www.myextensionN.com), since the storage for this isn't limited.
>> Should this "attack" be addressed somehow?

Heh.

> We could introduce a maximal number of content setting rules that can
> exist at any time (similar to cookies). Otoh, why would an extension
> do this instead of encoding the data in a single history entry?

Putting some kind of very large hard limit on the number of items
seems like a good thing to do.

But there should be little reason to resort to this. We provide proper
storage options to extension developers and there is no penalty (in
terms of scary UI) for using it to store whatever you want:

http://code.google.com/chrome/extensions/manifest.html#permissions
(see 'unlimitedStorage').

(we don't display any warning UI for requesting this permission)

- a

[Aaron Boodman]

This is exciting!

I wasn't able to comment on the doc, so here we go:

- Nit: We use permissions like 'contentSettings' instead of 'content_settings'

- I assume this is going to use the system you've been working on that
makes the setting for a given preference deterministic when multiple
extensions are involved? If so, will there be UI somewhere that tells
users that an extension (or which extension?) is managing a given
setting?

- Why not use match patterns
(http://code.google.com/chrome/extensions/match_patterns.html) for the
patterns? It seems to cover all the use cases you have in mind, and
developers will already be familiar with it.

- I prefer separate getters/setters for the misc settings.

- For some reason, I *don't* prefer separate modules for the non-misc
settings. I realize this is inconsistent with the above. Maybe it is
because it feels more egregious to dump a bunch of booleans into a
settings object than a bunch of more complex rules. Sorry :-/.

- a

[Jochen Eisinger]

On Sun, Dec 12, 2010 at 10:55 AM, Aaron Boodman <a...@chromium.org> wrote:

This is exciting!

I wasn't able to comment on the doc, so here we go:

- Nit: We use permissions like 'contentSettings' instead of 'content_settings'

- I assume this is going to use the system you've been working on that
makes the setting for a given preference deterministic when multiple
extensions are involved? If so, will there be UI somewhere that tells
users that an extension (or which extension?) is managing a given
setting?

- Why not use match patterns
(http://code.google.com/chrome/extensions/match_patterns.html) for the
patterns? It seems to cover all the use cases you have in mind, and
developers will already be familiar with it.

I want to comment on these two points: The reason for not using match patterns is that the rules created in this API could be directly displayed and managed in our existing UI.

An alternative would be to use match patterns and e.g. gray out the content exception editor in our UI with a note saying that you're using extensions to control these settings. In that case, using match patterns would be a perfectly reasonable thing to do.

I think I'd prefer the latter, wdyt?

-jochen

[Bernhard Bauer]

On Wed, Dec 15, 2010 at 12:14, Jochen Eisinger <joc...@chromium.org> wrote:
>
>
> On Sun, Dec 12, 2010 at 10:55 AM, Aaron Boodman <a...@chromium.org> wrote:
>>
>> This is exciting!
>>
>> I wasn't able to comment on the doc, so here we go:
>>
>> - Nit: We use permissions like 'contentSettings' instead of
>> 'content_settings'
>>
>> - I assume this is going to use the system you've been working on that
>> makes the setting for a given preference deterministic when multiple
>> extensions are involved? If so, will there be UI somewhere that tells
>> users that an extension (or which extension?) is managing a given
>> setting?
>>
>> - Why not use match patterns
>> (http://code.google.com/chrome/extensions/match_patterns.html) for the
>> patterns? It seems to cover all the use cases you have in mind, and
>> developers will already be familiar with it.
>
> I want to comment on these two points: The reason for not using match
> patterns is that the rules created in this API could be directly displayed
> and managed in our existing UI.

There's also the same reason we didn't use match patterns for the
regular content settings UI, which is that it's not clear which rule
would take precedence in case of conflicts.

> An alternative would be to use match patterns and e.g. gray out the content
> exception editor in our UI with a note saying that you're using extensions
> to control these settings. In that case, using match patterns would be a
> perfectly reasonable thing to do.

So, wou want to disable *all* exception editing as soon as you install
one extension which sets exceptions? Wouldn't that greatly reduce the
value of content settings in general?

Bernhard.

[Jochen Eisinger]

On Wed, Dec 15, 2010 at 1:56 PM, Bernhard Bauer <bau...@chromium.org> wrote:

On Wed, Dec 15, 2010 at 12:14, Jochen Eisinger <joc...@chromium.org> wrote:
>
>
> On Sun, Dec 12, 2010 at 10:55 AM, Aaron Boodman <a...@chromium.org> wrote:
>>
>> This is exciting!
>>
>> I wasn't able to comment on the doc, so here we go:
>>
>> - Nit: We use permissions like 'contentSettings' instead of
>> 'content_settings'
>>
>> - I assume this is going to use the system you've been working on that
>> makes the setting for a given preference deterministic when multiple
>> extensions are involved? If so, will there be UI somewhere that tells
>> users that an extension (or which extension?) is managing a given
>> setting?
>>
>> - Why not use match patterns
>> (http://code.google.com/chrome/extensions/match_patterns.html) for the
>> patterns? It seems to cover all the use cases you have in mind, and
>> developers will already be familiar with it.
>
> I want to comment on these two points: The reason for not using match
> patterns is that the rules created in this API could be directly displayed
> and managed in our existing UI.

There's also the same reason we didn't use match patterns for the
regular content settings UI, which is that it's not clear which rule
would take precedence in case of conflicts.

I think it would be easy to define some order that developers would understand.

 

> An alternative would be to use match patterns and e.g. gray out the content
> exception editor in our UI with a note saying that you're using extensions
> to control these settings. In that case, using match patterns would be a
> perfectly reasonable thing to do.

So, wou want to disable *all* exception editing as soon as you install
one extension which sets exceptions? Wouldn't that greatly reduce the
value of content settings in general?

An alternative would be to display the match patterns but have them not-editable?

[Aaron Boodman]

On Wed, Dec 15, 2010 at 4:14 AM, Jochen Eisinger <joc...@chromium.org> wrote:
>
>
> On Sun, Dec 12, 2010 at 10:55 AM, Aaron Boodman <a...@chromium.org> wrote:
>>
>> This is exciting!
>>
>> I wasn't able to comment on the doc, so here we go:
>>
>> - Nit: We use permissions like 'contentSettings' instead of
>> 'content_settings'
>>
>> - I assume this is going to use the system you've been working on that
>> makes the setting for a given preference deterministic when multiple
>> extensions are involved? If so, will there be UI somewhere that tells
>> users that an extension (or which extension?) is managing a given
>> setting?
>>
>> - Why not use match patterns
>> (http://code.google.com/chrome/extensions/match_patterns.html) for the
>> patterns? It seems to cover all the use cases you have in mind, and
>> developers will already be familiar with it.
>
> I want to comment on these two points: The reason for not using match
> patterns is that the rules created in this API could be directly displayed
> and managed in our existing UI.
>
> An alternative would be to use match patterns and e.g. gray out the content
> exception editor in our UI with a note saying that you're using extensions
> to control these settings. In that case, using match patterns would be a
> perfectly reasonable thing to do.
>
> I think I'd prefer the latter, wdyt?

Oh, I didn't realize you were intending to have extension-supplied
settings editable. It seems like this would create some complexity. I
think UX wise I would personally prefer just saying "extension xyz is
controlling. [ ] go back to manual control".

- a

[Bernhard Bauer]

On Wed, Dec 15, 2010 at 13:44, Jochen Eisinger <joc...@chromium.org> wrote:
>
>
> On Wed, Dec 15, 2010 at 1:56 PM, Bernhard Bauer <bau...@chromium.org> wrote:
>>

>> On Wed, Dec 15, 2010 at 12:14, Jochen Eisinger <joc...@chromium.org>

>> wrote:
>> >
>> >
>> > On Sun, Dec 12, 2010 at 10:55 AM, Aaron Boodman <a...@chromium.org> wrote:
>> >>
>> >> This is exciting!
>> >>
>> >> I wasn't able to comment on the doc, so here we go:
>> >>
>> >> - Nit: We use permissions like 'contentSettings' instead of
>> >> 'content_settings'
>> >>
>> >> - I assume this is going to use the system you've been working on that
>> >> makes the setting for a given preference deterministic when multiple
>> >> extensions are involved? If so, will there be UI somewhere that tells
>> >> users that an extension (or which extension?) is managing a given
>> >> setting?
>> >>
>> >> - Why not use match patterns
>> >> (http://code.google.com/chrome/extensions/match_patterns.html) for the
>> >> patterns? It seems to cover all the use cases you have in mind, and
>> >> developers will already be familiar with it.
>> >
>> > I want to comment on these two points: The reason for not using match
>> > patterns is that the rules created in this API could be directly
>> > displayed
>> > and managed in our existing UI.
>>

>> There's also the same reason we didn't use match patterns for the
>> regular content settings UI, which is that it's not clear which rule
>> would take precedence in case of conflicts.
>
> I think it would be easy to define some order that developers would
> understand.

Why don't we upgrade user-defined exceptions to match patterns then?

>
>>
>> > An alternative would be to use match patterns and e.g. gray out the
>> > content
>> > exception editor in our UI with a note saying that you're using
>> > extensions
>> > to control these settings. In that case, using match patterns would be a
>> > perfectly reasonable thing to do.
>>

>> So, wou want to disable *all* exception editing as soon as you install
>> one extension which sets exceptions? Wouldn't that greatly reduce the
>> value of content settings in general?
>
> An alternative would be to display the match patterns but have them
> not-editable?

I would prefer that, yes. If we're going to have policy-controlled
exceptions at some point, we could just as well get exceptions from
the ExtensionPrefStore (or whatever it's called these days ;-) ), and
treat them the same way.

As long as that only refers to the exceptions added by an extension.
It would suck not to be able to add exceptions for youtube.com, just
because an extension is managing your content settings for google.com,
for example.

Bernhard.

>
> - a

[Jochen Eisinger]

What would happen if you enter the youtube.com exception? Would it be automatically converted to a match pattern (and so the user can't edit it anymore)?

Note that there also several other UI elements that can create content settings apart from the exception editor.

[Bernhard Bauer]

There is no reason not to be able to edit match patterns, it's just
exceptions set by an extension that should be read-only (which may or
may not be match patterns). I think the pattern format to use is
orthogonal to how to treat user-defined vs extension-defined
exceptions. I just would like the format to be consistent for both
kinds of exceptions, that's why I proposed to change the format for
user-defined exceptions as well, if we end up using match patterns.

Bernhard.

[Jochen Eisinger]

Ok, I think you're right, both systems should use the same format.

What about allowing match patterns in the extension, but translating them to content setting patterns internally, so they can easily be displayed in the UI etc?

[Bernhard Bauer]

And throw some sort of error if the match pattern can't be converted
to a content setting pattern? I think I'd be okay with that.

Bernhard.

[Jochen Eisinger]

Yep, we'd be throwing errors for some patterns anyway, e.g. geolocation that doesn't support patterns at all.

Aaron, wdyt?

[Peter Kasting]

On Wed, Dec 15, 2010 at 5:44 AM, Jochen Eisinger <joc...@chromium.org> wrote:

On Wed, Dec 15, 2010 at 1:56 PM, Bernhard Bauer <bau...@chromium.org> wrote:

There's also the same reason we didn't use match patterns for the

regular content settings UI, which is that it's not clear which rule
would take precedence in case of conflicts.

I think it would be easy to define some order that developers would understand.

We talked about this a long time when creating the content exceptions, because we had precisely the same thoughts: "let's use match patterns, we already have code and definitions for them and they're nice and flexible".  In the end we could not come up with a sane and obvious preference order that made any sense to view/edit.

Thus I strongly advise NOT using match patterns here, and sticking with our existing grammar.  It was not randomly created.  I am opposed to the "use match patterns and convert to content setting patterns" idea too; that imposes a pointless burden on both the implementation and the extension author.  Please, just use the syntax we have today!

I also think you should make geolocation and notifications, which use a tuple of patterns, modifiable by extensions.  As for user control over extension-set data, I suggest showing the content exceptions editor as normal, and allowing user edits to user-created entries; then below them showing a separate, read-only list of exceptions set by extension X.

PK

[Jochen Eisinger]

On Wed, Dec 15, 2010 at 9:31 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Wed, Dec 15, 2010 at 5:44 AM, Jochen Eisinger <joc...@chromium.org> wrote:

On Wed, Dec 15, 2010 at 1:56 PM, Bernhard Bauer <bau...@chromium.org> wrote:

There's also the same reason we didn't use match patterns for the

regular content settings UI, which is that it's not clear which rule
would take precedence in case of conflicts.

I think it would be easy to define some order that developers would understand.

We talked about this a long time when creating the content exceptions, because we had precisely the same thoughts: "let's use match patterns, we already have code and definitions for them and they're nice and flexible".  In the end we could not come up with a sane and obvious preference order that made any sense to view/edit.

Thus I strongly advise NOT using match patterns here, and sticking with our existing grammar.  It was not randomly created.  I am opposed to the "use match patterns and convert to content setting patterns" idea too; that imposes a pointless burden on both the implementation and the extension author.  Please, just use the syntax we have today!

I'm not sure the burden would be that big. But I don't have a strong opinion about what pattern format to use.

 

I also think you should make geolocation and notifications, which use a tuple of patterns, modifiable by extensions.  As for user control over extension-set data, I suggest showing the content exceptions editor as normal, and allowing user edits to user-created entries; then below them showing a separate, read-only list of exceptions set by extension X.

Yes, extensions would be able to set and modify geolocation and notification as well.

 

PK

[Aaron Boodman]

Are there any mocks or design docs that describe the user experience
that you're aiming for, after all this work is done? I'm having
trouble envisioning the interplay between extension-provided settings
and user overrides/exceptions.

- a

[Jochen Eisinger]

There are no mocks. My initial idea was that the extension API would use the same pattern format as the existing content settings, so it would directly modify the content setting rules, i.e. no changes are required to our current UI.

That way, an extension could implement additional UI (like a button that disables javascript, reloads the page, and enables it again), but the user could always just go to the content settings editor and see all existing rules and edit them.

-jochen

[Mike Perry]

On Nov 30, 7:07 pm, Jochen Eisinger <joc...@chromium.org> wrote:
> I've put together a proposal for a content settings extension API at
>

> https://docs.google.com/a/google.com/document/d/1IIQLPtUAuLBHjUHGVsGw...

>
> the purpose of this API is to expose Chromium's content setting
> sub-system to extension, so they can implement different usage models
> than what we've implemented in Chromium.

Hi Jochen!

I have a couple of directly related comments, and a couple of
tangential comments (in my crusade against declarative models :).

Overall, the API looks like a nice parallel to Firefox's
nsIContentPolicy. But I did notice a couple of minor omissions that I
know have been useful in Firefox land. TYPE_FONT is useful for
blocking css font requests, which have seen a number of exploits in
the past (the renderer for which is a horrible beast rewritten from
pascal to single-threaded C to reentrant C). NoScript blocks these
requests by default, for example. TYPE_MEDIA is useful for similar
reasons to block potentially buggy audio and video codecs, which
NoScript provides Click-To-Play place holders for. Individual plugin
control would be nice here, too, so that you could allow for example,
Gnash, but block Java, silverlight, pdf, and flash by default.

Also, for using the content policy as a true content policy, this API
seems to fit well as a declarative model. However, not many Firefox
addons actually use the content policy as merely a simple URL-based
load/block decision point. This is possibly due to incompleteness in
the other APIs that lead people to resort to the content policy
because the other APIs don't provide the right DOM context, but I
think also that addons need to make some semi-complicated decisions
about what to load and when and how to load it. For example, NoScript,
RequestPolicy, and AdBlock Plus all make decisions based on not only
element URL, but also DOM context and often sourcing document info, as
well. Both NoScript and Torbutton also have notions of per-tab
properties, which means that tab-id is another possible decision
source (though Torbutton on Chrome wouldn't need this info any more
because it would operate differently).

It appears that all of this context could be added to your pattern
matching and still remain a declarative model, with the expense of
more properties to check per request.

However, this brings me into my tangential comments about declarative
models: in addition to the difficulties of envisioning and capturing
the full expressiveness possible with programmatic APIs, declarative
models can have poor performance properties once they see heavy use.
For example, HTTPS Everywhere now has rules for something like 2000
websites, for a total of ~15k worth of regexps that would need to be
checked for each URL, and about 10 new sites are being added per week
by the community. AdBlock Plus has even more. RequestPolicy tends to
have fewer, but that is only because of the lack clear endorsement of
user-contributed policies by the author.

Each of these addons have implemented their own optimization system to
avoid checking every single rule property, for every single request.
For HTTPS Everywhere, we were able to perform some wildcard tricks
with the host name to achieve O(1) lookups of the relevant ruleset to
reduce the regexps per URL from ~15000 to under 10. I believe Adblock
plus has more involved optimizations to structure their filters into
hierarchies, because they must also check the path independent from
the host name. RequestPolicy only needs to check destination rules if
the source host name matches. In general, I suspect most addons may
have their own domain-specific optimizations to reduce the number of
patterns and attributes that are checked.

Declarative models might not be able to achieve this as easily, since
they need to be both expressive enough to cover functionality, and yet
narrow enough to easily schedule and optimize application and
evaluation order.

Furthermore, if you install 3-4 Adblock/HTTPS Everywhere/RequestPolicy-
style addons that each register their own filter model, it seems to me
that a pure declarative API is going to cause the network layer to
block until all addons' models are evaluated in succession. Either
that, or you need a worker thread pool to do processing of these
filter models in parallel, at which point optimization by condensing
rule evaluation seems more difficult. If you instead were to dispatch
the request as a call out to each addon in parallel (as I describe at
the bottom of my WebRequest review:
https://groups.google.com/a/chromium.org/group/chromium-extensions/browse_thread/thread/17ea6efa15bfea0a),
you just wait for the slowest addon to complete its processing. This
also makes performance problems easier to diagnose from both a user
and developer point of view, because things shouldn't get
incrementally slower as you add more addons - there will instead be a
clear culprit that is the obvious bottleneck when enabled/disabled.


I guess I'm not saying that the declarative models can't work. I'm
just very wary that it will be as easy to design one that is as
flexible as a programmatic model from an extension's point of view.
But I guess if someone came out and said "Well, the IPC overhead for
even one RTT to the extension processes takes as long as it would take
to evaluate 15000 regular expressions per URL", or if there is general-
purpose optimization for condensing these match patterns, then it
would make more sense.

[Peter Kasting]

On Thu, Dec 16, 2010 at 8:55 PM, Mike Perry <mikeperr...@gmail.com> wrote:

Also, for using the content policy as a true content policy, this API

seems to fit well as a declarative model. However, not many Firefox
addons actually use the content policy as merely a simple URL-based
load/block decision point.

This proposal is not intended to address dynamic needs.  You should look at the network extension APIs for programmatic requests.  It is the API we're designing for AdBlock, NoScript, HTTPS Everywhere, etc., not this one.

Let's not make the content settings surface any broader than it already is.

PK