Proposal: extension API: activating extensions through Omnibox

[Zoran Rilak]

Hi devs,

this is a proposal for a new extension API to would allow invoking
extensions by entering text in the Omnibar. In brief:

[☆| gtalk dilbert ] // user enters "gtalk dilbert"
in the omni-bar and hits Enter

// behind the scenes, ExtensionsService is queried for one that has
registered "gtalk" as its trigger keyword.
// The extension is then sent the contents of Omnibox, which it uses
to do its thing, for example:

+---------------+ // extension opens Gmail, pops ups a chat with
Dilbert
|Gmail | // and focuses the textarea.
| .......... |
| .......... |
| +-------+
| |Dilbert|
| | |
| | |
| |[# ]|
+-------+-------+

This would allow extensions to register interest in processing user
input from the Omnibox. The basic premise is simple enough; I for one
love the Omnibar and off the top of my head I can think of half a
dozen different ways to put it to even heavier use through extensions
(see [1] for a couple more use cases), but I suspect it would be a
fairly major feature to consider adding. I'm willing to dig down into
the code base and start implementing it but I'd really appreciate a
few pointers first. Should this idea be pursued? Does it have a
future in Chromium?

---
[1] http://groups.google.com/a/chromium.org/group/chromium-discuss/browse_thread/thread/267ca3820b450807
More thoughts on this and a few use cases written in a case of a
severe sleep deprivation; caveat emptor.

[Aaron Boodman]

Hi Zoran,

We've definitely thought along these lines before, and I personally
think something like this would be a really great addition to the
extension system.

A few thoughts I've had on this idea:

* It would be neat if the keyword ("gtalk" in your example) were
derived automatically from the extension name, and if the omnibox were
clever about matching. So if you have an extension named "Google
Talk", typing "gt" or "gtalk" or "googletalk" would all match. This
also elegantly handles the issue of conflicts.

* It would be cool if the extension could interact with the omnibox
session, rather than just accepting all the input. The way I imagine
this working is that if you type "gtalk", you get all the usual
suggestions, from your history, bookmarks, etc, along with an
additional suggestion for the "Google Talk" extension you have
installed which declares it interacts with the omnibox:

[☆| gt ]
-----------------------------------------------------
[ icon | http://www.google.com/talk ]
[ icon | gtalk ]
[ icon | Google Talk ]
...

If you select the "Google Talk" entry and press "Tab", you enter a new
mode where you've accepted that you're only dealing with the
extension. At this point, the extension could expose additional
information to help the user enter the query. For example, it could
expose how many arguments it wants, or help with autocompleting, or
whatever. So, for the gtalk extension, maybe it takes one argument
which is the person to talk to. The omnibox might change to:

[ icon | Google Talk: <contact name> ]
-----------------------------------------------------
[ icon | Google Talk: b...@gmail.com ]
[ icon | Google Talk: na...@gmail.com ]
[ icon | Google Talk: s...@gmail.com ]
...

And of course as you type, the system could keep sending the current
query to the extension, which could refine the suggestions.

===

As for how to get started, the best thing to do is to fix some smaller
bugs first to get the hang of working on Chromium. You'll have to work
up to bigger features over time. If you're interested, there's more
information on how to get started here:

http://dev.chromium.org/developers

- a

> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
>    http://groups.google.com/a/chromium.org/group/chromium-dev
>

[Antony Sargent]

If you're interested in getting a few easier patches in to learn some of the process and codebase (and extensions system in particular), you can search for "feature:extensions helpwanted" at crbug.com, or just follow this link:

http://code.google.com/p/chromium/issues/list?q=feature:extensions+helpwanted

[Peter Kasting]

On Mon, Feb 22, 2010 at 12:02 AM, Aaron Boodman <a...@google.com> wrote:

The devil is in the details here.  The current omnibox code is not particularly amenable to extensions, and any interface we design would have two major problems:

* Very fragile w.r.t. interactions with the rest of the omnibox items

* May destroy our ability to modify and fix ranking functions

The basic issue is that minimizing the first problem would be most easily done by allowing extensions to register as AutocompleteProviders using the existing internal interface, but that means exposing our (currently very strange) ranking functions to extensions, which causes the second problem.  Any other solution to the first problem has lots of vexing edge cases around how the omnibox knows what is supposed to happen and displays it to the user.  Consider if the user already has a keyword "gtalk", for example; which action takes priority?

There are numerous other issues that would need to be solved as well, such as what to do when there are a large number of entries shown by extensions, how to deal with extensions that both want to handle the same input, how to ensure that extensions don't break inline autocompletion by having differing rankings synchronously versus asynchronously (or by doing a blocking operation during the synchronous pass), etc.

For this reason, I suggest that anyone wanting to design an API here become familiar with how the existing system works in enough detail to understand the pitfalls involved.

PK

[Munjal Doshi]

May be make the users type extension:galk ... (or something similar) to trigger extension from an omnibox? Of course that makes the user type more, but avoids problems Peter mentions.

-Munjal

--

[Peter Kasting]

On Mon, Feb 22, 2010 at 11:27 AM, Munjal Doshi <mun...@chromium.org> wrote:

May be make the users type extension:galk ... (or something similar) to trigger extension from an omnibox? Of course that makes the user type more, but avoids problems Peter mentions.

It avoids some issues, although it's still possible that a user wants to search Google for something about "extension:", so it doesn't mean it's safe for us to completely change over from the normal mechanism.  Forcing such a prefix is somewhat unfortunate in terms of UX, though.

PK 

[Aaron Boodman]

What if you use <tab> to disambiguate entering extension mode from
doing a search. gtalk<tab>foo would search send "foo" to the gtalk
extension, gtalk<space>foo uses the normal omnibox providers. I think
this also eliminates the need for the "extension:" prefix.

- a

[Zoran Rilak]

Aaron and Peter, thanks for your very informative replies. I
certainly don't intend to start submitting patches for something as
intuitively complex to implement as this, even without Peter's quick
list of pitfalls. It helps to know that I am thinking in the right
direction, though.

Peter—
what if the extensions don't register as AutocompleteProviders
themselves, but are offered through an internal provider which knows
about extensions' keywords (or names)? That'd keep the ranking system
flexible.

Z

[Peter Kasting]

On Mon, Feb 22, 2010 at 11:52 AM, Aaron Boodman <a...@google.com> wrote:

What if you use <tab> to disambiguate entering extension mode from

doing a search. gtalk<tab>foo would search send "foo" to the gtalk
extension, gtalk<space>foo uses the normal omnibox providers. I think
this also eliminates the need for the "extension:" prefix.

This doesn't help you at all, because tab and space both trigger the existing keyword provider, and have to in order for "tab to search" to work.  This also doesn't address how you surface the extension capabilities to the user in the dropdown before the user gets to the point of typing <tab>.

PK 

[Peter Kasting]

On Mon, Feb 22, 2010 at 12:07 PM, Zoran Rilak <zoran...@gmail.com> wrote:

what if the extensions don't register as AutocompleteProviders
themselves, but are offered through an internal provider which knows
about extensions' keywords (or names)?  That'd keep the ranking system
flexible.

That doesn't really buy you very much.  If you allow extension-provided items to intermix, you still have to solve the same issues (whatever ranking affordance we give extensions becomes something we have to map consistently to the "real" rankings forever after), and even if you don't, you at least have to solve extensions ranking against each other.

This whole area is a total minefield.

PK

[Aaron Boodman]

Sheesh, don't be such a downer :).

I've sketched out some of my thoughts in more detail (thanks Zoran for
inspiring me to think about this again), here:

http://docs.google.com/Doc?docid=0AYUOdFfLVYFRZGZqNnQ2N2RfODFnMzg2a2NmZA&hl=en

I think the idea of entering a mode that exclusively interacts with
the extension solves some of the issues, but maybe I'm just not seeing
them.

Peter, Zoran, I've given you edit access explicitly so that you can
comment. Anyone else, just request access and I'll approve it.

Thanks,

- a