Re: Proposal: Search API

[Matthew Phillips]

Hi Ben, my question is what does this API provide that the system can't
already do? On desktop Firefox the Awesomebar seems to work fine without
any APIs. Without knowing how it works I am wondering what is missing from
it?

I prefer solutions that require as little developer work as possible as
it's a barrier to adoption. Can someone chime in on how the Awesomebar
indexes and what this API would provide over top of that?

[Ben Francis]

On Mon, Mar 18, 2013 at 1:56 PM, Matthew Phillips
<mat...@phillipsoft.biz>wrote:

> Hi Ben, my question is what does this API provide that the system can't
> already do?

A standard interface by which a web app can search the data inside another
web app (or itself).

> On desktop Firefox the Awesomebar seems to work fine without any APIs.
> Without knowing how it works I am wondering what is missing from it?

Desktop Firefox is a desktop application built using XUL and the data for
the awesomebar is stored in a relational database using the Places system
which can be queried via query APIs
https://developer.mozilla.org/en-US/docs/Querying_Places - these are not
web standards and are only accessible from within the XUL application
itself.

The browser app on B2G is built using only web technologies and uses a
simplified version of the Places system, implemented using a non-relational
IndexedDB database. IndexedDB is not ideal for doing full text search, but
it is doable. What is not currently possible (except perhaps by using an
app-specific API using PostMessage) is for another app (e.g. the homescreen
app) to query the data stored in this data store. In particular, it would
not be possible to query the data stored inside an app when that app is not
currently running, which is required for some of the use cases described in
my proposal.

I prefer solutions that require as little developer work as possible as
> it's a barrier to adoption. Can someone chime in on how the Awesomebar
> indexes and what this API would provide over top of that?
>

Desktop apps can use whatever solution they like for storing and querying a
local search index. For a web app, the options are more limited. A
standardised method that works cross-origin between web apps does not
currently exist. A standarised API could significantly reduce the amount of
work necessary for developers to build a search index for their app, and
also make it possible to expose that search index to other apps.

If it's possible to achieve the use cases I describe with existing web
standards, I'd like to do that rather than create another specialised
offline storage API. But right now, I don't think it is.

Ben

--
Ben Francis
http://tola.me.uk

[Matthew Phillips]

*Didn't copy the list the first time, sorry.

> Desktop apps can use whatever solution they like for storing and querying
a local search index.

A desktop app cannot index the contents of another desktop app without
cooperation from that other app. This is the crux of my argument; a web app
CAN index other web apps; see Google, Bing, Blekko.

> For a web app, the options are more limited. A standardised method that
works cross-origin between web apps does not currently exist. A standarised
API could significantly reduce the amount of work necessary for developers
to build a search index for their app, and also make it possible to expose
that search index to other apps.

An app's index is already exposed to other apps in the form of its rendered
HTML. The way to make it easier for a 3rd party to index it is with
microformats, not a new JavaScript API.

[Ben Francis]

On Tue, Mar 19, 2013 at 4:50 PM, Matthew Phillips
<mat...@phillipsoft.biz>wrote:

> a web app CAN index other web apps; see Google, Bing, Blekko.
>

> An app's index is already exposed to other apps in the form of its
> rendered HTML. The way to make it easier for a 3rd party to index it is
> with microformats, not a new JavaScript API
>

The examples you cite are the kinds of "online search providers" I describe
as being potentially out of scope for this API.

The "offline search provider" use cases I'm talking about are installed web
apps on a device which may not even have a server side component, and
certainly don't have crawlable public web pages. Indexing and querying
would need to work completely offline.

Your response is interesting though, because it makes me realise how
client-side we have become in our thinking when developing apps for Firefox
OS.

Ben

[Matthew Phillips]

Doesn't the Homescreen app use the Browser API to launch apps? If so why
can't it index them through that?

> Your response is interesting though, because it makes me realise how
client-side we have become in our thinking when developing apps for Firefox
OS.

My concern is that you're trying to mimic the way native APIs tend to work
instead of doing things the web way. Remember that proprietary native
platforms are operating under a different set of constraints.

What I don't want is to throw a bunch of extra work at app developers (and
a JavaScript search API would be a ton of extra work) when existing
solutions already exist (or are easily adapted).

[Dale Harvey]

I think what Mathew Phillips is saying is that with mozApps permissions and
systemXHR you can identify what apps have been installed on the device and
search their contents.

On 19 March 2013 17:44, Ben Francis <b...@krellian.com> wrote:

> On Tue, Mar 19, 2013 at 4:50 PM, Matthew Phillips
> <mat...@phillipsoft.biz>wrote:
>
> > a web app CAN index other web apps; see Google, Bing, Blekko.
> >
> > An app's index is already exposed to other apps in the form of its
> > rendered HTML. The way to make it easier for a 3rd party to index it is
> > with microformats, not a new JavaScript API
> >
>
> The examples you cite are the kinds of "online search providers" I describe
> as being potentially out of scope for this API.
>
> The "offline search provider" use cases I'm talking about are installed web
> apps on a device which may not even have a server side component, and
> certainly don't have crawlable public web pages. Indexing and querying
> would need to work completely offline.
>

> Your response is interesting though, because it makes me realise how
> client-side we have become in our thinking when developing apps for Firefox
> OS.
>

> Ben
> _______________________________________________
> dev-webapi mailing list
> dev-w...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-webapi
>

[Ben Francis]

On Tue, Mar 19, 2013 at 5:53 PM, Matthew Phillips
<mat...@phillipsoft.biz>wrote:

> Doesn't the Homescreen app use the Browser API to launch apps? If so why
> can't it index them through that?

The apps API is used to launch apps, and the browser API is used to contain
and navigate them. The Browser API doesn't give the creator of an iframe
any access to the actual content inside the iframe which means the system
app can't just look inside the iframe to look at the data.

As Dale rightly points out, you could potentially get the HTML content of
the app using SystemXHR.

In the case of packaged apps I'm not sure how well this would work over the
app:// protocol which uses UUIDs in the URL. Does the apps API expose these
UUIDs when enumerating the installed apps?

In the case of hosted apps, presumably you would need to be authenticated
as the user in order to crawl the user's data and you'd need an Internet
connection to do so. I assume SystemXHR wouldn't be able to access offline
pages from appcache given that apps live in different data jars. Another
consideration is that the kinds of web apps we have in Gaia are not very
crawlable by nature, because they require user events to trigger JavaScript
in order to actually retrieve data from a local store and add it into the
DOM.

One other concern is that fetching, crawling and indexing web pages is
quite a resource hungry task which isn't brilliantly suited to running
locally on a mobile device.

I would love to implement this is a more web-like way if it's possible. But
that might require that our web apps in B2G become more web-like too!
Currently they don't have a real URI on the Internet and don't have a
server-side component.

Ben

[Ben Francis]

On Fri, Mar 15, 2013 at 6:15 PM, Ben Francis <b...@krellian.com> wrote:

> For the B2G project this would allow for the implementation of an
> "awesomebar" for the whole OS, which can search across content stored in
> multiple apps.
>

To help understand these proposed use cases, you might want to look at the
"RocketBar" concept
http://gordonbrander.github.com/rocket-bar-demo/annotated/ that the Firefox
OS UX team has been working on, with a prototype here
http://gordonbrander.github.com/rocket-bar/

Ben

[Matthew Phillips]

Hey Ben, sorry for taking so long to get back to you. You make a good
point about battery consumption and crawling, I hadn't considered that but
it's enough to make a JS API like you're proposing more palpable for me.
If we do it that way I would like it to be something simple. Something to
the effect of:

navigator.search.register(function(searchTerm, done) {
// Do async things to get results.
done(results);
});

And for placing a search:

var request = navigator.search.open('Yankees');
request.oncomplete = function(results) {
assert(Array.isArray(results));
};

[Mounir Lamouri]

On 15/03/13 18:15, Ben Francis wrote:
> Hi there,
>
> I'd like to propose some early ideas for a new API which provides a search
> index for the content of a web app where that index can be queried not only
> by the app itself, but by other apps which have the required permission.
>
> One use case of this API would be for a web app to be able to provide a
> unified search interface for the content stored across multiple apps on a
> device, but using a standard interface which would provide an open way for
> apps to opt-in to being search "providers".
>
> At the platform level you could think of this API as a kind of client-side
> Open Search[1].

>
> For the B2G project this would allow for the implementation of an
> "awesomebar" for the whole OS, which can search across content stored in

> multiple apps. This could be similar to Spotlight in iOS [2] and the Google
> search bar on the homescreen of Android 4.x (an interface to Google Now
> [3]) which both allow the user to search local data across apps on the
> device.
>
> [...]

Hi Ben,

Thank you for this message. This is a kind of problem we have been
looking at recently. It is currently not clear how we could solve it but
we have two ideas that we could use:
- some kind of data-source mechanism where apps can register as
providing a data-source of some sorts and other app would ask to access it;
- an API to communicate between two applications:
https://wiki.mozilla.org/WebAPI/Inter_App_Communication

Those two ideas are at a very early stage, the former one doesn't even
have something written down. But I believe both of them should help
solving your use cases. It wouldn't be as simple as a "Search API" but
they should give you the tools you need.

The benefits of the former is that it might help solving the problem we
have with some APIs that contain data like SMS API. We could simply make
the SMS database a data-source and allow some applications to access it.

The later proposition is more generic and we might or might not need it
anyway. However, it goes with serious security concerns (you don't want
any app to access any other app data).

Thanks,
--
Mounir

[Dietrich Ayala]

On Wednesday, March 27, 2013 8:15:33 AM UTC-7, Mounir Lamouri wrote:
> The benefits of the former is that it might help solving the problem we
>
> have with some APIs that contain data like SMS API. We could simply make
>
> the SMS database a data-source and allow some applications to access it.

This would be fantastic!

> The later proposition is more generic and we might or might not need it
>
> anyway. However, it goes with serious security concerns (you don't want
>
> any app to access any other app data).

How is the access issue for inter-app communication different from the inter-app datasource sharing?

[Mounir Lamouri]

On 28/03/13 20:54, Dietrich Ayala wrote:
>> The later proposition is more generic and we might or might not need it
>>
>> anyway. However, it goes with serious security concerns (you don't want
>>
>> any app to access any other app data).
>
> How is the access issue for inter-app communication different from the inter-app datasource sharing?

The main difference between both is the semantic behind datasource
sharing. You know what action is being done. Also, the provider of the
data can allow r/w/rw and the requester can ask for a specific set of
access privileges too. You could also have some system datasource behind
a permission (like SMS or Contacts).

In other words, it is easier to ask the user "Do you want this
application to be able to READ your SMS." or "Do you want this
application to be able to MODIFY your Contacts.". But you can hardly say
"Do you agree for this application and that application to exchange
information." because that means nothing and we can hardly say more.

If we put aside the user prompting part, it is probably quite the same
problem scope.

--
Mounir

[Jonas Sicking]

I think there are a lot of really cool ideas here. But I think this in
general is a tremendously complex feature, at least to do well.

What would be "pretty easy" to support would be an API which allows us to
start an app in the background, hand it a string that the user has typed in
the search bar, and then wait for it to return a set of results.

This has the advantage that it allows apps to implement their own searching
however they want. They can even search data that's only residing
server-side. And it could implement arbitrary smarts about fuzzy matching
for misspelled words, plurals etc.

However this has the severe problem that we have to start all apps that
support searching every time the user tries to search. This is likely
simply not acceptable on the type of hardware that we're currently running.

Another solution would be to enable applications to store data in a
database that the system can read from directly. However this is *really*
complex. First off, we need something like full-text indexing, which is a
big feature in and of itself. I.e. we need to do all the fuzzy matching,
word tokenizing and so on that you generally need for free-text searching.

On top of that you ideally want to do ranking. But that's a function of the
found document as well as the search a string that was used. I have no idea
how you'd do that without running application logic.

Finally you need a way to map the found result to something that can be
displayed in UI. We could certainly force the app to store the UI that
should be displayed with each stored entry, but that means duplicating a
lot of data.

All of this is certainly solvable. But its a very big undertaking. It would
be interesting to look at how other platforms have created these types of
solutions.

Early versions of iOS didn't allow searching of third party app data, is
that something they support now? My current Jelly Bean device does not.

The work that we are doing for the DataStores API is potentially a step
towards solving some of these issues. But there's still a long way to go.

/ Jonas