Question: RESTful search

[Colton Pierson]

Is there a reason that /noun/search?q=foo+bar should be used versus /
noun?q=foo+bar for search? If so, what might it be? My guess at the
moment is because search is a verb the functionality should be
isolated from the noun - however that guess is countered by the fact
that search is merely a system of filtering the nouns.

[landlessness]

solid point.

thinking out loud: with /search required you could be smarter about the params.

for example, with /noun?q=... you would have to support a superset of params where conflicts like q=red+fur&furColor=blue could happen.

whereas if you did /search?q=... you could disallow any other params.

i'll be thinking about this one some more, i like that /noun?q= is terse.

[Ed Anuff]

My thought is that /noun represents a collection and /noun?q=... is a query against it.  By convention, q=... typically represents a search-style query (i.e. the contents of a form input box) while other parameter attributes might be filters.  So, you might be able to do /noun?q=color+eq+red or you might do /noun?color=red

[Jeff Schmidt]

While admittedly a read (GET) action, I still think more than real
basic search capability stands outside of CRUD. When going beyond
simply providing search terms, a whole slew of parameters can be
required. You are searching things (nouns), but there is not SEARCH
method in HTTP, and searching is an action. I like the /noun/search
idea.

As far as the parameter metadata goes, given my experience with Solr,
it can get quite extensive. Solr uses q for the search terms/
expression, but there are many more parameters for specifying results
pagination, faceting highlighting etc. Not all of these capabilities
need be made visible via your API, but depending on your requirements,
many of them may be, particularly, pagination and faceting. I may not
want all of these search related parameters showing up in the WADL for
GET on my /noun resource, when it otherwise has a CRUD orientation.
Using /noun/search keeps that stuff isolated.

What are opinions/experience with this? I certainly want to implement
something sensible. ;)

Cheers,

Jeff

[lpa]

You could keep the "/noon/search" resource to serve an OpenSearch
description of your search interface on "noon"
http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_3
Then do the search on "/noon?q=..."

[Luke Stokes]

I'm new to all this and trying to learn quickly, but the approach I
was thinking of taking involved using the plural form of the noun
which (in my mind) implies a listing of resources. Search then becomes
just a way to filter those resources.

Example:

/users

I POST to that to create a new user (thus creating /users/1)

But if I GET to /users, then I have a listing of users, and thus a
search. It's true, we'll still have to document how the search and
filter parameters work, but it makes more sense to me than creating a /
user/search "resource" because the search doesn't actually represent
anything (unless, or course, the API provided a way to save those
searches as actual resources, but that seems odd to me considering the
contents would be constantly changing).

Am I thinking about this wrong?

[Sam Ramji]

Luke:

What will you do when GET /users returns 10,000 records?

I suspect that in most applications, you will want to narrow this to the working set you care about, such as users named "John", users whose accounts are "expired", etc.  Pulling the set of all users seems like a mismatch to nearly all use cases that I can think of for a non-trivial application.

Cheers,

Sam

[Luke Stokes]

It will definitely include pagination.

Check this at around minute 41:
http://blog.apigee.com/detail/slides_for_restful_api_design_second_edition_webinar/

Brian says to use a search "verb" only when searching across multiple
resources and that "normally" just doing /dogs/ would be fine.

As for narrowing things down, the approach I was thinking about would
be to let the URI do that for me. I know there are different opinions
on this, but my thought was:

/users/1/stores would just return the stores that user has access to.
Also, if someone wanted to work with a store, say /users/1/stores/1 it
makes security and authorization super simple. My OAuth token will
tell me which user I am and the scope tells me which store I can work
with so I don't have to hit the database at all to reject an invalid
request, I just inspect the url.

[Will Norris]

[quick introduction since I've been lurking for a few weeks, but haven't posted yet.  I'm an engineer at Google and did the majority of the design work on the Google+ API and a few other Google APIs]

For Google+, we designed search pretty much how Luke is describing... as a filter on a collection.  The idea being that it would be equally applicable to any collection of resources.  So if wanted to search across all comments, you could do something like:

  /comments?q=foo

Or you could search across comments that were left in reply to a specific activity:

  /activities/123/comments?q=foo

Conceptually, I think this fits the REST philosophy better than most alternatives.  We currently only support search on the global people and activities collections, but this is the approach we took (see people.search[0] and activities.search[1]).  I don't know that we'll actually be able to add search capabilities to more specific collections (simply due to how search is implemented in our backends), but we're still designing for the possibility.

[0]: https://developers.google.com/+/api/latest/people/search

[1]: https://developers.google.com/+/api/latest/activities/search

hope that helps,

Will

[Sam Ramji]

The "implicit scope through OAuth" that narrows results to the logged-in user is a good one.  It's something that can greatly help app developers write apps that behave in a smart, personalized way without doing much work.

As long as there are not too many returned records, I don't see a problem with your approach.  But think hard about the number of records a mobile developer wants to deal with to generate a page or action, and think about what bandwidth, radio power, and battery/CPU will be used to consume all your responses.  Pagination is often not enough.

Cheers,

Sam

[Ed Anuff]

Thanks Will, I'm in total agreement on this approach as well.  That's the approach we're using on the Usergrid API too.

[Ed Anuff]

Pagination is a good enough starting point.  You're raising a separate issue, Sam, which is the way that it's hard to predict the way a search API can be expected to be used.  In my experience, your search endpoint is potentially one of your heaviest hit so it needs to be really really fast.  I've seen more than a few situations where the performance for a search that could return a lot of results is relatively slow, even where you're only returning the first 10 results.  To give you an example, suppose you're using the search API for a typeahead find feature.  The first few characters you enter would matching against thousands or millions of records.  So, you do need to think about that sort of thing with your search endpoints and quickly return some sort of response that indicates more specificity is needed if you can't process and return the first 10 results quickly.

[Ed Anuff]

Someone at FB wrote a good article on this kind of thing: http://www.facebook.com/note.php?note_id=389105248919

[landlessness]

helen and i are working on an ebook for web api design and this discussion has forced a rev on the search section.

we're going to suggest the following for scoped search:

/dogs?q=foo+bar

and for global search:

/search?q=foo+bar

doing global search as just a query param felt like it would be hard capability to discover and document. whereas /search?q= felt easy to discover and simple.

colton thanks for suggesting this nugget, ed, luke, will, et al thanks for the great chatter. 

[Jack Repenning]

On Feb 10, 2012, at 3:55 PM, landlessness wrote:

we're going to suggest the following for scoped search:

/dogs?q=foo+bar

and for global search:

/search?q=foo+bar

Not sure I follow. Is the notion that this app has dogs and cats and parrots and peacocks and so on, and that "GET /dogs?q=color_blue" is the same as "GET /search?q=color_bluw+species_dog"?

Jack Repenning

Pssssssssttttt . . . . . . . there are no memes!

[landlessness]

sorry for being unclear. i was alluding to the design problem we tackled in this video: http://www.youtube.com/watch?v=QpAhXa12xvU

objects like dogs, comments, owners, veterinarians, etc.

the GET below would return content from all objects types:

/search?q=walk

these GETs would be scoped

/veterinarians?q=walk

/owners?q=walk

/dogs?q=walk

-b

[Darrel Miller]

Brian,

Having a root api document that contains something like,

<link rel="urn:acme:globalsearch" href="/search?q={keywords}" />
<link rel="urn:acme:dogsearch" href="/dogs?q={keywords}" />

means that developers can probably guess how to search the site.
There is no need to reference "API documentation", just do a GET on
the root of the API and see what it has to offer.

Or if you don't want to invent your own search link relations, you
could always use the OpenSearch standard. All this work has been done
before, there really is no need to re-invent the wheel for every new
API.

Trying to create "web wide" URL conventions is going to be painful task.

Darrel

[Carlos Eberhardt]

This is how we are thinking about it. I like how it fits in my head. 

Relatedly, has anyone been thinking about using atompub for responses for these collections? Seems like a good fit, and it ties in nicely with OpenSearch. Not sure I'm recognizing all the gotchas, though.

[guivalerio]

I am working on an API at the moment and the way we have it designed
to perform searches for users for example is like described by Will
and Luke:

/users?q=john+smith

The thing is that I should also allow the app client to send to the
API some parameters to filters the list even further.
E.g. The scenario i am working on is when a user is searching for
other users so he can add them as friends.

1. the search results shouldn't return the user who is performing the
search;
2. it shouldn't return users who are already friends with the user;
3. it shouldn't return users who have a membership request pending;

So basically I the client needs to send the user's ID.

I am using the following:

/users?q=john&ignore={user_id}

I believe this a good way of implementing it...... any thoughts?

Cheers!
G

[Dale McCrory]

How about something like:
/users/@me/suggested?q=john = all users suggested for me filtered by "john"
/users/@me = all users associated with me

I like the use of selectors for user context association in URLs.

Dale

[sune jakobsson]

Hi. You might pick up some inspiration from the OMA presence enabler here:  http://www.openmobilealliance.org/Technical/release_program/docs/CopyrightClick.aspx?pck=ParlayREST&file=V2_0-20110111-C/OMA-TS-ParlayREST_Presence-V1_0-20110111-C.pdf 

Sune