Started Draft 4

[DeWitt Clinton]

Hi all,

I just created Draft 4:

  http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_4

Currently it is a verbatim clone of Draft 3.

There are a few small extensions floating around that we can now fold into the core spec.  The rule for changes is still the same -- everything we put in Draft 4 should be backward compatible and not break existing clients, and the changes should be useful in the 80% case.

So let's get this thread going again -- what are some of the things that should be added or clarified in Draft 4?

-DeWitt

[Niall Kennedy]

I have a few quick questions and observations to kick things off.

Is A9.com still the best location for OpenSearch namespace URI?
OpenSearch.org seems like a better choice.

What is the status of IANA registration of the OpenSearch MIME type
application/opensearchdescription+xml?

Should OpenSearch model its IANA considerations after Atom's example?
http://www.apps.ietf.org/rfc/rfc4287.html#sec-7

Why is AdultContent so liberal in its accepted values? 5 accepted
values for a negative result. Specifying a single desired behavior
will help clients and publishers meet spec.


Thanks,
-Niall

[DeWitt Clinton]

Responses inline.

On Wed, Oct 1, 2008 at 11:51 PM, Niall Kennedy <ni...@niallkennedy.com> wrote:

I have a few quick questions and observations to kick things off.

Is A9.com still the best location for OpenSearch namespace URI?
OpenSearch.org seems like a better choice.

Agreed.  Personally I'd love to move it, and plan on putting future specs under the opensearch.org namespace, but for the 1.1 series we don't want to break things arbitrarily, so for this draft things will have to stay under a9.com.

What is the status of IANA registration of the OpenSearch MIME type
application/opensearchdescription+xml?

Frank Ellermann kindly started a draft that could be filed for registration, but I believe there was a little bit of work left to be done.  See:
 
  http://tools.ietf.org/html/draft-ellermann-opensearch-01

Should OpenSearch model its IANA considerations after Atom's example?
http://www.apps.ietf.org/rfc/rfc4287.html#sec-7

Yes, the Ellermann draft does it this way.
 

Why is AdultContent so liberal in its accepted values? 5 accepted
values for a negative result. Specifying a single desired behavior
will help clients and publishers meet spec.

No particular reason, other than people like to express truthiness in all sorts of ways.  Probably not a good decision of mine in retrospect.

I'll run a test later and see what currently exists out there in description documents on the web.  If there isn't much variance in the ways people are using it in the real world (if they're using it at all) then we can discuss dropping the extra values.

I just checked and AtomPub uses "yes" and "no" as the two valid values for its one boolean attribute.  Unfortunately, those aren't in the 5 that AdultContent currently accepts.


Thanks for the feedback.  Please keep it coming.

-DeWitt

[DeWitt Clinton]

On Thu, Oct 2, 2008 at 12:18 AM, DeWitt Clinton <dew...@opensearch.org> wrote:

I just checked and AtomPub uses "yes" and "no" as the two valid values for its one boolean attribute.  Unfortunately, those aren't in the 5 that AdultContent currently accepts.

Actually, I take that back.  "yes" and "no" are valid, and they mean what you'd expect them to.

-DeWitt

[Steven Livingstone-Perez]

I need to read the document in full - and I am doing so - but in short...

Per the adult content - I wonder whether we could make this polymorphic, or
employ another element and use something like the Content Labels [1], [2]
(the replacement for PICS).

I 100% agree with making the basic selection fairly simple - I wonder
whether there is an option to allow users to extend this if necessary.

Thoughts?

steven
http://livz.org

[1] http://contentlabel.org/
[2] http://www.w3.org/2004/12/q/doc/content-labels-schema.htm

[Steven Livingstone-Perez]

I added some thoughts below. This is the first time I have read through the spec (some really exciting things in there!) and I wasn’t involved in any prior discussions so some parts may be wayward (apologies up front), but maybe one or two useful things will come out of it.

 

Btw, the link to Plone on “OpenSearch_software” has an extra “http” at the front, breaking the link.

 

steven

http://livz.org

 

 

 

OpenSearch Observations

------------------------

 

Namespace

=========

 

The namespace for a future draft could probably be moved to http://www.opensearch.org – this should affect any compatibility with previous version written against the old namespace.

 

 

Security

========

I know his is perhaps a bit early stage and certainly a major “add” but I wonder whether some security considerations could be considered. When advertising your OpenSearch description document you may wish to indicate that authentication may be required – perhaps against all or a subset of domains. In doing this, a client who is using the description document could use something like oAuth to get valid tokens that could be passed with the search and would then allow querying against backend sources as an authenticated user.

 

This would be useful on two fronts :

 

(1) that many sites don’t seem keen on handing out syndicated search results and perhaps would be more at ease with some pre-authentication and

(2) you could use other personalization features to provide contextually relevant results to the client.

 

 

 

Contact

=======

Just a thought, but I wonder whether content could be of multiple types such as “Email”, “XMPP” and so on. I have ideas on where that could go but a bit out of scope for just now I think.

 

 

 

Tags

====

I like tags, but often run into difficulties such as “New York” and so on. I wonder whether we could use a comma or semi-colon delimiter or give guidance on how to mark multiple words as a single logical work (such as syntax such as New_York).

 

 

 

The “Query” element

===================

I’d like to see this paragraph in the document expanded on a little (perhaps it will be a little more obvious when I read later on the in the document).

 

 

 

The "Developer" element

=======================

I’d be interested in where this came from, but I do wonder whether Name and Contact child elements would be valuable here – for example for technical support.

 

 

 

The "Attribution" element

=========================

I could see this extending for a host of reasons.

 

1.            In a search feed I can imagine the number of sources could be quite large and many ask that you identify this. 256 characters wouldn’t be enough for some searches.

 

2.            Going back to my security point above. If would could identify the sources of the data within the feeds, we could tie oAuth into this and allow users to query protected  (and perhaps personalized) content.

Perhaps these two points could be added to a separate element and something like an IDREF to point at the appropriate source, but worth pointing out.

 

I do wonder whether this could be extended to add optional data about the sources such as URL, LastCache (because many real-time RSS sources don’t allow you to hit them all the time so the user may sometimes have to get data that is slightly out of data and this may be worth indicating), NextUpdate (the converse to LastCache).

 

 

 

The "SyndicationRight" element

==============================

How does “closed” work in the real world?

 

 

 

 

The "AdultContent" element

==========================

Per the adult content - I wonder whether we could make this polymorphic, or employ another element and use something like the Content Labels (the replacement for PICS).

 

I 100% agree with making the basic selection fairly simple - I wonder whether there is an option to allow users to extend this if necessary

 

 

 

The "Language" element

======================

Would this not have to be incorporated into the OpenSearchDescription ? So using the xml:lang attribute we could allow others to pull in a description in their language? I’m not sure how this would be implemented – by separate URL’s for requesting the description document for an appropriate language? Are there any sites that provide descriptions in multiple languages and how is that currently achieved?

 

 

 

OpenSearch 1.1 parameters

=========================

It may be useful to have a “GUID” as a parameter in here. This could be used to refer to state on the server associated with the user search. I am working with FAST just now which uses some facilities which are highly dependent on state (otherwise you need to re-execute multiple queries to get to the “starting position” of your current query).

 

State isn’t always great, but in some cases for performance reasons it is clearly needed and an optional GUID could be used to permit OpenSearch state (short or long term depending on implementation).

 

 

 

Suggested Template

==================

This is simply something that occurred to me based on some work I’ve done recently. Many sites that provide syndicated data interpret what is to go into the elements differently – many putting fully marked up data into the “Content” section. This ignores the fact that sites provide feeds in Atom and RSS (in my case I normalized to Atom for all feeds).

It may be nice if in the description of the OpenSearch if they could optionally specify one or more Xslt templates that could be used to display a row from a given source. This could be particularly useful when combining data from multiple sources into a single output. The output of a single Xslt for an Atom source is varied – some sites add media, others have different semantic interpretations of the elements and so on.

 

 

 

Finally

=======

Based on a client I am working with who wishes to syndicate their data to partners securely with a large FAST backend database, there are a number of other features such as categories, similarity searching and so on I could imagine being part of syndicated search interface, but at the moment this may only complicate things so I’ll ignore adding anything on this for now.

[Steven Livingstone-Perez]

An errata to what I mentioned on Content Labels – POWDER is what it’s now termed and could be useful.

 

http://www.w3.org/TR/powder-formal/

 

Perhaps some of it could be useful – e.g. the tagset.

 

steven

http://livz.org

[BrandonLive]

Things I would like to see added to Draft 4:

1) The "rel" attribute on the <Url> element, which we talked about in
the past. Default value is "search" or "results" or something.
Ideally, alternates could be things like:
"suggestions" - search suggestions
"list" - a list of all items contained in this search scope (basically
a "*" query)
"preview" - to accomplish IE8's preview page goal
etc.

2) "MaximumResultsCount" or something along those lines. Some
sources cap the number of results returned, such as Live.com which
maxes out at 200 or 250 (I forget exactly).
This would be useful for client UI to tell the user upfront (before a
query is initiated) how many results are being returned. Servers
could also communicate to clients that they don't want more than this
number of results to be enumerated.


Thanks,
Brandon

On Oct 2, 4:53 am, "Steven Livingstone-Perez" <webl...@hotmail.com>
wrote:
> An errata to what I mentioned on Content Labels - POWDER is what it's now

> termed and could be useful.
>
> http://www.w3.org/TR/powder-formal/
>

> Perhaps some of it could be useful - e.g. the tagset.

>
> steven
>
> http://livz.org
>
> From: opens...@googlegroups.com [mailto:opens...@googlegroups.com] On
> Behalf Of Steven Livingstone-Perez
> Sent: 02 October 2008 10:39
> To: opens...@googlegroups.com
> Subject: [opensearch:171] Re: Started Draft 4
>
> I added some thoughts below. This is the first time I have read through the
> spec (some really exciting things in there!) and I wasn't involved in any
> prior discussions so some parts may be wayward (apologies up front), but
> maybe one or two useful things will come out of it.
>
> Btw, the link to Plone on "OpenSearch_software" has an extra "http" at the
> front, breaking the link.
>
> steven
>
> http://livz.org
>
> OpenSearch Observations
>
> ------------------------
>
> Namespace
>
> =========
>

> The namespace for a future draft could probably be moved tohttp://www.opensearch.org- this should affect any compatibility with

> previous version written against the old namespace.
>
> Security
>
> ========
>
> I know his is perhaps a bit early stage and certainly a major "add" but I
> wonder whether some security considerations could be considered. When
> advertising your OpenSearch description document you may wish to indicate

> that authentication may be required - perhaps against all or a subset of

> Contact child elements would be valuable here - for example for technical

> in their language? I'm not sure how this would be implemented - by separate

> URL's for requesting the description document for an appropriate language?
> Are there any sites that provide descriptions in multiple languages and how
> is that currently achieved?
>
> OpenSearch 1.1 parameters
>
> =========================
>
> It may be useful to have a "GUID" as a parameter in here. This could be used
> to refer to state on the server associated with the user search. I am
> working with FAST just now which uses some facilities which are highly
> dependent on state (otherwise you need to re-execute multiple queries to get
> to the "starting position" of your current query).
>
> State isn't always great, but in some cases for performance reasons it is
> clearly needed and an optional GUID could be used to permit OpenSearch state
> (short or long term depending on implementation).
>
> Suggested Template
>
> ==================
>
> This is simply something that occurred to me based on some work I've done
> recently. Many sites that provide syndicated data interpret what is to go

> into the elements differently - many putting fully marked up data into the

> "Content" section. This ignores the fact that sites provide feeds in Atom
> and RSS (in my case I normalized to Atom for all feeds).
>
> It may be nice if in the description of the OpenSearch if they could
> optionally specify one or more Xslt templates that could be used to display
> a row from a given source. This could be particularly useful when combining
> data from multiple sources into a single output. The output of a single Xslt

> for an Atom source is varied - some sites add media, others have different

[Steven Livingstone-Perez]

"MaximumResultsCount"

True, Google returns 1000 records and FAST returns 1010 (by default) - would
be useful as users sometimes try to go further than this and get nothing
back.

[BrandonLive]

Hey Dewitt -

I know you're very busy with your real job and all, but I was just
wondering if you had any idea when some of the Draft 4 updates might
start to get included.

On Oct 2, 11:22 pm, "Steven Livingstone-Perez" <webl...@hotmail.com>
wrote:

> tohttp://www.opensearch.org-this should affect any compatibility with

> > may only complicate things so I'll ignore adding anything on this for now.- Hide quoted text -
>
> - Show quoted text -

[Niall Kennedy]

I think the best path forward is to create separate threads for each
proposed feature. We need to identify possible changes, work through
any issues raised by such changes, and document possible
implementations for the benefit of both publishers and consuming
agents.

In other words, how can "what if" be turned into "here's how" for
others to poke holes and move ideas towards incorporation. Ultimate
changes fall to Dewitt in his BDFL role, but there is some definite
prep work for interested parties.

So far I see a few possible proposal areas for changes, edits, and
clarifications in 1.1 based on this thread:

* AdultContent
* Url
* Tags

-Niall

> > tohttp://www.opensearch.org-thisshould affect any compatibility with

[DeWitt Clinton]

Thanks, Niall.  Agreed wholeheartedly.  I would love for people to take the initiative and create a wiki page or start some threads about specific proposals.

A few I will bring to the table myself:

  - Clarification of language regarding paging and indexes.  Not a change of behavior, just better editorial to clear up ambiguity.
  - Addition of two moz-specific features, UpdateUrl and UpdateInterval, to the core description document spec and namespace
  - Addition of JSON encoded responses if we can agree on a format

-DeWitt

[Steven Livingstone-Perez]

Agree. A wiki would be nice to pull discussion together.

 

http://opensearch.wetpaint.com/

 

I have structured and cross-referenced according to the current discussion. Please feel free to hack away.

 

-steven

[DeWitt Clinton]

Also, opensearch.org is a wiki.

-DeWitt

[Steven Livingstone-Perez]

Ah crap – didn’t realize it was open to anyone to add to.

 

I put the content I had on that wetpaint wiki here:

 

http://www.opensearch.org/User_talk:Weblivz

 

Needs cleaning up a little as I lost all formatting. I will have the WetPaint site removed.

[DeWitt Clinton]

Thanks!  I didn't mean to imply the wetpaint site was a bad idea, just mentioning that opensearch.org was also a wiki if people wanted to use it.  I've locked down most of the actual spec pages to prevent random edits, but the rest of the site is open and fair game for us to play with.

-DeWitt

[Steven Livingstone-Perez]

Yes, having a central location for this probably makes more sense (as does locking down the key pages!).

 

Anyone - feel free to copy, merge, change, obliterate what I have added as things progress.

[Christian Ledermann]

On Thu, 2008-10-09 at 06:06 -0700, DeWitt Clinton wrote:
> Also, opensearch.org is a wiki.

yes please keep it in one place.

>
> -DeWitt
>
> On Thu, Oct 9, 2008 at 1:39 AM, Steven Livingstone-Perez
> <web...@hotmail.com> wrote:
> Agree. A wiki would be nice to pull discussion together.
>
>
>
> http://opensearch.wetpaint.com/
>
>

opensearch.org is the better place to discuss that, but if you have good
suggestions how to streamline it I am sure you will not meet much
resistance.

>
>
--
Best Regards,

Christian Ledermann

[lexpauld]

Can you explain the test you run to see what currently exists?

Where on the web do you look?

I have bookmarked some at
http://delicious.com/mitrepauld/osdfxml

Paul

On Oct 2, 3:18 am, "DeWitt Clinton" <dew...@opensearch.org> wrote:
> Responses inline.
>
> On Wed, Oct 1, 2008 at 11:51 PM, Niall Kennedy <ni...@niallkennedy.com>wrote:
>

...

>
> I'll run a test later and see what currently exists out there in description
> documents on the web.  If there isn't much variance in the ways people are
> using it in the real world (if they're using it at all) then we can discuss
> dropping the extra values.
>

...