Thank you for publishing this to the group.
There is indeed a lot of closeness here, which is in the first place due to the syntactic similarities of SPARQL and N3 (variables etc).
Also, the timing rather coincides.
I started working on this for my master’s thesis, available at http://lib.ugent.be/fulltxt/RUG01/001/418/670/RUG01-001418670_2010_0001_AC.pdf
There, you see how I embed N3 expressions in OWL-S.
The first paper was presented at ISWC’10, and detailed our use of SPARQL endpoints: https://biblio.ugent.be/input/download?func=downloadFile&fileOId=1168482
The framework using N3 to express functional relations was accepted for MTAP in October 2010 : http://www.springerlink.com/content/kn92617280296473/
As you can see, the emphasis is on composition with N3 reasoners. We closely work together with Jos De Roo, the author of the Eye reasoner.
Then, Thomas published a paper for WS-REST 2011: http://ws-rest.org/2011/proc/a3-steiner.pdf
That paper influenced the move away from OWL-S to a complete N3-based solution, which is the one mentioned in the slides.
We also submitted a journal article on RESTdesc, which is currently pending review, wherein we cite LOS. I first heard about LOS when meeting Barry at ESWC, when the NWeSP paper had already been submitted.
I’m really sorry if this has caused any confusion. Apparently, great minds think alike ;-)
The NWeSP conference was the first time we talked publicly about RESTdesc, so this might explain why this only surfaces now.
I’m looking forward to discussing this with you.
Cheers,
Ruben
Previously ESWC 2010 had turned down Reto and my paper, but a helpful
reviewer pointed out Marco Luca Sbodio which used the SPARQL patterns
with variable quantification in OWL-S back in 2007. (Great minds,
indeed, think alike)
Our first LOS paper, with this added related work, was resubmitted, and
had already been published at FIS in September 2010
(http://www.linkedopenservices.org/publications/FIS2010.pdf).
Our other "great minds think alike" moment was when I moved from
Innsbruck to Karlsruhe and realised Andreas and Sebastian were working
on something very similar. Their first publication was also in September
2010, at I-SEMANTICS
(http://www.aifb.kit.edu/images/4/4a/Triplify-2010-ssp-aha-taking-the-lids-off-data-silos.pdf).
Barry
On 20.10.2011 16:33, Ruben Verborgh wrote:
> Hi Barry,
> Hi all,
>
> Thank you for publishing this to the group.
>
> There is indeed a lot of closeness here, which is in the first place due to the syntactic similarities of SPARQL and N3 (variables etc).
> Also, the timing rather coincides.
>
> I started working on this for my master�s thesis, available at http://lib.ugent.be/fulltxt/RUG01/001/418/670/RUG01-001418670_2010_0001_AC.pdf
> There, you see how I embed N3 expressions in OWL-S.
>
> The first paper was presented at ISWC�10, and detailed our use of SPARQL endpoints: https://biblio.ugent.be/input/download?func=downloadFile&fileOId=1168482
>
> The framework using N3 to express functional relations was accepted for MTAP in October 2010 : http://www.springerlink.com/content/kn92617280296473/
> As you can see, the emphasis is on composition with N3 reasoners. We closely work together with Jos De Roo, the author of the Eye reasoner.
>
> Then, Thomas published a paper for WS-REST 2011: http://ws-rest.org/2011/proc/a3-steiner.pdf
> That paper influenced the move away from OWL-S to a complete N3-based solution, which is the one mentioned in the slides.
>
> We also submitted a journal article on RESTdesc, which is currently pending review, wherein we cite LOS. I first heard about LOS when meeting Barry at ESWC, when the NWeSP paper had already been submitted.
>
> I�m really sorry if this has caused any confusion. Apparently, great minds think alike ;-)
> The NWeSP conference was the first time we talked publicly about RESTdesc, so this might explain why this only surfaces now.
>
> I�m looking forward to discussing this with you.
I hope they're more than syntactic - the quantification you expect I
believe is exactly what LOS and LIDS have in mind. (I.e. this is a
common semantics.)
Getting more service descriptions out there in this form can only
benefit us all - whether we apply (FOL?) rule-based reasoning, query
decomposition (as in LIDS) or manual composition over a triple space
(which is my interest).
One concern that I have about including the (HTTP) 'plumbing' into the
service descriptions is that it (presumably) breaks an implicit
fundamental of Linked Service validation - that the input (/output)
graph pattern can be applied, as an ASK query, to the input (/output) to
validate the message contents. Presumably these are not populated in the
RDF if this is the form the message takes?
This is why I sketched (probably incorrectly) how I believe a model like
the MSM can be used to record this information in RDF form.
In general (and I apologise for not having read both papers properly
yet), it looks like you don't require that the messages are (at least
when requested) represented in RDF. Certainly Thomas' service seems only
to be made available in JSON. Is this the case? If not, is there any
formal relationship between the messages and the descriptions?
Barry
I hope they're more than syntactic - the quantification you expect I
believe is exactly what LOS and LIDS have in mind. (I.e. this is a
common semantics.)
There is. The argumentation I used for the presentation why we use N3, is because of quantification.
I found this LOS example somewhere:
geo:search rdf:type posm:Service;posm:hasInputMessage [sawsdl:modelReference _:input];posm:hasOutputMessage [sawsdl:modelReference _:output];sawsdl:loweringSchemaMapping_:input rdf:value "?x geo:name ?name ."ˆˆsparql:GraphPattern ._:output rdf:value "?x wgs84:lat ?lat; wgs84:long ?lng ."ˆˆ..
Since RDF doesn't have quantification, LOS uses a kind of metalevel, by resorting to strings that contain SPARQL, which does provide quantification.
This is the part where RESTdesc differs. Below is a draft of what it would look like:
@prefix geo: <http://ws.geonames.org/>.@prefix wgs84: <http://www.w3.org/2003/01/geo/wgs84_pos#>.@prefix http: <http://www.w3.org/2006/http#>.@prefix tmpl: <http://purl.org/restdesc/http-template#>.
{ ?point geo:name ?name. }
=>{ _:request http:methodName "GET";
tmpl:requestURI ("/places/" ?name);http:resp [ tmpl:represents ?point ].?point wgs84:lat ?lat; wgs84:long ?lng.}.
The main difference here is that the meaning is defined by the N3 specification. The "wgs84:lat", for example, is just a URI. The meaning is integrated in RESTdesc.SPARQL strings, on the contrary, form a (kind of) separate world, that requires separate interpretation.
So a big difference, according to me, is integrated versus separate.
Getting more service descriptions out there in this form can only
benefit us all - whether we apply (FOL?) rule-based reasoning, query
decomposition (as in LIDS) or manual composition over a triple space
(which is my interest).
Yes, and this is really cool. Our approach was developed with composition in mind. "Composition" here means really automated composition of course, in the sense of creating a chain of services that fulfills a complex function. (As opposed to the parameter matching that requires human intervention.)
Therefore, we have close contacts with Jos De Roo, developer of the EYE reasoner, which we use to create these compositions. Another benefit here is that EYE is just a regular N3 reasoner without RESTdesc-specific stuff. Still, it manages to create complex compositions. (See the MTAP journal paper.)
One concern that I have about including the (HTTP) 'plumbing' into the
service descriptions is that it (presumably) breaks an implicit
fundamental of Linked Service validation - that the input (/output)
graph pattern can be applied, as an ASK query, to the input (/output) to
validate the message contents. Presumably these are not populated in the
RDF if this is the form the message takes?
That's correct.
In general (and I apologise for not having read both papers properly
yet), it looks like you don't require that the messages are (at least
when requested) represented in RDF.
Indeed. The thing is, we are concerned with general service descriptions for services that input anything and output anything. We consider the choice "RDF or not" to be on the level of content negotiation. This is just a philosophy of course.
Certainly Thomas' service seems only
to be made available in JSON. Is this the case? If not, is there any
formal relationship between the messages and the descriptions?
There can be a formal relationship, but this is not required.
Cheers,
Ruben
> There are three separate issues here:
>
> 1) I don't know what you mean regarding "wgs84:lat" - its semantics, beyond being a particular identified predicate, come from the associated schema, and inference over that is not part of n3. (Here SPARQL has the advantage of the work on entailment regimes.)
What I meant here is also part of the metalevel argument. The “wgs84:lat” part of the string has no meaning implied from the syntax. You’ll have to explicitly interpret the string to connect meaning to the predicate. (At this moment, I’m not really sure how the wgs4 prefix is then tied to the string, but that’s a separate issue.) And this interpretation is not a part of the standard RDF interpretation; you need a separate interpreter to understand the SPARQL strings.
In Notation3, the wgs84:lat is a regular property that has meaning in the language itself. The property is interpreted by the same mechanism that interprets the whole description. Therefore, wgs84:lat means something to the N3 parser (it’s a property), whereas with the SPARQL literal, it is just a part of a string (and even not identified individually).
> 2) I take your point about the uniform semantics applied across the whole description in one language (which is why LIDS abused CONSTRUCT queries, to get this in SPARQL, in my eyes), but to me you're also abusing rules. Your rules introduce unbound variables precisely because they don't (as these descriptions shouldn't) capture the whole semantics of the services and are no more sensibly executable than LIDS pseudo-transforms (CONSTRUCTs).
I believe there’s somewhat a misunderstanding here, partly due to the way I formulated the RESTdesc version of the service description. When rapidly getting the description together, I accidentally used variable syntax for latitude and longitude. Although this is strictly spoken not incorrect (“unbound” variables are just anonymous nodes), here is a RESTdesc description that more closely fits the examples of my slides and the paper:
@prefix geo: <http://ws.geonames.org/>.
@prefix wgs84: <http://www.w3.org/2003/01/geo/wgs84_pos#>.
@prefix http: <http://www.w3.org/2006/http#>.
@prefix tmpl: <http://purl.org/restdesc/http-template#>.
{ ?point geo:name ?name. }
=>
{ _:request http:methodName "GET";
tmpl:requestURI ("/places/" ?name);
http:resp [ tmpl:represents ?point ].
?point wgs84:lat _:lat; wgs84:long _:lng.
}.
Maybe let me walk you through this description, to show you that it indeed captures the whole semantics of the service.
A possible English translation of the above rule would be
IF you have a point with a certain name
THEN you can perform an HTTP GET request to /places/{name of the point},
which will return a response representing a point, that has a certain latitude and longitude.
As you can see, this description provides both:
- the functional aspect (what the service does);
- the operational aspect (how you should invoke the service).
To see this in action, let’s try this with a point, for instance:
<http://example.org/Paris> geo:name "Paris”.
If we put this together with the service description in a reasoner, we get (after formatting):
#eye --nope --pass http://n3test.restdesc.org/restdesc/geonames/{places.n3,myplace.n3}
<http://example.org/Paris> <http://ws.geonames.org/name> "Paris";
wgs84:lat [];
wgs84:long [].
_:x :requestURI ("/places/" "Paris");
http:methodName "GET";
http:resp [:represents <http://example.org/Paris> ]].
Which reads as: there exists a GET request to /places/Paris which will return a representation of Paris. And this presentation includes the latitude and the longitude. Of course, we don’t know what the exact coordinate values are, but we know how to obtain them. (Note that you can use the reasoner proof to verify which request supplies what values.)
We also get the exact same input with the description of my previous mail (using the unbound variables, which are equally anonymous nodes):
#eye --nope --pass http://n3test.restdesc.org/restdesc/geonames/{places_unbound.n3,myplace.n3}
So I’m not sure if your point about abusing rules still holds: I’m using rules in a standard way that can be used by a standard N3 reasoner, and they tell me what request I have to make to obtain my goal. I think it’s the most correct use of rules there is.
> 3) There actually is a "meta" separation, which you've lost, between message metadata and message contents. I understand though that, since you don't deal with RDF messages (as below), this distinction is less meaningful for you.
Yes, but this is another metadata level. Then, you are talking about the RDF response, which is a document, and therefore indeed on another level. You cannot escape this fact directly. But this is why we use the “represents” predicate, which brings this meta level back to our reasoning level, so we can talk about what is inside the resource.
For instance, in the example above, the response might be in RDF format (or JSON or HTML or whatever), but we say that it will have a latitude and longitude. Of course, an RDF agent would indicate its preference of RDF using content negotiation.
Cheers,
Ruben
On 10/21/11 12:42 AM, "Ruben Verborgh" <ruben.v...@ugent.be> wrote:
>Hi Barry,
>
>> There are three separate issues here:
>>
>> 1) I don't know what you mean regarding "wgs84:lat" - its semantics,
>>beyond being a particular identified predicate, come from the associated
>>schema, and inference over that is not part of n3. (Here SPARQL has the
>>advantage of the work on entailment regimes.)
>
>What I meant here is also part of the metalevel argument. The ³wgs84:lat²
>part of the string has no meaning implied from the syntax. You¹ll have to
>explicitly interpret the string to connect meaning to the predicate. (At
>this moment, I¹m not really sure how the wgs4 prefix is then tied to the
>string, but that¹s a separate issue.) And this interpretation is not a
>part of the standard RDF interpretation; you need a separate interpreter
>to understand the SPARQL strings.
>In Notation3, the wgs84:lat is a regular property that has meaning in the
>language itself. The property is interpreted by the same mechanism that
>interprets the whole description. Therefore, wgs84:lat means something to
>the N3 parser (it¹s a property), whereas with the SPARQL literal, it is
>just a part of a string (and even not identified individually).
I agree that the LOS input and output descriptions reside on another
level, by being encoded as strings. And the individual elements of the
graph patterns can not be interpreted as long as you are on the level of
the rdf description. But as a whole, the strings not meaningless: it can
be derived from the rdf that they represent input and output (personally I
additionally tag them with the datatype sparql:graphpattern). So all the
information on how to interpret them is given. As Barry mentioned the
patterns are used for things like validation, discovery and composition,
but for these tasks a special interpreter has to be created anyway.
For me the appeal of using rdf descriptions lies in the integration into
the LOD cloud. A service resource (i.e. the URI that identifies a service,
against which rdf input data can be posted) can be resolved via HTTP GET
and point (303) to the rdf service description. This directly follows the
third LOD principle and allows arbitrary agents to deal with service
resources like with any other URI in the LOD cloud (only the ones
concerned with service validation etc. need to be able to dive in the
deeper level of the graph patterns to fulfill their task).
Btw. I completely agree with the critique about the namespaces in the
graph patterns. This is something a description completely on one level
(like RESTdec) solves automatically. At the moment I assume the namespace
of the rdf description can be applied to the patterns, which is a horrible
break of the different levels. I would propose to describe the namespace
the service operates under with a additional property (and possibly
datatype):
[] a msm:service;
:serviceNamespace "PREFIX wgs84: ..."^^sparql:Namespace
This would also push the namespace into the other level and be helpful
when writing compositions (but thats a completely different story).
Regards,
Steffen
Thanks for taking the time to dive into this.
> I agree that the LOS input and output descriptions reside on another
> level, by being encoded as strings. And the individual elements of the
> graph patterns can not be interpreted as long as you are on the level of
> the rdf description. But as a whole, the strings not meaningless: it can
> be derived from the rdf that they represent input and output (personally I
> additionally tag them with the datatype sparql:graphpattern). So all the
> information on how to interpret them is given.
This is indeed a valid point: all information is given and they contain a meaning. But that meaning has to be enforced manually, i.e., by adding a SPARQL pattern interpreter.
> As Barry mentioned the
> patterns are used for things like validation, discovery and composition,
> but for these tasks a special interpreter has to be created anyway.
The benefit of the RESTdesc descriptions in N3 is that special tools are not necessary: we just use N3 reasoners. For discovery, we use HTTP OPTIONS (see below).
> For me the appeal of using rdf descriptions lies in the integration into
> the LOD cloud.
Yes, it is true that the RDF description integrates directly into the LOD cloud.
But that same cloud is indeed an RDF cloud – RDF only. The SPARQL strings have no meaning there anyway. This means the important semantics do not get integrated. So I don’t think there’s much of a benefit there.
I’ve been thinking along similar lines previously in the aformentioned MTAP paper. We embedded the preconditions and postconditions as string literals inside an OWL-S description. This was fully RDF, so it integrated with the LOD cloud. But the string literals would just be literals, and the important meaning gets lost. This was not what we wanted, so we moved on to N3.
I often compare this to HTML plus JavaScript. You can have a perfectly valid HTML document with syntactically correct JavaScript. But if you have a browser with JavaScript support disabled, you only see the HTML. The script just does not get interpreted. So if it is required for the page to render correctly (which is obviously a bad idea), then the end result is wrong. Although the HTML is perfectly valid and fits into the rest of the Web.
There is a similar thing with expressions inside RDF string literals. Yes, they integrate beautifully. But the important functional semantics get lost, so the integration doesn’t bring you anything. Since our focus is on the functionality, we had to find another way.
> A service resource (i.e. the URI that identifies a service,
> against which rdf input data can be posted) can be resolved via HTTP GET
> and point (303) to the rdf service description.
The thing is, we target resource-oriented (“REST”) Web services. So there’s not really a URI that identifies the service, only the URI that identifies your answer. The service is not our resource, it results are.
For example, with the places description, we would just have:
http://example.org/places/Paris
This is a resource, not a service. This means: there might be some service behind this, it might be a static page, it might be something else… but that doesn’t matter to the user. The user just gets the resource, and we describe what the result means.
Because our service is not the resource, we don’t put the description there. Instead, we ask to each resource what options it has.
For example: an HTTP OPTIONS request to http://example.org/places would return the aforementioned description.
In short: RESTdesc describes how to access resources and what it means to access a certain resource. (Which of course, in practice, often maps to services internally, so that’s why it’s a service description method.)
> This directly follows the
> third LOD principle and allows arbitrary agents to deal with service
> resources like with any other URI in the LOD cloud (only the ones
> concerned with service validation etc. need to be able to dive in the
> deeper level of the graph patterns to fulfill their task).
So this indeed seems another important distinction: we don’t see services as resources. Thanks for clarifying this.
Best,
Ruben
Indeed. But the two languages that need interpreters (RDF and SPARQL)
are Semantic Web / Linked Data standards, unlike Notation3, which is
clearly not.
>> As Barry mentioned the
>> patterns are used for things like validation, discovery and composition,
>> but for these tasks a special interpreter has to be created anyway.
> The benefit of the RESTdesc descriptions in N3 is that special tools are not necessary: we just use N3 reasoners. For discovery, we use HTTP OPTIONS (see below).
Yes, I was going to say this myself in reply to Steffen. I really like
the OPTIONS way of giving a description. My LOS implementations use
JAX-RS (having changed from Restlet), which gives a (non-semantic)
default (WADL) implementation for OPTIONS. It was always an aim, if I'd
stayed at the university, to have a student look at a LOS-JAX-RS
extension to (semi-)automatically give an RDF response to this.
What is not clear to me (I should check into it) is whether an Accept
header is expected for OPTIONS.
Ironically I'd followed DBPedia in (somewhat incorrectly) using text/n3
as the primary human-readable RDF media type in LOS implementation.
Otherwise you could foresee (though unfortunately there is no SPARQL
Internet Media type - only for the result formats) a system where you
could get LOS, LIDS and RESTdesc descriptions from:
OPTIONS /service
Accept: application/x-turtle
(LOS)
OPTIONS /service
Accept: text/n3
(RESTdesc)
and
OPTIONS /service
Accept: application/x-sparql (or something)
(LIDS)
Of course the REST way to do this is with extended types - e.g.,
text/los-turtle (if/when the text/turtle registration is accepted),
text/restdesc-n3, etc. (I must admit I got this very wrong, and used an
unregistered secondary HTTP header, in the following paper:
http://ceur-ws.org/Vol-596/paper-09.pdf)
>
>> For me the appeal of using rdf descriptions lies in the integration into
>> the LOD cloud.
+1
> Yes, it is true that the RDF description integrates directly into the LOD cloud.
> But that same cloud is indeed an RDF cloud � RDF only. The SPARQL strings have no meaning there anyway. This means the important semantics do not get integrated. So I don�t think there�s much of a benefit there.
Nonsense. The LOD cloud is drawn from CKAN and SPARQL endpoints are an
important part of CKAN registration (LOD = resolvability + dumps +
SPARQL, for me).
See Linked Data Principle 3: http://www.w3.org/DesignIssues/LinkedData.html
> I�ve been thinking along similar lines previously in the aformentioned MTAP paper. We embedded the preconditions and postconditions as string literals inside an OWL-S description. This was fully RDF, so it integrated with the LOD cloud. But the string literals would just be literals, and the important meaning gets lost. This was not what we wanted, so we moved on to N3.
OWL-S always had this problem. At least RIF and SPARQL literals have the
advantage over SWRL (the OWL-S recommendation) and N3 literals in using
standard languages.
> I often compare this to HTML plus JavaScript. You can have a perfectly valid HTML document with syntactically correct JavaScript. But if you have a browser with JavaScript support disabled, you only see the HTML. The script just does not get interpreted. So if it is required for the page to render correctly (which is obviously a bad idea), then the end result is wrong. Although the HTML is perfectly valid and fits into the rest of the Web.
To me that's a good thing. There is useful information in the pure RDF
part of the MSM-LOS description I showed. That information is hidden in
a N3 rule (requiring a FOL reasoner) in the equivalent RESTdesc.
> There is a similar thing with expressions inside RDF string literals. Yes, they integrate beautifully. But the important functional semantics get lost, so the integration doesn�t bring you anything. Since our focus is on the functionality, we had to find another way.
Doesn't "bring anything"? They "get lost"? I just don't understand that
point of view. As I said above, the entire semantics "get lost" in a
non-standard (indeed, stigmatised) language when everything is thrown
into N3. I don't say this lightly, by the way (in fact I apologise): I
spent years getting papers rejected on exactly this "non-standard
language" point, using OCML and WSML, when no standard language
supported rules! I imagine you'll be similarly upset by my assertion
(though RIF does exist now).
>> A service resource (i.e. the URI that identifies a service,
>> against which rdf input data can be posted) can be resolved via HTTP GET
>> and point (303) to the rdf service description.
> The thing is, we target resource-oriented (�REST�) Web services. So there�s not really a URI that identifies the service, only the URI that identifies your answer. The service is not our resource, it results are.
Yes, I somewhat agree with you here... I think 'result' is wrong though.
URIs identify resources (here RDF and REST agree) - they do not identify
representations, i.e. your 'results'. Representations are what go in
HTTP bodies, and when you GET them they're subject not just to the
resource identified, but the media type requested (Accepted).
> For example, with the places description, we would just have:
>
> http://example.org/places/Paris
>
> This is a resource, not a service. This means: there might be some service behind this, it might be a static page, it might be something else� but that doesn�t matter to the user. The user just gets the resource, and we describe what the result means.
This sounds naive. Do RESTdesc descriptions perhaps, like LIDS, only
cover the retrieval of representations (i.e. HTTP GET). Or do you
describe also PUT/PATCH/POST/DELETE?
> Because our service is not the resource, we don�t put the description there. Instead, we ask to each resource what options it has.
> For example: an HTTP OPTIONS request to http://example.org/places would return the aforementioned description.
> In short: RESTdesc describes how to access resources and what it means to access a certain resource. (Which of course, in practice, often maps to services internally, so that�s why it�s a service description method.)
Again, I totally agree with the use of OPTIONS (but worry about both the
REST and the LD use of Accept headers - LD current practice is
completely naive to this, almost all crawlers being unaware of extension
types; REST practice may also not apply Accept headers to OPTIONS requests).
>
>> This directly follows the
>> third LOD principle and allows arbitrary agents to deal with service
>> resources like with any other URI in the LOD cloud (only the ones
>> concerned with service validation etc. need to be able to dive in the
>> deeper level of the graph patterns to fulfill their task).
> So this indeed seems another important distinction: we don�t see services as resources. Thanks for clarifying this.
What do you see them as?
(By the way, I apologise a second time if my tone seems combative - it
is usual for me - I think this is an excellent discussion and I hope
it's already clear that I'm thinking how these approaches can be brought
together and we can get some cross-fertilisation)
Barry
I want to focus on other points than using SPARQL or N3 or whatever to represent rules. I think RESTdesc is an interesting approach, but there are still some points that are not clear for me how they work:
- Mixing of description of functionality and access method, which is powerful, because you can construct the service call URIs in a flexible way, but it can be confusing for humans/hard for computers to seperate the functionality part from the access part, in situations where only functionality is of interest. I don't know whether this is a good or bad thing, but it is different from LIDS and LOS.
- I assume the vocabulary "tmpl" needs specialized interpretation, which is not automatically done by N3 reasoners; namely the string concatenation for constructing URIs and the represents predicate are vital parts for calling a service and creating a relation to the input data, but have to be interpreted at a special level (it is nice to know that there exists an http request that represents a solution to the information need that I have; but until now this are only descriptions in a RDF vocabulary and I have to know that and how to create an actual HTTP request for the service call).
- While RESTdesc gives a description of the access method to the service, it does not help with interpreting the result. Saying that the result represents a point leaves room open for returning the point in any format including binary (meaning non text-based, non human-readable results). But this does not have to be bad, and you say that with content negotiation, one could specify that one wants RDF back, but here I see a problem: now returning an RDF document, which contains: ":point1 :lat ....; :long ...", how could the receiving software know that it is interested in the :point1 entity in the output? For us (humans) this can be solved by intuition, but the machine only sees a list of triples describing an arbitrary number of resources (imagine a more complex service result) and does not have a clue where to start, i.e., what is the root element of the result.
This are just some new thoughts, that we can discuss.
LIDS note: LIDS gave up using SPARQL CONSTRUCT queries as descriptions, due to limited expressiveness and mainly due to the confusion this generated. We are now also using an RDF description with SPARQL patterns in strings (see our ESWC 2011 paper [1]). However an alternative would be to express SPARQL patterns using SPIN in RDF, which would however not solve the semantic mismatch between RDF and the rules.
Sebastian
[1]: Sebastian Speiser, Andreas Harth: Integrating Linked Data and Services with Linked Data Services. ESWC (1) 2011: 170-184
I'd forgotten this. Many thanks for jumping in, Sebastian.
> However an alternative would be to express SPARQL patterns using SPIN in RDF, which would however not solve the semantic mismatch between RDF and the rules.
Indeed. I actually had a service (with a little more error checking that
SPIN's own endpoint) that wrapped the inter-translation of plain text
SPARQL and SPIN, actually managing the graph patterns as resources (with
full POST/PUT/DELETE support). It was my idea that service descriptions
could just include a URI (e.g., pointed to by a modelRef predicate from
MSM), and then you can get them with content negotiation (which should
always be the solution when there are variants).
I don't know if this service if still running at KIT where
linkedopenservices.org points, but I'm thinking about moving where this
point.
Barry
I was aware of the approach to use OPTIONS and I see the appeal of it (I
might also look into accept headers for OPTION).
>
>and
>
>OPTIONS /service
>Accept: application/x-sparql (or something)
>
>(LIDS)
>
>Of course the REST way to do this is with extended types - e.g.,
>text/los-turtle (if/when the text/turtle registration is accepted),
>text/restdesc-n3, etc. (I must admit I got this very wrong, and used an
>unregistered secondary HTTP header, in the following paper:
>http://ceur-ws.org/Vol-596/paper-09.pdf)
which in my opinion would be the only way to achieve a system as described
above: By using only application/x-turtle and text/N3 I could not foresee
if I get different descriptions (from different approaches) of the same
service or the same description just differently serialized.
>
>
>>
>>> For me the appeal of using rdf descriptions lies in the integration
>>>into
>>> the LOD cloud.
>
>+1
>
>> Yes, it is true that the RDF description integrates directly into the
>>LOD cloud.
>> But that same cloud is indeed an RDF cloud RDF only. The SPARQL
>>strings have no meaning there anyway. This means the important semantics
>>do not get integrated. So I don¹t think there¹s much of a benefit there.
>
>Nonsense. The LOD cloud is drawn from CKAN and SPARQL endpoints are an
>important part of CKAN registration (LOD = resolvability + dumps +
>SPARQL, for me).
>
>See Linked Data Principle 3:
>http://www.w3.org/DesignIssues/LinkedData.html
>
>
>> I¹ve been thinking along similar lines previously in the aformentioned
>>MTAP paper. We embedded the preconditions and postconditions as string
>>literals inside an OWL-S description. This was fully RDF, so it
>>integrated with the LOD cloud. But the string literals would just be
>>literals, and the important meaning gets lost. This was not what we
>>wanted, so we moved on to N3.
>
>OWL-S always had this problem. At least RIF and SPARQL literals have the
>advantage over SWRL (the OWL-S recommendation) and N3 literals in using
>standard languages.
>
>
>> I often compare this to HTML plus JavaScript. You can have a perfectly
>>valid HTML document with syntactically correct JavaScript. But if you
>>have a browser with JavaScript support disabled, you only see the HTML.
>>The script just does not get interpreted. So if it is required for the
>>page to render correctly (which is obviously a bad idea), then the end
>>result is wrong. Although the HTML is perfectly valid and fits into the
>>rest of the Web.
>
>To me that's a good thing. There is useful information in the pure RDF
>part of the MSM-LOS description I showed. That information is hidden in
>a N3 rule (requiring a FOL reasoner) in the equivalent RESTdesc.
>
>
>> There is a similar thing with expressions inside RDF string literals.
>>Yes, they integrate beautifully. But the important functional semantics
>>get lost, so the integration doesn¹t bring you anything. Since our focus
>>is on the functionality, we had to find another way.
>
>Doesn't "bring anything"? They "get lost"? I just don't understand that
>point of view. As I said above, the entire semantics "get lost" in a
>non-standard (indeed, stigmatised) language when everything is thrown
>into N3. I don't say this lightly, by the way (in fact I apologise): I
>spent years getting papers rejected on exactly this "non-standard
>language" point, using OCML and WSML, when no standard language
>supported rules! I imagine you'll be similarly upset by my assertion
>(though RIF does exist now).
>
>
>>> A service resource (i.e. the URI that identifies a service,
>>> against which rdf input data can be posted) can be resolved via HTTP
>>>GET
>>> and point (303) to the rdf service description.
>> The thing is, we target resource-oriented (³REST²) Web services. So
>>there¹s not really a URI that identifies the service, only the URI that
>>identifies your answer. The service is not our resource, it results are.
>
>Yes, I somewhat agree with you here... I think 'result' is wrong though.
>URIs identify resources (here RDF and REST agree) - they do not identify
>Representations
Which is why I return the description behind a 303 rather than a 200. This
obviously appeals more to the Linked Data than the REST principles, in it
makes a differentiation between information and non-information URIs (i.e.
I accept that there are URIs that identify representations rather than
resources, however the services only return resource identifying URIs).
But again I don't disagree with the use of HTTP OPTIONS.
Now for the identification of service results (and their representations)
I use the LIDS URI approach, which is to extend the service (resource) URI
with key/value pairs (that map to the graph patterns) and add an hash tag
(to make it a non-information URI). Please note that I just see this as an
easy technical solution to create these URIs, since I don't want to impose
any semantic in the structure of the URI. This is to say, that I could
also use other URIs generated in any way, which actually would be formally
superior, because it would allow me to get rid of this "hash-tag trick"
and make the differentiation between information and non-information URI
again in the HTTP header.
The services devised in this way completely adhere to the LOS principles,
but also incorporate the main LIDS ideas (actually everything, but
CONSTRUCT queries as descriptions and the notion of "endpoints").
Examples are the service behind http://bing2rdf.appspot.com and the
services behind http://vocab.cc .
>, i.e. your 'results'. Representations are what go in
>HTTP bodies, and when you GET them they're subject not just to the
>resource identified, but the media type requested (Accepted).
>
>
>> For example, with the places description, we would just have:
>>
>> http://example.org/places/Paris
>>
>> This is a resource, not a service. This means: there might be some
>>service behind this, it might be a static page, it might be something
>>elseŠ but that doesn¹t matter to the user. The user just gets the
>>resource, and we describe what the result means.
>
>This sounds naive. Do RESTdesc descriptions perhaps, like LIDS, only
>cover the retrieval of representations (i.e. HTTP GET). Or do you
>describe also PUT/PATCH/POST/DELETE?
>
>
>> Because our service is not the resource, we don¹t put the description
>>there. Instead, we ask to each resource what options it has.
>> For example: an HTTP OPTIONS request to http://example.org/places would
>>return the aforementioned description.
>> In short: RESTdesc describes how to access resources and what it means
>>to access a certain resource. (Which of course, in practice, often maps
>>to services internally, so that¹s why it¹s a service description method.)
>
>Again, I totally agree with the use of OPTIONS (but worry about both the
>REST and the LD use of Accept headers - LD current practice is
>completely naive to this, almost all crawlers being unaware of extension
>types; REST practice may also not apply Accept headers to OPTIONS
>requests).
>
>
>>
>>> This directly follows the
>>> third LOD principle and allows arbitrary agents to deal with service
>>> resources like with any other URI in the LOD cloud (only the ones
>>> concerned with service validation etc. need to be able to dive in the
>>> deeper level of the graph patterns to fulfill their task).
>> So this indeed seems another important distinction: we don¹t see
>>services as resources. Thanks for clarifying this.
>
>What do you see them as?
>
>(By the way, I apologise a second time if my tone seems combative - it
>is usual for me - I think this is an excellent discussion and I hope
>it's already clear that I'm thinking how these approaches can be brought
>together and we can get some cross-fertilisation)
I totally agree: the discussion is more than interesting and the (not so)
different approaches could really profit from each other
Steffen
>
>Barry
Minor note, text/turtle is a registered mime type [1], and
application/x-turtle doesn't exist :) (text/n3 is also registered [2])
[1] http://www.iana.org/assignments/media-types/text/turtle
[2] http://www.iana.org/assignments/media-types/text/n3
Best,
Nathan
Cheers,
Barry
I was aware of the approach to use OPTIONS and I see the appeal of it (I might also look into accept headers for OPTION).
Of course the REST way to do this is with extended types - e.g., text/los-turtle (if/when the text/turtle registration is accepted), text/restdesc-n3, etc. (I must admit I got this very wrong, and used an unregistered secondary HTTP header, in the following paper: http://ceur-ws.org/Vol-596/paper-09.pdf)which in my opinion would be the only way to achieve a system as described above: By using only application/x-turtle and text/N3 I could not foresee if I get different descriptions (from different approaches) of the same service or the same description just differently serialized.
Yes, I somewhat agree with you here... I think 'result' is wrong though. URIs identify resources (here RDF and REST agree) - they do not identify RepresentationsWhich is why I return the description behind a 303 rather than a 200.
Now for the identification of service results (and their representations) I use the LIDS URI approach, which is to extend the service (resource) URI with key/value pairs (that map to the graph patterns) and add an hash tag (to make it a non-information URI).
[...] The services devised in this way completely adhere to the LOS principles, but also incorporate the main LIDS ideas
> Indeed. But the two languages that need interpreters (RDF and SPARQL) are Semantic Web / Linked Data standards, unlike Notation3, which is clearly not.
I have two points for this.
The first being the stereotypical argument “yeah, but N3 is on its way to become a standard”, which of course is not really relevant here :)
The second argument is somewhat more interesting. We agree that there will be two kinds of use cases:
1. Cases in which N3 will be supported
2. Cases in which N3 will not be supported
In case N3 is supported, we just serve the RESTdesc descriptions.
In cases where N3 is not supported we do something else. As you know, N3 rules themselves are not RDF-compliant. But the result of executing an N3 rule, is regular RDF. So to those clients, we serve the result of the execution of the rule.
For example:
If we perform an HTTP OPTIONS to http://example.org/places/Paris with an Accept header preferring N3 (* I’ll get to Accept headers later), then we get the description:
{ ?point geo:name ?name. }
=>
{ _:request http:methodName "GET";
tmpl:requestURI ("/places/" ?name);
http:resp [ tmpl:represents ?point ].
?point wgs84:lat _:lat; wgs84:long _:lng.
}.
If we perform an HTTP OPTIONS to http://example.org/places/Paris with an Accept header preferring RDF, the service returns the result of executing the rule:
<http://example.org/Paris> <http://ws.geonames.org/name> "Paris";
wgs84:lat [];
wgs84:long [].
_:x :requestURI ("/places/" "Paris");
http:methodName "GET";
http:resp [:represents <http://example.org/Paris> ]].
This is just plain RDF, with all the benefits we have been discussing. This is another important advantage of RESTdesc.
Of course, there are some subtleties to this. For example: how does the client get to the /places/Paris URI? (Short answer: hyperlinks / Link header).
> I really like the OPTIONS way of giving a description.
That’s really cool, because we got some negative reactions from reviewers about that, which will probably lead us to also include other methods. All credits for the idea of OPTIONS go to Thomas, BTW, I first saw this in his REST-WS vision paper.
> What is not clear to me (I should check into it) is whether an Accept header is expected for OPTIONS.
The thing is, the result body of OPTIONS is – as you know – vaguely described in the HTTP specification. So I can imagine they are also being vague about accept headers:
"The response body, if any, SHOULD also include information about the communication options. The format for such a body is not defined by this specification, but might be defined by future extensions to HTTP.”
It seams reasonable to have an Accept header, and in fact, I can only think of negative arguments of not using it.
>
> Ironically I'd followed DBPedia in (somewhat incorrectly) using text/n3 as the primary human-readable RDF media type in LOS implementation. Otherwise you could foresee (though unfortunately there is no SPARQL Internet Media type - only for the result formats) a system where you could get LOS, LIDS and RESTdesc descriptions from:
>
> OPTIONS /service
> Accept: application/x-turtle
>
> (LOS)
>
> OPTIONS /service
> Accept: text/n3
>
> (RESTdesc)
>
> and
>
> OPTIONS /service
> Accept: application/x-sparql (or something)
>
> (LIDS)
My opinion is a bit different about this – but I can see arguments for both sides and I might change my mind.
For example, from what I wrote above, the aim would be that “text/n3” returns the RESTdesc descriptions, and “text/turtle” the result of executing the rule for clients that do not understand N3.
Of course, I do not want to reserve neither “text/n3” nor “text/turtle” for RESTdesc exclusively. For example, “text/turtle” could return LOS and RESTdesc, since all this is just RDF.
> Of course the REST way to do this is with extended types - e.g., text/los-turtle (if/when the text/turtle registration is accepted), text/restdesc-n3, etc.
And this is exactly where I see the point then for differentiation.
- Does the client ask for text/turtle? Then he might get LOS, RESTdesc, both, or something else.
- Does the client ask for text/los-turtle? He gets LOS.
- Does the client ask for text/restdesc-turtle? He gets RESTdesc.
Needlessly to say, this combines nicely with Accept header priorities.
The only drawback to this, I think, is the lack of an RFC for this (at least, to my knowledge). I don’t think that there is a standard for expressing the link between extended types and regular types. For the XML world, they do have something like application/xhtml+xml, but I don’t think there is more (correct me if I’m wrong).
>> Yes, it is true that the RDF description integrates directly into the LOD cloud.
>> But that same cloud is indeed an RDF cloud – RDF only. The SPARQL strings have no meaning there anyway. This means the important semantics do not get integrated. So I don’t think there’s much of a benefit there.
>
> Nonsense. The LOD cloud is drawn from CKAN and SPARQL endpoints are an important part of CKAN registration (LOD = resolvability + dumps + SPARQL, for me).
Okay, I might have expressed myself badly here. I meant that there is no syntactic meaning for a string with SPARQL contents in it.
For example, the string "?x wgs84:lat ?lat; wgs84:long ?lng .” does, for the RDF document, not refer to wgs84:lat (the point I tried to make earlier).
So yes, the SPARQL string does have a meaning, in the sense that is expressed in a standard Semantic Web language.
But this meaning is not enforced by the RDF standard. So in that sense, there is no meaning to that string. (But of course, there is if you interpret it… but that is not in the RDF standard.)
I think this is related to the issue of interpreting the meaning of prefixes inside these strings. Because they live in a separate world, you have to make the connection somewhere. Now I’m sure that this is possible in a simple way, but this would again be non-standard.
> At least RIF and SPARQL literals have the advantage over SWRL (the OWL-S recommendation) and N3 literals in using standard languages.
Just a side note: N3 is fully RIF-compatible. In fact, one of the recognized implementations of RIF reasoners is the EYE N3 reasoner.
> To me that's a good thing. There is useful information in the pure RDF part of the MSM-LOS description I showed. That information is hidden in a N3 rule (requiring a FOL reasoner) in the equivalent RESTdesc.
So here, I would point again to the fact that we can return the result of the N3 rule execution.
>> There is a similar thing with expressions inside RDF string literals. Yes, they integrate beautifully. But the important functional semantics get lost, so the integration doesn’t bring you anything. Since our focus is on the functionality, we had to find another way.
>
> Doesn't "bring anything"? They "get lost"? I just don't understand that point of view. As I said above, the entire semantics "get lost" in a non-standard (indeed, stigmatised) language when everything is thrown into N3. I don't say this lightly, by the way (in fact I apologise): I spent years getting papers rejected on exactly this "non-standard language" point, using OCML and WSML, when no standard language supported rules! I imagine you'll be similarly upset by my assertion (though RIF does exist now).
Very good point. I would also bring up the “result of the N3 rule execution” thing here.
We both needed a way to express functional relationships, that cannot be expressed in RDF directly. You have chosen to put the functionality in a standard language, wrapped inside a string. So you still have RDF then for the rest of the description.
We have chosen to leave RDF and use a superset of it, while providing a fallback to (full) RDF for non-N3 clients (which I didn’t mention before, so without that fact, your argumentation fully stands).
>> The thing is, we target resource-oriented (“REST”) Web services. So there’s not really a URI that identifies the service, only the URI that identifies your answer. The service is not our resource, it results are.
>
> Yes, I somewhat agree with you here... I think 'result' is wrong though. URIs identify resources (here RDF and REST agree) - they do not identify representations, i.e. your 'results'. Representations are what go in HTTP bodies, and when you GET them they're subject not just to the resource identified, but the media type requested (Accepted).
The point is that our results *are* resources. We state, for example, that the result is “the coordinates of the place with the given name”. But that coordinates can be RDF, they can be plaintext, they can be anything… RESTdesc explains what the result means, not what it looks like.
Another example: we might have a service that returns a photograph of that place. That phorograph might be a JPEG image, PNG, or we might even (with proper redirect of course ;) return RDF metadata about the photograph.
> This sounds naive. Do RESTdesc descriptions perhaps, like LIDS, only cover the retrieval of representations (i.e. HTTP GET). Or do you describe also PUT/PATCH/POST/DELETE?
No, we also have scenarios for non-safe and non-idempotent methods, and in fact, this is an essential point. My slides did not cover these (I only had a 10 minute slot) but the sample application at http://restdesc.no.de/ does.
>> So this indeed seems another important distinction: we don’t see services as resources. Thanks for clarifying this.
>
> What do you see them as?
As something behind the scenes. For us, it doesn’t matter that there is a service that maps place names to coordinates. What matters, is that clients can understand they can access a resource which gives them those coordinates.
> (By the way, I apologise a second time if my tone seems combative - it is usual for me - I think this is an excellent discussion and I hope it's already clear that I'm thinking how these approaches can be brought together and we can get some cross-fertilisation)
No need to apologise (and I do apologise if my tone seems combative). I really appreciate the effort you put into this discussion, which is indeed excellent. Thanks a lot!
Ruben
> - Mixing of description of functionality and access method, which is powerful, because you can construct the service call URIs in a flexible way, but it can be confusing for humans/hard for computers to seperate the functionality part from the access part, in situations where only functionality is of interest. I don't know whether this is a good or bad thing, but it is different from LIDS and LOS.
Very valid point. Providing the client the means to construct a URI can be vary handy, but we kind of neglect the hypermedia constraint there. So this is why, after our initial journal paper (under revision), we introduced a way to leave URI construction to the server, using hypermedia links (e.g., Link headers). Here, the functionality is separated from URI construction, which is in accordance with the hypermedia constraint. However, we still have the HTTP verbs than and other modalities. But the verbs can be an indication of the implications of the operation (safe, idempotent…).
Another thing is that mixing description and access method, has been the source of many problems in the past. Software got compiled against a WSDL document, and if the WSDL was modified, the whole thing would break.
We specifically target runtime decisions. Our goal is that a client discovers at runtime what service it needs and how to access it. Hypermedia links would encourage this even more.
> - I assume the vocabulary "tmpl" needs specialized interpretation, which is not automatically done by N3 reasoners; namely the string concatenation for constructing URIs and the represents predicate are vital parts for calling a service and creating a relation to the input data, but have to be interpreted at a special level (it is nice to know that there exists an http request that represents a solution to the information need that I have; but until now this are only descriptions in a RDF vocabulary and I have to know that and how to create an actual HTTP request for the service call).
Good point as well. The tmpl vocabulary is meant as a shortcut, to keep descriptions short and concise. In the long version, the RESTdesc description would indeed contain the concatenation etc… but this would be really repetitive and inelegant.
Another benefit of using N3 surfaces here: we can express in N3 rules what the tmpl predicates mean. Then, a standard N3 reasoner takes the description and the tmpl rules, and will derive the full description, including concatenations etc. This would translate everything in the regular HTTP vocabulary, which gives you everything you need for the request, without needing to understand special predicates.
So in a sense, we will also have an N3 ontology for the tmpl vocabulary, containing the substantiation of its meaning in rules (in addition to the English explanation in the current OWL ontology.)
> - While RESTdesc gives a description of the access method to the service, it does not help with interpreting the result. Saying that the result represents a point leaves room open for returning the point in any format including binary (meaning non text-based, non human-readable results). But this does not have to be bad, and you say that with content negotiation, one could specify that one wants RDF back, but here I see a problem: now returning an RDF document, which contains: ":point1 :lat ....; :long ...", how could the receiving software know that it is interested in the :point1 entity in the output?
Well, the answer here is that it shouldn’t return ‘:point1', but ':point rdf:label “Paris”’ or something similar in the answer. This question of representation should be handled by the server. The server should be responsible for returning a response that the client can interpret. “:point1” is not a good choice in that aspect. Furthermore, the hypermedia constraint asks that services also provide the controls to navigate through resources. This is not the case if you call your output “:point1” and do not provide any link to the thing this represents.
You could say that this is a really naive or idealistic assumption, and it is. But currently, I’m targeting services that do thing by the book. For example, I also heavily rely on correct resource-oriented behavior.
Another answer could be that we specifically enforce a certain representation in the description, but this is not the task of the description as far as I’m concerned. And it does not seem RESTful.
> This are just some new thoughts, that we can discuss.
Thanks for that, I’m looking forward to your ideas on this.
Best,
Ruben
I think the descriptions themselves should make that clear in some way as well. I believe there must be a way to relate a description to its service, i.e., a description must stand on its own, regardless of how it is retrieved. The client should be able to interpret and use a description, even if it doesn’t know where the description came from.
> Yes, I somewhat agree with you here... I think 'result' is wrong though.
> URIs identify resources (here RDF and REST agree) - they do not identify
> Representations
Just to be clear: we fully agree on that. Our result is a resource, not a representation. See my NWeSP slides.
Cheers,
Ruben
This needs some thought though - where we have changeable results (our
first and recurring example was the weather service):
1) If you want to identify the result you should have a link between the
'generic' (but canonicalised) URI (identifying the weather station, or
some general point or geometry) and the results for a particular point
in time
2) If you do identify that (the 12:00 weather report from Baden Airpark
on 2011-10-22), should that not be resolvable in future (meaning keeping
a history in the service implementation)
Many other points need discussion, but I get on the plane soon, so I
hope we can talk in person
Barry
Weather is indeed an interesting example. This is where we thought about using the Content-location header:
"The Content-Location entity-header field MAY be used to supply the resource location for the entity enclosed in the message when that entity is accessible from a location separate from the requested resource's URI. A server SHOULD provide a Content-Location for the variant corresponding to the response entity; especially in the case where a resource has multiple entities associated with it, and those entities actually have separate locations by which they might be individually accessed, the server SHOULD provide a Content-Location for the particular variant which is returned."
> 2) If you do identify that (the 12:00 weather report from Baden Airpark on 2011-10-22), should that not be resolvable in future (meaning keeping a history in the service implementation)
You could indeed keep a history. To make things practical, you could (in case of the weather service) store only the last x days, and give a 410 Gone for older ones.
> Many other points need discussion, but I get on the plane soon, so I hope we can talk in person
Just arrived in Bonn, staying until Friday morning. Looking forward to talk!
Ruben
Being more a REST than SemWeb guy I have to admit that I don't really like
the use of HTTP OPTIONS. First off all OPTIONS is not supported by all
servers, secondly it is defined as follows:
"The OPTIONS method represents a request for information about the
communication options available on the request/response chain identified by
the Request-URI. This method allows the client to determine the options
and/or requirements associated with a resource, or the capabilities of a
server, without implying a resource action or initiating a resource
retrieval." [RFC2616]
What you are doing is a resource retrieval IMHO. You retrieve a RESTdesc
description. I think a HTTP HEAD that will return a Link header pointing to
your RESTdesc document would be far better. This would also allow you to
link to your RESTdesc document from a normal HTML page, for instance.
Another advantage of doing it that way is that the response is cacheable.
Responses of OPTIONS requests are NOT cacheable.
Why do you do a OPTIONS / and not a OPTIONS *? What happens if the "service"
is in a subdirectory and not visible from the root directory? All those
problems would be resolved by using a link header.
Using OPTIONS with an accept header as generic as text/N3 isn't really
helpful either. How would a machine client know how to use it or what the
relation of that response is to the request he issued?
> > - While RESTdesc gives a description of the access method to the
> service, it does not help with interpreting the result. Saying that the
> result represents a point leaves room open for returning the point in
> any format including binary (meaning non text-based, non human-readable
> results). But this does not have to be bad, and you say that with
> content negotiation, one could specify that one wants RDF back, but
> here I see a problem: now returning an RDF document, which contains:
> ":point1 :lat ....; :long ...", how could the receiving software know
> that it is interested in the :point1 entity in the output?
>
> Well, the answer here is that it shouldn't return ':point1', but
> ':point rdf:label "Paris"' or something similar in the answer. This
> question of representation should be handled by the server. The server
> should be responsible for returning a response that the client can
> interpret. ":point1" is not a good choice in that aspect. Furthermore,
> the hypermedia constraint asks that services also provide the controls
> to navigate through resources. This is not the case if you call your
> output ":point1" and do not provide any link to the thing this
> represents.
This is a crucial thing which I think is currently not really addressed by
RESTdesc. If you would like to generalize RESTdesc you have to process the
responses and be able to relate them to the resource you are looking for.
For JSON, an approach like SEREDASj [1] might help you (it's still work in
progress). You might also want to look into JSON-LD [2] and the follow-up
paper of SEREDASj [3].
[1] http://bit.ly/seredasj
[2] http://www.json-ld.org/spec/latest/json-ld-syntax/
[3] http://bit.ly/sparql2rest
--
Markus Lanthaler
@markuslanthaler
Thanks for taking the time to reply to this, since your feedback from a REST background is really valuable.
> Being more a REST than SemWeb guy I have to admit that I don't really like
> the use of HTTP OPTIONS. First off all OPTIONS is not supported by all
> servers, secondly it is defined as follows:
>
> "The OPTIONS method represents a request for information about the
> communication options available on the request/response chain identified by
> the Request-URI.
Serving RESTdesc in OPTIONS actually started on the idea of the above paragraph. We specifically provide "information about the communication options available": e.g., a description might tell that POST is available under a certain precondition. This offers far more functionality than the Allow header, which only tells what methods are supported (which BTW actually differs from the literal meaning of “allowed”).
> This method allows the client to determine the options
> and/or requirements associated with a resource,
So this is actually what we do, the requirements being the precondition of our description (antecedent of the rule).
> or the capabilities of a server,
This accounts for the postcondition part of the description: e.g., the server is capable of providing location data of a place.
> without implying a resource action
Also true.
> or initiating a resource retrieval." [RFC2616]
And this is subject to discussion of course… On thing is sure: we do not retrieve the resource of the URI, so in that sense, this is correct.
Whether or not the description is a resource, is open to discussion. (For me, it is a meta-resource.)
But I think that any kind of body of an OPTIONS request can be seen as a resource, so I assume the “resource retrieval” is about the original resource, which we correctly do not serve as a result of the OPTIONS call.
> What you are doing is a resource retrieval IMHO. You retrieve a RESTdesc description.
I understand you point of view, yet I think it is indeed a matter of viewpoint.
If you have a server that serves location information, what are your resources? Possible answers: places, coordinates, … etc.
But I don’t think “service descriptions” belong in that list.
> I think a HTTP HEAD that will return a Link header pointing to
> your RESTdesc document would be far better. This would also allow you to
> link to your RESTdesc document from a normal HTML page, for instance.
> Another advantage of doing it that way is that the response is cacheable.
> Responses of OPTIONS requests are NOT cacheable.
Valid point. I think it could make sense to provide an additional way to access the description with a regular GET. However, this would indeed make it a resource. Link headers are a good candidate to get to the description in that case. Still, I’m not convinced of mixing resources with meta-resources.
We should also bear in mind that OPTIONS being not cacheable was written in the specification when there was no use yet for the response body.
> Why do you do a OPTIONS / and not a OPTIONS *? What happens if the "service"
> is in a subdirectory and not visible from the root directory? All those
> problems would be resolved by using a link header.
Now this is a good point, and we actually do use Link headers for that.
Simple example: a server with location information. The / resource is a list of locations, the /Paris gives information about Paris;
How this works now is:
- an OPTIONS request to / gives the description about the current resource (i.e., GET returns a list of locations)
- requesting the / resource (with any method) also provides links to (some of) the locations, in any form (HTML links, RDF, Link headers…), for example /Paris
- an OPTIONS request to /Paris gives the description about the current resource (i.e., PUT updates it information)
However, be aware that the “service” is not a resource and will therefore never be visible. It is the resources that we have to make visible, and therefore, we indeed use links (in any form). The beauty of this is that your existing structure just remains as-is. You have your resources, and interlink them as you see fit. That doesn’t change. Descriptions are added transparently with the OPTIONS method. Your linking structure is based on your resources, your content, and not on the way you describe them.
> Using OPTIONS with an accept header as generic as text/N3 isn't really
> helpful either. How would a machine client know how to use it or what the
> relation of that response is to the request he issued?
Well, if a machine client states it accepts text/n3, it should be able to interpret text/n3.
If it does, it can just read the N3 and figure out the relation by itself.
If it doesn’t, it should ask for something different instead.
>> - While RESTdesc gives a description of the access method to the
>> service, it does not help with interpreting the result. Saying that the
>> result represents a point leaves room open for returning the point in
>> any format including binary (meaning non text-based, non human-readable
>> results). But this does not have to be bad, and you say that with
>> content negotiation, one could specify that one wants RDF back, but
>> here I see a problem: now returning an RDF document, which contains:
>> ":point1 :lat ....; :long ...", how could the receiving software know
>> that it is interested in the :point1 entity in the output?
>>
>> Well, the answer here is that it shouldn't return ':point1', but
>> ':point rdf:label "Paris"' or something similar in the answer. This
>> question of representation should be handled by the server. The server
>> should be responsible for returning a response that the client can
>> interpret. ":point1" is not a good choice in that aspect. Furthermore,
>> the hypermedia constraint asks that services also provide the controls
>> to navigate through resources. This is not the case if you call your
>> output ":point1" and do not provide any link to the thing this
>> represents.
>
> This is a crucial thing which I think is currently not really addressed by RESTdesc.
> If you would like to generalize RESTdesc you have to process the
> responses and be able to relate them to the resource you are looking for.
I didn’t see that as a goal of RESTdesc. RESTdesc explains what the result of your action will mean, what you obtain by executing that action.
Creating self-contained resources is just a reasonable thing to do.
This is even true on the human Web. If I request an HTML for /Paris, I don’t expect to see:
48°50'N, 02°20’E
But I expect to see:
The geographical coordinates of Paris are 48°50'N, 02°20’E.
Or, quoting Fielding:
"REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.”
Although Fielding is referring to the message, it makes sense to have the body of that message, which is the representation of a resource, also self-descriptive.
It makes even sense with the hypermedia constraint:
"I mean the simultaneous presentation of information and controls such that the information becomes the affordance through which the user (or automaton) obtains choices and selects actions.”
When presenting a user with
48°50'N, 02°20’E"
the controls on this data (for example, hyperlink to “favorite” or “share”) are meaningless to him, because he don’t have the necessary information to make a choice.
However, if the user gets
The geographical coordinates of Paris are 48°50'N, 02°20’E."
then the controls on this data do have meaning.
So I don’t consider this a job for RESTdesc, which assumes RESTful principles for the underlying resource. I consider this a task of the resource creator.
I also think that, once you are describing how to interpret a result, this is tied to a particular representation. E.g., the description for RDF and CSV would be different. This seems wrong to me: REST is about resources, not representations, and so is RESTdesc.
> For JSON, an approach like SEREDASj [1] might help you (it's still work in
> progress). You might also want to look into JSON-LD [2] and the follow-up
> paper of SEREDASj [3].
Thanks for the pointer, Markus. I don’t know if you remember, we’ve had an interesting e-mail conversation about SEREDASj in the past. So hereby I’m fulfilling my promise of keeping you up to date ;)
Cheers,
Ruben
For me it is a resource :-) I haven't heard of a meta-resource before.
> But I think that any kind of body of an OPTIONS request can be seen as
> a resource, so I assume the "resource retrieval" is about the original
> resource, which we correctly do not serve as a result of the OPTIONS
> call.
Well.. True and false. You could do a normal HTTP GET / with an Accept of
text/N3 and return your RESTdesc description there. In the end it's just a
specific format meant for machine-consumption instead of human-consumption,
right?
> I understand you point of view, yet I think it is indeed a matter of
> viewpoint.
> If you have a server that serves location information, what are your
> resources? Possible answers: places, coordinates, . etc.
> But I don't think "service descriptions" belong in that list.
Think of it more like HTML. If you browse HTML pages you get a whole lot of
"unnecessary" stuff, not just the raw data. But all those hyperlinks and
forms are needed to guide the agent (which is in that case a human) through
the web page/application/service (whatever you would like to call it). I see
service descriptions like the one RESTdesc produces like a homepage, i.e.
entry point, for a machine agent. The hyperlinks or link templates are then
used to guide the agent through the state machine of the service (HATEOAS).
> > I think a HTTP HEAD that will return a Link header pointing to
> > your RESTdesc document would be far better. This would also allow you
> to
> > link to your RESTdesc document from a normal HTML page, for instance.
> > Another advantage of doing it that way is that the response is
> cacheable.
> > Responses of OPTIONS requests are NOT cacheable.
>
> Valid point. I think it could make sense to provide an additional way
> to access the description with a regular GET. However, this would
> indeed make it a resource. Link headers are a good candidate to get to
> the description in that case. Still, I'm not convinced of mixing
> resources with meta-resources.
What's the main advantage of using OPTIONS instead of a Link header? The
only "advantage" I see at the moment is that you don't have to assign it its
own IRI. I doubt if that's really such an advantage as it greatly affects
visibility.
> However, be aware that the "service" is not a resource and will
> therefore never be visible. It is the resources that we have to make
> visible, and therefore, we indeed use links (in any form). The beauty
> of this is that your existing structure just remains as-is. You have
> your resources, and interlink them as you see fit. That doesn't change.
> Descriptions are added transparently with the OPTIONS method. Your
> linking structure is based on your resources, your content, and not on
> the way you describe them.
That's exactly the point. Even if I would know the IRI of the resource I
would have no way to find out where it's descriptions are. The only think I
could do is to try it brute-force-like on every resource. By having a Link
header you really *show* the client that there is a description for that
resource available somewhere.
> > Using OPTIONS with an accept header as generic as text/N3 isn't
> really
> > helpful either. How would a machine client know how to use it or what
> the
> > relation of that response is to the request he issued?
>
> Well, if a machine client states it accepts text/n3, it should be able
> to interpret text/n3.
> If it does, it can just read the N3 and figure out the relation by
> itself.
> If it doesn't, it should ask for something different instead.
What I meant here is that text/n3 isn't specific enough. Most clients would
probably also understand application/xml or application/json. The question
is whether the client can make use of the contents. For such a specific use
case as RESTdesc I would thus use a more specific MIME type such as
text/restdesc+n3
> > This is a crucial thing which I think is currently not really
> addressed by RESTdesc.
> > If you would like to generalize RESTdesc you have to process the
> > responses and be able to relate them to the resource you are looking
> for.
>
> I didn't see that as a goal of RESTdesc. RESTdesc explains what the
> result of your action will mean, what you obtain by executing that
> action.
>
> ...
>
> So I don't consider this a job for RESTdesc, which assumes RESTful
> principles for the underlying resource. I consider this a task of the
> resource creator.
This seems to be a common practice these days in research papers :-) The
question I have then, what will I use that system for? What's the benefit of
such descriptions if I can't get the real raw data I'm interested in. It's
kind of similar as if you would ask me a question and I tell you that I know
that Victor knows the answer so you go and ask him. But he replies in
Russian. What was the win? I really like the straightforward concise
description in N3 but if I can't make any use of its results why should I
use it?
> > For JSON, an approach like SEREDASj [1] might help you (it's still
> work in
> > progress). You might also want to look into JSON-LD [2] and the
> follow-up
> > paper of SEREDASj [3].
>
> Thanks for the pointer, Markus. I don't know if you remember, we've had
> an interesting e-mail conversation about SEREDASj in the past. So
> hereby I'm fulfilling my promise of keeping you up to date ;)
Yes, I remember and I find your research quite interesting and closely
related to mine. So please don't take the above as criticism but as (a
hopefully for both of us valuable) discussion.
Best,
Markus
I just coined the term actually :)
But what I mean is that descriptions – in my mindset – do not belong to the core resources of a server.
> You could do a normal HTTP GET / with an Accept of
> text/N3 and return your RESTdesc description there. In the end it's just a
> specific format meant for machine-consumption instead of human-consumption,
> right?
I’m afraid not. GET / should represent the resource with URI http://example.org/, regardless of what the Accept header is.
For example, if http://example.org/ represents the weather in Rome, then an Accept header of text/n3 gives that weather in Notation3.
An OPTIONS /, however, gives you the description.
> Even if I would know the IRI of the resource I would have no way to find out where it's descriptions are.
Well actually, you do… the description of a resource can be obtained by doing an OPTIONS request on that resource’s URI. That’s the advantage.
> The only think I could do is to try it brute-force-like on every resource.
> By having a Link header you really *show* the client that there is a description for that resource available somewhere.
This point is really good, and it makes me seriously consider of implementing this as a complement to OPTIONS.
In essence, by using the OPTIONS method, I want to send the signal that this could be an interesting method for service discovery. But I totally agree with your paragraph for the Web as it is now. As long as we are the only ones using OPTIONS, you’d indeed just have to try it. But if this method would become a common practice (which I believe is a great idea, for the arguments mentioned before), it would really work out well.
But considering that this is not the case, it would be interesting (and for some clients or servers, even necessary) to provide a fallback method using the Link header. This could serve as a transitionary measure.
Once the OPTIONS method for service descriptions becomes a common practice (hypothetically speaking), the extra link header could just be dropped, leaving the agreement that “OPTIONS x = description of resource with URI x”.
> What I meant here is that text/n3 isn't specific enough. Most clients would
> probably also understand application/xml or application/json.
Big difference here: Notation3 is a semantic format. The content actually has a well-defined meaning, as opposed to JSON, which is just a structure.
So they would not “understand” JSON, but just parse it. Notation3, on the other hand, can be “understood” (or well, let’s say “interpreted”).
> What's the benefit of
> such descriptions if I can't get the real raw data I'm interested in. It's
> kind of similar as if you would ask me a question and I tell you that I know
> that Victor knows the answer so you go and ask him. But he replies in
> Russian. What was the win? I really like the straightforward concise
> description in N3 but if I can't make any use of its results why should I
> use it?
Oh you can. But RESTdesc assumes that you’re having a RESTful service underneath. If that’s not the case, then you’ll end up with not-so concise descriptions.
In a RESTful service, your real raw data should speak for itself. But of course, I’m stating this from my Semantic Web background.
What RESTdesc basically says is: design your server contents according to RESTful principles. And then describe it.
But don’t blame RESTdesc for having a bad RESTful design ;)
> Yes, I remember and I find your research quite interesting and closely
> related to mine. So please don't take the above as criticism but as (a
> hopefully for both of us valuable) discussion.
And that’s how I take it – thanks for participating in this discussion. I highly appreciate and learn from your input!
Ruben
Well.. from just looking at http://restdesc.no.de/ there doesn't seem to be
a difference between the two. One retrieves an interlinked HTML
representation and OPTIONS returns IMO the same in Notation3.
> Once the OPTIONS method for service descriptions becomes a common
> practice (hypothetically speaking), the extra link header could just be
> dropped, leaving the agreement that "OPTIONS x = description of
> resource with URI x".
"OPTIONS x = description of resource with URI x"? So it's metadata about the
resource x, right? According to the HTTP spec this metadata (or
metainformation how they call it) should be put in headers.
The HTTP protocol is a request/response protocol. A client sends a
request to the server in the form of a request method, URI, and
protocol version, followed by a MIME-like message containing request
modifiers, client information, and possible body content over a
connection with a server. The server responds with a status line,
including the message's protocol version and a success or error code,
followed by a MIME-like message containing server information, entity
metainformation, and possible entity-body content.
[RFC 2616, 1.4 Overall Operation]
If the metadata is too big to be put into the header the logical consequence
would be to just put a link in the header. At least that's how understand
it.
> > What I meant here is that text/n3 isn't specific enough. Most clients
> would
> > probably also understand application/xml or application/json.
>
> Big difference here: Notation3 is a semantic format. The content
> actually has a well-defined meaning, as opposed to JSON, which is just
> a structure.
> So they would not "understand" JSON, but just parse it. Notation3, on
> the other hand, can be "understood" (or well, let's say "interpreted").
True, but nevertheless it would be too general to request just N3. What you
would really intent to do is to request a RESTdesc description. Of course at
the moment that's the same since you are the only one using OPTIONS that
way.
> In a RESTful service, your real raw data should speak for itself. But
> of course, I'm stating this from my Semantic Web background.
> What RESTdesc basically says is: design your server contents according
> to RESTful principles. And then describe it.
Isn't that a oxymoron? If the data would really speak for itself why would
you then need a description of it?
--
Markus Lanthaler
@markuslanthaler
This should not be the case:
$curl http://restdesc.no.de/ -I
HTTP/1.1 200 OK
X-Powered-By: Express
Link: </photos>; rel="index"
Content-Type: text/html; charset=utf-8
Content-Length: 208
Connection: keep-alive
As you can see, the GET response returns the root resource, which links to the photos index page (both in HTML and link headers).
curl http://restdesc.no.de/ -I -H "Accept: text/n3"
HTTP/1.1 406 Not Acceptable
X-Powered-By: Express
Link: </photos>; rel="index"
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
If we try to get text/n3, we get 406, because the server does not have an N3 representation for that page. (note how it still links to the photos index page).
$curl http://restdesc.no.de/ -I -X OPTIONS
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/n3
Content-Length: 0
Connection: keep-alive
The OPTIONS response returns an empty description, because there is nothing to describe.
> "OPTIONS x = description of resource with URI x"? So it's metadata about the
> resource x, right? According to the HTTP spec this metadata (or
> metainformation how they call it) should be put in headers.
>
> The HTTP protocol is a request/response protocol. A client sends a
> request to the server in the form of a request method, URI, and
> protocol version, followed by a MIME-like message containing request
> modifiers, client information, and possible body content over a
> connection with a server. The server responds with a status line,
> including the message's protocol version and a success or error code,
> followed by a MIME-like message containing server information, entity
> metainformation, and possible entity-body content.
> [RFC 2616, 1.4 Overall Operation]
>
> If the metadata is too big to be put into the header the logical consequence
> would be to just put a link in the header. At least that's how understand
> it.
Good point, this is indeed a possible interpretation. For that reason, I think it makes sense to offer the description also as a resource.
>> In a RESTful service, your real raw data should speak for itself. But
>> of course, I'm stating this from my Semantic Web background.
>> What RESTdesc basically says is: design your server contents according
>> to RESTful principles. And then describe it.
>
> Isn't that a oxymoron? If the data would really speak for itself why would
> you then need a description of it?
It is an oxymoron indeed. For GET. We argue this in http://www.lsi.upc.edu/~tsteiner/papers/2011/efficient-runtime-service-discovery-nwesp2011.pdf
For POST and PUT, however, the situation is different:
“According to the Linked Data principles [29], RDF URIs need to be dereferencable, which means that an HTTP GET request to them should return useful information about the concept they identify. […] This dereferencing implied by Listing 4 can be specified explicitly by the server, as shown in Listing 5. While unnecessary for simple GET requests, it is required to express the functionality of other HTTP methods such as PUT and POST.”
More concretely: you would not want to find out that you have added a comment to a photograph only after you did that.
So RESTdesc doet not describe the data (which describes itself), it describes what the result of an HTTP request will be. There might be different reasons for not trying that request (even GET) directly: it might be slow, there might be many of them, requests might cost money…
RESTdesc tells you in advance what the result will mean, so you can decide if you need it.
And if you decide you need it (using RESTdesc or any other way), then you just obtain that result and it explains itself.
Cheers,
Ruben
That's true. I might mixed up some things.. A question: Why are you not using RESTdesc to describe the link to /photos here? /photos returns a list of photoIds which would actually be quite useful to know.
> $curl http://restdesc.no.de/ -I -X OPTIONS
> HTTP/1.1 200 OK
> X-Powered-By: Express
> Content-Type: text/n3
> Content-Length: 0
> Connection: keep-alive
>
> The OPTIONS response returns an empty description, because there is
> nothing to describe.
Actually it's too empty :-P You don't even return the allowed methods..
> RESTdesc tells you in advance what the result will mean, so you can
> decide if you need it.
> And if you decide you need it (using RESTdesc or any other way), then
> you just obtain that result and it explains itself.
Yes, and it does it in a straightforward way. That's what I really like about this approach. Good work. Looking forward to seeing more :-)
Cheers,
Markus
Interesting. We were actually thinking about what to return here. In theory, a client could discover all links by following the link headers, but this would involve *a lot* of communication (to the extent of being ridiculous).
But then we questioned ourselves: what do we return? One “level" deep? Two levels?
In the source code (https://github.com/tomayac/restdesc), you can see that we have a mechanism for returning “everything deeper then the current path”. This is the approach you also see at the /photos, /photos/1, /photos/1/persons etc. Note how the amount of descriptions diminishes in each step.
We explicitly disabled this for the root, but I have changed this behavior now following your comment.
Note, however, that this is still an open question instead of a final answer… one could argue easily that, for a large number of different resources, returning all descriptions at the root becomes similarly infeasible as following link headers. You could limit that to two or three levels, though. One benefit on the other hand, is that it scales in the number of different resource *types*, not resources (which should be manageable).
>> $curl http://restdesc.no.de/ -I -X OPTIONS
>>
>> The OPTIONS response returns an empty description, because there is
>> nothing to describe.
>
> Actually it's too empty :-P You don't even return the allowed methods..
Haha, indeed :) Thanks for reporting. Was fixed by enabling the description mechanism on the root.
>> RESTdesc tells you in advance what the result will mean, so you can
>> decide if you need it.
>> And if you decide you need it (using RESTdesc or any other way), then
>> you just obtain that result and it explains itself.
>
> Yes, and it does it in a straightforward way. That's what I really like about this approach. Good work. Looking forward to seeing more :-)
Great. Thanks for the feedback, Markus, greatly appreciated!
Best,
Ruben
Unfortunately my meeting went past the end of lunch.
Can we instead try to agree some time to meet up tomorrow? I'll make
myself available at any point by means of an apology for today.
Barry
No worries, and sorry for disturbing you while in a meeting.
Tomorrow is fine, any point works for me as well. I’m looking forward to coordinate our ideas.
Best,
Ruben
Best,
Ruben
On 27 Oct 2011, at 08:29, Craig Knoblock wrote:
> I need to leave at 1pm to catch a train, but I'll try to get to lunch on the early side so that we have some time before I go.
> Craig
>
> On Oct 26, 2011, at 2:22 PM, Thomas Steiner wrote:
>
>> How about tomorrow over lunch break? We could try to get a table
>> upstairs with hopefully less noise than downstairs. Ruben and me would
>> highly appreciate the opportunity to exchange ideas with you guys.
>>
>> Best,
>> Tom
>>
>> Thank God not sent from a BlackBerry, but from my iPhone