FedX 3.1 Release

[Andreas Schwarte]

Hello,

We are happy to announce the new minor release FedX 3.1.

The new release provides an improvement to work with SPARQL endpoints which do not support SPARQL ASK queries (e.g., DBpedia). FedX heavily uses these ASK queries for its source selection mechanism. With the new change endpoints can be configured to use SELECT queries instead. Please see the provided examples and documentation for details.

The new release can be obtained from our website: http://www.fluidops.com/fedx/

To keep up to date with FedX developments, you can follow the discussions on this mailing list.

We welcome your feedback and comments.

On behalf of the FedX team,
 Andreas

[Nate A]

Am Donnerstag, 9. Oktober 2014 14:23:22 UTC+2 schrieb Andreas Schwarte:

The new release provides an improvement to work with SPARQL endpoints which do not support SPARQL ASK queries (e.g., DBpedia). FedX heavily uses these ASK queries for its source selection mechanism. With the new change endpoints can be configured to use SELECT queries instead. Please see the provided examples and documentation for details.

Hello Andreas,

cool that you have made a new release. But I couldn't find in the documentation, where to configure whether an endpoint supports ASK queries or not. When I've tested the new code it seamed as if FedX your automatically use SELECT queries since it send the following to dbpedia:

SELECT * WHERE {?-anon-1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?Concept .  } LIMIT 1

but unfortunately dbpedia returned:

Virtuoso 37000 Error SP030: SPARQL compiler, line 3: syntax error at '?' before '-'

SPARQL query:
define sql:big-data-const 0 
#output-format:text/html
define sql:signal-void-variables 1 define input:default-graph-uri <http://dbpedia.org> SELECT * WHERE {?-anon-1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?Concept .  } LIMIT 1

because it can't handle “?-” variables. Is there a way of telling FedX how the anon-variables should look like? Is there a reason for not just using ?anon1, which would work at least for dbpedia.

Any a second question is there also a switch to manually decide whether POST or GET method should be used?

Thank you very much

Natanael

[Andreas Schwarte]

Hi Natanael,

in the doc folder of the distribution you can find information on how to do such configuration:

particularly in

doc/FedX_Documentation.html

doc/pages/dataConfig.html

Below is the relevant snippet from the documentation - please see the provide doc for the full help:

SPARQL Federation:

@prefix fluid: <http://fluidops.org/config#>.

<http://DBpedia> fluid:store "SPARQLEndpoint";
fluid:SPARQLEndpoint "http://dbpedia.org/sparql";
fluid:supportsASKQueries "false" .

<http://SWDF> fluid:store "SPARQLEndpoint";
fluid:SPARQLEndpoint "http://data.semanticweb.org/sparql".

Note: if a SPARQL endpoint does not support ASK queries, the endpoint can be configured to use SELECT queries instead using fluid:supportsASKQueries "false". This is for instance useful for Virtuoso based endpoints like DBpedia. Moreover note that for convenience the public DBpedia endpoint is automatically configured to use SELECT queries

Best,

 Andreas

--
You received this message because you are subscribed to the Google Groups "Information Workbench Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iwb-discussio...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nate A]

Thank you very much for your answer.

Is there an option or is it planed for selecting the request type? E.g. http://lod.b3kat.de/sparql does not support POST queries but works very good with GET queries.

And another option for specifying the possible content types would be good as well. E.g. http://europeana.ontotext.com/sparql/ does support JSON and XML but not CSV or TSV, while FedX seams to ask for one of the latter.

Another issue as mentioned before is, that some endpoints don't support variables like ?-anon-1. Is there a reason for not just using ?anon1, which would work at least for dbpedia?

Thank you

Natanael

[Andreas Schwarte]

Hi Natnael,

thank you for providing this useful feedback. Such feedback from users helps us in understanding current problems and in a next step to find solutions.

Please see my inline comments with some explanation and hints.

On Tue, Nov 11, 2014 at 4:51 PM, Nate A <arn...@gmail.com> wrote:

Thank you very much for your answer.

Is there an option or is it planed for selecting the request type? E.g. http://lod.b3kat.de/sparql does not support POST queries but works very good with GET queries.

And another option for specifying the possible content types would be good as well. E.g. http://europeana.ontotext.com/sparql/ does support JSON and XML but not CSV or TSV, while FedX seams to ask for one of the latter.

First some technical explanation: under the hood FedX uses the Sesame SPARQLRepository for the communication with SPARQL endpoints. I just tried to use an isolated SPARQL repository (i.e. without FedX being involved) against above b3kat endpoint and exactly saw the problem that you have found.

For the b3kat endpoint my proposal would be the following: you could contact the Sesame mailing list to ask for the best practices (how to deal with such endpoints). Moreover, it looks like this is a BigData sparql endpoint. I don't know which version they are using, but we have successfully used FedX with a SPARQL endpoint provided by BigData. As a side note: FedX supports also to use custom (vendor specific) repository implementations. See the endpoint documentation as well as the javadoc for com.fluidops.fedx.structures.Endpoint for some further information.

For the europeana endpoint I am seeing very weird behavior locally: using the SPARQL repository test I can only evaluate simple queries when executing with a JRE8, otherwise (with a JRE7) I get a HTTP 500. I will further investigate what is going on here. However, in principle the SparqlRepository (and hence FedX) seems to work properly for this endpoint. What you could try: the FedX release ships the 2.7.3 release of Sesame, locally I have used the newer 2.7.10 version. You could try to exchange the libraries.

Besides that another hint: if this is an OWLIM/GraphDB instance, they typically expose a Sesame HTTPRepositoy compatible endpoint. Maybe it is an option for you to use the Sesame HTTP Repository. Please see the "doc\pages\dataConfig.html" file in the RemoteRepository example for details.

 

Another issue as mentioned before is, that some endpoints don't support variables like ?-anon-1. Is there a reason for not just using ?anon1, which would work at least for dbpedia?

I am not able to reproduce this. Can you give me the initial query (or a simplification) that you are trying to submit? FedX internally does not do any renaming of variables (except for appending suffixes).

I hope these explanations help a bit. I will try to look further into the problems.

Best,

 Andreas

[Nate A]

Am Donnerstag, 13. November 2014 09:13:04 UTC+1 schrieb Andreas Schwarte:

Hi Natnael,

thank you for providing this useful feedback. Such feedback from users helps us in understanding current problems and in a next step to find solutions.

Please see my inline comments with some explanation and hints.

On Tue, Nov 11, 2014 at 4:51 PM, Nate A <arn...@gmail.com> wrote:

Thank you very much for your answer.

Is there an option or is it planed for selecting the request type? E.g. http://lod.b3kat.de/sparql does not support POST queries but works very good with GET queries.

And another option for specifying the possible content types would be good as well. E.g. http://europeana.ontotext.com/sparql/ does support JSON and XML but not CSV or TSV, while FedX seams to ask for one of the latter.

First some technical explanation: under the hood FedX uses the Sesame SPARQLRepository for the communication with SPARQL endpoints. I just tried to use an isolated SPARQL repository (i.e. without FedX being involved) against above b3kat endpoint and exactly saw the problem that you have found.

For the b3kat endpoint my proposal would be the following: you could contact the Sesame mailing list to ask for the best practices (how to deal with such endpoints). Moreover, it looks like this is a BigData sparql endpoint. I don't know which version they are using, but we have successfully used FedX with a SPARQL endpoint provided by BigData. As a side note: FedX supports also to use custom (vendor specific) repository implementations. See the endpoint documentation as well as the javadoc for com.fluidops.fedx.structures.Endpoint for some further information.

For the europeana endpoint I am seeing very weird behavior locally: using the SPARQL repository test I can only evaluate simple queries when executing with a JRE8, otherwise (with a JRE7) I get a HTTP 500. I will further investigate what is going on here. However, in principle the SparqlRepository (and hence FedX) seems to work properly for this endpoint. What you could try: the FedX release ships the 2.7.3 release of Sesame, locally I have used the newer 2.7.10 version. You could try to exchange the libraries.

Besides that another hint: if this is an OWLIM/GraphDB instance, they typically expose a Sesame HTTPRepositoy compatible endpoint. Maybe it is an option for you to use the Sesame HTTP Repository. Please see the "doc\pages\dataConfig.html" file in the RemoteRepository example for details.

Unfortunately I can't find any information about a RemoteRpository provided by europeana.

 

 

Another issue as mentioned before is, that some endpoints don't support variables like ?-anon-1. Is there a reason for not just using ?anon1, which would work at least for dbpedia?

I am not able to reproduce this. Can you give me the initial query (or a simplification) that you are trying to submit? FedX internally does not do any renaming of variables (except for appending suffixes).

Sending this query: select distinct ?Concept where {[] a ?Concept} LIMIT 100 to http://dbpedia.org/sparql vie FedX results in:

FedX Cli 3.1 (build 101)

Running Query 1

ERROR: Query 1 could not be evaluated: 

com.fluidops.fedx.exception.OptimizationException: @ sparql_dbpedia.org_sparql - Virtuoso 37000 Error SP030: SPARQL compiler, line 1: syntax error at \'?\' before \'-\'

SPARQL query:

define sql:big-data-const 0 SELECT * WHERE {?-anon-1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?Concept .  } LIMIT 1. Subquery: SELECT * WHERE {?-anon-1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?Concept .  } LIMIT 1

',

)%  

This can also be reproduce with curl:

curl http://dbpedia.org/sparql\?query\=SELECT+%2A+WHERE+%7B%3F-anon-1+%3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23type%3E+%3FConcept+.++%7D+LIMIT+1.

Virtuoso 37000 Error SP030: SPARQL compiler, line 1: syntax error at '?' before '-'

SPARQL query:

define sql:big-data-const 0 SELECT * WHERE {?-anon-1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?Concept .  } LIMIT 1.%   

 

I hope these explanations help a bit. I will try to look further into the problems.

I will also try the sesame update and see if this helps for europeana.

Thank you

Natanael

[Andreas Schwarte]

Hi Natanael,

Thanks for providing this query, now it is clear what happens:  the Sesame parser creates a variable ?-anon for the token "[]". Since FedX reuses the provided variable names, apparently you run into the above error. I will provide a fix with the next release.

For now you can simply use a standard variable, e.g. SELECT DISTINCT ?Concept WHERE { ?instance a ?Concept } LIMIT 100

This should give the exact same results.

Best,

 Andreas

[Marija Djokic Petrovic]

Hello Andreas,

I am trying to run a simple command: ./cli.sh -d examples/DBpediaSemanticWebDogFood.ttl @q examples/q1.txt, but I am getting the following error: "Data config 'examples/DBpediaSemanticWebDogFood.ttl' could not be loaded: Repository sparql_SWDF could not be initialized: org.openrdf.query.QueryEvaluationException: Connection timed out."

Any information about the FedX service availability? What is the official website now?

Thanks,
Marija

[Andreas Schwarte]

Hello Marija,

 

the example querys are formulated against public SPARQL endpoints (dbpedia, semantic web dog food, …). It might well be that the SPARQL endpoints are no longer available, which is also indicated by the connection timeouts you are seeing in your queries.

 

If you want to run the provided example queries, consider setting up a local SPARQL endpoint for the no-longer available public ones. The datasets should still be available on the internet.

 

Currently there is no official website for FedX available. Questions can still be raised on this mailing list.

 

Hope this helps,

Andreas

--

You received this message because you are subscribed to the Google Groups "Information Workbench Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to

iwb-discussio...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marija Djokic Petrovic]

Hello Andreas,

Thanks for your answer. I managed to run my own SPARQL query on FedX.

I am currently working on my PhD and I am trying to compare SPARQL query results from my application with the results from FedX.

In my application, there is a user interface for providing inputs (i.e. finding targets for given compound), while SPARQL query is generated and executed in background. Is there some similar option through API or some interface on FedX, that provides similar functionality? If I understood correctly, FedX deals only with given SPARQL queries with optimizations for execution performance? 

Many thanks,

Marija

To unsubscribe from this group and stop receiving emails from it, send an email to iwb-discussion+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Information Workbench Discussions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to iwb-discussion+unsubscribe@googlegroups.com.

[priyan...@gmail.com]

Hi

Can someone point me to the link to download the FedX. I am redirectied to Veritas website and cannot find anything there.

To unsubscribe from this group and stop receiving emails from it, send an email to iwb-discussio...@googlegroups.com.

[Aneeq Bokhari]

Hi Andreas. 

down vote

favorit

I'm doing federation query exercise. I want the list of films that stars with leonardio di caprio and also the list of directors that worked with him.

I have to design federated SPARQL query over Wikidata and Dbpedia. I have to run the query through FEDx.

Here's the query. This is done in Eclipse.

   PREFIX wd: <http://www.wikidata.org/entity/> 
            PREFIX wdt: <http://www.wikidata.org/prop/direct/>
            PREFIX dbo: <http://dbpedia.org/ontology/>  
            PREFIX dbr: <http://dbpedia.org/resource/>  
            PREFIX rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
            PREFIX wikibase: <http://wikiba.se/ontology#>
            PREFIX bd: <http://www.bigdata.com/rdf#>

SELECT ?film  ?director   WHERE 
{
     ?film  wdt:P161 wd:Q38111.
    ?film  wdt:P57 ?director.
   ?film rdf:type dbo:Film .  
   ?film dbo:starring dbr:Leonardo_DiCaprio .
} LIMIT 10

When I ran the code, I get an error "Rate limit exceeds".

Can you  explain what is the error and also I think there might be some problem with the query. Can you help me out? 

Thanks 

[aschwarte10]

Hello,

 

thanks for your interest in FedX.

 

The error you are seeing sounds like a message directly coming from one of the SPARQL endpoints. Maybe one of the endpoints only accepts a limited number of requests?

 

Have you been able to submit very simplistic queries? Do you see details in the logs at which endpoint it fails? Do you maybe have some more information (stacktrace etc?). You may also try to run the query using the FedX CLI.

 

Best,

Andreas

[Aneeq Bokhari]

Hi.

There is an error in the query but I fixed it. when running on java. I got the same error  "Rate limit exceeds". But when I run the code couple of times,  the error is gone and shows me the number of records in the output.  
At  "https://stackoverflow.com/questions/42567516/wikidata-request-limit-for-sparql-queries"
Person mentioned  "Single query is currently limited to 1 minute runtime.

Also, each IP is limited to 5 concurrent requests currently. There's no limit on how many sequential requests can be processed.These limits may be adjusted depending on capacity and traffic."