Test server architecture

[Andrew Matthews]

Hi All,

Now we have a few others on the discussion group, I thought it might
be a good time to raise a not particularly pressing issue that I want
to address.

I've been using SemWeb's in-built support for triple stores. That is I
have two avenues for getting to the triple store - via an in-memory
triple store or through SPARQL to (currently) a semweb server running
under ASP.NET. My problem is that SemWeb uses a very cut-down
implementation of Jos de Roo's Euler reasoning engine. It also lacks
much of the reasoning support that was implicit in the rules
specification of N3 (at least as far as I can tell from the docs up at
the W3C).

I spent a very long time hunting around for the perfect test framework
before I gave up and settled for the SPARQL compliance of Ryan
Levering's SPARQL Engine that is provided by SemWeb. What I wonder is
if there is a better performing and more powerful triplpe store out
there that I can use for this. I guess I don't care how it is
implemented (they mostly seem to be Java) and I know Jena is usable
for industrial scale applications. But if there are others I could
use, what are they?

Andrew Matthews

[Andrew Matthews]

hi chris,

 

To be honest I've never bothered to use virtuoso. I know that some of their engineers have used LinqToRdf successfully - and the latest round of changes that will (very soon) be in version 0.7 were requested by them.

 

When I was doing my initial researching on semantic web products and platforms I was disappointed to learn that there was nothing natively .NET other than SemWeb (which has no pretensions to being a complete platform the way virtuoso does). So I have focussed on producing SPARQL. Everyone's going to support that, so I don't need to target specific products. I DO know that those guys are pretty vigorous - Kingsley Idehen told me off for using Jena for my remote SPARQL testing rather than Virtuoso. :-)

The LinqToRdf documentation is a little bit lacking (hey, I'm a programmer, right! ;). If you have any questions, or need any help, just post questions via the discussion forum (so everyone gets the benefit of the answers). I've copied this email through to the forum, so you now have the email address. The web interface is at: http://groups.google.com/group/linqtordf-discuss. I've just been using it for announcements so far, but perhaps it'd be better to talk more...

 

NetworkGraphs looks great - I'd love to hear from you about how you go, and how it perfoms. I do worry about performance on these systems. Having a high performance, scalable and clustered, fail-over-able, persistent back-end that I can use in LinqToRdf would be good to throw back at the traditionalist relational database crowd. :-) nobody seems to be thinking too much about how you'd use semantic web technologies in enterprise computing environments (i.e. how you could use them in _preferance_ to RDBMSs).

 

Do you plan to support SPARQL Update? That's rising up my LinqToRdf features priority list...

 

Good Luck and Let me know your thoughts and experiences on LinqToRdf.

 

Andrew

On Fri, May 23, 2008 at 4:22 AM, Chris Richard <chris....@gmail.com> wrote:

What were your thoughts on Virtuoso?

I'm working with Sesame, trying to create a high-performance
distributed triple store underneath their API and SPARQL support.
Eventually I'd like to integrate Networked Graphs (http://isweb.uni-
koblenz.de/Research/NetworkedGraphs) as this seems to be a simple and
powerful way to develop re-usable rules.

Anyways, awesome project. I'm going to check out this linqtordf
documentation.

[Chris Richard]

Heh, did I 'Reply to author' again? If so please paste my reply into
your response again, it was not my intent to do this.

On May 22, 4:45 pm, "Andrew Matthews" <matthews.and...@gmail.com>
wrote:

> hi chris,
>
> To be honest I've never bothered to use virtuoso. I know that some of their
> engineers have used LinqToRdf successfully - and the latest round of changes
> that will (very soon) be in version 0.7 were requested by them.
>
> When I was doing my initial researching on semantic web products and
> platforms I was disappointed to learn that there was nothing natively .NET
> other than SemWeb (which has no pretensions to being a complete platform the
> way virtuoso does). So I have focussed on producing SPARQL. Everyone's going
> to support that, so I don't need to target specific products. I DO know that
> those guys are pretty vigorous - Kingsley Idehen told me off for using Jena
> for my remote SPARQL testing rather than Virtuoso. :-)

> The LinqToRdf documentation is a little bit lacking (hey, I'm a *programmer*,

> right! ;). If you have any questions, or need any help, just post questions
> via the discussion forum (so everyone gets the benefit of the answers). I've
> copied this email through to the forum, so you now have the email address.
> The web interface is at:http://groups.google.com/group/linqtordf-discuss.
> I've just been using it for announcements so far, but perhaps it'd be better
> to talk more...
>
> NetworkGraphs looks great - I'd love to hear from you about how you go, and
> how it perfoms. I do worry about performance on these systems. Having a high
> performance, scalable and clustered, fail-over-able, persistent
> back-end that I can use in LinqToRdf would be good to throw back at the
> traditionalist relational database crowd. :-) nobody seems to be thinking
> too much about how you'd use semantic web technologies in enterprise
> computing environments (i.e. how you could use them in _preferance_ to
> RDBMSs).
>
> Do you plan to support SPARQL Update? That's rising up my LinqToRdf features
> priority list...
>
> Good Luck and Let me know your thoughts and experiences on LinqToRdf.
>
> Andrew

> On Fri, May 23, 2008 at 4:22 AM, Chris Richard <chris.rich...@gmail.com>

> wrote:
>
>
>
> > What were your thoughts on Virtuoso?
>
> > I'm working with Sesame, trying to create a high-performance
> > distributed triple store underneath their API and SPARQL support.
> > Eventually I'd like to integrate Networked Graphs (http://isweb.uni-
> > koblenz.de/Research/NetworkedGraphs) as this seems to be a simple and
> > powerful way to develop re-usable rules.
>
> > Anyways, awesome project. I'm going to check out this linqtordf

> > documentation.- Hide quoted text -
>
> - Show quoted text -

[Andrew Matthews]

---------- Forwarded message ----------
From: Chris Richard <chris....@gmail.com>
Date: Fri, May 23, 2008 at 11:42 PM
Subject: Re: Test server architecture
To: Andrew Matthews <matthew...@gmail.com>


Argh. I had a lengthy response but IE went and crashed on me. I'll try
to do the highlights quickly:

I've seen Kingsley get a little overzealous on the SemWeb DL too.
Virtuoso is a powerful system nonetheless, as you can see in sites
like dbpedia and in various benchmarks.

I ended up on Sesame due to the lack of mature .NET SemWeb frameworks.
With SPARQL protocol (and things like LinqToRdf) I don't see that as
too much of a problem anymore. I believe Sesame has their own REST-
based update interface, probably relatively similar to SPARQL Update.

Sesame has a nice model where you can plug triple providers into their
query and API support. Basic integration involves a simple pattern
matching interface (find(subj, pred, obj, ctx)) which I've got working
well enough to load the entire dbpedia dataset and do simple querying.
Advanced integration involves analyzing the query expression tree and
executing optimized queries against your indexes and so on. I've got a
lot of work to do in this area before the system will be really
usable.

>nobody seems to be thinking
> too much about how you'd use semantic web technologies in enterprise
> computing environments

Completely agree. I think the sky's the limit in terms of potential
here. Have you seen this presentation? http://media.allgames.com/wilshire/semantic.wmv

One thing I wondered about for LinqToRdf was whether it would be
possible to use anonymous types, or schema analysis+code generation to
get a completely dynamic experience where you don't even need to
define your annotated classes. Admittedly a very under-developed idea
but Elmo (Sesame sub-project, http://openrdf.org/doc/elmo/1.0/) has a
similar ability to generate Java classes from schema or dynamically
create types based on the query results. Anyways, just food for
thought.

The idea of LinqToRdf from Silverlight with DataTemplate bindings
makes me salivate. Have you tried LinqToRdf on Silverlight 2? Any
known blockers?

Chris

[Andrew Matthews]

Hi Chris,

 

LinqToRdf still has a long way to go, I don't have the time or resources to put it through the kind of quality assurance regime I'd like. Still, we have a little time before s/w hits the bigtime, so hopefully I can attract some volunteers before then. As Fabrice Marguere pointed out, you'll only ever get volunteers after the hard work has been done...

 

For what it's worth, LinqToRdf started out working in In-Proc mode using a proprietary query language invented by Joshua Tauberer called RSQuary. I extended it to support SPARQL and tried to ensure that the system could be extended without too much misery. Perhaps you could target Sesame (if you IKVMed it).?.

 

Re anonymous types: do you mean projections within a query? If so, then yes it can. A good example is found on line 192 here. If you mean to generate the classes from metadata at runtime, then no. Although I am working on a supplementary project called RdfMetal, which will use SPARQL DESCRIBE queries to get enough metadata to generate a C# entity model. It's early days for that as well, but I don't think it will be too hard. In previous ORM products, I have made use of RDBMS metadata to generate  dynamic assemblies in the way you suggest, but I don't think that it would make for a very pleasant development experience... (thinking of intellisense).

 

SilverLight - no, I haven't played with it. I know that there is a version of the SemWeb library for Silverlight. SilverLight supports System.Core and System.Data.Linq. However, SemWeb SPARQL requires IKVM (IIRC), which is about 10 MB. So I guess it would be a bit painful... maybe worth some time experimenting though... Are You Volunteering? You might be able to strip out semweb SPARQL altogether...

Shiny new version out BTW: http://aabs.wordpress.com/

 

Andrew
 

On Fri, May 23, 2008 at 11:42 PM, Chris Richard <chris....@gmail.com> wrote:

Argh. I had a lengthy response but IE went and crashed on me. I'll try
to do the highlights quickly:

I've seen Kingsley get a little overzealous on the SemWeb DL too.
Virtuoso is a powerful system nonetheless, as you can see in sites
like dbpedia and in various benchmarks.

I ended up on Sesame due to the lack of mature .NET SemWeb frameworks.
With SPARQL protocol (and things like LinqToRdf) I don't see that as
too much of a problem anymore. I believe Sesame has their own REST-
based update interface, probably relatively similar to SPARQL Update.

Sesame has a nice model where you can plug triple providers into their
query and API support. Basic integration involves a simple pattern
matching interface (find(subj, pred, obj, ctx)) which I've got working
well enough to load the entire dbpedia dataset and do simple querying.
Advanced integration involves analyzing the query expression tree and
executing optimized queries against your indexes and so on. I've got a
lot of work to do in this area before the system will be really
usable.

>nobody seems to be thinking
> too much about how you'd use semantic web technologies in enterprise
> computing environments

Completely agree. I think the sky's the limit in terms of potential
here. Have you seen this presentation? http://media.allgames.com/wilshire/semantic.wmv

One thing I wondered about for LinqToRdf was whether it would be
possible to use anonymous types, or schema analysis+code generation to
get a completely dynamic experience where you don't even need to
define your annotated classes. Admittedly a very under-developed idea
but Elmo (Sesame sub-project, http://openrdf.org/doc/elmo/1.0/) has a
similar ability to generate Java classes from schema or dynamically
create types based on the query results. Anyways, just food for
thought.

The idea of LinqToRdf from Silverlight with DataTemplate bindings
makes me salivate. Have you tried LinqToRdf on Silverlight 2? Any
known blockers?

Chris

On May 22, 4:45 pm, "Andrew Matthews" <matthews.and...@gmail.com>
wrote:

> hi chris,
>
> To be honest I've never bothered to use virtuoso. I know that some of their
> engineers have used LinqToRdf successfully - and the latest round of changes
> that will (very soon) be in version 0.7 were requested by them.
>
> When I was doing my initial researching on semantic web products and
> platforms I was disappointed to learn that there was nothing natively .NET
> other than SemWeb (which has no pretensions to being a complete platform the
> way virtuoso does). So I have focussed on producing SPARQL. Everyone's going
> to support that, so I don't need to target specific products. I DO know that
> those guys are pretty vigorous - Kingsley Idehen told me off for using Jena
> for my remote SPARQL testing rather than Virtuoso. :-)

> The LinqToRdf documentation is a little bit lacking (hey, I'm a *programmer*,

> right! ;). If you have any questions, or need any help, just post questions
> via the discussion forum (so everyone gets the benefit of the answers). I've
> copied this email through to the forum, so you now have the email address.
> The web interface is at:http://groups.google.com/group/linqtordf-discuss.
> I've just been using it for announcements so far, but perhaps it'd be better
> to talk more...
>
> NetworkGraphs looks great - I'd love to hear from you about how you go, and
> how it perfoms. I do worry about performance on these systems. Having a high
> performance, scalable and clustered, fail-over-able, persistent
> back-end that I can use in LinqToRdf would be good to throw back at the
> traditionalist relational database crowd. :-) nobody seems to be thinking
> too much about how you'd use semantic web technologies in enterprise
> computing environments (i.e. how you could use them in _preferance_ to
> RDBMSs).
>
> Do you plan to support SPARQL Update? That's rising up my LinqToRdf features
> priority list...
>
> Good Luck and Let me know your thoughts and experiences on LinqToRdf.
>
> Andrew

> On Fri, May 23, 2008 at 4:22 AM, Chris Richard <chris.rich...@gmail.com>

> wrote:
>
>
>
> > What were your thoughts on Virtuoso?
>
> > I'm working with Sesame, trying to create a high-performance
> > distributed triple store underneath their API and SPARQL support.
> > Eventually I'd like to integrate Networked Graphs (http://isweb.uni-
> > koblenz.de/Research/NetworkedGraphs) as this seems to be a simple and
> > powerful way to develop re-usable rules.
>
> > Anyways, awesome project. I'm going to check out this linqtordf

> > documentation.- Hide quoted text -
>
> - Show quoted text -

--
Regards,

Andrew Matthews
http://aabs.wordpress.com