Browsing the data sets?

[PaulG]

From a previous posting really - I just feel this needs a topic of its
own.

At the moment we can only search for data sets, which is OK as its
only early days, we understand that, I can also understand why some
hierarchy of terms is not presented, as that'd be just as confusing -
loads of headings with nothing in them.

However as there is no speedy reply forthcoming, I am assuming - maybe
wrongly - that maybe they haven't worked how best to handle this yet.

So, what method, or hierarchy would we expect to see the data sets
presented in, so they are "brows-able" and could inspire connections
between sets? (don't say random, or then again - maybe that'd get the
old synapses twanging... )

[Kalvir Sandhu]

Hey Paul and all,

Great site so far, like the design.

I too found it hard to navigate around the data, at the moment I would to be able to filter by these

- Date range in datasets, I noticed that a lot of the data is for ranges of dates. "As a developer I want to create something with latest data, daily snapshots, etc", so how can i find data sets that are just reports and ones that are updated

- Format, is the data an API, CSV, XML files, etc. As a developer I can only work with API based data sets.

- Location - does the data have lat/lngs, addresses, etc. As a developer I want to build something that is location specific and need to find data that can work in this.

The problem with this is obviously how to categorise all the data sets, well perhaps as a first project we create a crowd sourced meta data tagger thingy, where a page loads an untagged dataset and users of the app can tag it - similar to Chris Thorpe's contentagger - that's if it is a problem to categorise the data in better ways.

Kalv.

[Philip John]

How about the ability to find data by searching the data itself as
well?

Phil

[Tom Morris]

On Thu, Oct 1, 2009 at 16:01, PaulG <foof...@gmail.com> wrote:
> So, what method, or hierarchy would we expect to see the data sets
> presented in, so they are "brows-able" and could inspire connections
> between sets? (don't say random, or then again - maybe that'd get the
> old synapses twanging... )
>

I'm not sure if many people on the list have done SPARQL much before,
but here is how I tend to explore a new SPARQL end-point.

We'll use the BBC programmes database as an example - it's at
http://api.talis.com/stores/bbc-backstage
The SPARQL endpoint is http://api.talis.com/stores/bbc-backstage/services/sparql

How do we explore it?

1. Query it to see what classes of data are available. In RDF, there
is a predicate called rdf:type which states that an individual is a
member of a class. It's generally done with RDF to ensure that
everything is a member of a class. Classes should be familiar to
anyone who either has understood set theory or who has done any
object-oriented programming. It's literally just a type of thing.

You can do a SPARQL query which gives you back a list of classes by
simply doing this:

SELECT DISTINCT ?class WHERE { ?object a ?class . }

You'll get back a list of all the different classes in the system. If
you are used to SQL, this is equivalent to getting a list of all the
tables in a database, except each table is referred to using a URI.

If the query takes a long time to respond, it might be because there's
a lot of data. So, before we go any further, you should learn the use
of the LIMIT and OFFSET commands. These are just there for paging
through results and not putting too much strain on the server. You use
them by simply putting them on subsequent lines after the main body of
the query, like this:

SELECT DISTINCT ?class WHERE { ?object a ?class . }
LIMIT 50

SELECT DISTINCT ?class WHERE { ?object a ?class . }
LIMIT 50
OFFSET 50

The first of these will return the first 50. The second will return
the second 50. You just keep adding to the OFFSET to page from that
point in the result set.

2. Next, I tend to see a list of properties that are used by whatever
class I'm interested in. Let's take a particular class - Series. The
URI is <http://purl.org/ontology/po/Series>

To see what properties there are available in that class, you simply need to do:

SELECT DISTINCT ?property
WHERE {
?object a <http://purl.org/ontology/po/Series> .
?object ?property ?x .
}
LIMIT 50

This will return a list of the first 50 properties it finds that are
used by members of that class. The way to read a SPARQL query is as a
list of distinct limitations on sentences with variables. So, here we
are saying get all the triples that match the first line - they are a
Series. Then see what properties are bound to them.

3. Finally, I tend to look at a few examples of whatever class I'm
using. To do that, you just do this query:

DESCRIBE ?object WHERE {
?object a <http://purl.org/ontology/po/Series> .
}
LIMIT 1

This returns the first instances of Series as RDF/XML. To see more,
you just add an OFFSET.

If you are going to be doing lots with RDF, I recommend that you
install cwm - http://www.w3.org/2000/10/swap/doc/cwm.html

cwm is sort of like sed or awk is for text - it lets you manipulate
RDF from the command line, and see it in different formats, like N3.
N3 is a much easier format to read than RDF/XML.

Beyond these very simple basics, you generally need to read the
'SPARQL Query Language for RDF' specification -
http://www.w3.org/TR/rdf-sparql-query/ - you can do some pretty
complex and cool queries with SPARQL. And there is more on the way - I
don't know if data.gov.uk or Talis are going to be implementing
SPARQL2 when it comes out - but SPARQL2 brings a whole load of stuff
that SQL users will be familiar with to SPARQL. Big things like the
ability to update (the equivalent of UPDATE and INSERT statements) and
to do things like SQL's SELECT COUNT. And much more. But that's Real
Soon Now apparently.

--
Tom Morris
<http://tommorris.org/>

[Kingsley Idehen]

All,

How does one actually get access to this Linked Data Space? I assume
there is some kind of registration process for accounts?

--

Regards,

Kingsley Idehen Weblog: http://www.openlinksw.com/blog/~kidehen
President & CEO
OpenLink Software Web: http://www.openlinksw.com

[Martin Stone]

Kingsley

you make a request to join the group and soon you get sent a signon and password for http://data.hmg.gov.uk/data

but you have to be extra special to be accepted ;-)

Martin Stone

2009/10/2 Kingsley Idehen <kid...@openlinksw.com>

[Ian Davis]

i think you should have received a login with your access to this
group - at least that's what happened when i signed up. Also remember
that this is all very nascent - not all the infrastructure is in
place, e.g. the web servers for the subdomains of data.gov.uk are not
active yet so the URIs arent responding to HTTP - its all coming soon.

> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
>

--
Ian Davis, Chief Technology Officer, Talis
tel: +44 (0) 870 400 5000
cell: +44 (0) 7525 941 919

I'm trialling Google Apps using a temporary email address. Email sent
to my usual address (ian....@talis.com) will still reach me.

[Al Sutton]

SPARQL sounds pretty heavyweight. Are there any studies on the costs
of scaling it up?

Amazon have done a lot of work with large datasets which could be of
use (http://aws.amazon.com/publicdatasets/), or even an lightweight
interface along the lines of their SimpleDB offering (http://aws.amazon.com/simpledb/
) might offer benefits in cost of scaling, simplicity of
implementation, and it would build on the work they've already done in
terms of scaling, access, and reliability.

Al.

--

* Looking for Android Apps? - Try http://andappstore.com/ *

======
Funky Android Limited is registered in England & Wales with the
company number 6741909. The registered head office is Kemp House,
152-160 City Road, London, EC1V 2NX, UK.

The views expressed in this email are those of the author and not
necessarily those of Funky Android Limited, it's associates, or it's
subsidiaries.

[Kingsley Idehen]

Martin Stone wrote:
> Kingsley
>
> you make a request to join the group and soon you get sent a signon
> and password for http://data.hmg.gov.uk/data
>
> but you have to be extra special to be accepted ;-)

Isn't this mailing list the group in question?

Kingsley

>
> Martin Stone
>
>
> 2009/10/2 Kingsley Idehen <kid...@openlinksw.com

> <mailto:kid...@openlinksw.com>>

>
>
> All,
>
> How does one actually get access to this Linked Data Space? I
> assume there is some kind of registration process for accounts?
>
> --
>
>
> Regards,
>
> Kingsley Idehen Weblog:
> http://www.openlinksw.com/blog/~kidehen

> <http://www.openlinksw.com/blog/%7Ekidehen>

[Kingsley Idehen]

Ian Davis wrote:
> i think you should have received a login with your access to this
> group - at least that's what happened when i signed up. Also remember
> that this is all very nascent - not all the infrastructure is in
> place, e.g. the web servers for the subdomains of data.gov.uk are not
> active yet so the URIs arent responding to HTTP - its all coming soon.
>

Ian,

I suspect the mail hit my spam box :-(

Do you have subject line and/or email address pattern for the mail?

Kingsley

[Kingsley Idehen]

Al Sutton wrote:
>
> SPARQL sounds pretty heavyweight. Are there any studies on the costs
> of scaling it up?
>
> Amazon have done a lot of work with large datasets which could be of
> use (http://aws.amazon.com/publicdatasets/), or even an lightweight
> interface along the lines of their SimpleDB offering

> (http://aws.amazon.com/simpledb/) might offer benefits in cost of

> scaling, simplicity of implementation, and it would build on the work
> they've already done in terms of scaling, access, and reliability.
>

When this data space settles, I expect there to be lots of Linked Open
Data Sets exposed as EC2 snapshots. I speculate because I am actually
building such things :-)

Kingsley

[Aaron Trevena]

2009/10/2 Kingsley Idehen <kid...@openlinksw.com>:

> Do you have subject line and/or email address pattern for the mail?

andrew...@cabinet-office.x.gsi.gov.uk

--
Aaron J Trevena, BSc Hons
http://www.aarontrevena.co.uk
LAMP System Integration, Development and Consulting

[Kingsley Idehen]

Aaron Trevena wrote:
> 2009/10/2 Kingsley Idehen <kid...@openlinksw.com>:
>
>> Do you have subject line and/or email address pattern for the mail?
>>
>
> andrew...@cabinet-office.x.gsi.gov.uk
>
>

Thanks!

Got it!

[Al Sutton]

Great!!!

One question though; Does it have to be EC2 images (i.e. bound to a
virtual machine with OS, etc., etc.)? Would it not be possible to do a
dump into SimpleDB or a simplistic format which could be imported into
any database a developer wished stored on S3?

Al.

[Timothy Lewy]

I had a fair bit of experience with SPARQL (and RDQL) a while back on the AKT project. The query language itself is very flexible and can't really be described as "heavy weight"; it is the complexity of the queries that defines how well performance will scale with the size of the knowledge base.

 

However, the language or interface at the front end seems a fairly mute point to me. There's no point botching a SPARQL interface onto a legacy dataset, it all needs to be in proper RDF. The FAQ suggests this is a longer term goal. If actually happenned it could be brilliant, but I am sceptical as to whether it could been done properly (dumping data sets on the web is cheap, doing serious refactoring is not). Lots of questions to be answered: Do we know what form the data is going to be stored in internally/presented as externally? There's obviously a lot of SWeb terminology being banded around, does this mean there is going to be a genuine commitment to publishing data in RDF against proper ontologies? Will URI's be referrable across sets? etc, etc.

---

Tim Lewy

 

 

2009/10/2 Al Sutton <a...@funkyandroid.com>

SPARQL sounds pretty heavyweight. Are there any studies on the costs of scaling it up?

Amazon have done a lot of work with large datasets which could be of use (http://aws.amazon.com/publicdatasets/), or even an lightweight interface along the lines of their SimpleDB offering (http://aws.amazon.com/simpledb/) might offer benefits in cost of scaling, simplicity of implementation, and it would build on the work they've already done in terms of scaling, access, and reliability.

[Ian Davis]

Timothy,

There is no botching of sparql onto legacy data. A lot of effort has
been put into translating the data into proper RDF and there is a lot
more to come. This is still early days!

The backend is currently supplied by Talis who specialise in RDF and
SPARQL. See http://www.talis.com/platform for more information about
our capabilities.

Ian

On Friday, October 2, 2009, Timothy Lewy <tim...@googlemail.com> wrote:
> I had a fair bit of experience with SPARQL (and RDQL) a while back on the AKT project. The query language itself is very flexible and can't really be described as "heavy weight"; it is the complexity of the queries that defines how well performance will scale with the size of the knowledge base.
>
>
> However, the language or interface at the front end seems a fairly mute point to me. There's no point botching a SPARQL interface onto a legacy dataset, it all needs to be in proper RDF. The FAQ suggests this is a longer term goal. If actually happenned it could be brilliant, but I am sceptical as to whether it could been done properly (dumping data sets on the web is cheap, doing serious refactoring is not). Lots of questions to be answered: Do we know what form the data is going to be stored in internally/presented as externally? There's obviously a lot of SWeb terminology being banded around, does this mean there is going to be a genuine commitment to publishing data in RDF against proper ontologies? Will URI's be referrable across sets? etc, etc.
>
>
> ---
> Tim Lewy
>
>

> 2009/10/2 Al Sutton <a...@funkyandroid.com <javascript:_e({}, 'cvml', 'a...@funkyandroid.com');>>

>
>
> SPARQL sounds pretty heavyweight. Are there any studies on the costs of scaling it up?
>
> Amazon have done a lot of work with large datasets which could be of use (http://aws.amazon.com/publicdatasets/), or even an lightweight interface along the lines of their SimpleDB offering (http://aws.amazon.com/simpledb/) might offer benefits in cost of scaling, simplicity of implementation, and it would build on the work they've already done in terms of scaling, access, and reliability.
>
> Al.
>
> --
>
> * Looking for Android Apps? - Try http://andappstore.com/ *
>
> ======
> Funky Android Limited is registered in England & Wales with the
> company number  6741909. The registered head office is Kemp House,
> 152-160 City Road, London,  EC1V 2NX, UK.
>
> The views expressed in this email are those of the author and not
> necessarily those of Funky Android Limited, it's associates, or it's
> subsidiaries.
>
>
>
>
> On 2 Oct 2009, at 18:32, Tom Morris wrote:
>
>

> don't know if data.gov.uk <http://data.gov.uk/> or Talis are going to be implementing

> SPARQL2 when it comes out - but SPARQL2 brings a whole load of stuff
> that SQL users will be familiar with to SPARQL. Big things like the
> ability to update (the equivalent of UPDATE and INSERT statements) and
> to do things like SQL's SELECT COUNT. And much more. But that's Real
> Soon Now apparently.
>
> --
> Tom Morris
> <http://tommorris.org/>
>
>
>

> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>

--

[Al Sutton]

I think we're coming at this from two different viewpoints. I see the access method as reasonably important because all the work put into collecting the data becomes wasted if it's not easily accessible. 

Looking at examples of re-usable data from Google, Yahoo, Microsoft, and the focus is on a light weight simplistic API which is purely focused on accessing the raw data. A simplistic API lowers the barrier to entry for developers and thus encourages innovation. Many developers I expect will use this data for "spare time" projects and therefore will most likely give things a miss if they have to learn a complex new technology or protocol just to gain access to the data.  

Personally I've always been a fan of KISS, so a stripped down simple access API is, to me, an important part of any data heavy project.

Al.

[Timothy Lewy]

Top down and bottom up, but I think we're on the same page.

 

Agreed on the simple API front - first step must always be to just get it out there, somehow (getting there already - we've got access to loads of data in various forms). Raw data is also good, keeping the original ensures we don't lose anything through someone elses interpretation.

 

All the rest comes in subsequent steps. Homogenise the data access; clean up the sources; provide increasingly powerful views on the data.

 

---

Tim

2009/10/2 Al Sutton <a...@funkyandroid.com>

[Alan Kelly]

I have to agree with this - primarily for me, access to this data is
for personal interest and hobby projects. A simple API like Twitter,
Facebook etc. offer would be preferrable. Google go a step further and
offer a complete set of libraries for different technologies. Using
the .NET assemblies I was able to put together a utility to query
address books for my members within an afternoon.

I am not familiar with SPARQL or some of the other things mentioned
here, although it looks similar to SQL - why not just use SQL, which
more developers are likely to be familiar with? Of course the lowest
common denominator would be TSC or binary data, let developers do what
they want with the simplest formats.

I do a lot of work with data from NASA etc. and that is usually either
a) a simple binary data format or b) standards compliant GIS data.
Both work well and are quick and easy to access. It's quicker (I
think) to read a note on how a file is structured than learning a new
language or API.

[Tom Morris]

On Fri, Oct 2, 2009 at 21:02, Alan Kelly <alan.kel...@gmail.com> wrote:
> I have to agree with this - primarily for me, access to this data is
> for personal interest and hobby projects. A simple API like Twitter,
> Facebook etc. offer would be preferrable. Google go a step further and
> offer a complete set of libraries for different technologies. Using
> the .NET assemblies I was able to put together a utility to query
> address books for my members within an afternoon.
>

If you are doing .NET, you should probably take a look at
http://razor.occams.info/code/semweb/

[Timothy Lewy]

The advantage of having SPARQL in the long term, is that you do not have to worry about your data sources and formats at all - you just infer over entire sets and get answers to complex queries very easily. While you are only doing simple mashups from a couple of sources, this isn't relevant. Once it gets much more sophisticated, where the data is correlated from all over the place, it is much more useful.

 

Tim

2009/10/2 Alan Kelly <alan.kel...@gmail.com>

[John Goodwin]

Yes agreed. Plus (IMHO) SPARQL is waaay easier to use than SQL - no horrible table joins. SPARQL is a really easy way to traverse the graph of data and find answers to questions across multiple datasets. This mash up:

http://www.johngoodwin.me.uk/boundaries/meshup.html

was made using SPARQL queries run over BBC music, BBC programmes, Ordnance Survey and DBpedia linked data. For example this:

http://www.johngoodwin.me.uk/boundaries/lutonbeeb.php

shows episodes of radio shows playing artists that originate from Luton by joining together the datasets mentions above. Was fairly straight forward to do.

John

 

--

Homepage: http://www.johngoodwin.me.uk
Blog: http://johngoodwin225.wordpress.com
Personal URI: http://www.johngoodwin.me.uk/me

Isn't it enough to see that a garden is beautiful without having to believe that there are fairies at the bottom of it too? - Douglas Adams

---

From: Timothy Lewy <tim...@googlemail.com>
To: uk-government-...@googlegroups.com
Sent: Friday, 2 October, 2009 9:21:56 PM
Subject: Re: Browsing the data sets?

[Kingsley Idehen]

Al Sutton wrote:
>
> Great!!!
>
> One question though; Does it have to be EC2 images (i.e. bound to a
> virtual machine with OS, etc., etc.)? Would it not be possible to do a
> dump into SimpleDB or a simplistic format which could be imported into
> any database a developer wished stored on S3?

You will have snapshots at your disposal covering:

1. Raw Data e.g. N3, Turtle, RDF/XML dumps
2. Virtuoso (Quad Store behind DBpedia [1] and many bubbles in the
Linked Open Data Cloud) [2]) backup dumps
3. Fully configured and optimized Virtuoso Database instances, so you
basically make an AMI and just start the DB server, after that you have
a live Linked Data Space (i.e., HTTP based URIs for each data object
that de-reference to structured metadata in negotiable representations;
of course, default representation will be: html+rdfa).

All of the above will take the form of EBS snapshots that you mount into
a given AMI. Thus, if SimpleDB can consume RDF model oriented data
formats, then #1 will serve you best. Of course, we can also expose #1
via S3 buckets.

Links:

1. http://dbpedia.org/About -- Virtuoso instance
2. http://lod.openlinksw.com -- LOD Cloud Cache instance
3. http://bbc.openlinksw.com -- BBC Linked Data Space (Virtuoso hosted
variant) .

Kingsley

[Rob Farnell]

I'm new to SPARQL, but I found an excellent source of data from Lee Feigenbaum's blog:
http://www.thefigtrees.net/lee/blog/2009/06/sparqling_at_semtech.html

SPARQL by example http://www.cambridgesemantics.com/2008/09/sparql-by-example/

SPARQL cheat sheet http://www.slideshare.net/LeeFeigenbaum/sparql-cheat-sheet

Regards,

Rob

[Leigh Dodds]

Fwding on from someone having problems accessing the list.

L.

---------- Forwarded message ----------
From: Paola Di Maio <paola....@gmail.com>
Date: 2009/10/2
Subject: Fwd: Browsing the data sets?
To: leigh...@talis.com


Leigh, I am signed up to this group, but my posts keep on bouncing
please repost for me, thanks
P

---------- Forwarded message ----------
From: Paola Di Maio <paola....@gmail.com>
Date: Fri, Oct 2, 2009 at 9:12 PM
Subject: Re: Browsing the data sets?

To: uk-government-...@googlegroups.com


Alan

I am new to SPARQL too, and not completely sold over to it yet, but from what I understand it has two advantages over SQL
as follows

1. it's more efficient in terms of writing queries (have seen some examples, its more compact, should be able to dig something up for you) - you could say it is marginally 'nicer'

2. SQL works when the data is in databses, the linked data model proposed here
spans across different datasets all over the place, hence the need to expose data  and
link it by using triples (rdf) and query it via sparql

(correct me anyone if wrong)

however I do agree that it would be nice to have high level interfaces with macros written out already
to query these datasets  so that we dont have to learn SPARQL ever (never liked sql either), just select some fields
and the query will self compile

(btw thanks for the tutorials, i am looking for them, possibly compiling a list at some point)


PDM

--
Paola Di Maio
**************************************************
Networked Research Lab, UK

***************************************************

--
Paola Di Maio
**************************************************
Networked Research Lab, UK

***************************************************

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________

--
Leigh Dodds
Programme Manager, Talis Platform
Talis
leigh...@talis.com
http://www.talis.com

[Timothy Lewy]

I've never used any of these Amazon services before. What is the advantage of delivering data in this manner?

 

Tim

2009/10/2 Kingsley Idehen <kid...@openlinksw.com>

[Leigh Dodds]

Hi,

2009/10/2 Alan Kelly <alan.kel...@gmail.com>

>
> I have to agree with this - primarily for me, access to this data is
> for personal interest and hobby projects. A simple API like Twitter,
> Facebook etc. offer would be preferrable. Google go a step further and
> offer a complete set of libraries for different technologies. Using
> the .NET assemblies I was able to put together a utility to query
> address books for my members within an afternoon.

There will be a simple API. As of next week you'll be able to do a
HTTP request for any of the URIs in the datasets and retrieve the data
as RDF/XML or JSON. So eventually retrieving data about a specific
school will be as simple as doing a GET on
http://education.data.gov.uk/id/school/_120805. No API keys or other
url munging required.

How do you find the URIs? well there will be SPARQL queries, which can
easily be parameterised to look up uris for different types of things
or based on different categories. There will also be a simple search
interface that'll let you submit simple key word searches and retrieve
a RSS feed with OpenSearch extensions to give you the results.

Overtime I'm sure additional options will become available.

The problem with providing just a "simple API" is that for services
like Twitter or Facebook they're typically dealing with data about
only a small number of different types of things, with a fixed set of
properties. So crafting an API to ask useful questions of the data
isn't that hard.

But what we're looking at here a process that will ultimately open up
1000s of datasets about a wide range of different resources. It'd be
time consuming to create a custom API for all of the different
variations. By using RDF and SPARQL, and supplementing them with
simple search interfaces, etc. the effort can focus on collecting and
modelling the data. And we get the benefit of using a uniform,
standardised API across the whole set.

The extra benefits that SPARQL will bring is allowing a much richer
set of queries than a simple API would allow. And I think this is an
important part of the overall infrastructure &legacy that this project
is aiming to build.

Cheers,

L.

[Al Sutton]

The main advantages are;

- Well known and well documented API: Numerous books, libraries, and web pages exist to help developers, and many developers of internet applications will have either worked with or be familiar with their offerings.

- Proven scalability: It's the design ethos behind Amazons Web services and there are numerous examples of large scale deployments (e.g. http://aws.typepad.com/aws/2008/05/amazon-simpledb.html)

- Resilience: They operate from several data centres in the US and EU so there is geographical as well as local failover  built in to the system.

- Cost: You pay for what you use instead of needing to pay for machines, maintenance resources, etc. before you know how heavily the dataset will be used. 

My experiences of it have been very good. My company uses EC2 because if we get load spikes we can have another virtual server up and running to share the load in a few minutes and we can bring several VMs online in parallel. 

Kingsleys' S3 dumps sound like a good start for getting the data into SimpleDB (I've not found an RDF import facility for it, but I'll hunt around a it more). If you combine the dumps with the "Requester pays" feature of S3 it means that developers will pay for their own usage, thus avoiding the need to pay for developers usage of the dumps. (http://docs.amazonwebservices.com/AmazonS3/latest/index.html?RequesterPaysBuckets.html)

Al.

[Al Sutton]

Leigh,

Good to hear things are going in both directions (i.e. simple API &
allowing complex queries). This should maximise the appeal to
developers.

Al.

--

* Looking for Android Apps? - Try http://andappstore.com/ *

======
Funky Android Limited is registered in England & Wales with the
company number 6741909. The registered head office is Kemp House,
152-160 City Road, London, EC1V 2NX, UK.

The views expressed in this email are those of the author and not
necessarily those of Funky Android Limited, it's associates, or it's
subsidiaries.

[Tim Lewy]

So, if i'm understanding things right, the data is going to be available in several ways:

 

- Raw data available in the original form

- RDF/XML etc available directly via SPARQL and HTTP

- Data dumps available through EC2

 

?

 

Tim

 

 

2009/10/3 Al Sutton <a...@funkyandroid.com>

[Richard Wallis]

Plus:    

"There will also be a simple search
interface that'll let you submit simple key word searches and retrieve
a RSS feed with OpenSearch extensions to give you the results."

Richard Wallis

Technology Evangelist - Talis

 

Please consider the environment before printing this email.

Find out more about Talis at www.talis.com

shared innovation^TM

Any views or personal opinions expressed within this email may not be those of Talis Information Ltd or its employees. The content of this email message and any files that may be attached are confidential, and for the usage of the intended recipient only. If you are not the intended recipient, then please return this message to the sender and delete it. Any use of this e-mail by an unauthorised recipient is prohibited.

Talis Information Ltd is a member of the Talis Group of companies and is registered in England No 3638278 with its registered office at Knights Court, Solihull Parkway, Birmingham Business Park, B37 7YB.

 

 

[Kingsley Idehen]

Timothy Lewy wrote:
> I had a fair bit of experience with SPARQL (and RDQL) a while back on
> the AKT project. The query language itself is very flexible and can't
> really be described as "heavy weight"; it is the complexity of the
> queries that defines how well performance will scale with the size of
> the knowledge base.
>
> However, the language or interface at the front end seems a fairly
> mute point to me. There's no point botching a SPARQL interface onto a
> legacy dataset, it all needs to be in proper RDF.

Depends on a variety of factors. Mapping a legacy (non RDF model based)
to RDF doesn't imply an improper end product. Its all about who is doing
the mapping and their commitment to granularity. Bottom line, this is a
function of who the data publisher is, and to some degree, the tools at
their disposal.

> The FAQ suggests this is a longer term goal. If actually happenned it
> could be brilliant, but I am sceptical as to whether it could been
> done properly (dumping data sets on the web is cheap, doing serious
> refactoring is not). Lots of questions to be answered: Do we know what
> form the data is going to be stored in internally/presented as
> externally? There's obviously a lot of SWeb terminology being banded
> around, does this mean there is going to be a genuine commitment to
> publishing data in RDF against proper ontologies? Will URI's be
> referrable across sets? etc, etc.

What I am assuming will happen here is a UK Govt data variant of
DBpedia[1] or BBC (re. the shape and form of a Linked Data Space for
Open Data). Thus, at the very least expect the following:

1. SPARQL endpoint -- seems to be there
2. Generic HTTP URIs for all data items in the Data Space supporting
HTML+RDFa, N3/Turtle, RDF/JSON, RDF/XML metadata representations -- TBD
3. Mechanism for expanding and enhancing schemas [2] [3] -- TBD
3. An archive dump URL -- which allows anyone with interest, skill set,
and technology to build 1&2

Links:

1. http://dbpedia.org
2. http://data-gov.tw.rpi.edu
3. http://data-gov.tw.rpi.edu/wiki/Data.gov_Catalog -- although cross
agency, similar approaches can be applied to Home Office specific data.

Kingsley
>
> ---
> Tim Lewy
>
>
> 2009/10/2 Al Sutton <a...@funkyandroid.com <mailto:a...@funkyandroid.com>>

> don't know if data.gov.uk <http://data.gov.uk/> or Talis are

> going to be implementing
> SPARQL2 when it comes out - but SPARQL2 brings a whole load of
> stuff
> that SQL users will be familiar with to SPARQL. Big things
> like the
> ability to update (the equivalent of UPDATE and INSERT
> statements) and
> to do things like SQL's SELECT COUNT. And much more. But
> that's Real
> Soon Now apparently.
>
> --
> Tom Morris
> <http://tommorris.org/>
>
>
>

[Kingsley Idehen]

Al Sutton wrote:
> I think we're coming at this from two different viewpoints. I see the
> access method as reasonably important because all the work put into
> collecting the data becomes wasted if it's not easily accessible.
>
> Looking at examples of re-usable data from Google, Yahoo, Microsoft,
> and the focus is on a light weight simplistic API which is purely
> focused on accessing the raw data. A simplistic API lowers the barrier
> to entry for developers and thus encourages innovation. Many
> developers I expect will use this data for "spare time" projects and
> therefore will most likely give things a miss if they have to learn a
> complex new technology or protocol just to gain access to the data.
>
> Personally I've always been a fan of KISS, so a stripped down simple
> access API is, to me, an important part of any data heavy project.

Al,

HTTP based Linked Data[1] is about KISS applied to open data access. The
problem right now is that this Linked Data Space isn't complete, it just
has a SPARQL endpoint. That said, as long as the SPARQL endpoint exposes
all the currently available data, anybody (with interest, skill set, and
technology) can generate a complete KISS style Linked Data Space, you
don't have to depend solely on the provider of the SPARQL endpoint.

Links:

1. http://en.wikipedia.org/wiki/Linked_Data
2. http://dbpedia.org/resource/Linked_Data
3. http://linkeddata.org/docs/how-to-publish -- How to publish Linked
Data (example Linked Data Deployment Platform here is Pubby)
4.
http://virtuoso.openlinksw.com/Whitepapers/html/vdld_html/VirtDeployingLinkedDataGuide.html
-- How to publish Linked Data (example Linked Data Deployment Platform
here is OpenLink Virtuoso).

Kingsley

>
> Al.
>
>
> On 2 Oct 2009, at 20:16, Timothy Lewy wrote:
>
>> I had a fair bit of experience with SPARQL (and RDQL) a while back on
>> the AKT project. The query language itself is very flexible and can't
>> really be described as "heavy weight"; it is the complexity of the
>> queries that defines how well performance will scale with the size of
>> the knowledge base.
>>
>> However, the language or interface at the front end seems a fairly
>> mute point to me. There's no point botching a SPARQL interface onto a
>> legacy dataset, it all needs to be in proper RDF. The FAQ
>> suggests this is a longer term goal. If actually happenned it could
>> be brilliant, but I am sceptical as to whether it could been done
>> properly (dumping data sets on the web is cheap, doing serious
>> refactoring is not). Lots of questions to be answered: Do we know
>> what form the data is going to be stored in internally/presented as
>> externally? There's obviously a lot of SWeb terminology being banded
>> around, does this mean there is going to be a genuine commitment to
>> publishing data in RDF against proper ontologies? Will URI's be
>> referrable across sets? etc, etc.
>>
>> ---
>> Tim Lewy
>>
>>

>> 2009/10/2 Al Sutton <a...@funkyandroid.com <mailto:a...@funkyandroid.com>>

>> don't know if data.gov.uk <http://data.gov.uk/> or Talis are

>> going to be implementing
>> SPARQL2 when it comes out - but SPARQL2 brings a whole load
>> of stuff
>> that SQL users will be familiar with to SPARQL. Big things
>> like the
>> ability to update (the equivalent of UPDATE and INSERT
>> statements) and
>> to do things like SQL's SELECT COUNT. And much more. But
>> that's Real
>> Soon Now apparently.
>>
>> --
>> Tom Morris
>> <http://tommorris.org/>
>>
>>
>>
>

[Kingsley Idehen]

Timothy Lewy wrote:
> The advantage of having SPARQL in the long term, is that you do not
> have to worry about your data sources and formats at all - you just
> infer over entire sets and get answers to complex queries very easily.
> While you are only doing simple mashups from a couple of sources, this
> isn't relevant. Once it gets much more sophisticated, where the
> data is correlated from all over the place, it is much more useful.

Like any Query Language and Data Access Protocol combo (e.g. SQL and
ODBC, JDBC, ADO.NET etc..), you can basically build and share Views for
high level data access esp. as SPARQL is also a Query Language & Data
Access Protocol combo plus Results Serialization format. When all is
said an done, Linked Data is about using resource URLs to share Views
applied to the triples / quads in an RDF store :-) Basically, its Webby
Report Writing & Sharing that covers the mundane all the way up to
sophisticated BI style cubes (which are called facets in the SemWeb realm).

The UK Govt. like the U.S. and Australian govts. is becoming a large
structured data publisher with granular HTTP based Data Access (Linked
Data) as a major bonus.

Kingsley

>
> Tim
>
> 2009/10/2 Alan Kelly <alan.kel...@gmail.com

> <mailto:alan.kel...@gmail.com>>

> <mailto:a...@funkyandroid.com>>

> >>> don't know if data.gov.uk <http://data.gov.uk/> or Talis are

> going to be implementing
> >>> SPARQL2 when it comes out - but SPARQL2 brings a whole load of
> stuff
> >>> that SQL users will be familiar with to SPARQL. Big things
> like the
> >>> ability to update (the equivalent of UPDATE and INSERT
> statements) and
> >>> to do things like SQL's SELECT COUNT. And much more. But
> that's Real
> >>> Soon Now apparently.
> >>>
> >>> --
> >>> Tom Morris
> >>> <http://tommorris.org/>
> >>
> >
> >
> >
>
>

[Kingsley Idehen]

All,

Collection of SPARQL tutorials:

1. http://delicious.com/kidehen/sparql_tutorial

[Richard Surey]

I'm new to SPARQL however it looks like a good idea. SQL would be in reams of joins after no time at all with these data quantities!

I'm just looking forward to the possibilities with augmented reality on mobile devices and this data.

[Kingsley Idehen]

Timothy Lewy wrote:
> I've never used any of these Amazon services before. What is the
> advantage of delivering data in this manner?

You get a service or personal-use specific data space in the EC2 cloud.
You don't have to squat with others when it comes to data usage etc..
Remember, the Internet's scale comes from federation rather than
centralization. Ironically, centralized services (the Software as a
Service model) achieve scale via domain specific federation :-)

So basically, Amazon gives an on/off style data space that serves your
specific needs.

Kingsley

>
> Tim
>
> 2009/10/2 Kingsley Idehen <kid...@openlinksw.com

> <mailto:kid...@openlinksw.com>>

>
>
> Al Sutton wrote:
>
>
> Great!!!
>
> One question though; Does it have to be EC2 images (i.e. bound
> to a virtual machine with OS, etc., etc.)? Would it not be
> possible to do a dump into SimpleDB or a simplistic format
> which could be imported into any database a developer wished
> stored on S3?
>
> You will have snapshots at your disposal covering:
>
> 1. Raw Data e.g. N3, Turtle, RDF/XML dumps
> 2. Virtuoso (Quad Store behind DBpedia [1] and many bubbles in the
> Linked Open Data Cloud) [2]) backup dumps
> 3. Fully configured and optimized Virtuoso Database instances, so
> you basically make an AMI and just start the DB server, after that
> you have a live Linked Data Space (i.e., HTTP based URIs for each
> data object that de-reference to structured metadata in negotiable
> representations; of course, default representation will be:
> html+rdfa).
>
> All of the above will take the form of EBS snapshots that you
> mount into a given AMI. Thus, if SimpleDB can consume RDF model
> oriented data formats, then #1 will serve you best. Of course, we
> can also expose #1 via S3 buckets.
>
> Links:
>
> 1. http://dbpedia.org/About -- Virtuoso instance

> 2. http://lod.openlinksw.com <http://lod.openlinksw.com/> -- LOD
> Cloud Cache instance
> 3. http://bbc.openlinksw.com <http://bbc.openlinksw.com/> -- BBC

> <foof...@gmail.com <mailto:foof...@gmail.com>>

> don't know if data.gov.uk <http://data.gov.uk/> or

> Talis are going to be implementing
> SPARQL2 when it comes out - but SPARQL2 brings a
> whole load of stuff
> that SQL users will be familiar with to SPARQL.
> Big things like the
> ability to update (the equivalent of UPDATE and
> INSERT statements) and
> to do things like SQL's SELECT COUNT. And much
> more. But that's Real
> Soon Now apparently.
>
> --
> Tom Morris
> <http://tommorris.org/>
>
>
>
>
>
> --
>
>
> Regards,
>
> Kingsley Idehen Weblog:
> http://www.openlinksw.com/blog/~kidehen

> <http://www.openlinksw.com/blog/%7Ekidehen>

> President & CEO
> OpenLink Software Web: http://www.openlinksw.com

> <http://www.openlinksw.com/>

>
>
>
>
>
>
>
>
> --
>
>
> Regards,
>
> Kingsley Idehen Weblog:
> http://www.openlinksw.com/blog/~kidehen

> <http://www.openlinksw.com/blog/%7Ekidehen>

> President & CEO OpenLink Software Web:

> http://www.openlinksw.com <http://www.openlinksw.com/>

[Kingsley Idehen]

Tim Lewy wrote:
> So, if i'm understanding things right, the data is going to be
> available in several ways:
>
> - Raw data available in the original form
> - RDF/XML etc available directly via SPARQL and HTTP
> - Data dumps available through EC2
>

Yes, I am saying that you would be able to start an EC2 AMI and then
mount volumes (derived from snapshots) of interest which may contain any
of the items above.

This is going to happen with DBpedia, LOD Cloud Data Sets (which will
ultimately include data from various Govts. that now adopt the HTP based
Linked Data approach).

Kingsley
> ?
>
> Tim
>
>
>
> 2009/10/3 Al Sutton <a...@funkyandroid.com <mailto:a...@funkyandroid.com>>

>
>
> Leigh,
>
> Good to hear things are going in both directions (i.e. simple API
> & allowing complex queries). This should maximise the appeal to
> developers.
>
>
> Al.
>
> --
>
> * Looking for Android Apps? - Try http://andappstore.com/ *
>
> ======
> Funky Android Limited is registered in England & Wales with the
> company number 6741909. The registered head office is Kemp House,
> 152-160 City Road, London, EC1V 2NX, UK.
>
> The views expressed in this email are those of the author and not
> necessarily those of Funky Android Limited, it's associates, or it's
> subsidiaries.
>
> On 3 Oct 2009, at 11:09, Leigh Dodds wrote:
>
>
> Hi,
>
> 2009/10/2 Alan Kelly <alan.kel...@gmail.com

> <mailto:alan.kel...@gmail.com>>

> leigh...@talis.com <mailto:leigh...@talis.com>
> http://www.talis.com <http://www.talis.com/>

[Kingsley Idehen]

Richard Wallis wrote:
> Plus:
> "There will also be a simple search
> interface that'll let you submit simple key word searches and retrieve
> a RSS feed with OpenSearch extensions to give you the results."

Basically, something like one of the following:

1. http://dbpedia.org/fct;
2. http://bbc.openlinksw.com;
3. http://lod.openlinksw.com;

where is all cases you start with "Full Text Patterns" but you end up
with the ability to navigate Linked Data along Entity Type or Entity
Property dimensions. It's important that exploration follows the Full
Text query in the general workflow to maximize the virtues of HTTP based
Linked Data.

So if I search on pattern: London, I can use the "Types" link to sort my
initial query result by Entity Types associated with the Text pattern.
Then, if required, filter further by clicking on the "Properties" link
so that Property Values or Origins filter the results even further. Once
you've completed filtering (act of disambiguation) you simply click on
"Show Distinct Results with Counts" or "Show Results" which present a
list of Entity Matches and their HTTP URIs. Then when you click these
URIs you get Descriptions or Links to "Statistics" which then provide a
holistic view of the data space from the Data Dictionary
(Schema/Ontology/Vocab) perspective unveiling things like: Direct &
Indirect Co-references etc..

This is what EAV/CR model (RDF model is an example of this) exploration
is about, and its also why its superior to the Relational Model when
dealing with ad-hoc data querying and meshing across disparate data sources.

Ironically, although users and some developers may not comprehend some
of the intrinsics of what I outline above, this is what they've come to
expect via Report Writing and BI tools from the RDBMS realm, so they
basically take a lot of this for granted, since they've been able to do
this using other DBMS technologies since the late '80's.

Remember, the only real novelty of Linked Data lies in the ingenuity of
the Generic HTTP URI that is implicitly bound to metadata (in myriad of
formats) of its referent (entity identified by an HTTP URI).

Kingsley

>
>
> Richard Wallis
> Technology Evangelist - Talis
>
> On 3 Oct 2009, at 15:10, "Tim Lewy" <tim...@googlemail.com

> <mailto:tim...@googlemail.com>> wrote:
>
>> So, if i'm understanding things right, the data is going to be
>> available in several ways:
>>
>> - Raw data available in the original form
>> - RDF/XML etc available directly via SPARQL and HTTP
>> - Data dumps available through EC2
>>
>> ?
>>
>> Tim
>>
>>
>>

>> 2009/10/3 Al Sutton <a...@funkyandroid.com <mailto:a...@funkyandroid.com>>

>>
>>
>> Leigh,
>>
>> Good to hear things are going in both directions (i.e. simple API
>> & allowing complex queries). This should maximise the appeal to
>> developers.
>>
>>
>> Al.
>>
>> --
>>
>> * Looking for Android Apps? - Try http://andappstore.com/ *
>>
>> ======
>> Funky Android Limited is registered in England & Wales with the
>> company number 6741909. The registered head office is Kemp House,
>> 152-160 City Road, London, EC1V 2NX, UK.
>>
>> The views expressed in this email are those of the author and not
>> necessarily those of Funky Android Limited, it's associates, or it's
>> subsidiaries.
>>
>> On 3 Oct 2009, at 11:09, Leigh Dodds wrote:
>>
>>
>> Hi,
>>
>> 2009/10/2 Alan Kelly <alan.kel...@gmail.com

>> <mailto:alan.kel...@gmail.com>>

>> leigh...@talis.com <mailto:leigh...@talis.com>

>> http://www.talis.com
>>
>>
>>
>
>
> Please consider the environment before printing this email.
>
>

> Find out more about Talis at www.talis.com <http://www.talis.com/>
>
> *shared innovation^TM *

>
>
> Any views or personal opinions expressed within this email may not be
> those of Talis Information Ltd or its employees. The content of this
> email message and any files that may be attached are confidential, and
> for the usage of the intended recipient only. If you are not the
> intended recipient, then please return this message to the sender and
> delete it. Any use of this e-mail by an unauthorised recipient is
> prohibited.
>
>
> Talis Information Ltd is a member of the Talis Group of companies and
> is registered in England No 3638278 with its registered office at
> Knights Court, Solihull Parkway, Birmingham Business Park, B37 7YB.
>
>
>

[PaulG]

Great, now it's obvious that one of the key skills for accessing LOD
is to understand the power of SPARQL queries... (I have seen some
twitter noise about this over the weekend).

And a great thanks to Tom Morris' comments and examples earlier - that
was just the advice I have been looking for.

"I'm not sure if many people on the list have done SPARQL much before,
but here is how I tend to explore a new SPARQL end-point. "

and Rob's links to Lee's tutorials and other references, I'll add in
my tuppence.

I am not sure how many other SPARQL n00bs (of whom I still count
myself, btw) would benefit from this advice, but the whole LOD thing
can seem a bit abstract when starting out.

If so, try starting out using DBpedia, it is basically Wikipedia re-
used as LOD.

If you can find things in Wikipedia, then you can do the same - and SO
much more by querying DBpedia.

If you get lost in DBpedia, and your SPARQL queries aren't getting you
the results you expect it is ever-so much easier to just keep going
back to the relevent Wikipedia page(s) to un-obfuscate the data and
see how it all "fits together".

One of the easiest places to get into DBpedia is the SNORQL gateway,
which sets some common preset namespaces for you:

http://dbpedia.org/snorql/

A good shoe-in might be to read a nice intro SPARQL, such as the ones
previously mentioned - or this one;

http://www.craigethomas.com/blog/2009/02/anatomy-of-a-sparql-query-part-1-select/

and then take a URL-based SNORQL query such as the one below:

(show known places in the Kirklees MBC area, get their postcodes, dial
codes where known)

http://dbpedia.org/snorql/?query=SELECT+%3Fplace+%3Flat+%3Flng+%3Fdial+%3Fabstract+%3Fpc%0D%0AWHERE+{%0D%0A++++++{+%3Fplace+dbpedia2%3Adistrict%3AKirklees+}+%0D%0AUNION+{+%3Fplace+dbpedia2%3AshireDistrict%3AKirklees+}+%0D%0AUNION+{+%3Fplace+dbpedia2%3AmetropolitanBorough%3AKirklees+}%0D%0AUNION+{+%3Fplace+dbpedia2%3AmetropolitanDistrict%3AKirklees+}+.%0D%0A++%3Fplace+dbpedia2%3Alatitude+%3Flat+.%0D%0A++%3Fplace+dbpedia2%3Alongitude+%3Flng+.%0D%0A+OPTIONAL+{+%3Fplace+dbpedia2%3AdialCode+%3Fdial+.}%0D%0A++%3Fplace+dbpedia2%3Aabstract+%3Fabstract+.FILTER+langMatches(+lang(%3Fabstract)%2C+%22en%22+)%0D%0A+OPTIONAL+{+%3Fplace+dbpedia2%3ApostcodeDistrict+%3Fpc+}+%0D%0A+%0D%0A}%0D%0A

Bookmark it, then just have a play with the values, click some of the
links in the results sets to see where they take you and you start to
appreciate some of the terminology being used, and I hope, some of the
power you can unleash.

Hope this helps someone, and I trust I am not advocating any poor
practices.

Put me right if I am, will ya?