Oxford Archaeology WFS Server

[jo.k...@googlemail.com]

Hi All,

Just to let you all know that OA now have their site database online
as a WFS server. There's no fancy interface at the moment (I'm working
on that), but you can connect with your favourite WFS client. Here's
the GetCapabilities request:

http://mapdata.thehumanjourney.net/cgi-bin/mapservwfs.cgi?service=wfs&request=getcapabilities&version=1.1.0

This is just a work in progress, with a whole series of issues
relating to problems of combining data from two separate databases
with few common fields, whether or not we should put live data on, and
whether there's any other data we could include. Anyhow, it's there,
so do with it as you wish.

Plans for a shiny web-interface are afoot, though I'm short of spare
time and having a few compilation issues. I'll keep you posted.

Let me/us know your thoughts/opinions

Cheers

Jo

[Leif Isaksen]

Hey Steve,

what about having it as a layer at Online Archeology? If everyone else joined in we might even get an inkling of what's been dug in the UK in the past 20 years! (or is that just crazy talk?)

There's a guide to converting e.g. NMR data kicking around in the files section of Antiquist. Any chance of a handy pointers list for setting up a WFS server, Jo? (i.e. what platform/mapping server/analgesic to use ;-)  )

L.

[Crispin Flower]

looks like Mapserver shooting 832 points with not much information attached :-(

seriously - congrats on this;  I failed miserably to get a WFS going last time I tried (about a year ago) - can you post your LAYER definition (or whole .map file) ?

 

re the comment about not knowing what has been dug and where, well most of the SMRs are pretty up-to-date with this info - would it be helpful to you antics if all that data was available as WMS/WFS ??  And for what purposes, and thus what level of detail?

 

cheers

Crispin

~~~~~~~~~~~~~~~~~~~~~
Crispin Flower
exeGesIS SDM Ltd.
Great House Barn
Talgarth
Powys
LD3 0AH
tel: 01874 713072/711145
fax: 01874 711156
email: Cris...@esdm.co.uk
URL: www.esdm.co.uk
~~~~~~~~~~~~~~~~~~~~~
This email and any files transmitted with it are private and intended solely for the use of the individual or entity to whom they are addressed.  If you are not the intended recipient, the email and any files have been transmitted to you in error and any copying, distribution or other use of the information contained in them is strictly prohibited.
Nothing in this email message amounts to a contractual or other legal commitment on the part of exeGesIS SDM Ltd unless confirmed by a signed communication.
exeGesIS SDM Ltd will make every effort to keep its network free of viruses.  However, the recipient of this message will need to scan this message, and any attachments, for viruses, as exeGesIS SDM Ltd can take no responsibility for any computer virus that might be transferred by this email.

Registered in England and Wales : Great House Barn, New Street, Talgarth, Brecon, Powys, LD3 0AH                Company Number : 3743089            VAT number : 736473122

 

---

From: anti...@googlegroups.com [mailto:anti...@googlegroups.com] On Behalf Of Leif Isaksen
Sent: 27 February 2007 18:31
To: anti...@googlegroups.com
Subject: [Antiquist] Re: Oxford Archaeology WFS Server

Hey Steve,

what about having it as a layer at Online Archeology? If everyone else joined in we might even get an inkling of what's been dug in the UK in the past 20 years! (or is that just crazy talk?)

There's a guide to converting e.g. NMR data kicking around in the files section of Antiquist. Any chance of a handy pointers list for setting up a WFS server, Jo? (i.e. what platform/mapping server/analgesic to use ;-)  )

L.

On 27/02/07, jo.k...@googlemail.com <jo.k...@googlemail.com> wrote:

[st...@online-archaeology.co.uk]

This is great news Jo.

Any chance of opening up a WMS service so I can just dump tiles onto
Google Maps?

Maybe we can work together to get a demo going and I can get a how to
guide so anyone can do it with Google Maps.

Of course if it was WMS anyone could also add a layer to Google Earth.

cheers,

Steve

On Feb 27, 6:08 pm, jo.k.c...@googlemail.com wrote:
> Hi All,
>
> Just to let you all know that OA now have their site database online
> as a WFS server. There's no fancy interface at the moment (I'm working
> on that), but you can connect with your favourite WFS client. Here's
> the GetCapabilities request:
>

> http://mapdata.thehumanjourney.net/cgi-bin/mapservwfs.cgi?service=wfs...

[benjami...@ufg.uni-kiel.de]

Hmmmm, I am pretty sure that here (i.e. in Germany) heritage managers
would freak out if their site data was released to the public with precise
coordinates -- for fear of looters.
I know of some pretty bad cases, where good ideas like this have gone
horribly wrong.
Are you planning to have some sort of access control, or maybe add some
fuzz to the site coordinates for public WFS layers?
Don't get me wrong, I think it's a great thing to have some quality
site data freely available. Just a thought.

Benjamin

[Leif Isaksen]

On 27/02/07, Crispin Flower <Cris...@esdm.co.uk> wrote:

looks like Mapserver shooting 832 points with not much information attached :-(

:-D
If I can take the liberty of speaking of Jo/OA's behalf, we thought it was best to get something out there, and only then worry about getting more data attached, rather then ponder about it for too long and never get started! It is admittedly a little skimpy tho, so we're intending to attach more data to the points (as well as add other layers) presently.

seriously - congrats on this;  I failed miserably to get a WFS going last time I tried (about a year ago) - can you post your LAYER definition (or whole .map file) ?

 

re the comment about not knowing what has been dug and where, well most of the SMRs are pretty up-to-date with this info - would it be helpful to you antics if all that data was available as WMS/WFS ??  And for what purposes, and thus what level of detail?

Now  that really would be cool.
Just thinking aloud here but, if I recall, councils have an obligation to make their SMR data available, but they charge for admin each and every time it's requested (which is every time someone does a Desk-Based Assessment). Oh, and they give to you in all kinds of crazy formats so then you've got a bunch of postprocessing to sort out too. Now if they were able to provide data in WMS/WFS so folks could just access it as needed in a single, standard, format then I, for one, would be a very happy puppy indeed. The obvious use is for DBAs - so the usual level of detail I guess - site, location, NGR,  parish, county, description, category, etc.

But here's the 64 million dollar question: I know a lot of arch techies who'd be delighted to see, say, Exegesis provide WFS/WMS functionality out of the box. But will the councils use it? Or, to put it bluntly, is it going to mess with their revenue stream? Any council folk out there (in particular) care to comment?

here's hoping ;-)

L.

[jo.k...@googlemail.com]

Hi All,

Many thanks for the comments. In no particular order:

1: Yes, there isn't very much information attached to the points at
the moment. That's because it's an amalgamation of two almost entirely
incompatible databases (for historic reasons, OA being made of two
separate entirely organisations until quite recently). If I had wanted
to get the data all nice and shiny, I could have done that, and
released it, ooh, next year sometime, whereas now I can change the
data quite happily at my leisure and have it available as well. It's a
proof of concept and an attempt to be more open, nothing more. In
fact- it's a prime example of why we need more adoption of standards
for describing what we do- at the moment it's full of terms like desk-
based assessments/DBAs/D.B.As/desk-top surveys all describing the same
thing. Tidying 30 years of data is quite a job...

2: Accuracy of the data- yes, we thought about the question of
looters, but the data isn't quite up to date (so hasn't got any live
sites on it), and is mostly only accurate to 100m or so. It's also
publically available via the NMR (which is how we actually got the
data for many of the sites), so we're not providing any extra
information than is available elsewhere. We might choose to make it
deliberately "fuzzy", or I think I can tune the WFS to only display
some information about each point in the xml. I'm not sure about that,
and it's certainly valuable to hear people's thoughts on the matter.

3: WMS- no problem. I didn't know how much call there was, but there's
no reason why I can't set it up to do both. I'll come back to you on
that.

4: Demos, guides to setting up mapping servers- no problem. Mapserver
is really very straightforward, it's usually the front ends that are a
drag.

Cheers for all the comments- I'll let you know as soon as we get more
functionality!

Jo

On Feb 27, 9:37 pm, "Leif Isaksen" <leif...@googlemail.com> wrote:

> > email: Crisp...@esdm.co.uk

> > URL:www.esdm.co.uk
> > ~~~~~~~~~~~~~~~~~~~~~
> > This email and any files transmitted with it are private and intended
> > solely for the use of the individual or entity to whom they are addressed.
> > If you are not the intended recipient, the email and any files have been
> > transmitted to you in error and any copying, distribution or other use of
> > the information contained in them is strictly prohibited.
> > Nothing in this email message amounts to a contractual or other legal
> > commitment on the part of exeGesIS SDM Ltd unless confirmed by a signed
> > communication.
> > exeGesIS SDM Ltd will make every effort to keep its network free of
> > viruses. However, the recipient of this message will need to scan this
> > message, and any attachments, for viruses, as exeGesIS SDM Ltd can take no
> > responsibility for any computer virus that might be transferred by this
> > email.
>
> > Registered in England and Wales : Great House Barn, New Street, Talgarth,
> > Brecon, Powys, LD3 0AH Company Number : 3743089
> > VAT number : 736473122
>

> > ------------------------------
> > *From:* anti...@googlegroups.com [mailto:anti...@googlegroups.com] *On
> > Behalf Of *Leif Isaksen
> > *Sent:* 27 February 2007 18:31
> > *To:* anti...@googlegroups.com
> > *Subject:* [Antiquist] Re: Oxford Archaeology WFS Server

>
> > Hey Steve,
>
> > what about having it as a layer at Online Archeology? If everyone else
> > joined in we might even get an inkling of what's been dug in the UK in the
> > past 20 years! (or is that just crazy talk?)
>
> > There's a guide to converting e.g. NMR data kicking around in the files
> > section of Antiquist. Any chance of a handy pointers list for setting up a
> > WFS server, Jo? (i.e. what platform/mapping server/analgesic to use ;-) )
>
> > L.
>

> > On 27/02/07, jo.k.c...@googlemail.com <jo.k.c...@googlemail.com> wrote:
>
> > > Hi All,
>
> > > Just to let you all know that OA now have their site database online
> > > as a WFS server. There's no fancy interface at the moment (I'm working
> > > on that), but you can connect with your favourite WFS client. Here's
> > > the GetCapabilities request:
>

> > >http://mapdata.thehumanjourney.net/cgi-bin/mapservwfs.cgi?service=wfs...

[Crispin Flower]

Hi all

Most SMRs/HERs are now looking to put their data on-line, and the delays in doing so are mainly financial hurdles (most do not have any uncomitted budget at all) and the time it takes to do things like this (given that they are very small cogs in very big wheels). But believe me it is happening and will advance rapidly in the next couple of years.

 

Loss of revenue stream is not really a factor here. More of factor is their well-founded concern that contractors will (understandably) want to save money/time/effort by using just the data that may be available on-line, in preference to discussing the area/site/contract with the SMR staff and getting all the info (often paper).  This is a well-observed phenomenon, and the quality of DBAs can plummet where the researcher relies on e.g. Canmore or whatever. This situation should mature - as people learn what info is appropriate for what purposes, and learn to serve up fit-for-purpose datasets.

WFS/WMS alone is only a small part of the answer here.

So in answer to Leif's question, yes local authorities will use these technologies in the near future, but we all need to think carefully about what is served up for what purposes, and to use that data responsibly. An interesting problem here is that these initiatives are often HLF-funded and have to be aimed at the public, but actually I suspect that contractors and academics are the more significant audience, so it is important that their needs are understood.

In a nut shell, archaeological info on the web will generally give you an index to stuff that you are usually going to have to research in more detail by other means.

 

re Jo's comment about the need to adopt standards for describing events - is that comment aimed at contractors/academics? 

The primary records of event data are the SMRs, with additional national inventories like the AIP and now OASIS, and all of these use reasonably well-developed standards for structure and terminology (not enough yet, but they are not bad).

I'm puzzled why OA are seeing themselves as curators of a dataset that anyone would want to look at, apart from for their own internal purposes and/or publicity.  Once OA have adequately described the work they have done (DBA, excavation, whatever), and then submitted that to the relevant SMR and OASIS, surely that is job done from their point of view. I mean, nobody should expect to have to ask OA what excavations have been undertaken in East Borsetshire in the last 5 years - they will ask the SMR who also know about the digs done by all the other contractors. I'm not criticising what you have done, because it is a very useful stimulant!

 

And Steve - I'm thinking it should be possible to get WFS onto Google maps - your post suggested maybe not - have you investigated this?? Perhaps performance would just be much worse than WMS?

 

cheers

Crispin

---

From: anti...@googlegroups.com [mailto:anti...@googlegroups.com] On Behalf Of Leif Isaksen

Sent: 27 February 2007 21:46

To: anti...@googlegroups.com
Subject: [Antiquist] Re: Oxford Archaeology WFS Server

On 27/02/07, Crispin Flower <Cris...@esdm.co.uk> wrote:

looks like Mapserver shooting 832 points with not much information attached :-(

:-D
If I can take the liberty of speaking of Jo/OA's behalf, we thought it was best to get something out there, and only then worry about getting more data attached, rather then ponder about it for too long and never get started! It is admittedly a little skimpy tho, so we're intending to attach more data to the points (as well as add other layers) presently.

seriously - congrats on this;  I failed miserably to get a WFS going last time I tried (about a year ago) - can you post your LAYER definition (or whole .map file) ?

 

re the comment about not knowing what has been dug and where, well most of the SMRs are pretty up-to-date with this info - would it be helpful to you antics if all that data was available as WMS/WFS ??  And for what purposes, and thus what level of detail?

Now  that really would be cool.
Just thinking aloud here but, if I recall, councils have an obligation to make their SMR data available, but they charge for admin each and every time it's requested (which is every time someone does a Desk-Based Assessment). Oh, and they give to you in all kinds of crazy formats so then you've got a bunch of postprocessing to sort out too. Now if they were able to provide data in WMS/WFS so folks could just access it as needed in a single, standard, format then I, for one, would be a very happy puppy indeed. The obvious use is for DBAs - so the usual level of detail I guess - site, location, NGR,  parish, county, description, category, etc.

But here's the 64 million dollar question: I know a lot of arch techies who'd be delighted to see, say, Exegesis provide WFS/WMS functionality out of the box. But will the councils use it? Or, to put it bluntly, is it going to mess with their revenue stream? Any council folk out there (in particular) care to comment?

here's hoping ;-)

L.

[st...@online-archaeology.co.uk]

Jo - thanks I look forward to the WMS and will rig up some test pages

Crispin - yes, it's possible to load data onto Gmaps via a WFS call.
You just need to parse the GML. But there are performance implications
you need to consider with Gmaps. Loading > 200 markers in one viewport
starts to degrade performance. You can use some clustering code to
help this though. From my tests, I prefer to overlay opaque layers of
PNG tiles from a WMS AJAX call. The user can toggle these layers on/
off on the client. You can then capture an onlick event per layer and
send the x,y back to a WFS call and populate the InfoWindow this way
with a single item. This reduces the load on the client considerably
and means you are actually targetting your data calls rather that
trying to parse too much at once.

Given this model and hopefully the introduction of more WMS/WFS
services from separate sources, it opens up some great opportunities
to present archaeological data to the end user at very low cost.

cheers,

Steve

> email: Crisp...@esdm.co.uk

> On 27/02/07, jo.k.c...@googlemail.com <jo.k.c...@googlemail.com>

[jo.k...@googlemail.com]

Why did we do it?

Because we wanted to. Why shouldn't we? This has both internal and
external uses, and all the while we are waiting for curators to put
their datasets on line and make them more available, this is one way
of getting our data out and about. We subscribe to the philosophy of
open data, which means we are open with our own datasets, and this was
a proof of concept experiment that we could do this easily. It's not
meant as a criticism or subversion of the standard curatorial process,
it's just a way of sharing data.

My comment about standards was not meant as a dig, but in fact a wry
remark that even in one organisation it's not possible to amalgamate
two datasets that do the same job without a lot of work sorting out
the consistency. That is partly an issue of training staff, and
building databases that don't allow inconsistency but we're talking
about 30 years of data and hindsight is a wonderful thing.

Jo

On Feb 28, 9:44 am, "Crispin Flower" <Crisp...@esdm.co.uk> wrote:

> email: Crisp...@esdm.co.uk

> On 27/02/07, jo.k.c...@googlemail.com <jo.k.c...@googlemail.com>

[Andrew Larcombe]

Hi Jo, all

http://www.andrewlarcombe.co.uk/index.php/oa

This is the OA WFS displayed on google maps via my server side proxy
which handles the clustering, minimizing stress on the client's browser.
Contra Steve (sorry) I think that this is a better approach than serving
up the data via WMS, mainly because you're separating out presentation
from content. Yes, you can style WMS layers using SLD, but it is a real
PITA, especially if you want to have dynamically generated SLDs.

Anyhow the above page is a bit rough and ready. A diamond with a '+1' in
its centre is a cluster of sites that can't be resolved at the present
zoom level. Click on the diamond to zoom in. A diamond with a dot in its
centre is a site. Click on it to display a popup window with the OA
project code and the site's code. Ideally there would be a link to
further information about the site, but I guess that might be for the
future...hint, hint ;)

Thanks to OA for making this available to the community.

Cheers,

Andrew

--
Andrew Larcombe
Freelance Geospatial, Database & Web Programming

web: http://www.andrewlarcombe.co.uk
email: and...@andrewlarcombe.co.uk
mob: +44 (7760) 258623
icq: 306690163

[Andrew Larcombe]

Leif Isaksen wrote:

> Now that really would be cool.
> Just thinking aloud here but, if I recall, councils have an obligation
> to make their SMR data available, but they charge for admin each and
> every time it's requested (which is every time someone does a Desk-Based
> Assessment). Oh, and they give to you in all kinds of crazy formats so
> then you've got a bunch of postprocessing to sort out too. Now if they
> were able to provide data in WMS/WFS so folks could just access it as
> needed in a single, standard, format then I, for one, would be a very
> happy puppy indeed. The obvious use is for DBAs - so the usual level of
> detail I guess - site, location, NGR, parish, county, description,
> category, etc.
>
> But here's the 64 million dollar question: I know a lot of arch techies
> who'd be delighted to see, say, Exegesis provide WFS/WMS functionality
> out of the box. But will the councils use it? Or, to put it bluntly, is
> it going to mess with their revenue stream? Any council folk out there
> (in particular) care to comment?

All of OAD's upcoming online HERs - SMR is soo 90's ;) - have WMS
functionality out-of-the-box. However the councils we've dealt with have
all specified this as a requirement for use internally, either for
linking with existing desktop GISs and/or WMS servers.

[st...@online-archaeology.co.uk]

Andrew -

No apologies needed, WMS is my personal preference as I said. The
reason is because if the user wants an holistic view of 1000s of
points across a wide area, the best method is using WMS and PNG
layers, not clustered points. Also if you want to show polygons on
Gmaps I think the best approach is clickable PNG layers rather than
processing all polygon co-ordinates.

cheers,

Steve

On Mar 1, 12:42 am, Andrew Larcombe <and...@andrewlarcombe.co.uk>
wrote:

> jo.k.c...@googlemail.com wrote:
> > Hi All,
>
> > Just to let you all know that OA now have their site database online
> > as a WFS server. There's no fancy interface at the moment (I'm working
> > on that), but you can connect with your favourite WFS client. Here's
> > the GetCapabilities request:
>

> >http://mapdata.thehumanjourney.net/cgi-bin/mapservwfs.cgi?service=wfs...

> icq: 306690163- Hide quoted text -
>
> - Show quoted text -

[jo.k...@googlemail.com]

Thanks for putting a nice little map up Andrew, you've beaten me to
it! We will have an interface on our website soon though.

My colleagues in Oxford have asked me to point out that the dataset
isn't exactly complete. There are a lot of old sites that we simply
don't have grid references for yet, but hopefully we should be able to
resolve this and get a more representative dataset up. Ho Hum

Jo

[Andrew Larcombe]

st...@online-archaeology.co.uk wrote:
> Andrew -
>
> No apologies needed, WMS is my personal preference as I said. The
> reason is because if the user wants an holistic view of 1000s of
> points across a wide area, the best method is using WMS and PNG
> layers, not clustered points. Also if you want to show polygons on
> Gmaps I think the best approach is clickable PNG layers rather than
> processing all polygon co-ordinates.

Yes, there's more that one way to skin a cat. What you say about
polygons is true - this is how we implemented a recent HER at OAD. At
high level points are drawn as javascript clickable objects on a map,
when zooming in beyond a certain level the sites polygons are drawn
using a wms layer.

[Andrew Larcombe]

jo.k...@googlemail.com wrote:
> Thanks for putting a nice little map up Andrew, you've beaten me to
> it! We will have an interface on our website soon though.
>
> My colleagues in Oxford have asked me to point out that the dataset
> isn't exactly complete. There are a lot of old sites that we simply
> don't have grid references for yet, but hopefully we should be able to
> resolve this and get a more representative dataset up. Ho Hum

No probs. I was stuck for 4 hours on the A34 towards Oxford yesterday
morning, then in meetings during the afternoon - felt I had to do
something productive that day!

Was hoping to add a dialog box to show site information, so let me know
as and when you get it!