GIS Interest Group
199 views
Skip to first unread message
James Griffin III
Dec 1, 2014, 12:08:01 PM12/1/14
to isla...@googlegroups.com
Hello Everyone,
For those who celebrated, I hope that your Thanksgiving holiday was a pleasant one.
Throughout the course of the summer, two projects scoped for the Lafayette College Libraries required that we implement a local solution using Web Mapping and Web Feature Services integrated with our Islandora ecosystem. Fortunately, the open source project GeoServer served our purposes, and our development efforts for these projects are well underway:
* http://digital.lafayette.edu/collections/purefood
* http://digital.lafayette.edu/collections/lebanesetown
Following the development of these prototypes, I would please like to propose the formation a GIS Interest Group.
Addressing our local institution, there is still much to be done regarding the development and testing of a solution which could be of use outside of our institution. I do not wish for this to be the case. Further, I would be more than open and willing to collaborate with any interested parties in improving upon *any* given codebase for a solution pack (ideally, one to which we could contribute from our work, and, which would permit us some degree of legacy compatibility).
Aside from GeoServer (which is currently also in use within the Omeka project Neatline), we have integrated many of the modules developed and maintained by within the (non-Islandora) Drupal community. Integral to our module functionality for Islandora is a GeoServer RESTful consumer which I've developed using Guzzle. I plan on finishing some test suites for this (drastically minimal) tool, and returning this to the Drupal community soon.
Please let me know of the state of any and all efforts related to the management of Shapefiles within the community. I would also be happy to supply technical details regarding how we serve Shapefile content managed within Object Datastreams.
Thank you very much, and have a pleasant day.
Sincerely,
James Griffin
For those who celebrated, I hope that your Thanksgiving holiday was a pleasant one.
Throughout the course of the summer, two projects scoped for the Lafayette College Libraries required that we implement a local solution using Web Mapping and Web Feature Services integrated with our Islandora ecosystem. Fortunately, the open source project GeoServer served our purposes, and our development efforts for these projects are well underway:
* http://digital.lafayette.edu/collections/purefood
* http://digital.lafayette.edu/collections/lebanesetown
Following the development of these prototypes, I would please like to propose the formation a GIS Interest Group.
Addressing our local institution, there is still much to be done regarding the development and testing of a solution which could be of use outside of our institution. I do not wish for this to be the case. Further, I would be more than open and willing to collaborate with any interested parties in improving upon *any* given codebase for a solution pack (ideally, one to which we could contribute from our work, and, which would permit us some degree of legacy compatibility).
Aside from GeoServer (which is currently also in use within the Omeka project Neatline), we have integrated many of the modules developed and maintained by within the (non-Islandora) Drupal community. Integral to our module functionality for Islandora is a GeoServer RESTful consumer which I've developed using Guzzle. I plan on finishing some test suites for this (drastically minimal) tool, and returning this to the Drupal community soon.
Please let me know of the state of any and all efforts related to the management of Shapefiles within the community. I would also be happy to supply technical details regarding how we serve Shapefile content managed within Object Datastreams.
Thank you very much, and have a pleasant day.
Sincerely,
James Griffin
Donald Moses
Dec 1, 2014, 10:48:12 PM12/1/14
to isla...@googlegroups.com
Hi James:
Any GIS content I've stored in an
Islandora repository has been static. Something I experimented with (don't know what
would
happen with D7/Islandora7 and my approach now would be different) was to relate a data object with two content models
- the basic image cmodel (which provides a preview jpg/tn of the map
layer) and the gis cmodel (which simply stores a zipped binary and the
original ARC metadata). This also provided a View tab and a Download tab
for the user.
This is an older Islandora/Drupal 6 install.
Below is the list of datastreams from a sample object. I didn't have any automated workflows for this ... all done manually.
Something I've looked at briefly and wondered if there was an opportunity to mash it up with Islandora was the Drupal Cartaro profile [1].
I'd be interested to learn more about how you integrate Islandora with GeoServer.
Thanks,
Donald
[1] http://cartaro.org/
Lingling Jiang
Dec 2, 2014, 8:45:37 AM12/2/14
to isla...@googlegroups.com
Hi James,
+1 for GIS Interest Group.
I have some basic knowledge about GIS but haven't have chance to dive in or do anything with Islandora. But I am very interested in what you have done to integrate different parts including islandora.
All the best,
Lingling Jiang
James Griffin III
Dec 8, 2014, 12:39:19 PM12/8/14
to isla...@googlegroups.com, Angel Nieves (anieves), Peter MacDonald, Greg Lord
Hello Everyone,
Thank you for the great degree of interest.
Currently, our existing implementation of Islandora GIS integration lies within several stages. I'll speak now more directly to the technologies involved in the Islandora ecosystem.
* Fedora Commons
Throughout the summer, we initially sought to involve Islandora community members while structuring the proposed Content Models implemented for the management of Shapefiles. The following documentation outlines our progress:
(Logical Data Models)
https://www.lucidchart.com/documents/view/bd3f98c5-7b6e-4f30-82cc-42e159c8d5b5
(The Ingestion Sequence)
https://www.lucidchart.com/documents/view/d6bf440e-1bb9-41ac-b2ea-eee20f94fdd1
(The Dependency Network Diagram)
https://www.lucidchart.com/documents/view/495053ec-410d-4173-8770-bba8902d1bfd
Addressing, the physical data models, I will refer to our existing implementation:
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/xml/islandora_sp_shapefile_cmodel.xml
Please note that I did attempt to implement the Object-Oriented functionality for derivative-generation found within Islandora 6 Content Model implementation:
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/libraries/ShapefileProcessor.php
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/libraries/FgdcProcessor.php
* GeoServer
GeoServer is a Java web application which we've successfully deployed into our staging environment for testing. The application itself it not at all needed for directly serving Datastream content into a JavaScript geovisualization widget (we're currently using OpenLayers; I shall outline this in a greater degree of detail within the context of discussing Drupal integration).
This was my primary reasoning underlying why two Modules were structured for our local solution pack: islandora_gis and islandora_gis_geoserver. islandora_gis simply serves either KML or GeoJSON Datastream content into OpenLayers.
GeoServer requires an additional component which I've also developed and tested, that being the GeoServer RESTful consumer. This is essentially a client which permits one to map either a Shapefile (managed within a Datastream) or a GeoTIFF (managed within an Islandora Large Image Object), and to create the related resource within GeoServer. The results yielded are WMS and WFS resources which can be integrated into OpenLayers as a "map layer". Further, the ability to submit Shapefiles into GeoServer to be managed as a collection of Features permits one to select, at a granular level, each feature specified within a given Shapefile.
Regarding the FGDC metadata, we simply transform this using the XSL Stylesheet developed and managed by the GeoHydra project. This permits one to map the FGDC Document into a MODS Document. I've drafted some initial MODS ingestion forms which should map to the essential FGDC elements for essential curation tasks.
* Drupal
The geoserver Module is primarily maintained by those managing and maintaining the Cartaro project. Cartaro essentially permits one to manage WFS Features as Drupal Content Nodes; The benefit to this approach is that it permit any given user to create and manage geospatial data from within the Drupal site.
This morning, I've released an early (functional) prototype to this team on Drupal.org:
https://www.drupal.org/node/2389233
I hope that this shall lead to some fruitful exchange, as their existing implementation simply leverages cURL bindings within PHP.
As a consequence to some of the limitations we've found when working with the OpenLayers Module, we've been forced to maintain a local fork of the Module itself:
Three essential features could not be implemented using the existing Drupal/OpenLayers API:
* Styling for individual map layers (e. g. rendering Features with certain colors)
* Overriding functionality for clicking on certain features (we've implemented the ability to generate thumbnails for Islandora Objects, and to link these Objects to individual features on the map)
* Symbolization logic (this has yet to be implemented, but essentially, we require the ability to override the functionality for rendering a given geospatial Feature as a point, geometric primitive, or polygon; We must render, in certain cases, a density plot using values within the attribute-value table for a given feature - this is supported by OpenLayers fully, but again, cannot be configured through any Drupal interface)
There are also additional feature requests and improvements pending, but our current focus is elsewhere until at least late January of 2015.
How this relates to the future of the GIS Interest Group is, I feel, straightforward. Firstly, I would ask of the community:
* Where are the faults within these Content Models? How can we work to yield forth Content Models which address the functional requirements for edge use cases?
* How best shall the integration of external WMS and WFS applications be undertaken (this would abstract the role of GeoServer for any WMS/WFS with the appropriate RESTful endpoints)
* OpenLayers seems to be one of the strongest candidates for a JavaScript visualization widget; Are there any alternatives which might be preferred?
* We default to using GeoJSON (in actuality, I have a preference for TopoJSON; but this will only be supported by OpenLayers 3) - are there any strong use cases for serving KML by default?
* How best to approach the myriad of issues regarding integrating OpenLayers (or, for that matter, any JavaScript visualization widget)?
** My preference here would be to work closely with contributors within the Drupal community, as applying pressure to leverage the abilities of those working with this code case actively seems to me to be more efficient
Please freely comment, as I'm hoping that these key points (as well as those identified by the community) shall be used to next construct our roadmap.
Thank you for your time.
Sincerely,
James
Thank you for the great degree of interest.
Currently, our existing implementation of Islandora GIS integration lies within several stages. I'll speak now more directly to the technologies involved in the Islandora ecosystem.
* Fedora Commons
Throughout the summer, we initially sought to involve Islandora community members while structuring the proposed Content Models implemented for the management of Shapefiles. The following documentation outlines our progress:
(Logical Data Models)
https://www.lucidchart.com/documents/view/bd3f98c5-7b6e-4f30-82cc-42e159c8d5b5
(The Ingestion Sequence)
https://www.lucidchart.com/documents/view/d6bf440e-1bb9-41ac-b2ea-eee20f94fdd1
(The Dependency Network Diagram)
https://www.lucidchart.com/documents/view/495053ec-410d-4173-8770-bba8902d1bfd
Addressing, the physical data models, I will refer to our existing implementation:
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/xml/islandora_sp_shapefile_cmodel.xml
Please note that I did attempt to implement the Object-Oriented functionality for derivative-generation found within Islandora 6 Content Model implementation:
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/libraries/ShapefileProcessor.php
https://github.com/LafayetteCollegeLibraries/islandora_dss_solution_pack_gis/blob/testing/islandora_gis/libraries/FgdcProcessor.php
* GeoServer
GeoServer is a Java web application which we've successfully deployed into our staging environment for testing. The application itself it not at all needed for directly serving Datastream content into a JavaScript geovisualization widget (we're currently using OpenLayers; I shall outline this in a greater degree of detail within the context of discussing Drupal integration).
This was my primary reasoning underlying why two Modules were structured for our local solution pack: islandora_gis and islandora_gis_geoserver. islandora_gis simply serves either KML or GeoJSON Datastream content into OpenLayers.
GeoServer requires an additional component which I've also developed and tested, that being the GeoServer RESTful consumer. This is essentially a client which permits one to map either a Shapefile (managed within a Datastream) or a GeoTIFF (managed within an Islandora Large Image Object), and to create the related resource within GeoServer. The results yielded are WMS and WFS resources which can be integrated into OpenLayers as a "map layer". Further, the ability to submit Shapefiles into GeoServer to be managed as a collection of Features permits one to select, at a granular level, each feature specified within a given Shapefile.
Regarding the FGDC metadata, we simply transform this using the XSL Stylesheet developed and managed by the GeoHydra project. This permits one to map the FGDC Document into a MODS Document. I've drafted some initial MODS ingestion forms which should map to the essential FGDC elements for essential curation tasks.
* Drupal
The geoserver Module is primarily maintained by those managing and maintaining the Cartaro project. Cartaro essentially permits one to manage WFS Features as Drupal Content Nodes; The benefit to this approach is that it permit any given user to create and manage geospatial data from within the Drupal site.
This morning, I've released an early (functional) prototype to this team on Drupal.org:
https://www.drupal.org/node/2389233
I hope that this shall lead to some fruitful exchange, as their existing implementation simply leverages cURL bindings within PHP.
As a consequence to some of the limitations we've found when working with the OpenLayers Module, we've been forced to maintain a local fork of the Module itself:
Three essential features could not be implemented using the existing Drupal/OpenLayers API:
* Styling for individual map layers (e. g. rendering Features with certain colors)
* Overriding functionality for clicking on certain features (we've implemented the ability to generate thumbnails for Islandora Objects, and to link these Objects to individual features on the map)
* Symbolization logic (this has yet to be implemented, but essentially, we require the ability to override the functionality for rendering a given geospatial Feature as a point, geometric primitive, or polygon; We must render, in certain cases, a density plot using values within the attribute-value table for a given feature - this is supported by OpenLayers fully, but again, cannot be configured through any Drupal interface)
There are also additional feature requests and improvements pending, but our current focus is elsewhere until at least late January of 2015.
How this relates to the future of the GIS Interest Group is, I feel, straightforward. Firstly, I would ask of the community:
* Where are the faults within these Content Models? How can we work to yield forth Content Models which address the functional requirements for edge use cases?
* How best shall the integration of external WMS and WFS applications be undertaken (this would abstract the role of GeoServer for any WMS/WFS with the appropriate RESTful endpoints)
* OpenLayers seems to be one of the strongest candidates for a JavaScript visualization widget; Are there any alternatives which might be preferred?
* We default to using GeoJSON (in actuality, I have a preference for TopoJSON; but this will only be supported by OpenLayers 3) - are there any strong use cases for serving KML by default?
* How best to approach the myriad of issues regarding integrating OpenLayers (or, for that matter, any JavaScript visualization widget)?
** My preference here would be to work closely with contributors within the Drupal community, as applying pressure to leverage the abilities of those working with this code case actively seems to me to be more efficient
Please freely comment, as I'm hoping that these key points (as well as those identified by the community) shall be used to next construct our roadmap.
Thank you for your time.
Sincerely,
James
kim pham
Dec 9, 2014, 10:04:38 PM12/9/14
to isla...@googlegroups.com, ani...@hamilton.edu, pmac...@hamilton.edu, gpl...@gmail.com
Definitely interested in being part of a GIS interest group to tackle those questions.
Donald Moses
Dec 10, 2014, 8:34:31 AM12/10/14
to isla...@googlegroups.com, ani...@hamilton.edu, pmac...@hamilton.edu, gpl...@gmail.com
Thank you for sharing this.
I think it would help me if I can get the modules and the GeoServer running on my local VM and test it with some sample data.
Re: the content models
* If all of the contents of the shapefile zip bundle are unzipped into islandora_shapefile_obj_cmodel objects is there a need to store the original ZIP bundle as the OBJ in islandora_shapefile_cmodel? Is it a convenience for sharing the original data package as a download option?
* I know there has been discussion about storing some XML inline (eg. the KML ds could be stored as Managed?) as it could potentially make the FOXML very large, especially if the ds are versioned.
I'll think some more on the cmodels.
I'm getting ahead of things, but are you utilizing Views GeoJSON [1] in any way? I'm wondering if that might allow folks to configure the mapping module of their choice? I need to get a better understanding of the GeoServer piece as that has tremendous potential.
Thanks again.
Donald
[1] https://www.drupal.org/project/views_geojson
I think it would help me if I can get the modules and the GeoServer running on my local VM and test it with some sample data.
Re: the content models
* If all of the contents of the shapefile zip bundle are unzipped into islandora_shapefile_obj_cmodel objects is there a need to store the original ZIP bundle as the OBJ in islandora_shapefile_cmodel? Is it a convenience for sharing the original data package as a download option?
* I know there has been discussion about storing some XML inline (eg. the KML ds could be stored as Managed?) as it could potentially make the FOXML very large, especially if the ds are versioned.
I'll think some more on the cmodels.
I'm getting ahead of things, but are you utilizing Views GeoJSON [1] in any way? I'm wondering if that might allow folks to configure the mapping module of their choice? I need to get a better understanding of the GeoServer piece as that has tremendous potential.
Thanks again.
Donald
[1] https://www.drupal.org/project/views_geojson
James Griffin III
Dec 12, 2014, 9:37:03 AM12/12/14
to isla...@googlegroups.com
Dear Donald,
Thank you for sharing this.
You're welcome.
I think it would help me if I can get the modules and the GeoServer running on my local VM and test it with some sample data.
If no one else objects, then I would propose that this be a common milestone that all parties must reach in order to be in a position to undertake the minimal degree of integration testing.
I can begin working to draft some of the documentation outlining our approach with deploying GeoServer. Please note that there is an issue (neither directly related to the base installation of Islandora nor Drupal itself) which inhibits one from "seeding" the GeoWebCache component of the GeoServer application (this ensures optimal performance). Attempting to seed the cache for geospatial data sets can easily exceed the maximum inode level on a given server.
If all of the contents of the shapefile zip bundle are unzipped into islandora_shapefile_obj_cmodel objects is there a need to store the original ZIP bundle as the OBJ in islandora_shapefile_cmodel? Is it a convenience for sharing the original data package as a download option?
Precisely. An alternative approach would be to simply provide a dynamically-compressed ZIP bundle.
I don't believe that we've reached a point where we have ingested any instances of islandora_shapefile_obj_cmodel Objects.
* I know there has been discussion about storing some XML inline (eg. the KML ds could be stored as Managed?) as it could potentially make the FOXML very large, especially if the ds are versioned.
This was an approach taken with the assumption that individuals were indexing their Objects using a Fedora Generic Search XSL Stylesheet which exclusively indexes inline XML datastreams. Obviously, the approach which you've outlined is superior to this, but I thought that I recalled that the Generic Search XSLT, by default, only selected inline XML datastreams for updates to Solr.
I'm getting ahead of things, but are you utilizing Views GeoJSON [1] in any way? I'm wondering if that might allow folks to configure the mapping module of their choice? I need to get a better understanding of the GeoServer piece as that has tremendous potential.
Unfortunately, we are not using this at the moment. It is certainly desirable, but at this point I'm much more concerned with lower-level architectural concerns for managing JSON Objects (I would much prefer to have JSON datastream content managed by MongoDB than locally within Fedora Commons; I'm uncertain as to whether or not a URL should be used with a redirection datastream, or, if something more complex which leverages some of the features of Fedora 4 should be used).
Thank you very much for the feedback, and I'll begin moving forward with the necessary documentation and specifications on a GitHub repository.
Sincerely,
James
James Griffin III
Dec 16, 2014, 2:54:14 PM12/16/14
to isla...@googlegroups.com, ani...@hamilton.edu, pmac...@hamilton.edu, gpl...@gmail.com
Dear Donald et. al.,
I've created a repository for the GIS Interest Group in the earlier stages of development. Please see the following:
https://github.com/LafayetteCollegeLibraries/Islandora-GIS-Interest-Group
For those still interested in the group, please freely fork and issue forth pull requests with your contact information. Should you include your contact information for GitHub, I can add you as a collaborator for the repository.
Following this message, I can look to begin outlining the installation process of the GeoServer web application within the wiki on the GitHub repository. This is essential for initial testing with our Islandora Modules.
Also, please note that I shall be relying upon the wiki for most technical documentation:
https://github.com/LafayetteCollegeLibraries/Islandora-GIS-Interest-Group/wiki
Further, I would like to please issue forth a call for user stories. These can be submitted to the GitHub repository either by directly editing the README.md, or, by posting a message to the group.
Thank you for your time and patience everyone.
I've created a repository for the GIS Interest Group in the earlier stages of development. Please see the following:
https://github.com/LafayetteCollegeLibraries/Islandora-GIS-Interest-Group
For those still interested in the group, please freely fork and issue forth pull requests with your contact information. Should you include your contact information for GitHub, I can add you as a collaborator for the repository.
Following this message, I can look to begin outlining the installation process of the GeoServer web application within the wiki on the GitHub repository. This is essential for initial testing with our Islandora Modules.
Also, please note that I shall be relying upon the wiki for most technical documentation:
https://github.com/LafayetteCollegeLibraries/Islandora-GIS-Interest-Group/wiki
Further, I would like to please issue forth a call for user stories. These can be submitted to the GitHub repository either by directly editing the README.md, or, by posting a message to the group.
Thank you for your time and patience everyone.
Sincerely,
James
On Wednesday, December 10, 2014 8:34:31 AM UTC-5, Donald Moses wrote:
James Griffin III
Dec 23, 2014, 3:43:32 PM12/23/14
to isla...@googlegroups.com, ani...@hamilton.edu, pmac...@hamilton.edu, gpl...@gmail.com
Hello Everyone,
I've just updated the wiki on the GitHub repository in order to reflect some basic information regarding our deployment of GeoServer, and its integration with our Islandora ecosystem:
I would please direct all interested parties to the following resources:
While the former page outlines the challenges which were faced in maintaining the existing deployment of GeoServer for our staging environment, the latter outlines our current approach to both managing geospatial materials within Fedora Commons, as well as how we implemented integration both for the OpenLayers and GeoServer Drupal Modules.
Thank you for your time, and the best holiday wishes to you and your own.
Sincerely,
James Griffin
James Griffin III
Jan 5, 2015, 12:26:27 PM1/5/15
to isla...@googlegroups.com, ani...@hamilton.edu, pmac...@hamilton.edu, gpl...@gmail.com
Hello Everyone,
A happy new year to all. In order to proceed with the progress of the Islandora GIS projects within the Lafayette College Libraries, I've been working to develop two Vagrant environments for the purposes of supporting development. As Information Technology Services currently supports Red Hat Enterprise Linux 6 (RHEL) as the preferred operating system and Puppet within our production, staging, and development environments, a VirtualBox image for CentOS 6.4 x86_64 (with, of course, Puppet for provisioning) has been used to implement these solutions.
Currently, they are under active development within the following repositories:
* https://github.com/jrgriffiniii/vagrant-geoserver
* https://github.com/jrgriffiniii/vagrant-cartaro
The former is an environment for GeoServer for a CentOS 6.4 x86_64 environment. It appeared to have been stable when I undertook testing from my own Fedora 19 environment (but I have yet to undertake testing within either Ubuntu or Debian).
Cartaro is still breaking, and under active development (issues arise with the provisioning of PostGIS and the OpenGDAL library). I aim to work to bring these projects to a certain degree of stability, and would greatly appreciate any feedback regarding their current state.
Thank you, and have a pleasant day.
Sincerely,
James
A happy new year to all. In order to proceed with the progress of the Islandora GIS projects within the Lafayette College Libraries, I've been working to develop two Vagrant environments for the purposes of supporting development. As Information Technology Services currently supports Red Hat Enterprise Linux 6 (RHEL) as the preferred operating system and Puppet within our production, staging, and development environments, a VirtualBox image for CentOS 6.4 x86_64 (with, of course, Puppet for provisioning) has been used to implement these solutions.
Currently, they are under active development within the following repositories:
* https://github.com/jrgriffiniii/vagrant-geoserver
* https://github.com/jrgriffiniii/vagrant-cartaro
The former is an environment for GeoServer for a CentOS 6.4 x86_64 environment. It appeared to have been stable when I undertook testing from my own Fedora 19 environment (but I have yet to undertake testing within either Ubuntu or Debian).
Cartaro is still breaking, and under active development (issues arise with the provisioning of PostGIS and the OpenGDAL library). I aim to work to bring these projects to a certain degree of stability, and would greatly appreciate any feedback regarding their current state.
Thank you, and have a pleasant day.
Sincerely,
James
George Duimovich
May 16, 2017, 2:24:06 PM5/16/17
to islandora
Hello all,
Regarding GIS integrations - any updates from anyone on examples, progress towards supporting GIS content and the use of FGDC / ISO 19115 metadata profile / content type?
George
Carleton University
Regarding GIS integrations - any updates from anyone on examples, progress towards supporting GIS content and the use of FGDC / ISO 19115 metadata profile / content type?
George
Carleton University
Reply all
Reply to author
Forward
0 new messages