GeoMesa

[Corvince]

Dear all,

I plan on using an ABM for a project, where we want to include and work with spatial data. Since I have read this feature was planned for mesa for a long time I am curious to what the current state of this is and if there are already any specific plans for this. 

I have limited expertise in combining Python and GIS, but have used this combination to generate some visualizations. I think Shapely is an excellent python package to work with arbitrary shapes in Python. "Work" in the sense of calculating distances, overlaps and buffers around shapes. However, Shapely itself is not concerned about data formats or any attributes of the shapes, only the geometry. 

But at least it is compatible with the geoJSON format for geometries, which I would recommend as a data standard anyway. 

The exact interactions between (still most commonly used) shape files, geoJSON and shapely needs to be elaborated and of course how this will tie into mesa. 

Conceptually I would say all this belongs to the space of an ABM (or mesa therefore), but I am curious to what functions would be needed. As mentioned above, intersections and distances immediately come to mind, but the definition of "neighbors" is much more uncertain. 

We would also need a new visualization type and decide if we want to allow combinations of spatial data and gridded data (what about raster data?)

I hope this thread can start a discussion about the way forward!

Best

Corvin

[David Masad]

Hi Corvin,

Yes, GeoMesa has been on the wishlist for a while, but as far as I know, nobody's really started to implement it. The main reason that we've discussed GeoMesa as being separate from core Mesa is because basically all spatial Python libraries rely on several C libraries which can be difficult to install and set up.

I think you're right that vector and raster data need to be largely handled separately. As far as I know, the main commonalities they both share are dealing with coordinate systems, and potentially projection issues. On the other hand, raster data is effectively a numeric grid, while vector data is continuous.

At the most basic level, I think vector shapes would be their own Mesa class, stored within a GeoSpatial space class (possibly a child class of ContinuousSpace?). We need to allow agents to check whether they're contained by a vector polygon (e.g. city boundaries), as well as to be *on* a point along a line (e.g. on a street). That part, at least, probably just requires a thin wrapper around Shapely (or a similar library).

By default, distance could just be in units of fractions of degrees, but it would be nice (and not too hard) to convert distances to meters based on where the coordinates are. Neighbors could be all objects within a certain distance, or all objects actually touching a given shape. I think Shapely has built-in ways to check for both.

In terms of visualization, we need to draw vectors in different colors / thicknesses, and to draw point agents (at least initially) in the same coordinate system. This is actually somewhere where d3 could probably be very useful.

Anyway, those are some thoughts off the top of my head. I think a helpful place to start would be to figure out how exactly mesa kits should work -- does it install into Mesa's existing namespace, how do we develop it and keep it separate, etc?

--
Project repos: github.com/projectmesa
---
You received this message because you are subscribed to the Google Groups "ProjectMesa" group.
To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa...@googlegroups.com.
To post to this group, send email to proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/8ac2ab8f-8304-4f28-a388-39845ce2b707%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jacqueline Kazil]

hey hey! 

I think David covered the basics. 

I think the first place to start would be to hack together a model using GIS, then to work backwards and create an abstraction of the pieces that can be reused.  At one point I was thinking leaflet, but I haven't researched it in awhile.

If you need to get feedback on a design, if you put something together, I can help find the experts to review and see if there are isn't a better approach or tooling to get more eyes on it. 

If this is something that you are looking to take on, there are other folks who are interested in as well --- we can talk about procedure on boarding. 

Let me know. 

-Jackie

P.S. I would be happy if you built this out. <3 ... no pressure. 

To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa+unsubscribe@googlegroups.com.

To post to this group, send email to proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/8ac2ab8f-8304-4f28-a388-39845ce2b707%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Project repos: github.com/projectmesa
---
You received this message because you are subscribed to the Google Groups "ProjectMesa" group.

To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa+unsubscribe@googlegroups.com.

To post to this group, send email to proje...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/CANyFgCBOBfSCoPTxXFjmjvZfKssDQ0e4990-egc6xx40yLFVaA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Jacqueline Kazil | @jackiekazil 

[Corvince]

Dear all,

thanks for your replies.

I have taken a few baby steps into this and I think most things will be more straight-forward than I thought.

Shapely really seems to be a good choice as a library for most of the actual. On the one hand we can use it to create really simple shapes (Points being the simplest) 

It also works nicely with geoJSON files. This means we can shift a lot of the difficult/nerve-wrecking part of GIS work (coordinate systems and transformations w.r.t. reference ellipsoid) outside of mesa and just say "look, however you do this, bring your date/shape files into the right format and we will do the rest". No need to re-invent the wheel here.

From there on things like nearest neighbor, intersections, distance, etc. will be straight forward using shapely.

Also things like "put agent randomly within this shape/within these coordinates" should be manageable.

But I quickly realized that GeoMesa will again raise the question of how to handle Patches. If we think of areas or roads these are neither part of the grid nor agents. Maybe this is a good opportunity to introduce Patches as static agents? I remember Patches came up in a different situation but I am not sure if there was a plan on how to proceed. 

Regarding the (non-)integration with mesa: I see and agree that because of the dependencies of difficult to install C libraries it should be somehow separate from base mesa. However since my knowledge of python packaging and best practices within the FOSS community is limited, I have no idea what will be the best way to go. For now I just work on a mesa branch, but am happy to migrate (and then publish) to a "mesa geo kit" repo once its structure is defined. 

To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa...@googlegroups.com.

To post to this group, send email to proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/8ac2ab8f-8304-4f28-a388-39845ce2b707%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Project repos: github.com/projectmesa
---
You received this message because you are subscribed to the Google Groups "ProjectMesa" group.

To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa...@googlegroups.com.

To post to this group, send email to proje...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/CANyFgCBOBfSCoPTxXFjmjvZfKssDQ0e4990-egc6xx40yLFVaA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Jacqueline Kazil | @jackiekazil 

[Corvince]

Hi there,

a quick update: Check out this GeoSchelling example with European NUTS regions :)

Pretty nice, huh? 

Downside is that it is mostly a proof of concept. As my time resources will be limited in the next months I think GeoMesa will take some more time, but stay tuned. At least I want to use it productively in Autumn, so there should be some progress. 

Upside is that I had to change surprisingly little from the Schelling example to get this to work, of course with the help of a new GeoSpace module, which does most of the work. 

This is how it works at the moment:

GeoMesa introduces patches, which are like agents, but do not act on themselves, i.e. no decision-making. They merely represent the landscape in which agents act. GeoMesa also assumes that every agent and every patch has a shape attribute, which is a shapely.geometry object. In the simplest case this would be a point. The shape also contains the coordinates/pos of an agent.

Functionality is provided that takes a GeoJSON object (and therefore potentially any spatial vector data) and turns it into a patch or agent (the NUTS regions in the example above). 

The GeoSpace module keeps a list of all patches and all agents on the "grid" and provides functions such as check for intersections between shapes, get all the neighboring shapes, etc. 

For the Schelling example, a patch is created for every NUTS region. Then, for every patch an agent is created with probability "density", which can be majority or minority with probability "minority_pc", just like in the original example. The shape of the agent will be the same as the patch. For my simplified example I then just count the number of similar and different neighbors and move if there are more different neighbors. Here is the gist of the example. The other modules are still too messy to put them online, sorry ;)

Now the two biggest issues so far are a 1) place to work on this and 2) the visualization. 

1) is related to where to publish mesa extensions/kits. This is something the mesa core team has to decide. I don't think there will be too many files so I am not sure if a separate repository would be necessary. I could also imagine putting it into mesa.kits.GeoMesa and have its dependencies as optional dependencies, if that is possible? I know this issue has been raised, I just want to push a bit ;)

2) is possibly related to 1)  and I have not looked into this yet. The above visualization was created with the python package folium which plots on a leaflet map. But this does not tie in with the way mesa works. But as every shape of GeoMesa can be represented as GeoJSON object, a vis module based on D3 or leaflet should be doable rather straight-forward. That is, if one is used to javascript, which I am not. 

There will be also some more specific questions on how to do some things mesa-style, but I think I will leave it like this for now

Best

Corvin