[ANN] github.com/JuliaGeo

324 views
Skip to first unread message

Sean Garborg

unread,
Jan 20, 2015, 11:28:58 AM1/20/15
to juli...@googlegroups.com
I just set it up and transferred Geodesy.jl over! Come on over and jump in.

Here are a couple of topics to discuss

1) Roadmap -- In the '[ANN] Geodesy.jl' thread, Yee Sian mentioned a few ideas for an organization roadmap: "I am using shapely/fiona/geopandas in python for now, and am interested to see julia alternatives. I think it will be great if we can have a JuliaGeo organization, with wrappers for GDAL (and OGR), GEOS, and NetCDF (similar to the toblerity project) as a first order priority." And Will mentioned updating GDAL.jl before moving it over. 

2) Contributors -- The group is still small. Once the organization has a few packages, people will be drawn in to the ecosystem through each individual package. Another announcement (about both julia-geo and JuliaGeo) on julia-users would be good, but I feel like the timing for that is once there are a couple more packages.

3) Package Guidelines -- This is something that will emerge as more contributors get involved, but it would be good to start out with something. Here's a go at a couple basics:
  • Travis enabled for PRs and commits, with Coveralls (test coverage tool).
  • Commitment to increasing documentation as the ecosystem coalesces.
  • Handling binary dependencies with BinDeps where it's an option.
4)  Down the road -- following in the footsteps of other Julia orgs and putting up a page that summarizes the ecosystem, and of course a customized version of the Julia logo specific to JuliaGeo ;)

Thoughts?

Excited to start building things with you all.

Fabian Gans

unread,
Jan 20, 2015, 11:45:06 AM1/20/15
to juli...@googlegroups.com
Thanks a lot for creating the group,

I would also move the NetCDF package over to the the julia-geo organization. In the last few weeks I have been re-writing many parts of the package (in branch major), so I would like to postpone moving it until I have merged the changes into master. I will let you know as soon as it is that far.

Another question: I just looked at the JuliaGeo github page, but couldn't do very much on it besides looking at the list of packages. Is there any forum or wiki attached to the organization where one can post or do we continue discussion here on the mailing list?


Yee Sian Ng

unread,
Jan 20, 2015, 2:18:42 PM1/20/15
to juli...@googlegroups.com
Just thought I'd mention first:
  1. there're potentially ambiguity issues with the JuliaGeo name (to mean any of Geo[metry], Geo[graphy], Geo[spatial], Geo[science], etc). FWIW, OSGeo stands for "Open Source Geospatial Foundation", so the natural fallback option is to rename it "JuliaGeospatial" (which is unfortunately an adjective, and not a noun). 
  2. there are packages for Geometry (such as Geometry2D, GeometricalPredicates, VoronoiDelaunay and BoundingBoxes), which might form their own Julia Organization (see the discussion here, here, here and here), or go into base julia
  3. JuliaGraphics seems to be facing similar issues getting traction, and everyone seems to be waiting and watching to see what will happen
For that reason, I'd suggest just maintaining the bindings to the corresponding libraries (in OSGeo), before we consider moving towards higher-level interfaces for working with julian types (depending on how it goes with the geometry packages)?

@Fabian, I've had a look at libgeos, and it seems like a sufficiently self-contained library, with a nice C interface. To my knowledge, shapely uses it (here and here), and I've been trying (without success) to use Clang.jl to generate the wrappers for it. If it's not too much trouble, can you generate a first pass (like you did with GDAL), and I'll have a look at it as and when I'm free to wrap it up?

Sean Garborg

unread,
Jan 20, 2015, 3:31:05 PM1/20/15
to juli...@googlegroups.com
@fabian, that sounds great.

Re: wikis, github organizations don't get a wiki, but I've seen a few approaches:
JuliaOpt -
JuliaStats -
BioJulia - 
Additional option -- add a repo like JuliaStats, but instead of only having conversations in issue threads, discussion in issues leads happens in parallel with building out either the repo body or repo wiki.

@yeesian Those are all good points.
1.)  If we rename the organization all links will be redirected. So the one reason we'd have to act now is if we think another group should take the short name `JuliaGeo` and we don't want any redirection failures from old urls in local repos or out-of-date METADTA. Thoughts?

3.) We may have to change predicates between Base.Graphics, those competing Geometry packages, etc., at some point. I don't think that needs to paralyze geospatial package development, but the route you suggested sounds best -- making the wrappers a priority (and trying / discussing / giving input into the best eventual Julia graphics and geometry predicates when we get there).

J Luis

unread,
Jan 20, 2015, 7:12:56 PM1/20/15
to juli...@googlegroups.com
Hi,

Just to tell that I have also an half backed wrapper to GMT. However, it's not yet fully working because there are still issues on the GMT API side that we hope to solve this spring

https://github.com/joa-quim/GMT.jl

Fabian Gans

unread,
Jan 21, 2015, 3:10:05 PM1/21/15
to juli...@googlegroups.com
@Yee,

I did a first try on getting starting the Clang wrappers again, it did not work immediately since we are few julia and Clang versions later than when I last used it, but I am optimistic to have an initial wrapper soon. I will let you know when I have something.

Fabian

J Luis

unread,
Jan 21, 2015, 4:26:42 PM1/21/15
to juli...@googlegroups.com
@Yee

I have bad news from the GEOS side. First, it has tons of headers and I don't know which are really needed. But the worst is that I tried to convert one as example and got Clang errors because those headers have C++ (a namespace {} declaration that errors in Clang).

Anyway the sequence of commands I use to make wrappers is like

using Clang.wrap_c
context = wrap_c.init(; header_library=x->"geos", clang_diagnostics=true, common_file="geos_h.jl");
wrap_c.wrap_c_headers(context, ["/Users/j/programs/geos-3.4.2/include/geos.h"])

The *_h.jl versions normally contain the constant, enums, struct declarations while the other hold ccall calls

Joaquim

Yeesian Ng

unread,
Jan 21, 2015, 4:57:37 PM1/21/15
to juli...@googlegroups.com
Hey thanks for this! I forgot to add: they suggest linking only to the geos_c.h header file (mentioned here), so it should be alot simpler than wrapping GDAL. I think this is how it's done in python (through shapely) too.

Yeesian Ng

unread,
Jan 21, 2015, 5:20:35 PM1/21/15
to juli...@googlegroups.com
Here's what I got by following what you did:

julia> context = wrap_c.init(; header_library=x->"geos", clang_diagnostics=true, common_file="geos_h.jl");
julia
> wrap_c.wrap_c_headers(context, ["/Users/yeesian/Downloads/geos-3.4.2/capi/geos_c.h"])
WARNING
: wrap_c_headers: deprecated
WRAPPING HEADER
: /Users/yeesian/Downloads/geos-3.4.2/capi/geos_c.h
WARNING
: Not wrapping InclusionDirective   stddef.h
WARNING
: Not wrapping InclusionDirective   geos/export.h
WARNING
: Not wrapping MacroInstantiation   GEOS_DLL
...
WARNING
: Not wrapping MacroInstantiation   GEOS_DLL
WARNING
: Not wrapping MacroInstantiation   GEOS_DLL
ERROR
: Index out of range or empty list
 
in getindex at /Users/yeesian/.julia/v0.3/Clang/src/cindex.jl:168
 
in wrap at /Users/yeesian/.julia/v0.3/Clang/src/wrap_c.jl:463
 
in wrap_header at /Users/yeesian/.julia/v0.3/Clang/src/wrap_c.jl:630
 
in run at /Users/yeesian/.julia/v0.3/Clang/src/wrap_c.jl:747
 
in wrap_c_headers at /Users/yeesian/.julia/v0.3/Clang/src/wrap_c.jl:777

J Luis

unread,
Jan 21, 2015, 6:14:19 PM1/21/15
to juli...@googlegroups.com
Hmm, I get a slightly different error (probably an older Clang.jl) but basically the same result. Sorry, don't more on this subject


IWARNING: convert{T}(p::Type{Ptr{T}},a::Array) is deprecated, use convert(p,pointer(a)) instead.

 in tokenize at /Users/j/.julia/v0.3/Clang/src/cindex/base.jl:276

 in wrap at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:439

 in wrap_header at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:505

 in wrap_c_headers at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:562

WARNING: Skipping empty struct: "GEOSGeom_t"

WARNING: Skipping empty struct: "GEOSPrepGeom_t"

WARNING: Skipping empty struct: "GEOSCoordSeq_t"

WARNING: Skipping empty struct: "GEOSSTRtree_t"

WARNING: Skipping empty struct: "GEOSBufParams_t"

WARNING: Skipping empty struct: "GEOSContextHandle_HS"

ERROR: Index out of range or empty list

 in getindex at /Users/j/.julia/v0.3/Clang/src/cindex.jl:166

 in wrap at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:371

 in wrap_header at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:505

 in wrap_c_headers at /Users/j/.julia/v0.3/Clang/src/wrap_c.jl:562

Martijn Visser

unread,
Jan 22, 2015, 7:39:07 PM1/22/15
to juli...@googlegroups.com
On Tuesday, January 20, 2015 at 5:28:58 PM UTC+1, Sean Garborg wrote:
4)  Down the road -- following in the footsteps of other Julia orgs and putting up a page that summarizes the ecosystem, and of course a customized version of the Julia logo specific to JuliaGeo ;)

Hope you don't mind that I started at point 4 (Down the road). What do you think about this logo? I still need to polish it up but your thoughts/feedback/other ideas are welcome.
Also registered JuliaGeo.org already, I can transfer it/point it somewhere when there is a page.

 
juliageo-globe-small.svg

Sean Garborg

unread,
Jan 23, 2015, 10:48:23 AM1/23/15
to juli...@googlegroups.com
Martijn, I really dig it!: github.com/JuliaGeo. I couldn't resist putting it up right away, but of course that's not my decision to make, so let's be open to other design ideas or ideas to tweak your design.

Thanks for taking care of JuliaGeo.org, too! Perhaps we could put a page up before too long -- say, once a couple wrappers are up and the roadmap is fleshed out a bit more and spelled out (as a work in progress), it could be useful to make the roadmap and discussions (and up-for-grabs projects and features) more discoverable.

Sean Garborg

unread,
Jan 27, 2015, 10:21:47 AM1/27/15
to juli...@googlegroups.com
As announced in another thread, looks like we have another package on the way in the form of GeoJSON.jl!

Returning to topic 3) at the top of the thread... There was a question in the other thread whether there would be consensus for moving GeoJSON under the JuliaGeo umbrella. GeoJSON seems common enough to that it wouldn't seem out of place to me, but any thoughts, GeoJSON-specific or general about the boundaries of scope of packages JuliaGeo should house?

And there are the questions above about quality criteria for moving a package into JuliaGeo. There's a conflict between encouraging submissions on one end and keeping packages under the org reliable on the other. Between putting lots of work on the author on one end and letting the author submit an incomplete package and leave all the tedious parts of development to the org's most active maintainers on the other.

Some orgs like a package to be reasonably mature before it gets moved over -- it seems that aspect of quality makes more sense as a criteria for inclusion among an existing set of mature packages, so I wouldn't argue for that here.

Promoting testing and documentation seems obvious. Once existing packages are more developed, wanting new packages' APIs and types to be consistent with the ecosystem (where reasonable) might be another criterion.

I did pretty horribly on the 'doc' criterion when getting Geodesy.jl ready for release. Maybe I considered the good test coverage and the simplicity of the code as a bit of a substitute, but mostly I was anxious to get fixes made there back upstream to OpenStreetMap.jl. Anyway, while I'm fine pushing for authors to include granular test coverage, and I know I've got some doc work to do on Geodesy.jl, I'm still a little hesitant to push too hard on the documentation requirements before Julia 0.4's new documentation system has been tested more broadly in packages. But I'm not sure that's the best stance.

Please weigh in.

Yeesian Ng

unread,
Jan 27, 2015, 4:32:41 PM1/27/15
to juli...@googlegroups.com
Thanks for bringing it up! I hadn't been thinking very carefully through such matters, Following Fabian's original thread, I think anything in mapschool.io is fair game for inclusion in JuliaGeo; and if there's any lingering doubt, ask (on this mailing list)?

I had been looking mostly at the design and organization of packages in Python (because they seem like the language of choice for the GIS developers I've been following), but will be happy to see things from a different perspective (e.g. from people who work with R/databases/etc).

For a start, I'm willing to continue to have a discussion over every new package that comes along (even bike-shedding over names), just to make sure we're converging towards a common objective (as an organization). We can always list some common guidelines later (put things into a wiki, etc), if we find ourselves repeating the same points.

Maybe it's easier to have standards on the "eventual quality standards" of packages in JuliaGeo. I had been implicitly looking out for
    - a clear scope/vision of what the package is trying to achieve, that
    - is relevant to the organization, and 
    - is orthogonal in contribution to existing packages
    - doesn't imply/require more effort* on my part (to get it "up to speed"), than I'm willing to put in

And so the priority (to me) at the moment isn't whether it has tests/docs/etc, but whether it is easy enough for others to follow what's happening. If the code is complex, provide an API. If the API is complex, provide some minimal working examples. If there's more examples/things-that-needs-to-be-explained, then I think it's fair for others to ask you for having tests/docs up before consideration for inclusion in JuliaGeo?

Sean Garborg

unread,
Jan 27, 2015, 5:10:20 PM1/27/15
to juli...@googlegroups.com
I think you're right that thinking through package scope, fit, and design (and seeking inspiration other ecosystems) is the right thing to focus on now. And that consideration for inclusion into JuliaGeo should be a fluid thing (as opposed to checking off a list of requirements).


More context for future cases when we may ask for tests, then I'll get off my horse:

I think having tests and/or examples (great point, even better!) is pretty important for lowering the barrier of entry to other contributors, which is even more important when an author is at a place where they're not up to putting additional effort into a package.

The first thing I do after 
Pkg.add("PkgIHaveNotUsed")
is ALWAYS
Pkg.test("PkgIHaveNotUsed")
followed by checking whether the underlying tests actually hint at correctness (as opposed to just testing that functions don't throw an error and maybe return the right object type).

I guess domain experts may have an easier time checking where a package is and isn't working, but for a generalist like me, it would be strange to start using a package without testing, especially in a young ecosystem where a few existing packages start failing tests almost every night, even on 0.3.

As a user, I'm happy to get a package back in working order, but I need a little guidance, and tests and examples are usually the closest thing to a spec there is.

For example, Ted's OpenStreetMap package was very well documented, but it was the tests and examples that let me start contributing, and the same goes for the contribution I made to GeoJSON this today -- only doable (for me) because you'd already added examples, comments, and especially tests.

Package authors are in the best position by far to write out initial tests and examples, and probably will need to have at least informal tests in place to refactor their package as it takes form (unless they're way smarter than I am), so I wouldn't feel bad about taking testing as a given.

Fabian Gans

unread,
Jan 30, 2015, 7:54:50 AM1/30/15
to juli...@googlegroups.com
Sorry for my long silence, I was busy last week but I finally looked at
the geos_c.h
Initially I got the same error message as Yee, but ignoring typedefs I
could at least wrap the functions in this header.
After that I manually added the enums and any opaque structs as Ptr{Void}.

Just clone this repo: https://github.com/meggart/GEOS_wrapper

I am not familiar with the library, so haven't tested anything. I hope
it helps...

Fabian


Yeesian Ng

unread,
Jan 30, 2015, 2:25:38 PM1/30/15
to juli...@googlegroups.com
Wow, this looks useful, thanks :D I'll see what I can make out of the GEOS library -- after I work on a port of TurfJS for operations on GeoJSON objects (if you're interested, see the effort here) -- I'll do a separate announcement when it's in a working state!

Sean Garborg

unread,
Feb 1, 2015, 11:56:48 AM2/1/15
to juli...@googlegroups.com
Looking forward to trying out Turf.jl!

Sean Garborg

unread,
Feb 1, 2015, 11:59:59 AM2/1/15
to juli...@googlegroups.com
Thanks to Martijn for tweaking the logo (attached for anyone else who needs it) and pointing JuliaGeo.org to the JuliaGeo github org page for now:
"
Glad you like the JuliaGeo logo. I polished it up a bit, see attachment. Now it aligns nicely with the JuliaLang org, and has a transparent background.
For now juliageo.org points to github.com/JuliaGeo, since that also has the mailing list link there.

-Martijn
juliageo.png
juliageo.svg

Yeesian Ng

unread,
Jun 24, 2015, 1:43:41 PM6/24/15
to juli...@googlegroups.com
Hi Fabian (and anyone who might be interested!),

May I enlist your help in generating wrappers for OGR this time? Here is the C API, and a tutorial of how it works in performing map projections.

On Wednesday, 21 January 2015 15:10:05 UTC-5, Fabian Gans wrote:
@Yeesian,

Fabian Gans

unread,
Jun 25, 2015, 4:22:19 PM6/25/15
to juli...@googlegroups.com, ngye...@gmail.com
Hello Yee,

I have a very busy week, so I probably won't look into this before Monday. Just let me know if anyone else was faster in running the wrapper.

Fabian


Reply all
Reply to author
Forward
0 new messages