DSTK site under new management

[Pete Warden]

Here's a blog post explaining the changes I've made recently, with the text included below.

http://petewarden.com/2015/09/27/openheatmap-and-datasciencetoolkit-under-new-management/

Thanks to everyone who's helped on this project, I'm happy to answer questions here or in the comments.

Pete

-----------------

I’ve been running OpenHeatMap and the Data Science Toolkit for quite a few years now, but a few months ago I realized I wasn’t able to keep maintaining them. I know a lot of people out there are still using them, so I looked around for a partner I could transfer the ownership to. After some discussions, I arranged a deal with the myheatmap.com team to transfer the sites to them, for no charge, in return for their agreement to keep supporting the existing community. For the last few weeks they’ve been handling the servers, support, and maintenance, and I’m very glad they were able to step in. The goal is to keep the existing free services supported, but give them the ability to expand in a more commercial direction too, so that the site becomes more self-sustaining. All OpenHeatMap support requests should now go to he...@openheatmap.com, which they administer.

The code behind DataScienceToolkit.org is all open-source on github, so that will continue to be available, but the DSTK site itself has an uncertain future. I’ve always tried to keep it open to anyone who wants to experiment with the APIs, but over the last year its come under denial-of-service level usage levels from a wide range of IP addresses. I spent some time learning firewall rules and attempting to block the problematic calls, but I wasn’t able to keep the levels low enough to keep the site consistently up. Since OpenHeatMap relies on the site as its geocoder, that meant the uploading there was also often unreliable. I came to the sad conclusion I didn’t have enough time to do the overhauling I’d need to deal with the problems, which is why I handed everything over to a team who can put in more time. The most common use of the DSTK was for geocoding US address, and with the Census Bureau now providing their own free API, that side of it became less essential too. The hosting of the large VMs unfortunately got lost when I shut down the jetpac.com site, so I’m afraid I don’t have those available any more.

Both of the sites were failed startup ideas that took on a life of their own, even though I was never able to make them commercial ventures. I’m hopeful that a fresh team with new ideas will be able to provide a better service to everyone who uses them. I’m grateful to everyone who’s been in touch over the years, I kept supporting the site for so long because I saw the amazing projects you were all using them for. My deep thanks go to the community that formed around the sites.

[Sourabh Antani]

Thank You Pete. 

Is anyone aware of another free/opensource service that can provide global geo-coding? 

Thanks

--
You received this message because you are subscribed to the Google Groups "dstk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dstk-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Patrick]

For an alternate way of doing open geocoding, I might suggest checking out OpenStreetMap nominatim. It's not always great, but it's a start.

OSGeo-Live may or may not also include some tools that fit the bill.

[Sourabh Antani]

Thanks a lot

[Noah Brod]

Thanks for all of your hard work on this Peter.  I would argue that DSTK still holds a special use value in that it seems to be the least restrictive free geocoding service out there (unlimited calls and batches).

 Is there a bittorrent link to the VM still floating around that works or any alternative host for the VM? 

[David Todd]

Somebody posted a link to the last dstk box image in a Github issue here:

https://github.com/petewarden/dstk/issues/57

The image was uploaded to somebody's Google Drive:

https://docs.google.com/uc?id=0Bw4DnUXiJwGPZUNNZnVLeG0yODA&export=download

I can confirm that the dstk_0.5.1.box downloaded from above does indeed work.  I've been using it for batch geocoding millions of addresses that can't go outside the network they are on.  It's doing about 4 million a day.

I believe ESRI has an ArcGIS geocoding service that isn't currently rate limited:

https://developers.arcgis.com/rest/geocode/api-reference/geocoding-find.htm

[cole.b...@gmail.com]

Hi all,

If anyone is interested I have written the geocoding software into a Docker image that is useful for free and offline geocoding: https://github.com/cole-brokamp/DeGAUSS

Thanks,
Cole