Experimental app: Share-a-Map (self-hosted OpenStreetMap)

121 views
Skip to first unread message

Dan Krol

unread,
Oct 16, 2022, 9:34:57 PM10/16/22
to Sandstorm-dev
Surprise! I made a map for Sandstorm:

https://apps.sandstorm.io/app/m3ctajcm6nnpce287r0a4t52ackzv7p7mmffrw88nge64fp0m8yh?experimental=true

It's called Share-a-map. (Or maybe TreasureMap? Keeping on the sandy theme. Or whatever, I'm open to ideas.)

Plan your trip

Available now are Boston and the New Hampshire seacoast (where I'm at currently). (Note: searching Boston will be less performant for this demo. And to that end, I recommend only downloading one of the areas per grain for now if you want to search.)

Download the region you want to visit, search for some places, and add them as bookmarks.

For New Hampshire, might I suggest:

* Strawbery Banke Museum (that's not a typo) - The original settlement in Portsmouth
* Kaffee Vonsolln - Where I bring my laptop and listen to 80s music
* Stone Church - An old church turned bar and music venue in Newmarket

Some things to search for in Boston:

* Freedom Trail - a tour, self guided or not, of various historical sites from the revolutionary era
* Trillium Brewing
* Harvard Square (just on the edge of the map)

But this should pretty much have every Point of Interest that OSM has.

Share the map with your trip companion, they can add some of their own destinations.

Once you're done, log in or share the grain with your phone, and export the bookmarks to Organic Maps, OSMAnd, etc.

So how does this work?

The map tiles - Long ago I tried the "usual" way which required postgres. Then one day something called Protomaps showed up on Hacker News. It's exactly what I needed - flat files that your browser downloads and renders. No need for a database at all! So far I use Protomaps' site to generate them. Eventually I'll have to generate it myself from raw OSM data.

Search - Usually OpenStreetMap uses something like Nominatim, which uses Elasticsearch. Another headache for Sandstorm. I found something called gazetteer that turns the raw OSM data into json basically, which I can work with.

The base world map (countries and US states) is using something called geojson, which is just polygons. The data is small enough that I just embedded them in the app.

I barely know this stuff and hacked it together! I bet it could be way better.

Oh, and as a bonus - downloading: Sandstorm powerbox requests don't support range headers so I chopped up the map data files for hosting and re-assemble them in-grain.

Next steps

What's left: So much!
* Data extraction.
    * Search: Gazetteer has quirks and is slow. I couldn't get Chicago or Montreal working yet.
    * Promomaps tiles from raw OSM (as mentioned)
* Search index (I'm literally doing a linear search now through a gzip'd text file, it'll get much better)
* UI improvements
* Rewrite the server in Golang. Speed, smaller app, concurrency for various things.
* Code on Gitlab (clean up my repo first). Start taking bug reports and feature requests from users.
* Hosting
    * Can I do this in perpetuity? Etc.
    * Domain for hosting map data
    * Bandwidth/hosting cost. S3 or equivalent?

What I could use:
* Some feedback - Make it more useful somehow? Shortest path to an MVP? Maybe start with a few select metro areas? Etc.
* Pointers from folks who know
    * OSM
    * Web UI
    * Leaflet (intersection of the above)
    * Search - current plan is to look for golang search index libraries and make something simple but much better than current linear

The future

Self-hosted OSM is a hot topic on Hacker News lately. This has the advantage of being way easier for the user than the full OSM stack. The experience will be more like OrganicMaps. I'm hoping we can post this and convince people that Sandstorm isn't dead, maybe get some more users and sponsors.

In the far future, this seems like a killer app for IPFS. Sandstorm installations sharing map data with each other. Originating server bandwidth wouldn't be a problem (in theory).

Jacob Weisz

unread,
Oct 16, 2022, 10:56:16 PM10/16/22
to sandst...@googlegroups.com
This is super neat! The map downloading feature feels super intuitive to me.

Not sure if I know how best to solve the data source issue, presumably the more you add and the more people use it, the harder serving it will be on your bandwidth, if there's not a public service we can piggyback off of in the right format that will be a struggle. I definitely think map data would be an interesting type of capability to let grains share with other grains, so I can store one copy of some map data on my server and let multiple instances of multiple apps consume it.

Random feedback:

- I would say putting some instruction how to start using the app somewhere in the app would be a good start: I tried several times to click on displayed names before trying to search for them, and they are not clickable unless they're searched.

- It's also very easy to click off of a search, losing both the pin point and the result list, which forces you to re-run the search to get back to where you just were. Being slow isn't too bad in itself, but having to run the search a couple times because you clicked off of it is rough. Any way of making results from searches stick for the rest of the session or until dismissed or something would probably save a fair bit of trouble.

- Not sure how much you can (or should) do to address this, but it's not useful when searching for a location that exists multiple times in the database. Search "Starbucks". It seems like you'll always go to one particular one, but there are many in the database, so there's presumably data being stored that isn't really being used effectively here.

- Sometimes if I click a bookmark, I have to click it twice for it to correctly center the map on the bookmark. Seems to only happen if I change the zoom level a bit before clicking on the bookmark.

- With the bookmarks list, there is no way to show all of your bookmarks on the map at once, and I feel like I might want to see how different bookmarks relate to each other on the map. While you aren't routefinding in here, I also imagine someone might want to reorder them as well, but trying to drag them drags the map instead.

--
  Jacob Weisz

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

David Chizmadia

unread,
Oct 17, 2022, 11:50:47 PM10/17/22
to Sandstorm Development
The first thought that came to mind for the data source issue would be to have a built-in tile sharing feature like BitTorrent (maybe TileTorrent) that could capitalize on network and physical topological locality to optimize both speed and need for a given tile. An interesting possibility here, once there exists Share-a-Map OSM-to-Tile-generator, would be be to have a distributed search for a tile that doesn't currently exist fail over to a tile-generator server for an on-demand creation of the tile.

--Dave Chizmadia

Dan Krol

unread,
Oct 20, 2022, 10:16:17 PM10/20/22
to David Chizmadia, Sandstorm Development
Regarding the Bandwidth issue, I also feel like that level of success would be a good problem to have! As far as bittorrent - I mentioned IPFS in my previous email. Same basic idea. I think there were some (perhaps ambitious) proposals to implement IPFS some day as a Sandstorm facility. But today you can't do BT or IPFS from a Sandstorm grain. Also note that I don't think Protomaps deals in the usual 256x256 OSM tiles. Each whole area is one .pmtiles file that doesn't take an awful lot of disk space. BTW another short term possibility - a CDN?

Regarding various UI things - I did as little as I could to make it viable enough for a demo. I definitely want to do more. But I'll take note and try to fix these specific ones next.

Clicking names on the map - That would be nice, but I'd need a reverse-lookup which is a little harder to implement, both backend and frontend. But I bet I could do it.

Results from searches getting accidentally dismissed too easily - So you'd like the search result marker to stay there until maybe you click a specific [x] next to it, or you search for something different? BTW I forgot to mention I'm using a plugin called leaflet-search which dictates some of that behavior. I've been able to tweak it but I may want to reimplement it from scratch.

Many Starbuckses - I could see about displaying the address in the search result. That's what these apps usually do. Further along I'm hoping I can bias results by distance from the current view.

Centering bookmarks - never noticed, I'll see about that.

Show all bookmarks at once - this was the plan for post-demo. I'll probably have bookmark sets as well that you can turn on and off. Unless each grain should act like a bookmark set. (Only if we offload the data to another grain like you described. Let's call it MapBase.)

So regarding MapBase - can we share files between grains today? Maybe it's easy then. Maybe I'll try it now if it's worth it, just for the heck of it. Problem is, it may be confusing overhead for users.

So maybe it'd be like this: You have a ShareAMap that lets you download maps remotely as is currently. But as an option in the dialogue it says "if you have a MapBase grain you can use those same maps in other ShareAMaps. click here to learn more". So the repeat user is thus encouraged to try out MapBase. The MapBase interface *also* lets you download map areas and generally see what maps you have. But from ShareAMAp, you can download the maps and send them straight to the MapBase, so you never actually have to open the MapBase grain. And then, any new ShareAMap grain, if you choose to connect to the MapBase grain on startup, it instantly imports all of the maps without asking what you want to do specifically (unless they have to be *stored* on the ShareAMap grain, in which case we should ask for space concerns). (interesting question would be ShareaMap handling of deleting maps from MapBase)

And then Datasette could use the same MapBase system.

Though.... I just thought of a problem. My map data bundles the Protomaps files with the search data. I don't think the search data is so generally useful across different apps until and unless I really formalize it. Right now I'm liable to change it to fit my specific needs.

BTW what do you all think of "TreasureMap"? Does it sound too much like a game or toy? I just don't like a generic sounding thing like Share-a-Map. It just came to mind once and stuck as a holdover.

Jacob Weisz

unread,
Oct 20, 2022, 11:40:21 PM10/20/22
to sandst...@googlegroups.com
So our example of implementing custom capabilities between grains is here: https://github.com/zenhack/sandstorm-filesystem

Implemented exactly as is you can share files between grains, but it may be worthwhile to define a specific thing for map data or something so only other grains offering compatible data would be selectable.

--
  Jacob Weisz

Daniel Krol

unread,
Oct 22, 2022, 3:32:33 PM10/22/22
to sandst...@googlegroups.com

Ah. File sharing seems more complicated than I was hoping. Probably not be worth digging into for a little while since I have enough hurdles ahead of me already before an MVP.

I fixed a couple of the reported bugs and made a couple other changes. I didn't catch the centering one, thanks for noticing!

From my changelog.md:

# 2022/10/22 - appVersion 2

* Make search marker not disappear so easily
* Allow search for accented characters (searching "caffe nero" will now find "Caffè Nero")

# 2022/10/22 - appVersion 3

* Search marker turns into bookmark marker on save
* Mostly fix bug where clicking on existing bookmark right after zoom doesn't get you to the right place
    * If you click while you're still mid-zoom, it will still be broken

Ian Denhardt

unread,
Oct 22, 2022, 3:38:27 PM10/22/22
to Daniel Krol, sandst...@googlegroups.com
Fwiw, for just sharing map data you can probably get away with something
simpler than the filesystem schema Jake linked to, which is oriented
around whole file trees; only the powerbox stuff is really essential
there.

But yeah, probably not worth the trouble for an MVP.

Quoting Daniel Krol (2022-10-22 15:32:29)
> Ah. File sharing seems more complicated than I was hoping. Probably not
> be worth digging into for a little while since I have enough hurdles
> ahead of me already before an MVP.
>
> I fixed a couple of the reported bugs and made a couple other changes.
> I didn't catch the centering one, thanks for noticing!
>
> From my changelog.md:
>
> # 2022/10/22 - appVersion 2
> * Make search marker not disappear so easily
> * Allow search for accented characters (searching "caffe nero" will now
> find "Caff� Nero")
> # 2022/10/22 - appVersion 3
> * Search marker turns into bookmark marker on save
> * Mostly fix bug where clicking on existing bookmark right after zoom
> doesn't get you to the right place
> * If you click while you're still mid-zoom, it will still be broken
>
> On 10/20/22 23:40, 'Jacob Weisz' via Sandstorm Development wrote:
>
> So our example of implementing custom capabilities between grains is
> here: [1]https://github.com/zenhack/sandstorm-filesystem
>
> Implemented exactly as is you can share files between grains, but it
> may be worthwhile to define a specific thing for map data or something
> so only other grains offering compatible data would be selectable.
>
> --
> Jacob Weisz
> [2]goo...@jacobweisz.com
> [4]https://apps.sandstorm.io/app/m3ctajcm6nnpce287r0a4t52ackzv7p7mmffrw
> 88nge64fp0m8yh?experimental=true
> postgres. Then one day something called [5]Protomaps showed up on
> Hacker News. It's exactly what I needed - flat files that your browser
> downloads and renders. No need for a database at all! So far I use
> Protomaps' site to generate them. Eventually I'll have to generate it
> myself from raw OSM data.
> Search - Usually OpenStreetMap uses something like Nominatim, which
> uses Elasticsearch. Another headache for Sandstorm. I found something
> called [6]gazetteer that turns the raw OSM data into json basically,
> [7]https://groups.google.com/d/msgid/sandstorm-dev/CAAWRcS8%2B1ApXtJC0U
> Q7MvjdSbetWqBq7ch3%2BXFLoa1pJjXjUOg%40mail.gmail.com.
>
> --
>
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
>
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [8]sandstorm-de...@googlegroups.com.
>
> To view this discussion on the web visit
> [9]https://groups.google.com/d/msgid/sandstorm-dev/809d772d-ecc3-43ef-b
> 0d6-a82e6b5d4ebbn%40googlegroups.com.
>
> --
>
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
>
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [10]sandstorm-de...@googlegroups.com.
>
> To view this discussion on the web visit
> [11]https://groups.google.com/d/msgid/sandstorm-dev/CAAWRcS-fAKNfHnedxR
> F52gtLU5kao84nTzmZKugq0W0sXBk_Bg%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [12]sandstorm-de...@googlegroups.com.
> To view this discussion on the web visit
> [13]https://groups.google.com/d/msgid/sandstorm-dev/3fb9ce35-2aee-4de4-
> 8cb8-739a63b855b6%40app.fastmail.com.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [14]sandstorm-de...@googlegroups.com.
> To view this discussion on the web visit
> [15]https://groups.google.com/d/msgid/sandstorm-dev/1d1d0f88-09e8-19b2-
> 3980-6dff77d2ed5a%40gmail.com.
>
> Verweise
>
> 1. https://github.com/zenhack/sandstorm-filesystem
> 2. mailto:goo...@jacobweisz.com
> 3. mailto:dave...@gmail.com
> 4. https://apps.sandstorm.io/app/m3ctajcm6nnpce287r0a4t52ackzv7p7mmffrw88nge64fp0m8yh?experimental=true
> 5. https://protomaps.com/
> 6. https://github.com/kiselev-dv/gazetteer/
> 7. https://groups.google.com/d/msgid/sandstorm-dev/CAAWRcS8%2B1ApXtJC0UQ7MvjdSbetWqBq7ch3%2BXFLoa1pJjXjUOg%40mail.gmail.com?utm_medium=email&utm_source=footer
> 8. mailto:sandstorm-de...@googlegroups.com
> 9. https://groups.google.com/d/msgid/sandstorm-dev/809d772d-ecc3-43ef-b0d6-a82e6b5d4ebbn%40googlegroups.com?utm_medium=email&utm_source=footer
> 10. mailto:sandstorm-de...@googlegroups.com
> 11. https://groups.google.com/d/msgid/sandstorm-dev/CAAWRcS-fAKNfHnedxRF52gtLU5kao84nTzmZKugq0W0sXBk_Bg%40mail.gmail.com?utm_medium=email&utm_source=footer
> 12. mailto:sandstorm-de...@googlegroups.com
> 13. https://groups.google.com/d/msgid/sandstorm-dev/3fb9ce35-2aee-4de4-8cb8-739a63b855b6%40app.fastmail.com?utm_medium=email&utm_source=footer
> 14. mailto:sandstorm-de...@googlegroups.com
> 15. https://groups.google.com/d/msgid/sandstorm-dev/1d1d0f88-09e8-19b2-3980-6dff77d2ed5a%40gmail.com?utm_medium=email&utm_source=footer

Jake W

unread,
Oct 23, 2022, 3:24:00 AM10/23/22
to Sandstorm Development
This build feels like it does what I expect of it much more reliably. Losing the search marker and having to re-search for something was really painful, and the current version it seems much, much harder to do that.

Dan Krol

unread,
Dec 9, 2022, 4:08:36 PM12/9/22
to Sandstorm Development
Okay it's cleaned up enough that I don't mind putting it on github:


Feel free to file Issues etc. It'll be a good motivator.

Also BTW I made a new release the other day with a small fix: I was unnecessarily escaping apostrophes in exported maps, leading to ' showing up on OrganicMaps.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/abd7c801-e7f4-4b34-b790-030729c184d4n%40googlegroups.com.

Dan Krol

unread,
Dec 14, 2022, 4:45:28 PM12/14/22
to Sandstorm Development
New version: Fast search!

For speedy search that was Sandstorm friendly, I was looking for "the sqlite of search". Turns out it's... sqlite. It can do full-text search using something called fts5. Probably won't compete with elastic search as far as smartness of interpreting queries but it's pretty much strictly better than what I had before (only exception I can think of is searching for a substring that isn't the beginning of a word, but removing that is probably a positive, actually). Plus it can handle double quotes (I manage this sort of carefully), not all words need to be in order ("bar book" will find "book & bar"), it's much more compact on disk, and most importantly it's much faster.

It's more compact in part because I ignore (always have) a lot of what gets downloaded. For the future we can trim the actual download as well for download speed, but that's low priority now.

For anybody following along at home, this will be a breaking change, you'll need to make new grains and download the map data again.

Jacob Weisz

unread,
Dec 14, 2022, 4:59:59 PM12/14/22
to sandst...@googlegroups.com
I'll try to test this out tonight. Thanks for specifying it's breaking: While I don't have any "production Share-A-Map data", I do tend to just update existing grains for testing!

--
  Jacob Weisz

Dan Krol

unread,
Dec 15, 2022, 12:04:27 AM12/15/22
to Jacob Weisz, sandst...@googlegroups.com
Just pushed another small update to add a progress bar for importing the search data into sqlite.

Dan Krol

unread,
Dec 22, 2022, 8:04:19 PM12/22/22
to Sandstorm-dev
# 2022/12/22 - appVersion 7

* Permissions for downloading maps and editing bookmarks
* All bookmark pins visible at the same time
* On startup, zoom to nicely fit all bookmarks (if you have any).
  * This will be nice for sharing read-only, since they'll probably only want to see the pins anyway.
  * Now that I've released this, I see that this is BROKEN on mobile. This is gonna be real fun to debug...
* If you download a map and click on the download pin again, it will now to zoom to that map
* Bookmarks list collapsable (will be a relief on the phone)
* Limit 50 for search results (in case a ton get returned from the db)

screenshot-1.png

Dan Krol

unread,
Dec 22, 2022, 9:03:44 PM12/22/22
to Sandstorm-dev
Okay fixed the mobile bug.

Jacob Weisz

unread,
Dec 23, 2022, 12:35:26 AM12/23/22
to sandst...@googlegroups.com
This is really quite pleasant! The latest release seems to have finished smashing most of any thoughts I particularly had about things I'd like to see tweaked.

How has the bandwidth issue been for you? Do you think with search improved some more areas may be workable with this now?

--
  Jacob Weisz

Dan Krol

unread,
Dec 23, 2022, 10:26:49 AM12/23/22
to Jacob Weisz, sandst...@googlegroups.com
Awesome, glad that I'm moving in the right direction. Thanks!

Bandwidth - I have no idea but I assume it's a non-issue right now. For all I know, you and I are the only users so far. The maps are just 10s of megs. The interesting question will be when it's on Hacker News.

The pipeline for search data is: raw OSM protobuf -> gzipped list of json objects -> sqlite database. The second conversion happens in the grain (it needs to, because there is one gzip file per area, but they all get combined into a single sqlite). Before my search speedup, the grains were just reading through all the downloaded gzips fore every search. The search speedup didn't have any effect on the download sizes and thus bandwidth.

That said, I could improve bandwidth by optimizing those gzip'd json objects, probably by quite a bit. Make it a csv, remove unneeded data. Make the files smaller.

But that isn't actually the main reason I only have two regions right now. That has to do with the first step, OSM protobuf-> gzipped list of json objects, which occurs as a preparation step to generate the data that the grains will download. I found an application called "gazetteer" for that. For whatever reason, it crashes for certain regions. I have to dedicate some time to that. I might be doing something wrong.

My big three hurdles at this point are:

* Figure out hosting (perhaps the simplest)
* Figure out the gazetteer problem for extracting search from OSM protobuf
* Figure out how to extract protomaps tile data from the same OSM protobuf (instead of manually getting it from the protomaps website)

After these, I think everything else will polish, dotting I's and crossing T's.

I'm actually looking at the protomaps part next. Once I can do that part on the command line, I can make a single map-preparation script. Then, I feel like I can more efficiently play with regions to see what gazetteer accepts, and in the meantime have more maps for the demo.

Dan Krol

unread,
Dec 23, 2022, 10:31:49 AM12/23/22
to Jacob Weisz, sandst...@googlegroups.com
will *be polish

Dan Krol

unread,
Jan 31, 2023, 2:01:29 PM1/31/23
to sandst...@googlegroups.com
With some (hopefully temporary) concessions, I can generate entire states. I now offer Illinois, Massachusetts, and New Hampshire. I have everything scripted so adding new regions is fairly trivial. I won't be doing the whole world yet but I'm open for requests! LMK if you want me to add your area and I'll cut a new version of the app.

Details: I now generate tiles from raw OSM data instead of downloading manually from Protomaps' website. The tiles don't look as good probably, because I had to dip my hand into tile generation (how to represent various regions and streets, etc) for reasons that are too boring for this email. I also found a new way to generate search data. Gazetteer was crashing on certain data. This new method (using osmium, a more basic tool) - I can export whole states, but I'm having issues exporting lines and areas at the moment, so for now I left that out. Also - I had to dip my hand into curating search data out of the mess of user-generated tags. I now understand what Gazetteer was good for. So in short: we now have full states, but (for now) regressions in how tiles look and in what search data is available.

Separately: with whole states, we'll start to see the effects of the corners I cut for search. As Jacob noted: Duplicate names, locations far away from the current view, etc. LMK how the experience is now. I'll have to do some serious thinking about this now. I don't want this "perfect" for release but it's probably not good enough yet.

I don't think this is a breaking change; the old smaller map files still show up in old grains and are still available for download for old versions of the app. But I'll delete them from the download server eventually, making this upgrade necessary to get map data again.

From changelog:

# 2023/01/31 - appVersion 9

* Switch search import format to csv
* Offer 3 entire U.S. states: IL, MA, NH (not just small regions).
  * Caveat 1: To get such big regions, I generate my own tiles now instead of downloading manually from Protomaps' website. However this required a fair amount of playing with how the data is represented. I now deem it *passable* but it could use a lot of improvement. See generate-data/ directory for details.
  * Caveat 2: The search data that I put in the CSV *only includes nodes* (points) for the time being. I have to figure out how to get the data I need out of the ways (lines and areas), which will include buildings and streets.
* Upgrade protomaps.js (necessary for the newly generated tiles) and leaflet (maybe not necessary but nice to have)
* Bugfix for exporting of .kmz file - Set the content type. Chromium on Android was appending .zip.

Dan Krol

unread,
Jan 31, 2023, 8:22:55 PM1/31/23
to sandst...@googlegroups.com
Okay I fixed Caveat 2 - you can search for streets and named buildings and stuff. This fix is just in the data; this works with appVersion 9.

Jacob Weisz

unread,
Feb 1, 2023, 2:37:42 AM2/1/23
to sandst...@googlegroups.com
All my local stuff is here! Woooo! This is really awesome.

--
  Jacob Weisz

Dan Krol

unread,
May 14, 2023, 7:47:13 PM5/14/23
to Jacob Weisz, sandst...@googlegroups.com
I'm up to appVersion 15. Versions 10-14 had a few features and UI changes that I didn't bother mentioning here. You can check out the changelog: https://github.com/orblivion/sandstorm-share-a-map/blob/master/changelog.md

However, with Version 15, all of North America is available. Europe is only about twice as much data as North America, and every other continent is smaller. So, I think I should be able to get the rest of the world without much trouble. And, from now on it will no longer require an app upgrade; it gets the region definitions from the server. I'll let you know when I get there.

The bad news is that (for now) I no longer have it cut up as states (or provinces). It's a bunch of random rectangles*. But since it's random rectangles, I also changed the UI to get rid of the download pins. Instead it shows the rectangles on the map so you know what you're getting, and you click on them to select them for download. UI around this could be improved, and I'm all ears for feedback, but I wanted this released ASAP. Let me know if you want the rectangles to be smaller. I went on the larger side because I found that if I download a ton of small regions it performs worse than one giant region. But I could probably cut them in half or so without much problem.

I'm curious how this affects performance, both on server and on laptop. My dev environment, which has to handle both at once, and is running both in a VM, got a little more exhausted than usual. When I install it on my VPS, and my laptop just runs the front end, it seems fine. Though it seems to take a decent amount of memory on the server, but maybe it always did that. I never checked before.

* The reason for the rectangles: I was previously downloading NH, MA and IL from a site called Geofabrik, but it's not feasible to download all of the administrative regions. They have a per-file daily limit. But their bandwidth limit isn't so strict; I think I can download all of the continents from them. Or maybe I could just try for the whole planet at once from another source. In any case, this means I have to split it up. Geofabrik describes how they themselves do their splitting, but they don't want to make their actual script publicly available. However they pointed me to a very simple tool that splits large regions into smaller rectangular regions with a specified maximum data size. I'd rather do that for now than take the time to recreate Geofabrik's splitting system.

Dan Krol

unread,
May 14, 2023, 7:52:04 PM5/14/23
to Jacob Weisz, sandst...@googlegroups.com
I wonder if this is going to take a toll on the demo server... Is there an issue if some people take up gigs of data during their demo use?

Jacob Weisz

unread,
May 14, 2023, 9:50:36 PM5/14/23
to Dan Krol, sandst...@googlegroups.com
There are probably a bunch of other ways to spend that much storage and bandwidth on the demo server anyways. I suspect it will be trivial, I don't know how often any given app gets demo'd generally anyways.

--
  Jacob Weisz

Reply all
Reply to author
Forward
0 new messages