Best Practices for Maps/KML Files Over 1000+ Placemarks

[Justin Fields]

I'm fairly new to mapping files and KML. I'm working with a fairly small dataset, but it is still over the 1000 placemark threshold that can become a problem. In planning for future growth, I'm trying to figure out the best practices for presenting the data I have. Right now, it is a single large KML file, specified in a website I have hosted. It works pretty well and users can pan, zoom and such through the data, from continent scale to street level.

As the data grows, which it is sure to, I want to make sure that I don't hit a hard limit that results in pages not loading, or loading without the full range of data when at a low level. I have been reading about network links and divided KML files. I'm trying to solidify my understanding to make sure I'm headed in the correct direction. From what I understand (which could easily be wrong), I can create a special high-level KML file, with a broad but shallow sampling of the data, for use at low zoom levels, like 2-3. Then, as they zoom in, I could swap to a different set of KML files, with more detail, but with North America (my target continent) broken up into a grid of lat-long based KML files. Depending on where they are focused, the relevant KML file will be displayed, as controlled by regions and bounding boxes. Depending on my eventual level of data, I might need an intermediary grid between the lowest zoom file and the street-level zoom files.

What happens when a viewer is zoomed in somewhat, but then pans substantially, crossing the boundary of a KML file? Does it fairly seamlessly transition to the neighboring file? Similarly, if they move around between locations and zoom levels, does the web page just swap underlying KML files as needed, with no user intervention?

My data is all stored in a relational database, so it is pretty easy for me to export portions of the data into KML files in flexible ways. But before I embark on that process, I wanted to see if I was doing this the right way. Also, are there any examples anyone can point to for a tutorial about this type of data and KML manipulation?

Thanks!

Justin 

[Jason M]

Hi Justin.

What you're describing can be done in KML using NetworkLinks and
Regions with appropriate Level Of Detais (LOD) settings.

You'd basically have a master KML with several NetworkLinks each with
different Region/Lod settings to load at an appropriate zoom level.

You can check out this tutorial for some details and a number of
examples:
http://code.google.com/apis/kml/documentation/regions.html

The KML Handbook discussing techniques in Chapter 8. Dealing with
Large Data Sets.
You can browse portions of the chapter online:
http://my.safaribooksonline.com/book/gis/9780321574404/dealing-with-large-data-sets/ch08lev1sec3

Hope that helps.
--jason

[Matt Fox]

Here's an example with 6.5 million placemarks based on Global
Geographic Names Database. I broke up the dataset into 2x2 degree
areas that load and unload as you pan around the globe.

http://www.gelib.com/gnis.htm

Matt

> You can browse portions of the chapter online:http://my.safaribooksonline.com/book/gis/9780321574404/dealing-with-l...
>
> Hope that helps.
> --jason

[Justin Fields]

Thanks for the replies, folks! It will take me a little while to read up and go through the samples, but I really appreciate knowing that it is possible and how to start going about it.

Justin

[Jason M]

A single large KML can have some real performance problems in Google Earth. To investigate the limits I've created a KML generator that generates 1000's of placemarks either globally distributed or clustered. Didn't really see major problems until it hit 100,000+ placemarks. 500K really makes things go badly.

However, this just shows that when a single KML is too large you should break it up in smaller KML files and use combination of NetworkLinks, radioFolder listItemTypes, and regions to limit what is loaded and displayed at any given time. Google Earth client can and does scale to many millions of placemarks but not loaded and visible all at once. Large KML needs to be designed in such a way to scale efficiently. Nested NetworkLink and regions (aka super overlays) is the way to deal with vast amounts of data.