KML layers partially loading or failing, and Google IP addresses making a large number of requests on a single KML layer load.

525 views
Skip to first unread message

arthur5005

unread,
Nov 24, 2011, 2:41:17 PM11/24/11
to Google Maps JavaScript API v3
I'm having a problem loading my KML layers into Google Maps v3. I find
that whenever I make a request to load a .KML file, that 80-90% of the
time the layer 'partially' loads (showing on some tiles, but not
others), and sometimes doesn't even load at all. I also find that
whenever I make a request to load a KML layer, that our server takes
on a very large number of requests from Google, which brings our
server to a crawl, and sometimes triggers a watch agent on our server
that believes it's some sort of DDOS attack. I expect Google to only
require the KML data once.

Here are our server logs for a single KML layer request
74.125.52.80 - - [22/Nov/2011:13:43:51 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.90 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.94 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.88 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.94 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.93 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.92 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.91 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.89 - - [22/Nov/2011:13:43:52 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.85 - - [22/Nov/2011:13:43:53 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.80 - - [22/Nov/2011:13:43:54 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.87 - - [22/Nov/2011:13:43:55 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.87 - - [22/Nov/2011:13:43:55 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.95 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.92 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:55 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.88 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.86 - - [22/Nov/2011:13:43:54 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.94 - - [22/Nov/2011:13:44:00 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.90 - - [22/Nov/2011:13:43:59 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.85 - - [22/Nov/2011:13:44:00 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.92 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.91 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.85 - - [22/Nov/2011:13:44:00 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.89 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.91 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.93 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.85 - - [22/Nov/2011:13:43:59 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.91 - - [22/Nov/2011:13:43:57 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.82 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.92 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.80 - - [22/Nov/2011:13:43:59 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.81 - - [22/Nov/2011:13:43:59 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.84 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.81 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.88 - - [22/Nov/2011:13:44:00 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.81 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.88 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.81 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.89 - - [22/Nov/2011:13:43:55 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.87 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.87 - - [22/Nov/2011:13:43:56 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.83 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.126.85 - - [22/Nov/2011:13:44:00 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"
74.125.52.89 - - [22/Nov/2011:13:43:58 -0800] "GET /
lmi_country_polygons/CA.kml HTTP/1.1" 200 45221 "-" "Kml-Google;
(+http://code.google.com/apis/kml), gzip"

Is there something wrong with how our server is delivering the kml
data? I'm pretty sure our headers are ok. The oddest thing is that
this was working about a month ago, without a hitch.

I have KML layers hosted at
http://www.3d-profile.com/lmi_country_polygons/<ISO 3166-1 2-Alpha
Country Code>.kml

Examples
http://www.3d-profile.com/lmi_country_polygons/US.kml
http://www.3d-profile.com/lmi_country_polygons/CL.kml


The javascript we're using is pretty basic stuff:
<script>

var MapTest = {};

MapTest.map = null;
MapTest.kmlLayer = null;

MapTest.init = function(){
var latlng = new google.maps.LatLng(0,-40);
var myOptions = {
zoom: 1,
center: latlng,
mapTypeId: google.maps.MapTypeId.ROADMAP
};
MapTest.map = new
google.maps.Map(document.getElementById('mapCanvas'),myOptions);

}
$(document).ready(function(){
$("#countrySelect").change(function(event){
var value = $(this).val();
if(value!=""){
if(MapTest.kmlLayer){
MapTest.kmlLayer.setMap(null);
}
MapTest.kmlLayer = new
google.maps.KmlLayer('http://www.3d-profile.com/
lmi_country_polygons/'+value+'.kml');
MapTest.kmlLayer.setMap(MapTest.map);
}
});
MapTest.init();
});
</script>

Joseph Elfelt

unread,
Nov 24, 2011, 4:35:30 PM11/24/11
to Google Maps JavaScript API v3
That's odd. I see different parts of your KML depending how I zoom
and pan the map:
http://www.mappingsupport.com/p/gmap4.php?q=http://www.3d-profile.com/lmi_country_polygons/US.kml

Does your KML validate?
Is it close to the max file size Google allows?
Does it work better if you compress it to a KMZ?
Are you using network links in your KML?

arthur5005

unread,
Nov 24, 2011, 6:34:16 PM11/24/11
to Google Maps JavaScript API v3
My KML Validates with:
http://googlemapsapi.blogspot.com/2007/06/validate-your-kml-online-or-offline.html
(I had to make a few minor adjustments, but it is now fully valid)

The KML files are no where near the max size, the largest being
287kilobytes I believe (Canada)
There are no network links my KML

I'll KMZ compressing these, but I'm more concerned about the number of
requests I'm getting from Google for the resource, is this normal?


On Nov 24, 1:35 pm, Joseph Elfelt <josephelf...@gmail.com> wrote:
> That's odd.  I see different parts of your KML depending  how I zoom

> and pan the map:http://www.mappingsupport.com/p/gmap4.php?q=http://www.3d-profile.com...

Joseph Elfelt

unread,
Nov 24, 2011, 10:55:43 PM11/24/11
to Google Maps JavaScript API v3
All very interesting.
Your file now displays perfectly and quickly with my app.
Perhaps the adjustments you made to your file fixed things.

Is Google still pinging your server big time?

Rich Taylor

unread,
Nov 28, 2011, 12:02:04 PM11/28/11
to google-map...@googlegroups.com
We've noticed similar behavior, sporadically.  My theory is that in certain cases, multiple google servers will request the same KML from our servers for the purposes of distributing the processing of the file or as part of their availability implementation.  Basically they're having several (regional?) servers ask for the same kml file from us, instead of asking for it once then replicating out to their other servers on their own.  This is a hypothesis, I would love to see a true explanation from someone in the know.

arthur5005

unread,
Nov 28, 2011, 4:14:29 PM11/28/11
to Google Maps JavaScript API v3
I'm still seeing partially loaded KML files today, and our server is
still seeing a flood of requests every time I load the layer, it's too
much for our server to bear at once as it's currently setup. :S

arthur5005

unread,
Nov 28, 2011, 4:17:00 PM11/28/11
to Google Maps JavaScript API v3
That was my theory as well for a bit, but I think it's more likely
that we're sending out the response to the request incorrectly, and
it's just a flood of retries from a myriad of other servers.

arthur5005

unread,
Nov 29, 2011, 4:39:54 PM11/29/11
to Google Maps JavaScript API v3
Not that I'm depending on this anymore. (I've refactored all my data
to be fetch-able via json and quickly parsed into a Polygon object via
Google maps v3 API), but I would be curious to hear what someone at
Google has to say about this behavior.

Thanks,
Arthur.

Jacinto Barcelona

unread,
Jan 17, 2013, 4:14:29 PM1/17/13
to google-map...@googlegroups.com
We're getting this same issue currently where multiple Google IPs are hitting our server (that builds and streams the kmz) multiple times (ranges from 10 to 30) per view/pan/zoom of the map. Tried the suggestion below without luck.

On Wednesday, 16 January 2013 08:04:37 UTC-7, Io Mihai wrote:
I've experienced the same problem. A flood of requests by the maps api when loading a kmz file, 10-30 requests by different google servers. I have 1-2MB kmz files and my bandwith was frequently over quota.

My application is on Google App Engine / python and the kmz files are stored in the Blobstore.

After refactoring all my javascript and the blobstore code I solved the issue by changing the way I included the maps api.
from:
to:

I guess not using the protocol confused the google maps cache.

I also added caching headers when serving the files from the blobstore.

expires_date = datetime.datetime.utcnow() + datetime.timedelta(1)
expires_str = expires_date.strftime("%d %b %Y %H:%M:%S GMT")
self.response.headers['Expires'] = expires_str
self.response.headers['Content-Type'] = 'application/vnd.google-earth.kmz'
self.response.headers['Cache-Control'] = 'public, max-age=80000'

I hope this helps.

Jacinto Barcelona

unread,
Jan 21, 2013, 9:27:17 AM1/21/13
to google-map...@googlegroups.com
Hi,

We have encountered the issue from this original thread that just started happening Jan 17th.  We have kmz layers on our map being generated from FME Server on demand. If layer is on, Google hits our FME server's url that generates kmz on the fly. The problem is about 5 Google IPs are requesting the same single kmz and the number of requests total to about 30 to 40.  We have tried the suggestion by lo Mihai to change the reference of the map API but that did not make any difference (changing the ref to https://maps.googleapis.com/maps/api/js?sensor=false).

Thoughts/suggestions for fix are appreciated.

barryhunter

unread,
Jan 21, 2013, 1:05:27 PM1/21/13
to google-map...@googlegroups.com
Are you sure the server is setting good caching headers? Allowing the content to be cached. 


If Googles servers cant cache the content, it pretty much has to download it for every tile. 


http://redbot.org/ is a good way to check. 

Jacinto Barcelona

unread,
Jan 21, 2013, 4:06:57 PM1/21/13
to google-map...@googlegroups.com
Hi Barry,
Thanks for the suggestion. Plugged in the URL to REDbot and got the following results.  It doesn't look like it's a caching issue.


  HTTP/1.1 200 OK
    Server: Apache-Coyote/1.1
    Content-Disposition: attachment; filename="lbls.kmz"
    Content-Type: application/vnd.google-earth.kmz
    Transfer-Encoding: chunked
    Date: Mon, 21 Jan 2013 20:42:39 GMT


General

  • The server's clock is correct.

Caching

  • This response allows all caches to store it.
  • This response allows a cache to assign its own freshness lifetime.
Reply all
Reply to author
Forward
0 new messages