Can anyone explain how the New York Times is able to show thousands of
markers on their Google Flash maps at once without clustering? Each
marker has hover info, popup info, and when clicked activates
JavaScript on the page.
Here is an example with 32,000 markers on one map, and it's super
fast:
The basic idea is that they create a highly-performant custom overlay,
and they likely draw many shapes into just one overlay, instead of one
shape per overlay. That technique is described by Redfin in the video
+ slides.
On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> Can anyone explain how the New York Times is able to show thousands of
> markers on their Google Flash maps at once without clustering? Each
> marker has hover info, popup info, and when clicked activates
> JavaScript on the page.
> Here is an example with 32,000 markers on one map, and it's super
> fast:
Flash actually has the ability to work with bitmaps, the main choke
point with Flash is the vector renderer, so if you have the markers as
bitmap data what you could do is have bitmap tiles, then copy the
markers using copy pixels. Finally do the collision detection ( mouse
rollover etc..) in action script. If you did it properly, you could
probably get it even smoother then the example you provided. Just be
aware that if you are using draw() it utilizes the vector renderer, so
you won't necessarily get any performance increases.
On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
wrote:
> The basic idea is that they create a highly-performant custom overlay,
> and they likely draw many shapes into just one overlay, instead of one
> shape per overlay. That technique is described by Redfin in the video
> + slides.
> - pamela
> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > Can anyone explain how the New York Times is able to show thousands of
> > markers on their Google Flash maps at once without clustering? Each
> > marker has hover info, popup info, and when clicked activates
> > JavaScript on the page.
> > Here is an example with 32,000 markers on one map, and it's super
> > fast:
On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> Flash actually has the ability to work with bitmaps, the main choke
> point with Flash is the vector renderer, so if you have the markers as
> bitmap data what you could do is have bitmap tiles, then copy the
> markers using copy pixels. Finally do the collision detection ( mouse
> rollover etc..) in action script. If you did it properly, you could
> probably get it even smoother then the example you provided. Just be
> aware that if you are using draw() it utilizes the vector renderer, so
> you won't necessarily get any performance increases.
> On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> wrote:
>> Hi Your Mapper-
>> The basic idea is that they create a highly-performant custom overlay,
>> and they likely draw many shapes into just one overlay, instead of one
>> shape per overlay. That technique is described by Redfin in the video
>> + slides.
>> - pamela
>> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
>> > Can anyone explain how the New York Times is able to show thousands of
>> > markers on their Google Flash maps at once without clustering? Each
>> > marker has hover info, popup info, and when clicked activates
>> > JavaScript on the page.
>> > Here is an example with 32,000 markers on one map, and it's super
>> > fast:
(Zoom in six or seven levels to see the individual markers replace
those clusters.)
I use TileLayerOverlay to present 2 million markers on the map. One
bitmap overlay is generated on-the-fly for each 256x256 tile. The
approach I use is as follows:
1.As you pan the map (at a local level), the API notifies me through
the ITileLayer.load Tile () interface method that new tiles are
emerging, and I return an empty Bitmap.
2. I then hold a reference to that bitmap and submit a query to the
spatial databox back-end for data for that tile
3. When the data comes back, through the reference to the Bitmap
object, I update the Bitmap with markers (and if necessary cluster
images too)
The challenge here is knowing when to discard the Bitmap object, as it
occupes 1/4 megabyte of memory. I use a heuristic based on assumptions
about how long Google holds on to their basemap tiles.
As Jonathan mentioned earlier, rendering duplicate bitmaps
(BitmapData) is somewhat faster than their Shape equivalents.
Also, as he mentioned, the "custom" approach to rendering markers
requires you to handle hover and click events, as well as draw the
info window. It's a lot of work.
Although Bitmaps offer some performance advantage over Shape overlays,
the bigger performance gain was to limit the number of overlays by
going with an "overlay per tile" instead of "overlay per marker". This
establishes a firm upper limit on the number of overlays in view,
regardless of the number of markers to display.
Finally, if you wish to use Google marker overlays, you can gain a bit
more performance by turning off shadows.
Hope this helps.
On Sep 17, 6:06 pm, "pamela (Google Employee)" <pamela...@gmail.com>
wrote:
> On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > Flash actually has the ability to work with bitmaps, the main choke
> > point with Flash is the vector renderer, so if you have the markers as
> > bitmap data what you could do is have bitmap tiles, then copy the
> > markers using copy pixels. Finally do the collision detection ( mouse
> > rollover etc..) in action script. If you did it properly, you could
> > probably get it even smoother then the example you provided. Just be
> > aware that if you are using draw() it utilizes the vector renderer, so
> > you won't necessarily get any performance increases.
> > On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> > wrote:
> >> Hi Your Mapper-
> >> The basic idea is that they create a highly-performant custom overlay,
> >> and they likely draw many shapes into just one overlay, instead of one
> >> shape per overlay. That technique is described by Redfin in the video
> >> + slides.
> >> - pamela
> >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> >> > Can anyone explain how the New York Times is able to show thousands of
> >> > markers on their Google Flash maps at once without clustering? Each
> >> > marker has hover info, popup info, and when clicked activates
> >> > JavaScript on the page.
> >> > Here is an example with 32,000 markers on one map, and it's super
> >> > fast:
Thanks for the great responses Pamela and Jonathan. Just what I was
looking for.
Pamela, I had seen your presentation online a few months ago but had
forgotten the details. Thanks for the link.
The bottom line seems to be that in order to show that many 'markers'
the solution is to create tiles/custom overlay and detect clicks to
show popups. Makes sense, I'll give it a try. More work than just
throwing a couple thousand markers at the API, but worth it.
Michael
On Sep 17, 6:06 pm, "pamela (Google Employee)" <pamela...@gmail.com>
wrote:
> On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > Flash actually has the ability to work with bitmaps, the main choke
> > point with Flash is the vector renderer, so if you have the markers as
> > bitmap data what you could do is have bitmap tiles, then copy the
> > markers using copy pixels. Finally do the collision detection ( mouse
> > rollover etc..) in action script. If you did it properly, you could
> > probably get it even smoother then the example you provided. Just be
> > aware that if you are using draw() it utilizes the vector renderer, so
> > you won't necessarily get any performance increases.
> > On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> > wrote:
> >> Hi Your Mapper-
> >> The basic idea is that they create a highly-performant custom overlay,
> >> and they likely draw many shapes into just one overlay, instead of one
> >> shape per overlay. That technique is described by Redfin in the video
> >> + slides.
> >> - pamela
> >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> >> > Can anyone explain how the New York Times is able to show thousands of
> >> > markers on their Google Flash maps at once without clustering? Each
> >> > marker has hover info, popup info, and when clicked activates
> >> > JavaScript on the page.
> >> > Here is an example with 32,000 markers on one map, and it's super
> >> > fast:
It does not include interactivity, but it could be built in, in the
example I used 100k markers distributed randomly through the world, I
have better explanations of everything in the class. Hope this helps
someone.
Jonathan
On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> Thanks for the great responses Pamela and Jonathan. Just what I was
> looking for.
> Pamela, I had seen your presentation online a few months ago but had
> forgotten the details. Thanks for the link.
> The bottom line seems to be that in order to show that many 'markers'
> the solution is to create tiles/custom overlay and detect clicks to
> show popups. Makes sense, I'll give it a try. More work than just
> throwing a couple thousand markers at the API, but worth it.
> > On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > Flash actually has the ability to work with bitmaps, the main choke
> > > point with Flash is the vector renderer, so if you have the markers as
> > > bitmap data what you could do is have bitmap tiles, then copy the
> > > markers using copy pixels. Finally do the collision detection ( mouse
> > > rollover etc..) in action script. If you did it properly, you could
> > > probably get it even smoother then the example you provided. Just be
> > > aware that if you are using draw() it utilizes the vector renderer, so
> > > you won't necessarily get any performance increases.
> > > On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> > > wrote:
> > >> Hi Your Mapper-
> > >> The basic idea is that they create a highly-performant custom overlay,
> > >> and they likely draw many shapes into just one overlay, instead of one
> > >> shape per overlay. That technique is described by Redfin in the video
> > >> + slides.
> > >> - pamela
> > >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > >> > Can anyone explain how the New York Times is able to show thousands of
> > >> > markers on their Google Flash maps at once without clustering? Each
> > >> > marker has hover info, popup info, and when clicked activates
> > >> > JavaScript on the page.
> > >> > Here is an example with 32,000 markers on one map, and it's super
> > >> > fast:
That's a terrific example, almost comical. And it shows the distortion
of the map projection with a lower density of points at the north and
south pole areas.
I downloaded the source code too. I have not used Flex nor do I have
it installed (I only have CS3 Web which has Flash), but did try to
open the files in a new project in the Flex 60-day demo version. It's
going to take a bit more time on my end to learn the new environment
and decide if Flex is worth it, but it's definitely promising.
Would it be possible to build a generic Flex app like the one you
made, where one could pass in a 'lightweight' JSON or XML file that
just had 100K location ID, lat, lon, and marker color fields, and the
app could create the map on the fly from that data? Even better would
be a Name field that could show on hover, and a link to a dynamic web
page based on the ID field.
Anyway, maybe just a rhetorical brainstorming question there. Thanks,
Michael
On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> It does not include interactivity, but it could be built in, in the
> example I used 100k markers distributed randomly through the world, I
> have better explanations of everything in the class. Hope this helps
> someone.
> Jonathan
> On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> > Thanks for the great responses Pamela and Jonathan. Just what I was
> > looking for.
> > Pamela, I had seen your presentation online a few months ago but had
> > forgotten the details. Thanks for the link.
> > The bottom line seems to be that in order to show that many 'markers'
> > the solution is to create tiles/custom overlay and detect clicks to
> > show popups. Makes sense, I'll give it a try. More work than just
> > throwing a couple thousand markers at the API, but worth it.
> > > On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > Flash actually has the ability to work with bitmaps, the main choke
> > > > point with Flash is the vector renderer, so if you have the markers as
> > > > bitmap data what you could do is have bitmap tiles, then copy the
> > > > markers using copy pixels. Finally do the collision detection ( mouse
> > > > rollover etc..) in action script. If you did it properly, you could
> > > > probably get it even smoother then the example you provided. Just be
> > > > aware that if you are using draw() it utilizes the vector renderer, so
> > > > you won't necessarily get any performance increases.
> > > > On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> > > > wrote:
> > > >> Hi Your Mapper-
> > > >> The basic idea is that they create a highly-performant custom overlay,
> > > >> and they likely draw many shapes into just one overlay, instead of one
> > > >> shape per overlay. That technique is described by Redfin in the video
> > > >> + slides.
> > > >> - pamela
> > > >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > > >> > Can anyone explain how the New York Times is able to show thousands of
> > > >> > markers on their Google Flash maps at once without clustering? Each
> > > >> > marker has hover info, popup info, and when clicked activates
> > > >> > JavaScript on the page.
> > > >> > Here is an example with 32,000 markers on one map, and it's super
> > > >> > fast:
> That's a terrific example, almost comical. And it shows the distortion
> of the map projection with a lower density of points at the north and
> south pole areas.
> I downloaded the source code too. I have not used Flex nor do I have
> it installed (I only have CS3 Web which has Flash), but did try to
> open the files in a new project in the Flex 60-day demo version. It's
> going to take a bit more time on my end to learn the new environment
> and decide if Flex is worth it, but it's definitely promising.
> Would it be possible to build a generic Flex app like the one you
> made, where one could pass in a 'lightweight' JSON or XML file that
> just had 100K location ID, lat, lon, and marker color fields, and the
> app could create the map on the fly from that data? Even better would
> be a Name field that could show on hover, and a link to a dynamic web
> page based on the ID field.
> Anyway, maybe just a rhetorical brainstorming question there. Thanks,
> Michael
> On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > So I wanted a challenge and actually built a class for handling large
> > amounts of markers, I have included the demo and source at:
> > It does not include interactivity, but it could be built in, in the
> > example I used 100k markers distributed randomly through the world, I
> > have better explanations of everything in the class. Hope this helps
> > someone.
> > Jonathan
> > On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> > > Thanks for the great responses Pamela and Jonathan. Just what I was
> > > looking for.
> > > Pamela, I had seen your presentation online a few months ago but had
> > > forgotten the details. Thanks for the link.
> > > The bottom line seems to be that in order to show that many 'markers'
> > > the solution is to create tiles/custom overlay and detect clicks to
> > > show popups. Makes sense, I'll give it a try. More work than just
> > > throwing a couple thousand markers at the API, but worth it.
> > > > On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > > Flash actually has the ability to work with bitmaps, the main choke
> > > > > point with Flash is the vector renderer, so if you have the markers as
> > > > > bitmap data what you could do is have bitmap tiles, then copy the
> > > > > markers using copy pixels. Finally do the collision detection ( mouse
> > > > > rollover etc..) in action script. If you did it properly, you could
> > > > > probably get it even smoother then the example you provided. Just be
> > > > > aware that if you are using draw() it utilizes the vector renderer, so
> > > > > you won't necessarily get any performance increases.
> > > > > On Sep 17, 9:46 am, "pamela (Google Employee)" <pamela...@gmail.com>
> > > > > wrote:
> > > > >> Hi Your Mapper-
> > > > >> The basic idea is that they create a highly-performant custom overlay,
> > > > >> and they likely draw many shapes into just one overlay, instead of one
> > > > >> shape per overlay. That technique is described by Redfin in the video
> > > > >> + slides.
> > > > >> - pamela
> > > > >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > > > >> > Can anyone explain how the New York Times is able to show thousands of
> > > > >> > markers on their Google Flash maps at once without clustering? Each
> > > > >> > marker has hover info, popup info, and when clicked activates
> > > > >> > JavaScript on the page.
> > > > >> > Here is an example with 32,000 markers on one map, and it's super
> > > > >> > fast:
I didn't build it into the class but here is a function you can add
and use to display the data after the markers are registered:
public function renderAll() {
clearRenderQueue();
for(var i:int = 0; i < _tiles.length; i++) {
cleanTile(_tiles[i][0]);
addToRenderQueue(_tiles[i][0], _tiles[i][1], _tiles[i][2])
}
}
This function will stop all current rendering, and the force all the
tiles to re-render. If you're loading from a server I would recommend
disabling the rendering under loadTile until your data is loaded. This
class is mostly useful for visualization, it really depends on what
you're doing. If you're interested in implementing interactivity, the
most efficient way would be to get the mouse latlng, then grab the
closest index sector and flip through it figuring out which marker is
closest to the mouse latlng. This way you would not have to flip
through all 100k markers trying to find the right marker.
On Sep 22, 5:24 pm, Kevin Macdonald <kevin.macdon...@pentura.ca>
wrote:
> Do you have any thoughts about how to populate the map with 100,000
> records fetched asynchronously from a server?
> On Sep 22, 4:59 pm, Your Mapper <schnue...@gmail.com> wrote:
> > Jonathan,
> > That's a terrific example, almost comical. And it shows the distortion
> > of the map projection with a lower density of points at the north and
> > south pole areas.
> > I downloaded the source code too. I have not used Flex nor do I have
> > it installed (I only have CS3 Web which has Flash), but did try to
> > open the files in a new project in the Flex 60-day demo version. It's
> > going to take a bit more time on my end to learn the new environment
> > and decide if Flex is worth it, but it's definitely promising.
> > Would it be possible to build a generic Flex app like the one you
> > made, where one could pass in a 'lightweight' JSON or XML file that
> > just had 100K location ID, lat, lon, and marker color fields, and the
> > app could create the map on the fly from that data? Even better would
> > be a Name field that could show on hover, and a link to a dynamic web
> > page based on the ID field.
> > Anyway, maybe just a rhetorical brainstorming question there. Thanks,
> > Michael
> > On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > So I wanted a challenge and actually built a class for handling large
> > > amounts of markers, I have included the demo and source at:
> > > It does not include interactivity, but it could be built in, in the
> > > example I used 100k markers distributed randomly through the world, I
> > > have better explanations of everything in the class. Hope this helps
> > > someone.
> > > Jonathan
> > > On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> > > > Thanks for the great responses Pamela and Jonathan. Just what I was
> > > > looking for.
> > > > Pamela, I had seen your presentation online a few months ago but had
> > > > forgotten the details. Thanks for the link.
> > > > The bottom line seems to be that in order to show that many 'markers'
> > > > the solution is to create tiles/custom overlay and detect clicks to
> > > > show popups. Makes sense, I'll give it a try. More work than just
> > > > throwing a couple thousand markers at the API, but worth it.
> > > > > On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > > > Flash actually has the ability to work with bitmaps, the main choke
> > > > > > point with Flash is the vector renderer, so if you have the markers as
> > > > > > bitmap data what you could do is have bitmap tiles, then copy the
> > > > > > markers using copy pixels. Finally do the collision detection ( mouse
> > > > > > rollover etc..) in action script. If you did it properly, you could
> > > > > > probably get it even smoother then the example you provided. Just be
> > > > > > aware that if you are using draw() it utilizes the vector renderer, so
> > > > > > you won't necessarily get any performance increases.
> > > > > >> The basic idea is that they create a highly-performant custom overlay,
> > > > > >> and they likely draw many shapes into just one overlay, instead of one
> > > > > >> shape per overlay. That technique is described by Redfin in the video
> > > > > >> + slides.
> > > > > >> - pamela
> > > > > >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > > > > >> > Can anyone explain how the New York Times is able to show thousands of
> > > > > >> > markers on their Google Flash maps at once without clustering? Each
> > > > > >> > marker has hover info, popup info, and when clicked activates
> > > > > >> > JavaScript on the page.
> > > > > >> > Here is an example with 32,000 markers on one map, and it's super
> > > > > >> > fast:
Hi peoples,
Sorry to bring this one back up after such a long time (3 days is
forever in forums :) but I just wanted to say thanks to Jonathan I was
struggling some with getting lots of markers (around 1000) to show up
on the map without getting seriously bogged down. I tried a few
different methods, initially I tried doing about a 1/4 of what you did
by using one overlay but still having a sprite for each asset so that
I could put listeners on them. Unfortunately this results in an
unbearable amount of latency when moving the map around although it
was a simple way to get listeners and more markers than I could
before, also this way the sprites can do some cool animation type
things pretty easily, whereas my next try I lost the ability to easily
animate the sprites but gained significantly in terms of
responsiveness. The next method I tried was using the same type of
thing as is explained here where I'm just doing the drawing for each
one in the overlay and getting rid of the sprites, this worked great,
dragging was extremely responsive however I had completely lost my
interactivity as you can't add a listener to the pixels you're
drawing, was looking into how 3d apps do hit detection, explained here
http://michaelvandaniker.com/blog/ but I just couldn't figure out how
that kind of hit detection works quickly enough (and am still not sure
it's a viable solution since the markers do need to move quite a bit)
so I started implementing you're idea posted here about using the
distance on mouse over unfortunately I didn't split my items into
sectors (just seemed complicated although yours does perform
phenomenally) so I run through every marker in the viewport. Anyways
I decided I should post what I've got for others to benefit from since
I benefited so much from you're post, can't thank you enough and it
came at the perfect time the 23rd is when I started looking into how
to deal with large groups of markers. In my implementation I pare out
the markers that are not going to appear in the current view port then
I cluster the markers that are in view using a clusterer I found out
on the internets (thanks to Kelvin Luck for that check it out here
http://www.appelsiini.net/2008/11/introduction-to-marker-clustering-w...)
and here's what I ended up with (right click to view source):
it's not perfect still acts a bit buggy in terms of the clustering
when panning because it needs to include any new markers that have
come into the view and re-cluster and depending on the pan position
the local position of the markers changes some so when you're zoomed
out really far and almost everything is a cluster there's a good
chance that panning will cause the markers to jump to different
clusters. My version isn't 100% clean but figured it still might be
useful for others to learn some of these concepts from and isn't quite
as complicated as Jonathan's class (not a bad thing since it works so
damn well but hard for beginners/intermediate programmers to
understand)
Thanks again,
Shaun
On Sep 22, 4:52 pm, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> I didn't build it into the class but here is a function you can add
> and use to display the data after the markers are registered:
> public function renderAll() {
> clearRenderQueue();
> for(var i:int = 0; i < _tiles.length; i++) {
> cleanTile(_tiles[i][0]);
> addToRenderQueue(_tiles[i][0], _tiles[i][1], _tiles[i][2])
> }
> }
> This function will stop all current rendering, and the force all the
> tiles to re-render. If you're loading from a server I would recommend
> disabling the rendering under loadTile until your data is loaded. This
> class is mostly useful for visualization, it really depends on what
> you're doing. If you're interested in implementing interactivity, the
> most efficient way would be to get the mouse latlng, then grab the
> closest index sector and flip through it figuring out which marker is
> closest to the mouse latlng. This way you would not have to flip
> through all 100k markers trying to find the right marker.
> On Sep 22, 5:24 pm, Kevin Macdonald <kevin.macdon...@pentura.ca>
> wrote:
> > Jonathan,
> > That's a lot of markers.
> > Do you have any thoughts about how to populate the map with 100,000
> > records fetched asynchronously from a server?
> > On Sep 22, 4:59 pm, Your Mapper <schnue...@gmail.com> wrote:
> > > Jonathan,
> > > That's a terrific example, almost comical. And it shows the distortion
> > > of the map projection with a lower density of points at the north and
> > > south pole areas.
> > > I downloaded the source code too. I have not used Flex nor do I have
> > > it installed (I only have CS3 Web which has Flash), but did try to
> > > open the files in a new project in the Flex 60-day demo version. It's
> > > going to take a bit more time on my end to learn the new environment
> > > and decide if Flex is worth it, but it's definitely promising.
> > > Would it be possible to build a generic Flex app like the one you
> > > made, where one could pass in a 'lightweight' JSON or XML file that
> > > just had 100K location ID, lat, lon, and marker color fields, and the
> > > app could create the map on the fly from that data? Even better would
> > > be a Name field that could show on hover, and a link to a dynamic web
> > > page based on the ID field.
> > > Anyway, maybe just a rhetorical brainstorming question there. Thanks,
> > > Michael
> > > On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > So I wanted a challenge and actually built a class for handling large
> > > > amounts of markers, I have included the demo and source at:
> > > > It does not include interactivity, but it could be built in, in the
> > > > example I used 100k markers distributed randomly through the world, I
> > > > have better explanations of everything in the class. Hope this helps
> > > > someone.
> > > > Jonathan
> > > > On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> > > > > Thanks for the great responses Pamela and Jonathan. Just what I was
> > > > > looking for.
> > > > > Pamela, I had seen your presentation online a few months ago but had
> > > > > forgotten the details. Thanks for the link.
> > > > > The bottom line seems to be that in order to show that many 'markers'
> > > > > the solution is to create tiles/custom overlay and detect clicks to
> > > > > show popups. Makes sense, I'll give it a try. More work than just
> > > > > throwing a couple thousand markers at the API, but worth it.
> > > > > > On Fri, Sep 18, 2009 at 1:55 AM, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > > > > Flash actually has the ability to work with bitmaps, the main choke
> > > > > > > point with Flash is the vector renderer, so if you have the markers as
> > > > > > > bitmap data what you could do is have bitmap tiles, then copy the
> > > > > > > markers using copy pixels. Finally do the collision detection ( mouse
> > > > > > > rollover etc..) in action script. If you did it properly, you could
> > > > > > > probably get it even smoother then the example you provided. Just be
> > > > > > > aware that if you are using draw() it utilizes the vector renderer, so
> > > > > > > you won't necessarily get any performance increases.
> > > > > > >> The basic idea is that they create a highly-performant custom overlay,
> > > > > > >> and they likely draw many shapes into just one overlay, instead of one
> > > > > > >> shape per overlay. That technique is described by Redfin in the video
> > > > > > >> + slides.
> > > > > > >> - pamela
> > > > > > >> On Thu, Sep 17, 2009 at 10:44 PM, Your Mapper <schnue...@gmail.com> wrote:
> > > > > > >> > Can anyone explain how the New York Times is able to show thousands of
> > > > > > >> > markers on their Google Flash maps at once without clustering? Each
> > > > > > >> > marker has hover info, popup info, and when clicked activates
> > > > > > >> > JavaScript on the page.
> > > > > > >> > Here is an example with 32,000 markers on one map, and it's super
> > > > > > >> > fast:
Hello Abyss,
Got my clustering working better if anyone is interested also solved
my issue with having lines that are greater than 32768 by making a
short line interpolation algorithm to cut it into segments and draw
lines for each segment, if the segmentation is necessary it draws from
the start point the first segment then it skips a bunch of space in
between and then draws the last segment because generally if you're
zoomed in and not near a marker you don't care about the middle of the
path you just want to see it leaving (or coming into) the marker and
going in some direction off the screen this greatly reduced the amount
of drawing otherwise it was drawing lines for all the elements that
weren't on screen, although it still does that at least it only draws
three lines rather than 10 for each one, I'll probably try to
eliminate the drawing of these lines completely if neither endpoint is
in view. Either way it's running pretty well with 1000 markers and
1000 lines, lots of wacky hacky heuristics, but it is working as good
if not better than the first map I tried, using the built-in markers
and overlays, with 8 markers and 5 lines. I lost my little animated
circle but I think overall it's worth it plus I think I can bring that
back for the roll over hover display since only one of these is on
screen at a time and it doesn't update position while dragging.
Big Thanks to the community for all your help and support, for a tech
community everyone playing with Google maps seems really
philanthropic, not nearly as many downers as most forums,
Shaun
Test project to show the problems with really long lines, to see the
issue up the numeric stepper by 1 to 32768 then slide the vertical
slider (either slider will cause the error but I'm sure this does it)
to something greater than 32768 and all of a sudden the line becomes
this huge shape, step the numeric stepper down and tweak the slider to
redraw and you'll see it works fine because now it's cutting it up
into lines that are no longer than 32767 hitting that last bit seems
to be really messing things up, that's all AS3 related though not a
map problem.
http://www.shaunhusain.com/LineLengthTest/ Test project to show a junkload of light markers, make them
interactive, draw lines between them, and cluster points that can be
zoomed in on by clicking the cluster...sweet.
http://www.shaunhusain.com/GoogleMapManyMarker/
On Sep 25, 2:34 pm, Shaun <shaun.hus...@gmail.com> wrote:
> Hi peoples,
> Sorry to bring this one back up after such a long time (3 days is
> forever in forums :) but I just wanted to say thanks to Jonathan I was
> struggling some with getting lots of markers (around 1000) to show up
> on the map without getting seriously bogged down. I tried a few
> different methods, initially I tried doing about a 1/4 of what you did
> by using one overlay but still having a sprite for each asset so that
> I could put listeners on them. Unfortunately this results in an
> unbearable amount of latency when moving the map around although it
> was a simple way to get listeners and more markers than I could
> before, also this way the sprites can do some cool animation type
> things pretty easily, whereas my next try I lost the ability to easily
> animate the sprites but gained significantly in terms of
> responsiveness. The next method I tried was using the same type of
> thing as is explained here where I'm just doing the drawing for each
> one in the overlay and getting rid of the sprites, this worked great,
> dragging was extremely responsive however I had completely lost my
> interactivity as you can't add a listener to the pixels you're
> drawing, was looking into how 3d apps do hit detection, explained herehttp://michaelvandaniker.com/blog/but I just couldn't figure out how
> that kind of hit detection works quickly enough (and am still not sure
> it's a viable solution since the markers do need to move quite a bit)
> so I started implementing you're idea posted here about using the
> distance on mouse over unfortunately I didn't split my items into
> sectors (just seemed complicated although yours does perform
> phenomenally) so I run through every marker in the viewport. Anyways
> I decided I should post what I've got for others to benefit from since
> I benefited so much from you're post, can't thank you enough and it
> came at the perfect time the 23rd is when I started looking into how
> to deal with large groups of markers. In my implementation I pare out
> the markers that are not going to appear in the current view port then
> I cluster the markers that are in view using a clusterer I found out
> on the internets (thanks to Kelvin Luck for that check it out herehttp://www.appelsiini.net/2008/11/introduction-to-marker-clustering-w...)
> and here's what I ended up with (right click to view source):
> http://www.shaunhusain.com/GoogleMapManyMarker/http://www.shaunhusain... > (just found out context menu doesn't seem to work on top of the map
> [for right click view source], might be due to something in my
> implementation but maybe this is what another post I replied to was
> referring to about the ContextMenu)
> it's not perfect still acts a bit buggy in terms of the clustering
> when panning because it needs to include any new markers that have
> come into the view and re-cluster and depending on the pan position
> the local position of the markers changes some so when you're zoomed
> out really far and almost everything is a cluster there's a good
> chance that panning will cause the markers to jump to different
> clusters. My version isn't 100% clean but figured it still might be
> useful for others to learn some of these concepts from and isn't quite
> as complicated as Jonathan's class (not a bad thing since it works so
> damn well but hard for beginners/intermediate programmers to
> understand)
> Thanks again,
> Shaun
> On Sep 22, 4:52 pm, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > Hey Kevin,
> > I didn't build it into the class but here is a function you can add
> > and use to display the data after the markers are registered:
> > This function will stop all current rendering, and the force all the
> > tiles to re-render. If you're loading from a server I would recommend
> > disabling the rendering under loadTile until your data is loaded. This
> > class is mostly useful for visualization, it really depends on what
> > you're doing. If you're interested in implementing interactivity, the
> > most efficient way would be to get the mouse latlng, then grab the
> > closest index sector and flip through it figuring out which marker is
> > closest to the mouse latlng. This way you would not have to flip
> > through all 100k markers trying to find the right marker.
> > On Sep 22, 5:24 pm, Kevin Macdonald <kevin.macdon...@pentura.ca>
> > wrote:
> > > Jonathan,
> > > That's a lot of markers.
> > > Do you have any thoughts about how to populate the map with 100,000
> > > records fetched asynchronously from a server?
> > > On Sep 22, 4:59 pm, Your Mapper <schnue...@gmail.com> wrote:
> > > > Jonathan,
> > > > That's a terrific example, almost comical. And it shows the distortion
> > > > of the map projection with a lower density of points at the north and
> > > > south pole areas.
> > > > I downloaded the source code too. I have not used Flex nor do I have
> > > > it installed (I only have CS3 Web which has Flash), but did try to
> > > > open the files in a new project in the Flex 60-day demo version. It's
> > > > going to take a bit more time on my end to learn the new environment
> > > > and decide if Flex is worth it, but it's definitely promising.
> > > > Would it be possible to build a generic Flex app like the one you
> > > > made, where one could pass in a 'lightweight' JSON or XML file that
> > > > just had 100K location ID, lat, lon, and marker color fields, and the
> > > > app could create the map on the fly from that data? Even better would
> > > > be a Name field that could show on hover, and a link to a dynamic web
> > > > page based on the ID field.
> > > > Anyway, maybe just a rhetorical brainstorming question there. Thanks,
> > > > Michael
> > > > On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
> > > > > So I wanted a challenge and actually built a class for handling large
> > > > > amounts of markers, I have included the demo and source at:
> > > > > It does not include interactivity, but it could be built in, in the
> > > > > example I used 100k markers distributed randomly through the world, I
> > > > > have better explanations of everything in the class. Hope this helps
> > > > > someone.
> > > > > Jonathan
> > > > > On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
> > > > > > Thanks for the great responses Pamela and Jonathan. Just what I was
> > > > > > looking for.
> > > > > > Pamela, I had seen your presentation online a few months ago but had
> > > > > > forgotten the details. Thanks for the link.
> > > > > > The bottom line seems to be that in order to show that many 'markers'
> > > > > > the solution is to create tiles/custom overlay and detect clicks to
> > > > > > show popups. Makes sense, I'll give it a try. More work than just
> > > > > > throwing a couple thousand markers at the API, but worth it.
On Wed, Sep 30, 2009 at 8:16 AM, Shaun <shaun.hus...@gmail.com> wrote:
> Hello Abyss,
> Got my clustering working better if anyone is interested also solved
> my issue with having lines that are greater than 32768 by making a
> short line interpolation algorithm to cut it into segments and draw
> lines for each segment, if the segmentation is necessary it draws from
> the start point the first segment then it skips a bunch of space in
> between and then draws the last segment because generally if you're
> zoomed in and not near a marker you don't care about the middle of the
> path you just want to see it leaving (or coming into) the marker and
> going in some direction off the screen this greatly reduced the amount
> of drawing otherwise it was drawing lines for all the elements that
> weren't on screen, although it still does that at least it only draws
> three lines rather than 10 for each one, I'll probably try to
> eliminate the drawing of these lines completely if neither endpoint is
> in view. Either way it's running pretty well with 1000 markers and
> 1000 lines, lots of wacky hacky heuristics, but it is working as good
> if not better than the first map I tried, using the built-in markers
> and overlays, with 8 markers and 5 lines. I lost my little animated
> circle but I think overall it's worth it plus I think I can bring that
> back for the roll over hover display since only one of these is on
> screen at a time and it doesn't update position while dragging.
> Big Thanks to the community for all your help and support, for a tech
> community everyone playing with Google maps seems really
> philanthropic, not nearly as many downers as most forums,
> Shaun
> Test project to show the problems with really long lines, to see the
> issue up the numeric stepper by 1 to 32768 then slide the vertical
> slider (either slider will cause the error but I'm sure this does it)
> to something greater than 32768 and all of a sudden the line becomes
> this huge shape, step the numeric stepper down and tweak the slider to
> redraw and you'll see it works fine because now it's cutting it up
> into lines that are no longer than 32767 hitting that last bit seems
> to be really messing things up, that's all AS3 related though not a
> map problem.
> http://www.shaunhusain.com/LineLengthTest/ > Test project to show a junkload of light markers, make them
> interactive, draw lines between them, and cluster points that can be
> zoomed in on by clicking the cluster...sweet.
> http://www.shaunhusain.com/GoogleMapManyMarker/
> On Sep 25, 2:34 pm, Shaun <shaun.hus...@gmail.com> wrote:
>> Hi peoples,
>> Sorry to bring this one back up after such a long time (3 days is
>> forever in forums :) but I just wanted to say thanks to Jonathan I was
>> struggling some with getting lots of markers (around 1000) to show up
>> on the map without getting seriously bogged down. I tried a few
>> different methods, initially I tried doing about a 1/4 of what you did
>> by using one overlay but still having a sprite for each asset so that
>> I could put listeners on them. Unfortunately this results in an
>> unbearable amount of latency when moving the map around although it
>> was a simple way to get listeners and more markers than I could
>> before, also this way the sprites can do some cool animation type
>> things pretty easily, whereas my next try I lost the ability to easily
>> animate the sprites but gained significantly in terms of
>> responsiveness. The next method I tried was using the same type of
>> thing as is explained here where I'm just doing the drawing for each
>> one in the overlay and getting rid of the sprites, this worked great,
>> dragging was extremely responsive however I had completely lost my
>> interactivity as you can't add a listener to the pixels you're
>> drawing, was looking into how 3d apps do hit detection, explained herehttp://michaelvandaniker.com/blog/but I just couldn't figure out how
>> that kind of hit detection works quickly enough (and am still not sure
>> it's a viable solution since the markers do need to move quite a bit)
>> so I started implementing you're idea posted here about using the
>> distance on mouse over unfortunately I didn't split my items into
>> sectors (just seemed complicated although yours does perform
>> phenomenally) so I run through every marker in the viewport. Anyways
>> I decided I should post what I've got for others to benefit from since
>> I benefited so much from you're post, can't thank you enough and it
>> came at the perfect time the 23rd is when I started looking into how
>> to deal with large groups of markers. In my implementation I pare out
>> the markers that are not going to appear in the current view port then
>> I cluster the markers that are in view using a clusterer I found out
>> on the internets (thanks to Kelvin Luck for that check it out herehttp://www.appelsiini.net/2008/11/introduction-to-marker-clustering-w...)
>> and here's what I ended up with (right click to view source):
>> http://www.shaunhusain.com/GoogleMapManyMarker/http://www.shaunhusain... >> (just found out context menu doesn't seem to work on top of the map
>> [for right click view source], might be due to something in my
>> implementation but maybe this is what another post I replied to was
>> referring to about the ContextMenu)
>> it's not perfect still acts a bit buggy in terms of the clustering
>> when panning because it needs to include any new markers that have
>> come into the view and re-cluster and depending on the pan position
>> the local position of the markers changes some so when you're zoomed
>> out really far and almost everything is a cluster there's a good
>> chance that panning will cause the markers to jump to different
>> clusters. My version isn't 100% clean but figured it still might be
>> useful for others to learn some of these concepts from and isn't quite
>> as complicated as Jonathan's class (not a bad thing since it works so
>> damn well but hard for beginners/intermediate programmers to
>> understand)
>> Thanks again,
>> Shaun
>> On Sep 22, 4:52 pm, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
>> > Hey Kevin,
>> > I didn't build it into the class but here is a function you can add
>> > and use to display the data after the markers are registered:
>> > This function will stop all current rendering, and the force all the
>> > tiles to re-render. If you're loading from a server I would recommend
>> > disabling the rendering under loadTile until your data is loaded. This
>> > class is mostly useful for visualization, it really depends on what
>> > you're doing. If you're interested in implementing interactivity, the
>> > most efficient way would be to get the mouse latlng, then grab the
>> > closest index sector and flip through it figuring out which marker is
>> > closest to the mouse latlng. This way you would not have to flip
>> > through all 100k markers trying to find the right marker.
>> > On Sep 22, 5:24 pm, Kevin Macdonald <kevin.macdon...@pentura.ca>
>> > wrote:
>> > > Jonathan,
>> > > That's a lot of markers.
>> > > Do you have any thoughts about how to populate the map with 100,000
>> > > records fetched asynchronously from a server?
>> > > On Sep 22, 4:59 pm, Your Mapper <schnue...@gmail.com> wrote:
>> > > > Jonathan,
>> > > > That's a terrific example, almost comical. And it shows the distortion
>> > > > of the map projection with a lower density of points at the north and
>> > > > south pole areas.
>> > > > I downloaded the source code too. I have not used Flex nor do I have
>> > > > it installed (I only have CS3 Web which has Flash), but did try to
>> > > > open the files in a new project in the Flex 60-day demo version. It's
>> > > > going to take a bit more time on my end to learn the new environment
>> > > > and decide if Flex is worth it, but it's definitely promising.
>> > > > Would it be possible to build a generic Flex app like the one you
>> > > > made, where one could pass in a 'lightweight' JSON or XML file that
>> > > > just had 100K location ID, lat, lon, and marker color fields, and the
>> > > > app could create the map on the fly from that data? Even better would
>> > > > be a Name field that could show on hover, and a link to a dynamic web
>> > > > page based on the ID field.
>> > > > Anyway, maybe just a rhetorical brainstorming question there. Thanks,
>> > > > Michael
>> > > > On Sep 22, 11:39 am, Jonathan Wagner <m...@jonathanwagner.ca> wrote:
>> > > > > So I wanted a challenge and actually built a class for handling large
>> > > > > amounts of markers, I have included the demo and source at:
>> > > > > It does not include interactivity, but it could be built in, in the
>> > > > > example I used 100k markers distributed randomly through the world, I
>> > > > > have better explanations of everything in the class. Hope this helps
>> > > > > someone.
>> > > > > Jonathan
>> > > > > On Sep 21, 9:53 am, Your Mapper <schnue...@gmail.com> wrote:
>> > > > > > Thanks for the great responses Pamela and Jonathan. Just what I was
>> > > > > > looking for.
>> > > > > > Pamela, I had seen your presentation online a few months ago but had
>> > > > > > forgotten
how can i read an XML with 10k markers...?
i want to show just groups of this 10k, selecting it with a combobox,
but when is loading the app, i have troubles..
any example of an app wich read an xml with 10k markers, but show no
more than 2k at once??
help please
For this, because of the size of the XML file, you will likely have to
load the XML file data into a database, then grab the relevant data as
KML or as API markers on the fly. Although I am speaking from a
standard Google Maps API perspective, and not the Flash API, which
might be able to load all the data at once and display it as needed
through flash as was done in these other examples.
On Oct 26, 6:08 pm, Leonardo Vallejos <leovalle...@gmail.com> wrote:
> how can i read an XML with 10k markers...?
> i want to show just groups of this 10k, selecting it with a combobox,
> but when is loading the app, i have troubles..
> any example of an app wich read an xml with 10k markers, but show no
> more than 2k at once??
> help please
but my markers are controlled by a filter pannel (checkboxes),
comboboxes and have a counter for the showing markers. So i can't to
use markerCluseterer or like that..
In that case the best thing to do is to use Javascript AJAX calls to
get data on the fly from the database, and return the total marker
count with the markers. Similar concept for if you are using the
Flash API, passing the right parameters in with Javascript as the user
clicks or refreshed.
Hey Leonardo,
A few things, I don't understand exactly what your problem is. I
think you were saying you can't use the clusterer because you're
changing your data set(or need to count them or something), but that
isn't true even if the data set changes dynamically you just need to
re-cluster the markers when the change occurs. Insofar as just
reading in the data from the XML file you can use e4x to parse the
nodes you want into an ArrayCollection, then to do the radio buttons
you can set different filterFunctions to the array collection
depending on which radio is selected, for check boxes have one
function that checks if a check box is selected and the item being
evaluated correlate.
so something along these lines
var ac:ArrayCollection=new ArrayCollection();
ac = xml..markerNodeName;
/*This filter function could be changed in a click handler for the
radio button groups if each of them is disjoint*/
ac.filterFunction=myFilterFunction;
private function myFilterFunction(object:Object):Boolean
{
if(someCB.selected)
{
if(object.someProperty=="someValue")
return true;
}
return false;
}
Then put a change listener on the checkboxes and have them call a
method that does
ac.refresh()
myMap.clearOverlays();
addMarkersBasedOnAC();
this will force it to use the filter function passing each element of
the array to it and if it returns false it won't "show" that element
in the ac, the element won't be actually removed and if you set
filterFunction=null then call refresh you'll have the original
collection (for example if they hit a select all filters button or
something).
If you need more help explain your problem in more detail, what does
"i have troubles.. " mean, I mean I'm troubled too, but that's going
to take years of psycho therapy, and I don't have the time or the
money for it :). Is the trouble that your app locks up when the
markers are loading from XML, is the XML local or being fetched from a
server, do you get any sort of errors or is it just terribly slow
performance?
BTW the marker clusterer just tries to find if markers are within some
radius of one another and then it puts them into a single object if
they are and returns you a array of arrays where each element of the
array has an array of the original markers that make up the cluster so
to count the total number of markers you could just check the length
of each cluster array and add them up, or potentially just operate on
the original data set I believe if you use a filter function as I
described length will report the correct value of the filtered down
collection.
Shaun
On Oct 28, 11:50 am, Your Mapper <schnue...@gmail.com> wrote:
> In that case the best thing to do is to use Javascript AJAX calls to
> get data on the fly from the database, and return the total marker
> count with the markers. Similar concept for if you are using the
> Flash API, passing the right parameters in with Javascript as the user
> clicks or refreshed.
Sahun, thank for answer. My system read a local XML and add the nodes to an ArrayCollection. Then i can show or hide the markers according the user selection in a filter pannel with checkboxes and a combobox. You can see how is working right now in this url: http://www.angelessalsa.cl/google/demo.html My problem is that demo it's working with 448 markers. I try it succesfully too with 1200 markers (though it worked a bit slower with the filters, but still acceptable) and I never work with more than that, but the XML file I read should be approximately 8 thousand markers.
So that's my problem, when i load the XML with the 8k markers into my IDE (eclipse), the process it's so slow and when i execute the system with that XML, the app don't load the markers and gives me the following error: "Error #1502: El tiempo de ejecución del script ha superado el tiempo de espera predeterminado de 15 segundos." ("Error #1502: The script execution time has exceeded the default timeout of 15 seconds").
