DC.JS Crossfilter and Leaflet

559 views
Skip to first unread message

Sim re

unread,
Sep 24, 2018, 2:14:46 AM9/24/18
to dc-js user group
Im trying to display my data on openstreet map with leaflet.

Im always getting these errors codes. 

Uncaught dc.errors.InvalidStateException ; Uncaught ReferenceError: _ is not defined at (index):266 at Object.<anonymous> (d3.js:1996) at Object.event (d3.js:504) at XMLHttpRequest.respond (d3.js:1949)

Anyone can help me to figure out what I am doing wrong?

Ive joined my code to this message 



index.html

Gordon Woodhull

unread,
Sep 24, 2018, 11:38:28 AM9/24/18
to dc.js user group
I forked the block you posted on SO (there's really no need to cross-post your questions as it just makes everything more confusing - the same people are here and on SO I promise you).


I fixed the d3 and dc depedencies, and added the data you sent by email.

The _ error is because wherever you got that code, it used lodash/underscore (clever names for that symbol). But you don't really need those libraries for .each(...), the following works just as well:

       allDim.top(Infinity).forEach(function (d) { ... })

You also had "map" when your map is called "mymap", and you initialized "loc" to the ID, and the ID obviously doesn't have fields like lat and lng

Once I shrunk the charts and expanded the iframe, the charts are visible and working in bl.ocks.org (too big for blockbuilder)]



There are still a lot of problems here. If you're just starting out with JavaScript, this is a pretty big mountain to start on! You're going to find that you'll need to add yet more code in order to get filtering to work and so on.

I definitely recommend using dc.leaflet.js which has already solved the dc.js - leaflet binding problem. dc.leaflet.js 0.3.* is compatible with dc.js 2.* and d3 v3 (like you are using here). I'm working on releasing a new version compatible with d3v5 and dc 3.*


 
--
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dc-js-user-group/ce64df82-e33f-45b0-9503-c9ec741d5383%40googlegroups.com.
<index.html>

Sim re

unread,
Oct 5, 2018, 6:27:22 AM10/5/18
to dc-js user group
Hi Gordon, 

I think I might need to get back to the basic :s. Im trying to display a part of your code that you send me the link "Markers with no clustering and selection by area". Im trying to display the dataset "demo2" without any success. 


What I am doing wrong here? 

Error : 


  1. HTML1524: Invalid HTML5 DOCTYPE. Consider using the interoperable form "!DOCTYPE html>".
    index%20-%20test%20markers%20with%20no%20clustering.html (1,1)
  2. HTML1521: Unexpected "/body>" or end of file. All open elements should be closed before the end of the document.
    index%20-%20test%20markers%20with%20no%20clustering.html (107,1)
  3. You are using d3.schemeCategory20c, which has been removed in D3v5. See the explanation at https://github.com/d3/d3/blob/master/CHANGES.md#changes-in-d3-50. DC is using it for backward compatibility, however it will be changed in DCv3.1. You can change it by calling dc.config.defaultColors(newScheme). See https://github.com/d3/d3-scale-chromatic for some alternatives.

Gordon Woodhull

unread,
Oct 5, 2018, 2:03:16 PM10/5/18
to dc.js user group
Aside from the dependencies all being broken, it looks like you were trying to apply the charts to the wrong data source - these charts were built to work with demo1.tsv but you were pulling in demo2.csv instead.


But rather than asking me to debug random examples you've pulled off the web, what are you really trying to do?


-- 
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.

Sim re

unread,
Oct 10, 2018, 12:44:19 AM10/10/18
to dc-js user group
Hi Gordon, 

Thanks for all your help once again! Im a on a long way learning path. 

As you can see, Ive joined my own datasets.... 

I dont get it why its not displaying and not refreshing on the map... 

What I dont get here? 

Simon


On Monday, September 24, 2018 at 1:14:46 PM UTC+7, Sim re wrote:

Sim re

unread,
Oct 10, 2018, 12:46:03 AM10/10/18
to dc-js user group

On Wednesday, October 10, 2018 at 11:44:19 AM UTC+7, Sim re wrote:
Hi Gordon, 

Thanks for all your help once again! Im a on a long way learning path. 

As you can see, Ive joined my own datasets.... 

I dont get it why its not displaying and not refreshing on the map... 

What I dont get here? 

Sim re

unread,
Oct 10, 2018, 12:46:13 AM10/10/18
to dc-js user group
On Wednesday, October 10, 2018 at 11:44:19 AM UTC+7, Sim re wrote:
Hi Gordon, 

Thanks for all your help once again! Im a on a long way learning path. 

As you can see, Ive joined my own datasets.... 

I dont get it why its not displaying and not refreshing on the map... 

What I dont get here? 

Gordon Woodhull

unread,
Oct 10, 2018, 12:46:18 AM10/10/18
to dc-js-us...@googlegroups.com
Did you mean to post a new link?
--
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.

Gordon Woodhull

unread,
Oct 10, 2018, 1:00:19 AM10/10/18
to dc.js user group
One peril of geospatial stuff is that it's ambiguous whether latitude or longitude should be first.

Leaflet expects "latlong" and that's what people tend to talk about, but that's strange because it's Y,X.

Reversing your coordinates causes markers to appear on the map, with filtering!

                .locationAccessor(d => d.key.split(',').reverse())



-- 
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.

Sim re

unread,
Oct 12, 2018, 6:07:34 AM10/12/18
to dc-js user group
Thanks! I will say its a bit confusing and non sense as a geographer! Well I deal with it ;) 

I have another question... Is it possible to have markers of different colors which follow the ones of my piechart?


Or if more simple:

2018 : Blue 
never covered:  black
other periods: red

Thanks 

On Monday, September 24, 2018 at 1:14:46 PM UTC+7, Sim re wrote:

Gordon Woodhull

unread,
Oct 12, 2018, 7:41:07 AM10/12/18
to dc.js user group
HI Simon,

Ah! If you're a geographer, you'll probably start learning more and more Leaflet as you go along. Amazing library.

In this case, the markers are just images which Leaflet calls "Icons".

In the dc.js coding style, everything is a function which encodes attributes of the data with visual attributes. So we have the method markerChart.icon() which gets/sets an accessor or encoding function. 

Actually the tricky part here is the crossfilter reduction. When you just use group.reduceCount() (the default), the only data that is available is key and value, where the value is an integer. So we have no data to color on available in the aggregation.

Since this is an important use-case, I've changed the demo page to color the markers.

In that data, the field we want to color on is "type", which has values like "solar", "wind", etc. It's probably a different field for you.

The reduction looks like:

      var facilitiesGroup = facilities.group().reduce(
          function(p, v) {
              p.type = v.type;
              ++p.count;
              return p;
          },
          function(p, v) {
              --p.count;
              return p;
          },
          function() {
              return {count: 0};
          }
      );

The difference between this and reduceCount is that instead of value being an integer, it's now an object with two fields, "type" and "count". We grab "type" from the first row we see (which only makes sense if there is only one row per bin, as is the case here). And we increment and decrement "count" when a row is added or removed.

Since the data format changed, we're going to need to tell the markerChart how to read a value, so that it can remove markers when they go to zero:

          .valueAccessor(d => d.value.count)

And now here's the fun part, assigning icons. In a quick search, the top library for "leaflet marker color" is this one:


This is a nice simple solution because you don't have to install any plugins or anything. You just change the iconUrls based on the data, to point at the CDN for this library:


          .icon(function(d) {
              var iconUrl;
              switch(d.value.type) {
              case 'solar':
                  break;
              case 'hydro':
                  break;
              case 'wind':
                  break;
              case 'biomass':
                  break;
              case 'gas_waste':
                  break;
              case 'hydro_waste':
                  break;
              default:
                  console.log('unk', d.value.type);
                  return new L.Icon.Default();
              }
              return new L.Icon({
                  iconUrl: iconUrl,
              });


I think there are more complex plugins which will allow changing the color dynamically, instead of just having certain colors. But I haven't looked at that today.

Hopefully this is enough to get you started!

Cheers,
Gordon


-- 
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.
Message has been deleted

Sim re

unread,
Oct 18, 2018, 2:53:21 AM10/18/18
to dc-js user group
Hi Gordon, 

Thanks it did work perfectly for my needs! 

I'm discovering leaflet and yes love it! Amazing stuff that I din't know it exist! Thanks again! If one day you come to Vietnam ill buy you a beer haha! 

I still have more questions for you ;)! 

This is my last update of the 'coverage' map: http://blockbuilder.org/renauld94/9f8da3820d18d4d8b863bdde969ef552

I'm trying to add a Pie Chart which will be filtering on the city.... I'm getting this error : 2main.js:48 Uncaught TypeError: Cannot read property 'length' of null

What is wrong here?


Second question: 

I figured out that it was easier for me to pre-build all my widgets seperatly to integrate them in my "final dashboard". 




Multiple Layers Example: search markers in multiple layers:


Labelling Geojson :


Layer Geojson selector




Ect...


Whats the best practice in order to build my final dashboard integrating all my widgets? Do you put everything in one HTML code? Diffrent folder with libraries? Does my application will have file size limits? 


Simon
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-group+unsub...@googlegroups.com.

Gordon Woodhull

unread,
Oct 18, 2018, 2:03:25 PM10/18/18
to dc.js user group
Hi Simon,

I'm not totally clear on your questions today.

I don't see the errors you are describing in this message or in the previous message. Did you fix the blocks already? They look spectacular!

Do you mean that you want to integrate all these maps together - have choropleth and marker layers all on one map? I'm not sure how to do that, but I'm sure it is possible.

Best practices are I guess whatever you feel best about maintaining. I guess I would try to at least break stuff out into functions so that I can read it, but that's really up to you. 

File size limits depend on where you plan to host this. I use github.io for some stuff and the limit there is 1GB per repo and 100MB per file - not sure if this is the same for gists (which underly blocks) as for repos.

Cheers,
Gordon


To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-gro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dc-js-user-group/7a012e87-07e9-42cd-b488-fb579c41e3e5%40googlegroups.com.

Message has been deleted

Sim re

unread,
Oct 18, 2018, 10:11:33 PM10/18/18
to dc-js user group
Hi Gordon,

Thanks for your advices! 


Im trying to display 2 pie charts. One for my coverage which is ok, and the second one which is the ''city''.... The second pie chart do not display and dont know why...

And ''Do you mean that you want to integrate all these maps together - have choropleth and marker layers all on one map? I'm not sure how to do that, but I'm sure it is possible'' Yes, thats what i want to do.

Cheers 


Simon
Hi Simon,

To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-group+unsubscribe...@googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "dc-js user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dc-js-user-group+unsub...@googlegroups.com.

Sim re

unread,
Oct 19, 2018, 4:21:25 AM10/19/18
to dc-js user group
I also have another major problem. 


At the end, my main goal is to have a dashboard for one country. 

It means that I want to filter down from my country to the  -> Provinces -> Cities -> District -> Ward  .... 

So Im gonna have endless charts Cities, Districts and Wards names... 

How i can overcome this problem? 

I was thinking

A ""pre-select scrolling"" menu on the provinces and cites....  After appliying only the crossfilter on district and wards ? 

Ive found one your fiddle : 

Could this work for my case? Or better options available? 

Thanks 
Reply all
Reply to author
Forward
0 new messages