D3 with SQL Server

[Glenn Maxwell]

Dear Friends,

I am looking for expert advice/help for my project. To summarize, I am looking to populate d3 charts with data in SQL server. I also have controls on my UI(combobox, radio button,checkbox) to filter the data.

I need some advice regarding:

1. I am aware that I will need to convert data from db into json format to populatethe chart. I want to understand, What language/platform should I use to create an interface between html and database? 

2. Should I go with d3.js or Highcharts?

Thanks,

Glenn

[Tito]

Hello Glenn

I am doing this exact same thing. What has worked for me is node.js + express.js  module. 

I have charts with d3,js getting data from mysql, mongodb and mssql.

[Drew Winget]

There are too many choices of how to translate http requests into database queries and back again. Better to just choose one and go with it. Tito's suggestion is a good one that has a great community behind it. The best way to get help doing so is to go on IRC and ask there for help with express and node.

As for d3/highcharts: although people often use them in similar ways, they are not really comparable except for a specific use case. If highcharts doesn't have a certain chart in its library, you won't be able to create the chart you're imagining. In d3, often simple charts can require a lot of seemingly meaningless set-up, so it really depends on what you want to do. If you just need some charts, and you don't need any interactivity beyond some basic filtering and tooltips, especially if you don't need to transition between charts, go with highcharts. If you need to do anything original, learn/use d3.

What are you trying to accomplish? Something like this budget breakdown would be easier to code from scratch than with highcharts, but it very easy and quick to do something similar with d3 (assuming you know how to use it well). It really helped me understand D3 to realise that it is not a charting library, or even a "data visualisation" library, but rather, an implementation of the "Grammar of Graphics", an epiphanic monograph on how conceptual drawings can be understood grammatically.

--
You received this message because you are subscribed to the Google Groups "d3-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to d3-js+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Glenn Maxwell]

Tito, that's great!

Can you share your code snippets for to me to take a look?

[Tito]

Glenn,

Try this 

https://github.com/yosiasz/med3

Put something together for you. You are on your own now son!

[Glenn Maxwell]

Drew - Thanks for your note. As of now I am just trying to implement basic charts with data in SQL server. Highcharts can fulfill the need.

I am new to javascript and learning it by doing. Because I am good at Python I wanted to use Python as my middle layer to transform Server data into Json. The couple of challenges I foresee are:

1. How to I translate and parse user selections to Python.
2. How do I Parse JSON to chart.

Best,

Glenn

[Christoph Pingel]

Hi Glenn,

I'm also good at Python, and I use d3.js in a huge ZOPE/Plone project. But this is something beyond the scope that you are probably looking for.

If you want to build your 'middle layer' in Python, I'd recommend to use a minimal Python framework like one of these:

http://codecondo.com/14-minimal-web-frameworks-for-python/

Of course you could also use django (https://www.djangoproject.com/) which is more complete and maybe also too much. 

In either case, you will need to download and use a database adapter to talk to your database. And you will need to learn how to handle data from form submissions, but this is usually trivial since the framework will convert the form data into some Python dict or list for you. 

HTH,

best regards,

Christoph

[Glenn Maxwell]

That's interesting Christoph. Is there a example on the web which i can follow?

Regards

[Nigel Legg]

Hi Glenn, I use python / django / postgres, though flask - in the list Christoph linked to - might be easier and quicker to pick up.  Both have straightforward tutorials which will get you started, and django has an active email list. 

Cheers, Nigel

mobile 07758 665575

skype: nigellegg

[Christoph Pingel]

You could use pyramid, which is minimal and extremely well documented. 

Here's a tutorial for a minimal web app, this could be a starting point. Explanations are thorough.

http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/firstapp.html

regards,

Christoph

[Drew Winget]

​If you just need to get things out of the database, and serve some simple pages with Python, I would recommend flask​.

[Drew Winget]

You shouldn't really need anything beyond what appears in the quickstart guide to establish the database connection you need to accomplish what youre looking for on the front end. APIs are the way to go. In my experience, frameworks that try to generate every string of HTML make it very difficult to separate the front end from the backend.

[Drew Winget]

http://flask.pocoo.org/docs/0.10/quickstart/

[Glenn Maxwell]

Chris/Drew,

Please let me know if I can accomplish the following with the Flask/Pyramid.

1. Pass parameters from javascript to sql query.

2. Render databse values on d3js charts on html.

Please also let me know:

1. Which editor is needed for proramming?

2. How much development effort do you expect?

3. Will the solution be Robust and Scale-able ?

Many Thanks,

Glenn

[matthew lange]

Christoph, interested to learn more about your experience with d3 in zope/Plone. Is your cms also querying SQL server? Do you have links?

[Christoph Pingel]

Matthew, there's nothing special with our usage of d3. We use it mostly for visualization in the context of quality management. 

The pattern is very common. A d3 visualization is called. 

Within the viz, there are usually lines like this:

d3.json("@@failurenet", function(json) {

  var causes_nodes = cluster.nodes(json.failurecauses);

  var effects_nodes = cluster.nodes(json.failureeffect);

This is the start of the creation of the actual visualization.

The interesting 'ZOPE relevant' part here is the URL of the browser view, '@@failurenet'. This is a browser view that returns JSON data. In our case, this is rendered in real time according to relationsships between content items that are modeled using ZOPE/Archetypes reference engine internally, but it could as well be the result of a SQL query. 

Since ZOPE uses URL traversal, the view URL is just appended to the current URL, like /a/b/@@failurenet or /a/b/c/@@failurenet and returns the JSON appropriate for the object at that URL in the hierarchy.

This is perhaps a little bit out of the scope of a d3 discussion, so if you need information about implementing something like this in ZOPE/Plone, you can contact me off list.

best regards

Christoph

[Christoph Pingel]

Hi Glenn,

Drew is right, Flask is probably most suited for your project.

2016-04-12 6:04 GMT+02:00 Glenn Maxwell <gm7...@gmail.com>:

Chris/Drew,

Please let me know if I can accomplish the following with the Flask/Pyramid.

1. Pass parameters from javascript to sql query.

Yes.

 

2. Render databse values on d3js charts on html.

Yes.

 

Please also let me know:

1. Which editor is needed for proramming?

 

This depends on you platform. I'm on Mac OS, and people use Sublime Text, BBEdit, PyCharm, etc.

For a project of this size, you don't need a IDE, any text editor will do.

 

2. How much development effort do you expect?

 

The question is too vague to give a serious answer, but as drew said, there's not much more required than following a 'getting started' tutorial. 

 

3. Will the solution be Robust and Scale-able ?

 

This depends on your implementation. I don't know flask, but in the case of pyramid the necessary conditions for robustness and scaleability are met.

May I suspect from your questions that you are not familiar with server side programming?

One recommendation would be to read the tutorials/documentation of the framework to get a clear picture of the cycle from a request to a response. And how the code you write (typically called a view or view callable) fits into this picture. 

best,

Christoph

[Phuoc Do]

My team at vida.io added support for SQL data source. You can program with d3.js and read data from SQL database. For more details:

http://blog.vida.io/2016/11/06/connect-d3-dot-js-visualization-with-sql-database/