will D3.js plugin be upgraded to version 4.x ?

[Silverfox]

Hello, 

I discovered TiddlyWiki few months ago and really enjoy it.

I enjoy it so much that, even f I did not program anything for around 20 years, I'd like to code again to realize an idea based on data visualization.

For that I started to use the D3JS official plugin, but I recently realized it is based on former V3.x version while V4.x is now the official D3JS one with nice new features I would like to use.

My skills are clearly limited... do you think the D3JS plugin will be updated for V4.x ? 

Is it complex to do ? Are there some recommendations / tutorial somewhere ?

Thank you for your help 

[BurningTreeC]

Hi @Silverfox

D3 seems to be easy to update

this is the repository where the source code is:

https://github.com/d3/d3

under "releases" there's the latest versio 5.0.0 to download

this is the tiddlywiki repository

https://github.com/Jermolene/TiddlyWiki5

and here's the d3 tiddlywiki plugin:

https://github.com/Jermolene/TiddlyWiki5/tree/master/plugins/tiddlywiki/d3/files


we could create a so-called "Pull request", updating the plugin with the latest version 5.0.0
and checking if the "cloudwidget" and "barwidget" need adjustments

Simon

[@TiddlyTweeter]

D3 (now in v5) is a great bit of gear. The TW plugin is really currently only a proof of concept.

Personally I'd use it extensively to make Word Clouds IF it were as bit easier to integrate into TiddlyWiki. To achieve that we need someone tech. happy to do the needed work. Especially methods to assemble Data to feed to it in an appropriate format.

Best wishes
Josiah

[BurningTreeC]

D3 (now in v4) is a great bit of gear. The TW plugin is really currently only a proof of concept.

now in v5
 

Personally I'd use it extensively to make Word Clouds IF it were as bit easier to integrate into TiddlyWiki. To achieve that we need someone tech. happy to do the needed work. Especially methods to assemble Data to feed to it in an appropriate format.

the current plugin as you mentioned comes with two widgets which are just demonstrations

updating d3js to v5 is the easy part

the cloudwidget and barwidget need work to be compatible
some sensible widgets should be created that accept tiddlywiki filters for visually showing data and connections


if we can make the two existing widgets work with the newest d3 or if we create two widgets that work better in the tw context, I think we can make that PR

[Silverfox]

I'm currently learning to build some other D3 widgets than the cloud and the bar, but as my skills are limited and there is no documentation, i do it mainly through trial and errors ...it takes time, but I think I can do it.

I'll publish my results when the results will be good enough.

But, for me, updating D3 to V5 looks much more scary, I don't feel I can do it at all...

[Jed Carty]

In my experience Three.js is easier to use than d3.js. Someone made a plugin for three.js but it would also need to be updated.

[BurningTreeC]

@Jed Carty

I think you're referring to http://rboue.tiddlyspot.com/

three.js is very nice, I used it to render molecules in 3d within tw

the purpose of d3.js appears to be a bit different - I think, representing data in a statistical manner, graphs, connections and so on

three.js seems to be a rendering library for 3d rendering of data, I guess you can also make graphs similar to 3d.js but 3d.js may be more appropriate because it does the things right out of the box?

What do you think?

@Silverfox

I could make a tiddlyspot page with an updated 3d.js plugin, without the cloudwidget and barwidget working
The widgets then would need to be adopted / it would need to figure out how to access the updated library correctly

If you're interested, I'll put that page online, you could drag the plugin to an empty wiki from there and try figuring out

BTC

[Silverfox]

@BurningTreeC 

Oh yes that would be great, I will then adapt the 2 demo widget and can add some others.

Thanks

 

[BurningTreeC]

@BurningTreeC 

Oh yes that would be great, I will then adapt the 2 demo widget and can add some others.

Thanks

 

@Silverfox

I've put it here: http://d3js500.tiddlyspot.com/

all the best,
Simon

[Silverfox]

Than you so much, l'll play with it during the next few days, and I'll share the result

Nicolas

[Silverfox]

I made few progress on barWidget, by correcting few details in the D3 code but still have a bug.

The (temporary) result is here : http://d3jsv5trial.tiddlyspot.com

It looks to be linked with the function transitionGrouped() ... 

When activated, all the x value are zero (or undefined)

I still need to investigate... if you have some ideas they are more than welcome

Nicolas

[BurningTreeC]

looks already better than before :)

the original barwidget was (maybe) made with the old API in mind and therefor now it's a bit complicated to make it work the way it's structured
but I don't know, I haven't looked at the API more deeply

What I've seen is that there are examples on d3js.org that use the new API, maybe it helps starting with such an example and working the barwidget into it

I'll keep updated on this thread, I'm also interested in the d3 plugin, but currently I don't find the time to help on this
When I find the time, I'll be happy to help

Simon

[Silverfox]

@BurningTreeC 

I tried to adapt this https://bl.ocks.org/mbostock/7607535 which is native D3.js V4.

I put it in http://d3jsv5trial.tiddlyspot.com, but i face the same kind of bug than with the previous trial.

All the data items that should be spread on the tiddler, are actually all concentrated because of non-defined data...

As a consequence, I wonder if there's a bug in your D3.js V5.0 that could explain it... 

But I don't know how to investigate it...

Apart of this I try to understand how to use css stylesheet from usual D3 files into TW...

[BurningTreeC]

Hi, is the svg._groups[0][0] correct?
I changed that in the barwidget, too , but just because it made an error disappear. I was sceptical if it was the right way..

I would investigate that

BTC

[BurningTreeC]

You can try another d3js version if you want to check if it's a bug

To do so you can head over to the github repository, go to releases and download a zipped release
Unzip it, look for the d3.min.js file and copy its content to the d3.js tiddler

[Silverfox]

Hi, is the svg._groups[0][0] correct?
I changed that in the barwidget, too , but just because it made an error disappear. I was sceptical if it was the right way..

I would investigate that

BTC

Yes it looks OK 

[Silverfox]

I'm not sure to understand (sorry, I'm a beginner)... 

you mean that you just need to copy/paste the content of d3.min.js in $:/plugins/tiddlywiki/d3/d3.js ?

there is no adaptation / modification ?

[BurningTreeC]

I'm not sure to understand (sorry, I'm a beginner)... 

you mean that you just need to copy/paste the content of d3.min.js in $:/plugins/tiddlywiki/d3/d3.js ?

there is no adaptation / modification ?

Yes you can simply copy it without modifying it

[BurningTreeC]

@Silverfox

I took a look at the chart in your demo with rightclick on the chart - inspect
there's a tab called "inspector" ... it shows all the html elements
If you find the chart elements, you see that the widget doesn't calculate correct values for the circle radius and the translate offsets

If I change those values temporarily within the inspector, the circles are shown
So the problem must be somewhere how the values are calculated

[BurningTreeC]

maybe it's this line in the widget

var diameter = +svg.attr("width");

[BurningTreeC]

maybe it's this line in the widget

var diameter = +svg.attr("width");

ok sorry ;) that's in the demo, too

I'm no javascript genius

[Silverfox]

Thank you for your support ... I learn a lot thanks to you...
I'll investigate and let you know

[Silverfox]

I have updated http://d3jsv5trial.tiddlyspot.com

I could implement the widget for Circle packing (still need to understand how to improve with CSS style rather than hardcoding), but I failed, so far, to implement Zoomable circle packing ( https://bl.ocks.org/mbostock/7607535).

I cannot understand why it works when using D3 script in an HTML page, and not on the D3 adaptation for TW.

I think that the problem start when we use d3.hierarchy... but I may be wrong

when I console.log  this: 

var root = d3.hierarchy(root) 

.sum(function(d) { return d.size; })

.sort(function(a, b) { return b.value - a.value; })

 

I obtain:

 

we can see that x, y and r are undefined, but I don't why... and I even don't understand how they should be generated

Would that be possible that some variable would need to be initialized just when using D3 into TW, and not into regular HTML ?

Any other idea ?

Nicolas

[Silverfox]

This post to inform you that I succeeded to implement the zoomable circle packing D3 widget (for D3.js v5.0.0)

You can see it here : http://d3jsv5trial.tiddlyspot.com

Nice isn't it ?

I have added a small tutorial for dummies (like me), it may push others to further combine D3.js and TW

Many thanks @BurningTreeC, I could have not done it without you

And I go on with my personal project...

[Jan]

Great!
I want more!!!
More displays like this and a little more explanation: Where does the data come from.
I would love to use something like this as a table of content: It would be great if the captions of the bubbles could open links (to Tiddlers)

Again: Great work
-Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/b070e47f-2584-4ed2-940c-47b2b2b89934%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[TonyM]

Silverfox,

I see great potential for this because TiddlyWiki can be used to represent many things, and new visualisations of what ever your tiddlywiki represents are valuable. This is especially true if it helps navigate real content.

Looking good.

Inclusions such as a field "bubble-caption" for custom titles when displayed as a bubble, or using the color field or icon specified in the tiddler representing the piece of data would be nice and quite simple to use?

Regards

Tony

Tony

[Silverfox]

This is exactly what I would like to do... 

In this case the data are here : http://d3jsv5trial.tiddlyspot.com/#flare.json, it's an example of nested data regularly used by people working on D3.js, but I would like this data to be generated by the content of TiddlyWiki

I still need to learn (and be helped) to reach it...

Nicolas

[BurningTreeC]

@Nicolas

You made great progress, awesome!

I've seen that one can use this to pass a filter to a widget and get the tiddler titles:

var tiddlers = $tw.wiki.filterTiddlers(this.myTiddlerFilter);


to get the filter:

CirclePackWidget.prototype.execute = function() {
    // Get the parameters from the attributes
    this.myDataTiddler = this.getAttribute("data");
    this.myTiddlerFilter = this.getAttribute("filter");
 };

Then you can make a json object from the tiddlers and their tags and fields to pass it to d3 afterwards

var jsondata = {};

if ($tw.utils.isArray(tiddlers)) {
for (var i=0; i < tiddlers.length; i++) {

var tiddler = $tw.wiki.getTiddler(tiddlers[i]);
console.log(tiddler);

// here we can build the jsondata from the tiddler
// var tags = tiddler.fields.tags;
// var title = tiddler.fields.title;
// var allTiddlerFields = tiddler.fields;

}

------------

The jsondata can be "filled" like so:

jsondata["entry1"] = "test";

or using the title stored in a variable:

var title = tiddler.fields.title;

jsondata[title] = ...

---------------------

I'm not sure what's the best way to do this, there's also $tw.wiki.getTiddlerAsJson in $:/core/modules/wiki.js - seems more comfortable to use:

exports.getTiddlerAsJson = function(title) {
	var tiddler = this.getTiddler(title);
	if(tiddler) {
		var fields = Object.create(null);
		$tw.utils.each(tiddler.fields,function(value,name) {
			fields[name] = tiddler.getFieldString(name);
		});
		return JSON.stringify(fields);
	} else {
		return JSON.stringify({title: title});
	}
};


...


What I'm thinking about is, what the jsondata should look like to create a meaningful, expressive tiddler graph
In your CirclePack for example, which tiddler field should create a group? I think a subfilter by tag could do that



Simon

[@TiddlyTweeter]

Impressive what you have done! Good stuff.

I would like this data to be generated by the content of TiddlyWiki

Right. That is the central issue.

If you can't do that then you might just as well use an existing online version to generate the output.

I think to get the numbers inputs you gonna need more than basic TW for anything more than really simple charts. It might be worth looking at Evan's formula plugin: https://evanbalster.com/tiddlywiki/formulas_0.1.0.html to see if it can help on that.

Just thoughts
Josiah

[David Gifford]

Just want to echo everyone's compliments. I always wanted someone to do something more with d3 and expand to other visualizations, but hesitated to ask. This is really cool.

Some things to consider as you tweak it:

1) I will definitely play sometime soon with the CSS and come up with some cool color schemes!
2) I am finding that when I click a white bubble, the title does not display until I click on its blue container to zoom back out. I realize now after experimenting that the trick is to only click on the blue containers to zoom in and out to see names. Might be good to indicate that just above the visualization, for newbies like me.
3) Is there a way to increase the indent for the contents of flare.json so that the levels are more pronounced and distinguishable? Even if it means lines running over, I think it would help users as they add their data. Currently it is hard to distinguish the levels of the parents and children.

[Silverfox]

Thank you all for your comments. 

I have worked few weeks ago on a TW macro that generates a json based on the TW table of content... it works but, I'm not really happy with it, as it requires to click on a button.

Since D3 V4.x, there is a new command (https://github.com/d3/d3-hierarchy#stratify) that will allow to use data from a simpler csv source than nested json. 

I'll work on implementing it, it should helps simplifying the transfer automatically TW generated data to D3 visualisation.

I'll publish all this in the coming days... but I first would like to concentrate on the feature that will allow to navigate thanks to the D3 chart (open tiddler by clicking on a bubble)

I'll open a new post when I'll be ready

[TonyM]

Silverfox,

Thanks for moving on that, CSV will make it more accessible and also integrate with TiddlyWiki even more.

Regards

Tony 

[Silverfox]

It now works with "flat" (non nested) json and also csv tiddler, see the demo here : http://d3jsv5trial.tiddlyspot.com/#D3%20ZCP%20widget%20based%20on%20flat%20csv.

This is done by some d3 methods that can parse and stratify data...

[@TiddlyTweeter]

Good stuff!

Small point. On hover I see "Noam" mentioned in the Seth ring but I don't see that name displayed in its sub-circle?

[Silverfox]

It's just a proof of concept so far... I still need to debug, improve, customize etc, before making a plug-in

[BurningTreeC]

Hi @Silverfox ,

I'm trying to make the base d3js library ready for the new tiddlywiki release

Do you think one or two of your widgets can be included? The old widgets have been in a demo-state anyway

I would also like to ask you if I can put a copy of your "how to" for making d3 widgets to the d3 demo page

What do you think?

all the best,

BTC