Pre-ANN: PlotlyJS.jl

[Eric Forgy]

Hi everyone,

In this post from last Wednesday, Hans-Peter pointed out that plotly.js was made open source (  https://plot.ly/javascript/open-source-announcement/ ).

Anyway, I tried my hands at writing a wrapper for the JS API. There already exists Plotly.jl, but that is an API to interact with the company's server requiring login info, etc. With the opening of the JS API, the interaction between Julia and Plotly can be much simpler, I think.

In the other post, I mused that it should be straightforward to hook up Plotly.js with Juno, so here is a snapshot:

To make it interactive, I used WebSockets.jl. One neat artifact of that is you can stream the charts to multiple connections. 

Here is a silly video demo of me controlling 4 browsers (only 3 are visible) including an iPhone from Juno: https://www.youtube.com/watch?v=mWDyyfVNqP0

The same concept should work with Vega.jl in Juno.

I call this a "Pre-announcement", because it is not really ready for the wild yet, but it is far enough that if some friendlies would like to have a look and help me get it in shape, that would be more than welcome.

Currently, I am struggling a bit to get the paths correct. If I "git clone" it and run it from the REPL, then I can make it work, but if I "Pkg.clone" it, the path to the html containing the scripts is wrong. 

Here are the lines in question:

# res = Response(open(readall,Pkg.dir("Plotly","src","web","plotly.html")))
res = Response(open(readall,"web/plotly.html"))

When run from the REPL (after CD'ing to the directory), the code will find plotly.html. But if I Pkg.clone, it can't find it. It seems to find it when I use the commented Pkg.dir, but then that doesn't work when I'm trying to test before committing to Github and Pkg.checkout etc. Any ideas?

Anyway, this has been kind of fun and if others can help improve it, that would be a great learning experience for me.

Best regards,

Eric

[Eric Forgy]

Oopsies. And here is the link :D 

https://github.com/EricForgy/PlotlyJS.jl

[Eric Forgy]

A shot of all 4 browsers (including Juno - which is using the browser-plus package btw) :)

[Tony Kelman]

Try using dirname(@__FILE__) instead of Pkg.dir

[Tom Breloff]

Thanks Eric. @joshday has been working on this as well in PlotlyLocal.jl, and I'm working on a plotly backend built into Plots.jl. If you have any interest in collaborating please let me know. 

[Sisyphuss]

So the following package is no longer useful?

https://github.com/plotly/Plotly.jl

[Tom Breloff]

Much of Plotly.jl is for interacting with the cloud API (login, credentials, etc). The remainder cannot be called directly so, for my purpose, it made more sense to just build the json directly. Others will likely still find this package to be useful. 

[Tom Breloff]

Eric: I just looked through your code.  It doesn't seem like there's very much plotly-specific code in there.  I wonder if you could generalize a little so that Bokeh or Vega could use your websocket approach as well.  That seems like it has uses beyond desktop visualizations.  I also think it would be great to separate plot-creation from plot-updating so that you could use websockets to update existing plots.  Let me know if you'd like to integrate your idea with Plots.jl.

ps - I love your music choice for your video... very dramatic

[Eric Forgy]

Hi Tom,

Thanks for your note. Yeah, sure. I am definitely interested in helping where I can. I started out working on Tony's fork of Plotly.jl since he had already converted to using Requests.jl, but by the time I got the WebSockets working, I realized I didn't need any of that code and could just interact with plotly.js directly, so gave it a new name :) With PlotlyJS, I'm pretty sure I can reproduce every example from their web page already. You simply need to construct Dicts that correspond to the JSON.

I submitted a PR to plotly.js too that would allow us to create multiple axes dynamically. Currently, the div name you provide needs to already exist in the DOM. In my PR, if you supply a name that isn't in the DOM, it will create a div with that name. Not sure if they will accept it though :)

I haven't spent much time with Bokeh and Vega, but expect it should be straightforward to use the same trick with those guys. In fact, I have my own JS graphing library I spent about 6 months developing a while ago I'd like to resurrect (boring demo alert http://youtu.be/IriE1ZP-uOM ) :)

I am also interested in updating existing charts. If we just change "newPlot" to "plot", it will append subplots to the current div, but that's not what I wanted. I should follow the same naming as plotly.js though I suppose. As I said, this is a "Pre-ANN" :D

For applications beyond visualizations, I may have need in my day job for something like this to get Matlab/Julia talking (without using MEX) so ideas are welcome :)

Sent from my iPhone

[Eric Forgy]

On Tuesday, November 24, 2015 at 2:03:20 AM UTC+8, Tony Kelman wrote:

Try using dirname(@__FILE__) instead of Pkg.dir

Thanks Tony. That did the trick. I am getting closer to having a decent package now.

You should be able to 

julia>Pkg.clone("https://github.com/EricForgy/PlotlyJS.jl.git")

julia> using PlotlyJS

Listening on 0.0.0.0:8000...

Then open any browser to http://localhost:8000/plot.ly and run

julia> include(Pkg.dir("plotlyJS","test","runtests.jl"))
 

You should see this nice interactive chart:

It can now handle both Plotly.plot and Plotly.newPlot. Feedback welcome.

[Randy Zwitch]

"The same concept should work with Vega.jl in Juno."

I don't use Juno, but I'm happy to accept a pull request to Vega.jl if there are people who want to integrate Juno as well. The behavior from the REPL is to generate a standalone webpage and open a browser, and plot inline in Jupyter Notebook (which is what I use).

[Sisyphuss]

Hello Eric, 

As I know, Atom does not implement the plot yet.

Does it mean that if I use Plotly.jl, I can plot in my Atom editor?

Best regards,

[Eric Forgy]

Hi Sisyphuss,

By now, there are several ways to get Julia working with Plotly. My solution launches its own HttpServer so it can work in any browser. Atom, being basically an "unsafe" browser, has a couple packages to let you open web pages. In my screenshots above, I installed one of these Atom packages and opened up the URL for the HttpServer my PlotlyJS launches, so yes, it works in Atom, but through a separate browser package.

If you look at my code, there is nothing much specific to Plotly in there and whatever is specific to Plotly can be easily removed. It simply connects Julia to your browser and lets you manipulate the DOM interactively similar to Escher, but in a way that gives you access to any JavaScript library. In this Pre-Ann, I am controlling both d3.js and plotly.js directly from the REPL, but the same idea will work for any JavaScript, so I might change the name to JavaScript.jl or something a little more creative :)