New golang Jupyter kernel and gonum

184 views
Skip to first unread message

Yu Watanabe

unread,
Jan 9, 2018, 7:46:44 AM1/9/18
to gonum-dev
Hi golang data scientists,

I'm developing a new jupyter notebook kernel for Go. I would like to announce it to golang data scientists and hear feedback about it.


Features:
  1. You can write and execute Go code interactively on browsers and CUI console.
  2. Jupyter Notebook integration
  3. Full Go language spec support.
  4. Code completion (Press Tab) and code inspection (Press Shift-Tab).
  5. Display images, HTML, JavaScript, SVG etc..
  6. It works perfectly with gonum packages (plot, mat, stat, ...)
I created an example notebook with gonum packages. You can run the notebook from this link:
You can just browse the notebook on nbviewer too.

If you think it's interesting, please go through the instructions in README.md (and star the repository on GitHub if you like it).
If you wonder about the advantages over another existing Go Jupyter kernel, read the "comparison" section in README.md.

I think REPL environment is indispensable (and missing) for data science in Go and I would like to know what do data scientists in Go think about this project.
Any feedback is welcome.

--
yunabe

Jan Mercl

unread,
Jan 9, 2018, 7:52:09 AM1/9/18
to Yu Watanabe, gonum-dev
On Tue, Jan 9, 2018 at 1:46 PM Yu Watanabe <yunabe...@gmail.com> wrote:

> Hi golang data scientists,

It would be nice to see at least scientists use the proper terminology.

Sorry for being OT.



--

-j

Yu Watanabe

unread,
Jan 9, 2018, 7:55:44 AM1/9/18
to gonum-dev
Screenshots:
=== code completion ===

=== code inspection ===




2018年1月9日火曜日 21時46分44秒 UTC+9 Yu Watanabe:

Sebastien Binet

unread,
Jan 9, 2018, 8:14:13 AM1/9/18
to Yu Watanabe, gonum-dev
hi yunabe,

On Tue, Jan 9, 2018 at 1:55 PM, Yu Watanabe <yunabe...@gmail.com> wrote:
Screenshots:
=== code completion ===

=== code inspection ===




nice!

so now we have 3 kernels for Go :)


gophernotes is LGPL because of the Go REPL it uses underneath (gomacro)
it's also a bit cumbersome to install because of the C++ 0MQ library.

I suspect yours also needs that C++ library.
but you do have very nice jupyter integration (code completion and code inspection!) and a nice license (BSD-3)

ng (BSD-2) and its jupyter frontend do not have (yet!) proper code completion/inspection but it's on the road map :P
but, IMHO, (I worked primarily on ng, so I may be biased) it has that very nice feature that it doesn't require the C++ library to be installed to work correctly (so it really is just a go-get away to play with)
see:


or, directly:

perhaps we could have a single jupyter package that is shared between these 3 frontends to deal with the rather complicated/under-documented jupyter messages protocol...

-s

--
You received this message because you are subscribed to the Google Groups "gonum-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gonum-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Yu Watanabe

unread,
Jan 10, 2018, 4:17:57 AM1/10/18
to Sebastien Binet, gonum-dev
Thank you for the feedback, Sebastien.

2018-01-09 22:13 GMT+09:00 Sebastien Binet <bi...@cern.ch>:
hi yunabe,

On Tue, Jan 9, 2018 at 1:55 PM, Yu Watanabe <yunabe...@gmail.com> wrote:
Screenshots:
=== code completion ===

=== code inspection ===




nice!

so now we have 3 kernels for Go :)


gophernotes is LGPL because of the Go REPL it uses underneath (gomacro)
it's also a bit cumbersome to install because of the C++ 0MQ library.

I suspect yours also needs that C++ library.

Yes
 
but you do have very nice jupyter integration (code completion and code inspection!) and a nice license (BSD-3)

ng (BSD-2) and its jupyter frontend do not have (yet!) proper code completion/inspection but it's on the road map :P
but, IMHO, (I worked primarily on ng, so I may be biased) it has that very nice feature that it doesn't require the C++ library to be installed to work correctly (so it really is just a go-get away to play with)

Good point. lgo works only on Linux and I accepted the dependency to C++ ZMQ because it relatively easy to install it with package managers.
But it's nicer if we can get rid of dependencies to C++ ZMQ library completely.
Do you know there are reliable ZMQ packages in pure Go?

I'm not familiar with ZMQ protocol and I just chose the package from http://zeromq.org/bindings:go.
My jupyter package is here:
It's isolated from Go execution mechanism and we can swap the backend easily.
Here is a simple example kernel using this package:

This package supports graceful code cancellation and kernel shutdown, code completion, inspection, etc... , which ng/gophernotes kernel does not look support now.
So, it might be helpful for ng too :)

---
yunabe

Sebastien Binet

unread,
Jan 10, 2018, 4:39:01 AM1/10/18
to Yu Watanabe, gonum-dev
On Wed, Jan 10, 2018 at 10:17 AM, Yu Watanabe <yunabe...@gmail.com> wrote:
Thank you for the feedback, Sebastien.
[...] 
but you do have very nice jupyter integration (code completion and code inspection!) and a nice license (BSD-3)

ng (BSD-2) and its jupyter frontend do not have (yet!) proper code completion/inspection but it's on the road map :P
but, IMHO, (I worked primarily on ng, so I may be biased) it has that very nice feature that it doesn't require the C++ library to be installed to work correctly (so it really is just a go-get away to play with)

Good point. lgo works only on Linux and I accepted the dependency to C++ ZMQ because it relatively easy to install it with package managers.
But it's nicer if we can get rid of dependencies to C++ ZMQ library completely.
Do you know there are reliable ZMQ packages in pure Go?

I'm not familiar with ZMQ protocol and I just chose the package from http://zeromq.org/bindings:go.

ng/jupyter uses this pure-Go package:

actually, it uses a fork of the above package:
that implemented the multi-part message of 0MQ.

(well, actually, ng/jupyter uses a vendored copy of sideshowdave7 with a few modifications on top. it's complicated :P)

 

My jupyter package is here:
It's isolated from Go execution mechanism and we can swap the backend easily.
Here is a simple example kernel using this package:

This package supports graceful code cancellation and kernel shutdown, code completion, inspection, etc... , which ng/gophernotes kernel does not look support now.
So, it might be helpful for ng too :)
definitely!

I am (daydreaming?) dreaming of having a pure-Go jupyter-messaging package that could be reused by all 3 kernels, and a pure-Go simple web server that would serve very simple jupyter notebooks (a bit like nteract) so playing with jupyter notebooks could be as easy as:
$> go get jupyter-server
on all Go-supported platforms, with no 3rd party C/C++ libraries to be installed.

that would be a great thing to create for Go.

-s

Yu Watanabe

unread,
Jan 12, 2018, 2:47:52 AM1/12/18
to Sebastien Binet, gonum-dev
At this point, I think the interface of my kernel and other kernels are not stable and it's not very good time to define a consolidated interface of Golang kernel.
For example, my jupyter package has an interface that defines methods for
- kernel_info
- execute_request
- complete/inspect

But it does not cover all protocols defined in Jupyter protocol and I think I will add more methods to the interface, which breakes the backward compatibility of the package.

But, of course, I hope we can define a stable interface and integrate implementation of jupyter package in Go!
 
and a pure-Go simple web server that would serve very simple jupyter notebooks (a bit like nteract) so playing with jupyter notebooks could be as easy as:
$> go get jupyter-server
on all Go-supported platforms, with no 3rd party C/C++ libraries to be installed.

I'm not sure how "pure-Go" is important for the future of data analysis and ML in Go because the core of many existing ML libraries are written in C/C++ (and wrapped with Python) and I'm sure these libraries can be fully rewritten in Go in the future. But, I agree that less dependencies to C++ is important for portability and easiness of installation :)

Thanks a lot,
yunabe
 

that would be a great thing to create for Go.

-s

--
You received this message because you are subscribed to a topic in the Google Groups "gonum-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gonum-dev/cXJLkRxEFWY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gonum-dev+unsubscribe@googlegroups.com.

Dan Kortschak

unread,
Feb 15, 2019, 7:56:53 PM2/15/19
to Yu Watanabe, Sebastien Binet, gonum-dev
Hi Yu,

I was just having a look at this and playing with the use on
mybinder.org. Would you be able to provide some instructions about how
a third party ipynb could use lgo to provide the kernel to mybinder and
so allow it to run?

I'm interested in using it to run a class that I have previously used
gophernotes for (but always had struggles with getting the students to
install quickly): github.com/kortschak/graphprac.

thanks
Dan

On Fri, 2018-01-12 at 16:47 +0900, Yu Watanabe wrote:
> 2018-01-10 18:37 GMT+09:00 Sebastien Binet <bi...@cern.ch>:
>
> >
> >
> >
> > On Wed, Jan 10, 2018 at 10:17 AM, Yu Watanabe <yunabe.public@gmail.
> > gonum-dev+...@googlegroups.com.
> >
> > For more options, visit https://groups.google.com/d/optout.
> >
--
CRICOS provider code 00123M

Yu Watanabe

unread,
Feb 18, 2019, 10:01:04 AM2/18/19
to Dan Kortschak, Sebastien Binet, gonum-dev
Hi Dan,

Thank you for the interest to my golang kernel.

I tried to set up mybinder config for your repository:

But it failed due to a build failure in your repository.
I sent a PR to fix the failure to you.

I'll retry to set up the environment again once the PR is merged.

----
yunabe

2019年2月16日(土) 9:56 Dan Kortschak <dan.ko...@adelaide.edu.au>:

Dan Kortschak

unread,
Feb 18, 2019, 5:58:47 PM2/18/19
to Yu Watanabe, Sebastien Binet, gonum-dev
Thanks Yu.

The Docker file in that graphprac-binder solves my question. There are
more breaking changes in the works for graph, and the class is not for
a while, so I'll do them all at once then. But when it is done, the
example there shows me what to do.

Dan
Reply all
Reply to author
Forward
0 new messages