JVM Magics

[Kyle Kelley]

Hey all,

A couple weeks ago a bunch of Scala kernel maintainers came together during the Notebook Enterprise Summit. One of the topics of discussion was making consistent magics across the various Scala & JVM kernels. Notes and proposals are here: https://github.com/nteract/nes/tree/master/jvm-kernel-magics

Ryan Blue created jvm-magics as a result of this: https://github.com/rdblue/jvm-magics

Please check it out if you're interested and post issues. When ready, we'd love to see this land in the jupyter org to support similarly to jvm-repr.

--

Kyle Kelley (@rgbkrk; lambdaops.com)

[Brian Granger]

Great news, thanks for sharing!

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgra...@calpoly.edu and elli...@gmail.com

[Ryan Blue]

Thanks for taking a look at this, everyone! I look forward to your feedback on this prototype.

It would be great to get this (or something like it) into the Jupyter community as a recommendation so that we can build magics that work in any Kernel. I have a SQL magic implementation that uses Spark that could be packaged this way. It is also a good way to implement a set of built-ins across kernels, like %%timeit.

--

Ryan Blue

Software Engineer

Netflix

[Doug Blank]

Ryan,

Very nice! 

We also have a set of magics that work across languages. It would be great to have a group to try to standardize these.

We implemented cross-language magics in metakernel [1], and there are some lessons learned that you might want to consider:

1. We allow languages to define magic prefixes and suffixes for magic, shell, and help

2. shell is a provided magic

3. Some languages allow "?" as a regular character, and so we allow the pattern "?object" to trigger help in those languages (eg, Scheme)

4. We provide some additional interlanguage support (like function regex) to support additional functionality (like the %%pipe magic [2]) for chaining function calls

5. There is also language-specific start-up code, including additional magics

6. A general way for one language to call another (%kernel, %kx)

7. A general way to use the ipython parallel/cluster tools so that all languages can run "in parallel" (%parallel, %px)

8. We added some additional magics that are very useful (%download, %get, %put, %%dot, etc)

9. A general way of using magics (must be at top, can be stacked)

10. The amazing %%% triple percent: makes the cell magic "sticky" so it applies to the rest of the cells (until you turn it off)

There is more, and probably needs more documentation. Feel free to contact me for more information or if you have additional comments. Metakernel was developed by me and Steven Sylvester and has had many contributions from many people over the years.

-Doug

[1] - https://github.com/calysto/metakernel 

[2] - https://github.com/Calysto/metakernel/blob/master/metakernel/magics/README.md#pipe

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAO4re1nLq8pY5LwvPeP9mTBr0s_t4oDJ5mQW06vv3WLz5Ur68Q%40mail.gmail.com.

[Pete Blois]

From a tooling perspective magics effectively declare a new superset of the language, which makes it more difficult to reuse existing tools for functionality such as linting, formatting and static analysis.

I'm curious how magics mesh with proposals like https://github.com/jupyter/enhancement-proposals/pull/26.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAAusYCjBTPWnS%2BdOTZuLg7v%3D6%2B9vMszycZp3grzakquRnD%3DHUg%40mail.gmail.com.