Re: [IPython-dev] Developing a MOOC with IPython notebooks

[William Stein]

Hi,

I just (seconds ago) added the ability to publish IPython notebooks
from SageMathCloud (https://cloud.sagemath.com). I've posted a quick
video I just made for you here illustrating how it works:

http://youtu.be/QcOK-LRCW38

Basically:
1. Create a SageMathCloud account, a new project, and upload or
create an ipython notebook.
2. Click the "Publish" button in the upper right corner of the
IPython notebook.
3. Share the link.
... or ...
Since SageMathCloud projects *are* just full Linux accounts, you can
use the Terminal via +New --> Terminal, or just ssh in (as explained
in project settings), then directly run nbconvert with whatever
options you want. Click the "i" to the left of any file or
directory tree to make it public.

SageMathCloud is kind of like http://tmpnb.org/, except it is a lot
less "tmp" -- every project is persistent, snapshotted every few
minutes (using bup), and there is chat, collaborators, simultaneous
editing for documents, etc. It's all open source now [1], though the
free hosted environment itself is very powerful (with 19 dedicated
servers plus cloud compute resources). We frequently had around 800
simultaneous running projects around finals recently...

[1] https://github.com/sagemath/cloud

William

On Mon, Dec 22, 2014 at 9:53 AM, Andrew Odewahn <ode...@oreilly.com> wrote:
> Hi, Anton. I've been working on something along a similar line: running a
> Notebook kernel in a Docker container via tmpnb and then connecting to it
> via a widget. This presentation has some of the details:
>
> http://odewahn.github.io/publishing-workflows-for-jupyter/#1
>
> Like Cyrille Rossant's work, it's still very experimental, but I've been
> pleased with the results so far. The biggest issues have been around
> getting it running for version 2 of the Notebook (currently it only works
> for version 1) and getting the CORS headers set up for tmpnb.
>
> Andrew
>
> ps -- here are a few examples of the output. (It only works in Chrome right
> now due to the CORS issues):
>
> http://sites.oreilly.com/odewahn/ipython-tutorial2/ch01.html
>
> http://sites.oreilly.com/odewahn/jem-test/ch01.html
>
>
>
> On Mon, Dec 22, 2014 at 11:47 AM, Cyrille Rossant
> <cyrille...@gmail.com> wrote:
>>
>> Hi Anton,
>>
>> > * What are the best practices for keeping notebooks in git? We've
>> > found nbdiff, but it seems limited: for example it doesn't do anything
>> > to diff markdown cells. Is there a better way?
>>
>> I am currently working on a solution that basically replaces JSON
>> .ipynb by regular Markdown .md. You loose all metadata and outputs,
>> but you keep Markdown cells and the input of code cells. It is still
>> highly experimental and unstable, you'll find the code here:
>> https://github.com/rossant/ipymd/tree/support-atlas -- let me know if
>> you're interested in trying it out or contributing.
>>
>> > * What are the current options for creating rich graphic content that
>> > doesn't rely on having a live kernel? I know of mpld3, ipywidgets, and
>> > bokeh. For us ipywidgets seem like the best solution so far, but it's
>> > a pet project of Jake Vanderplas that didn't see any action in around
>> > half a year. So I wonder if there is anything more alive in that
>> > respect, that'd be great. Perhaps in v3?
>>
>> I'm also very interested in this use case, but as far as I know
>> there's nothing ready yet... We plan to support this in VisPy, but we
>> haven't started to work on it yet.
>>
>> I suppose you could work something out based on tmpnb.org -- you get a
>> temporary live kernel in the cloud launched just for you.
>>
>> Cyrille
>> _______________________________________________
>> IPython-dev mailing list
>> IPyth...@scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPyth...@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[c.d. mclean]

tmpnb.org also serves its notebooks with the option of connecting to a Julia kernel ...

i have not yet watched the video yet, but i am hoping this is an option available with

a few clicks.

tmpnb.org also serves terminals with a few clicks ...

some of us knew that SMC was already effectively ahead of the curve

on this stuff; nicely done, William !!!

[William Stein]

On Mon, Dec 22, 2014 at 12:21 PM, c.d. mclean <cdmcle...@gmail.com> wrote:
>
> tmpnb.org also serves its notebooks with the option of connecting to a Julia
> kernel ...
>
> i have not yet watched the video yet, but i am hoping this is an option
> available with
> a few clicks.

Sadly, not quite yet...

>
> tmpnb.org also serves terminals with a few clicks ...
>
>
> some of us knew that SMC was already effectively ahead of the curve
> on this stuff; nicely done, William !!!

Thanks for the encouragement.

>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-cloud" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-cloud+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sage-cloud/30e94084-9d9a-4558-af8c-e73bb82d4fa2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[William Stein]

On Tue, Dec 23, 2014 at 4:46 AM, Anton Akhmerov
<anton.a...@gmail.com> wrote:

> On Mon, Dec 22, 2014 at 7:39 PM, William Stein <wst...@gmail.com> wrote:
>> Hi,
>>
>> I just (seconds ago) added the ability to publish IPython notebooks
>> from SageMathCloud (https://cloud.sagemath.com).
>

> Actually I should've considered SMC. Looking at the FAQ, it seems that
> it is suitable for most our needs, the only unclear thing being the
> software setup.
> Basically in addition to the regular scipy-stack we also need the
> kwant package (http://kwant-project.org), which is in pypi.

Since it was so easy, kwant is now installed for all SMC projects:

https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2014-12-23-kwant.html


Note: on import of kwant, it says "RuntimeWarning: The installed SciPy
does not use UMFPACK. Instead, SciPy will use the version of SuperLu
it is shipped with. Performance can be very poor in this case." Can
you run tests and let me know if this is a problem? If you know how
to rebuild scipy so it uses UMFPACK, I can fix this, of course. (With
Sage -- which I know much better than scipy -- there are a million
subtle performance dependencies like this, and it's best to get
feedback from an expert when one hits one of them.)


> Unfortunately it also requires the MUMPS linear algebra package, so
> overall the installation process would be somewhat involved I imagine.
>
> Do you know if we would be able to set it up, so that the course users
> wouldn't have a need to install Kwant separately?

Regarding kwant -- done!

Regarding MUMPS, is that this somewhat strange seeming "public-domain"
library, where you fill out a form and get an email link?

http://mumps.enseeiht.fr/index.php?page=dwnld#license

Anyway, my model with SMC is to pre-install a very wide range of
packages, available for everybody, motivated by what people need. I
have a script [1] that installs everything into a custom Python
install (=Sage), each time I upgrade to a new version (though I leave
the old versions in case people need them).

> Some extra questions: Is it possible use single cell servers with
> custom code pre-executed?
> Is it possible to embed those in an IFrame?

I don't know -- I've cc'd Andrey and Jason, the single cell server
devs, and the sage-cell mailng list.

[1] https://github.com/sagemath/cloud/blob/master/build.py

William

>
> Thanks,
> Anton

[William Stein]

On Tue, Dec 23, 2014 at 8:00 AM, William Stein <wst...@gmail.com> wrote:
> On Tue, Dec 23, 2014 at 4:46 AM, Anton Akhmerov
> <anton.a...@gmail.com> wrote:
>> On Mon, Dec 22, 2014 at 7:39 PM, William Stein <wst...@gmail.com> wrote:
>>> Hi,
>>>
>>> I just (seconds ago) added the ability to publish IPython notebooks
>>> from SageMathCloud (https://cloud.sagemath.com).
>>
>> Actually I should've considered SMC. Looking at the FAQ, it seems that
>> it is suitable for most our needs, the only unclear thing being the
>> software setup.
>> Basically in addition to the regular scipy-stack we also need the
>> kwant package (http://kwant-project.org), which is in pypi.
>
> Since it was so easy, kwant is now installed for all SMC projects:
>
> https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2014-12-23-kwant.html
>
>
> Note: on import of kwant, it says "RuntimeWarning: The installed SciPy
> does not use UMFPACK. Instead, SciPy will use the version of SuperLu
> it is shipped with. Performance can be very poor in this case." Can
> you run tests and let me know if this is a problem? If you know how
> to rebuild scipy so it uses UMFPACK, I can fix this, of course. (With
> Sage -- which I know much better than scipy -- there are a million
> subtle performance dependencies like this, and it's best to get
> feedback from an expert when one hits one of them.)

Hi,

I removed the pip version and installed Kwant system-wide using the
Ubuntu Packages suggested at http://kwant-project.org/install

Now kwant works fine without the UMFPACK warning.

-- William

[William Stein]

On Tue, Dec 23, 2014 at 8:53 AM, Anton Akhmerov
<anton.a...@gmail.com> wrote:
> Hi,

>
> On Tue, Dec 23, 2014 at 5:48 PM, William Stein <wst...@gmail.com> wrote:
>> I removed the pip version and installed Kwant system-wide using the
>> Ubuntu Packages suggested at http://kwant-project.org/install
>>
>> Now kwant works fine without the UMFPACK warning.
>

> Perfect, this means that also MUMPS is installed through the Ubuntu
> repos (it's actually indeed public domain, they just want to collect
> user emails).
> Sadly Umfpack in Scipy is now gone due to software rot and GPL <->
> Python world incompatibility.
>
> Thanks a lot for providing Kwant in SMC. This makes SMC really useful
> both for the Kwant team and the MOOC I'm organizing.

You're very welcome. Please don't hesitate to ask if you have any
additional SMC-related needs!

-- William

>
> Anton

[Andrey Novoseltsev]

On Tuesday, 23 December 2014 09:01:19 UTC-7, William Stein wrote:

On Tue, Dec 23, 2014 at 4:46 AM, Anton Akhmerov
<anton.a...@gmail.com> wrote:
> Some extra questions: Is it possible use single cell servers with
> custom code pre-executed?
>  Is it possible to embed those in an IFrame?

I don't know -- I've cc'd Andrey and Jason, the single cell server
devs, and the sage-cell mailng list.

I am not sure what exactly do you want to do, but it is possible to use init.sage file, linked cells with some of them auto-evaluated, or just your own version of Sage with some patching (note that SageCell already patches Sage to some extend, so you need to apply your changes on top).

Regarding IFrame, you probably just should try and see what happens. I don't know of any reason why cells there would work any different from the other pages.

Thank you,
Andrey