notebooks in docs

[Skipper Seabold]

Just seeing if it works before I merge the PR for the automatic doc
building. Only tested on firefox and chrome.

http://statsmodels.sourceforge.net/notebooks/examples/index.html#notebook-examples

I think this will make example writing much less tedious. Just drop
the output-less .ipynb in examples/notebooks/ and that's it.

Should I go ahead and move all the files in ipynb to be automatically
created as part of this PR. I guess also remove the redundant examples
using the ipython directive? Downside is these won't be automatically
built unless you set it up in tools/nbgenerate.py (until nbconvert is
in IPython), but I don't know how many people build the docs locally
anyway. There is a warning though if they aren't generated.

Skipper

[josef...@gmail.com]

Given that the current examples work nicely, I wouldn't rush into it,
until we have a bit more experience with the notebook conversion.

The notebooks look nice, I didn't even know yet about some of the
content that you put in.

I don't think the rmagic in M-estimators will work for poor guys like
me that don't have rpy2 installed. (An incentive to merge my
MM-estimation branch)

Is it possible to tweak the css styles for the converted notebooks
(like section titles)?

Some of the notebooks are a bit long for my taste without a TOC. Will
there be a way to eventually get a toc into the side bar?

Josef


>
> Skipper

[josef...@gmail.com]

There are Vincent's notebooks that are linked from the wiki page.
They, or some of them should be candidates for inclusion into the new
notebook documentation.

(I'm still not a notebook person, if we convert more examples, then I
need to figure out a way how to get them as python modules without
having to start a notebook each time.)

Josef

>
> Josef
>
>
>>
>> Skipper

[Vincent Arel]

All those are in the statsmodels/examples/ipynb folder already, so
Skipper's script can just convert this folder's content.

> (I'm still not a notebook person, if we convert more examples, then I
> need to figure out a way how to get them as python modules without
> having to start a notebook each time.)

You can do:

git clone https://github.com/ipython/nbconvert

And then simply call nbconvert2.py from the cloned directory:

./nbconvert2.py python notebook.ipynb > notebook.py

And you'll get a nice working script/module file

>
> Josef
>
>>
>> Josef
>>
>>
>>>
>>> Skipper

[Skipper Seabold]

On Thu, May 2, 2013 at 9:07 PM, <josef...@gmail.com> wrote:
> On Thu, May 2, 2013 at 8:56 PM, <josef...@gmail.com> wrote:
>> On Thu, May 2, 2013 at 7:15 PM, Skipper Seabold <jsse...@gmail.com> wrote:
>>> Just seeing if it works before I merge the PR for the automatic doc
>>> building. Only tested on firefox and chrome.
>>>
>>> http://statsmodels.sourceforge.net/notebooks/examples/index.html#notebook-examples
>>>
>>> I think this will make example writing much less tedious. Just drop
>>> the output-less .ipynb in examples/notebooks/ and that's it.
>>>
>>> Should I go ahead and move all the files in ipynb to be automatically
>>> created as part of this PR. I guess also remove the redundant examples
>>> using the ipython directive? Downside is these won't be automatically
>>> built unless you set it up in tools/nbgenerate.py (until nbconvert is
>>> in IPython), but I don't know how many people build the docs locally
>>> anyway. There is a warning though if they aren't generated.
>>
>> Given that the current examples work nicely, I wouldn't rush into it,
>> until we have a bit more experience with the notebook conversion.

IMO, it's much more robust than the ipython_directive because of the
refactored IPython code it's built on. It's much easier for me to get
my head around the new IPython internals than the more or less hacked
together ipython_directive.

>>
>> The notebooks look nice, I didn't even know yet about some of the
>> content that you put in.
>>
>> I don't think the rmagic in M-estimators will work for poor guys like
>> me that don't have rpy2 installed. (An incentive to merge my
>> MM-estimation branch)
>>
>> Is it possible to tweak the css styles for the converted notebooks
>> (like section titles)?

Yeah, the CSS for the notebooks is in nbviewer.min.css.

>>
>> Some of the notebooks are a bit long for my taste without a TOC. Will
>> there be a way to eventually get a toc into the side bar?

Yes, this is all under very active development. You could also easily
make a TOC with some pretty simple javascript (or in the nbgenerate
conversion).

Feel free to edit the notebooks. I'm writing these kinds of things
*much* differently now and will probably work on these here and there.

>
>
> There are Vincent's notebooks that are linked from the wiki page.
> They, or some of them should be candidates for inclusion into the new
> notebook documentation.
>

Are they still? I couldn't find links that worked anywhere, but I
didn't try that hard.

> (I'm still not a notebook person, if we convert more examples, then I
> need to figure out a way how to get them as python modules without
> having to start a notebook each time.)

I just taught a whole semester using the notebooks. They were *very*
popular and effective.

Skipper

[Skipper Seabold]

We could also just commit the python version of the notebooks. Should we?

Skipper

[josef...@gmail.com]

maybe not, I don't really like duplicate and generated code in the repo.

(last time I tried to write a notebook from scratch I got impatient
and worked on the notebook and the script in spyder at the same time,
until I had no idea anymore which is which. Thankfully, Firefox
crashed and took the notebooks with it :)

Josef

>
> Skipper

[Skipper Seabold]

Speaking of...

https://github.com/ipython/ipython/commit/b69eb67e69fd0ca9ec7840181a1baf50ab26a0ad

Skipper

[Skipper Seabold]

On Thu, May 2, 2013 at 11:11 PM, Skipper Seabold <jsse...@gmail.com> wrote:
> On Thu, May 2, 2013 at 9:07 PM, <josef...@gmail.com> wrote:
>> There are Vincent's notebooks that are linked from the wiki page.
>> They, or some of them should be candidates for inclusion into the new
>> notebook documentation.
>>
>
> Are they still? I couldn't find links that worked anywhere, but I
> didn't try that hard.

Found them. The links in the documentation are broken though.

https://github.com/statsmodels/statsmodels/wiki/Examples

Skipper

[josef...@gmail.com]

I think we might be going in a circle, the notebook examples look like
a conversion from the current documentation examples.

Josef


>
> Skipper

[Skipper Seabold]

I'm not sure I understand what you mean. Vincent took the current
examples, added some, made notebooks and pointed nbviewer at them
right?

The difference now is that we only have checked-in examples without
output in the repo.

Skipper

[josef...@gmail.com]

Yes, I thought initially there are some additional notebooks.
Rendering Vincent's notebooks for the documentation as you are able to
do now, would duplicate the existing examples on the website.

I didn't realize before that that is the question, and that we already
have duplicate versions in the repo.

I like the current examples with TOC better than the notebooks, so
maybe we retire the ipython_directive a bit later.

BTW: The rendered notebooks look better in Firefox and Google Chrome
than in Internet Explorer.

Josef

>
> Skipper

[Vincent Arel]

On Thu, May 2, 2013 at 11:32 PM, Skipper Seabold <jsse...@gmail.com> wrote:
> On Thu, May 2, 2013 at 11:11 PM, Skipper Seabold <jsse...@gmail.com> wrote:
>> On Thu, May 2, 2013 at 9:07 PM, <josef...@gmail.com> wrote:
>>> There are Vincent's notebooks that are linked from the wiki page.
>>> They, or some of them should be candidates for inclusion into the new
>>> notebook documentation.
>>>
>>
>> Are they still? I couldn't find links that worked anywhere, but I
>> didn't try that hard.
>
> Found them. The links in the documentation are broken though.

Except for "Getting Started", all the wiki notebooks can be found
here: https://github.com/statsmodels/statsmodels/tree/master/examples/ipynb

The master notebooks might even be more current.

Vincent

>
> https://github.com/statsmodels/statsmodels/wiki/Examples
>
> Skipper

[VincentAB]

Also, you would probably get a TOC for free by converting to rst and feeding those files to sphinx like the rest of the docs instead of converting to html 

Vincent

[Skipper Seabold]

The rST output did not look good, but it's an option in nbgenerate. I
need to look more at the nbconvert2 internals.

Skipper

[Skipper Seabold]

You also have to deal with managing external files from the rST
output, which I didn't really like.

Skipper

[josef...@gmail.com]

Skipper,

Unless you really want to get into nbconvert2 developement, we could
just leave it as you have it in your branch now, and come back to this
in half a year. Maybe, we can get some improvements then for "free".

Josef

[Skipper Seabold]

I don't want to get into nbconvert development, the API just isn't
documented all that well yet because it's not finalized.

I'm pretty much counting on free enhancements. In any case, there's
not really a good way to have the rst be flexible in the sphinx build.
I looked into trying to use a template, but it seems like that's
mostly just for the autosummary, etc. or for the built-in layout.html,
etc. I'm fairly happy with the results using the raw directive and a
little javascript.

I'll likely merge this after a few additions at some point
today/tomorrow. I'll need to do some setup on the build box to make
sure it works.

Skipper

[Thomas Haslwanter]

I am not sure if this is what you are asking for: but when you start the notebook with the "--script" option, a python file is automatically saved, together with the notebook.

[Jan Schulz]

Hi!

On 16 May 2013 19:45, Thomas Haslwanter <thomas.h...@gmail.com> wrote:
> I am not sure if this is what you are asking for: but when you start the
> notebook with the "--script" option, a python file is automatically saved,
> together with the notebook.

This has some problems (e.g. magic commands are not transformed) and
is probably going away in a future version:
https://github.com/ipython/ipython/issues/3295#issuecomment-17678062

Jan