Converting python log to Notebook

[Mounsif Mehdi]

Hello,

I'm running learning simulations on my computer. Running simulations one after another is counter-productive since they don't last very long and do not provide enough time to focus on other things. I wrote a script to write a log, which worked fine but lacked visuals. Hence, I used PyLaTeX to generate a pdf of my results with plots, but it also lacks flexibility, in particular because of figures positionning.

Then, the inspiration was here and I thought of Jupyter. Has anyone ever experiences automating things with Jupyter ? Any ideas how I could handle this ?


Thanks !

[Roland Weber]

Sounds like nbconvert is what you're looking for...

https://nbconvert.readthedocs.io/en/latest/

[M Pacer]

nbconvert + Jupyter does not do much for image positioning out of the gate. However, please open an issue on nbconvert and we can try to figure out the right way to go about
1. storing the metadata in the notebook 

2. extracting that metadata from the notebook somewhere using an nbconvert Preprocessor

3. exposing the metadata in the correct way in a Jinja template 

nbconvert also allows you to execute notebooks on the command line using the --execute flag, or programmatically in python with the ExecutePreprocessor API. Then, if you wanted to directly handle the plots, you could extract images written during the plotting calls from the resulting outputs. From that point, you could rearrange the images into subplots as you would like. 

You also might want to look at papermill, it sounds like that's solving a similar class of problems, though it is deploying the notebooks in parallel in containerized environments. NB: you're running things locally, you'll have to worry about side-effects if you try to parallelise this using the standard nbconvert API.

Does that give you a good idea of where to go for next steps? If not, is there anything else I can help answer?

On Wed, Feb 14, 2018 at 11:22 PM, Roland Weber <rolw...@de.ibm.com> wrote:

Sounds like nbconvert is what you're looking for...

https://nbconvert.readthedocs.io/en/latest/

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscribe@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/be7f8439-16df-4ed2-a41f-d42b21bdbfd8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Kyle Kelley]

Thanks for mentioning papermill M!

Just a note to add on for papermill - you don't have to run them in parallel or in containerized environments. It's a command line tool to execute a notebook which supports setting parameters, recording values for later recall (within the notebook file). Running in parallel is a matter of running papermill more than once. 😁

Docs for papermill: http://papermill.readthedocs.io/en/latest/

-- Kyle

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAM3SX44DZOBam4VuTCKaNu%2BtLh4goCobacFLB99KX5acZF5Sag%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kyle Kelley (@rgbkrk; lambdaops.com)