newbie trouble importing notebooks

[Andy Davidson]

Hi 

I have a couple of python functions defined in one notebook that I would like to reuse in another. I found the following direction.

http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Importing%20Notebooks.html

https://github.com/jupyter/notebook/blob/master/docs/source/examples/Notebook/Importing%20Notebooks.ipynb

My challenge is I have a lot of cells that define straight python code. These cells often load large data files and are very slow. Ideally I would like to to only import functions. It looks like the heavy lifting is done by NotebookLoader() bellow. Maybe there is a clever way to only execute function definitions? Is there a way to get the abstract syntax tree for the code in a cell and pick out the function definitions?

As a newbie my hack would be to require functions be defined in their own cell. Next before execute check the cell code for lines beginning with ‘def'

class NotebookLoader(object):
    """Module Loader for Jupyter Notebooks"""
    def __init__(self, path=None):
        self.shell = InteractiveShell.instance()
        self.path = path

    def load_module(self, fullname):
        """import a notebook as a module"""
        path = find_notebook(fullname, self.path)

        print ("importing Jupyter notebook from %s" % path)

        # load the notebook object
        with io.open(path, 'r', encoding='utf-8') as f:
            nb = read(f, 4)


        # create the module and add it to sys.modules
        # if name in sys.modules:
        #    return sys.modules[name]
        mod = types.ModuleType(fullname)
        mod.__file__ = path
        mod.__loader__ = self
        mod.__dict__['get_ipython'] = get_ipython
        sys.modules[fullname] = mod

        # extra work to ensure that magics that would affect the user_ns
        # actually affect the notebook module's ns
        save_user_ns = self.shell.user_ns
        self.shell.user_ns = mod.__dict__

        try:
          for cell in nb.cells:
            if cell.cell_type == 'code':
                # transform the input to executable Python
                code = self.shell.input_transformer_manager.transform_cell(cell.source)
                # run the code in themodule
                exec(code, mod.__dict__)
        finally:
            self.shell.user_ns = save_user_ns
        return mod

P.s. I found Importing%20Notebooks.html hard to use. It might be easier of the code as split out and this notebook and put in the standard distribution of jupyter.  “how to import notebooks.ipynb” then becomes 2 lines

Import importNotebookHelper

Import myNotebook

Kind regards

Andy

[Matthias Bussonnier]

Hi Andy,

I would suggest to look at https://github.com/ipython/ipynb , which is
supposed to do what you want by walking the AST.
We need to polish it and advertise it more, feedback (and
contributions) would be welcome.
--
Matthias

> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/D51565AA.571A2%25Andy%40SantaCruzIntegration.com.
> For more options, visit https://groups.google.com/d/optout.

[Andy Davidson]

Hi Matthias

ipynb is what I was looking for. I noticed two small problems. (I am not sure where/how to report them)

1) You can not import functions that have doc strings in them.

Here is the error msg

 File "<unknown>", line 13
    """uses Gauss's method for summing integers http://wmueller.com/precalculus/advanced/hint4_3_6.html"""
    ^
IndentationError: unexpected indent

2) I needed to use import 'as'. I think the https://github.com/ipython/ipynb readme.md needs to be update.

Here is an example

import ipynb.fs.defs.myMathFunctions as mmf

mmf.quickSum(5)

many thanks

[Matthias Bussonnier]

Hi Andy,

Yes, feel free to report a bug there, we can try to follow-up on the
repository and track it appropriately.

Thanks !
--
M

> https://groups.google.com/d/msgid/jupyter/5e2fc660-31b1-47bf-ada5-49e20b12c896%40googlegroups.com.

[Andy Davidson]

Hi Matthias

I am running into a path problem. I have started my juypter notebook server with the current working directory = ~/myWork. I am working on a notebooks at the following path ~/myWork/a/b/notebook1.ipynb Is there a way I can import definitions from a notebook in another directory under ~/mywork

thanks

Andy

p.s. I saw the doc about 'relative imports' how ever I could not get it to work

[Matthias Bussonnier]

I'm unsure about relative import, I haven't touched ipynb in a while.
I would suggest raising an issue on the repository.
--
M

On Fri, Apr 14, 2017 at 1:40 PM, Andy Davidson

> https://groups.google.com/d/msgid/jupyter/045b102d-fb0d-4303-9216-0da9672a5a85%40googlegroups.com.

[Andy Davidson]

Hi Matthias

Note sure how to debug the following error. Attached is the notebook I tried to import. They are in the same directory

Any ideas?

Thanks

Andy

<div id="inner-editor"><br class="Apple-interchange-newline">#from ipynb.fs.defs.week1 import patternToNumber import ipynb.fs.defs.week1 as wk1 #import ipynb.fs.defs.week1 as week1 # import ipynb.fs from .defs.week2 \ # import foo from .defs.week1 \ # import patternToNumber # import ipynb.fs # from ../week2/week2 import patternToNumber from .defs.week1 # # base 4 to base 10 # # TODO copied from week1 notebook # def patternToNumber(pattern) : # exponent = 0 # ret = 0 # for nucleotide in reversed(pattern): # digit = 0 # if nucleotide == "C" : # digit = 1 # elif nucleotide == "G" : # digit = 2 # elif nucleotide == "T" : # digit = 3 # ret = ret + digit * 4**exponent # #print("ret:{0}, digit:{1}, exponent:{2} nucleotide:{3}" # # .format(ret, digit, exponent, nucleotide)) # exponent = exponent + 1 # return ret </div>

#from ipynb.fs.defs.week1 import patternToNumber

import ipynb.fs.defs.week1 as wk1

​

#import ipynb.fs.defs.week1 as week1

# import ipynb.fs from .defs.week2 \

#     import foo from .defs.week1 \

#         import patternToNumber 

​

# import ipynb.fs

​

# from ../week2/week2 import patternToNumber from .defs.week1

​

# # base 4 to base 10

# # TODO copied from week1 notebook

# def patternToNumber(pattern) :

#     exponent = 0

#     ret = 0

#     for nucleotide in reversed(pattern):

#         digit = 0

#         if nucleotide == "C" :

#             digit = 1

#         elif nucleotide == "G" :

#             digit = 2

#         elif nucleotide == "T" :

#             digit = 3

#         ret = ret + digit * 4**exponent

#         #print("ret:{0}, digit:{1}, exponent:{2} nucleotide:{3}"

#         #         .format(ret, digit, exponent, nucleotide))

#         exponent = exponent + 1

#     return ret

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-14-2eb451ec0e6d> in <module>()
----> 1 from ipynb.fs.defs.week1 import patternToNumber
      2 #import ipynb.fs.defs.week1 as wk1
      3 
      4 #import ipynb.fs.defs.week1 as week1
      5 # import ipynb.fs from .defs.week2 \

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in _find_and_load(name, import_)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/__init__.py in get_code(self, fullname)
     60                     ))
     61                 return self.source_to_code(
---> 62                     self._get_filtered_ast(code_from_ipynb(nb)),
     63                     self.path
     64                 )

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/__init__.py in _get_filtered_ast(self, source)
     47         """
     48         tree = ast.parse(source)
---> 49         tree.body = [n for n in tree.body if self._filter_ast_node(n)]
     50         return tree
     51 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/__init__.py in <listcomp>(.0)
     47         """
     48         tree = ast.parse(source)
---> 49         tree.body = [n for n in tree.body if self._filter_ast_node(n)]
     50         return tree
     51 

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/__init__.py in _filter_ast_node(self, node)
     38 
     39         if isinstance(node, ast.Assign):
---> 40             return all([t.id.isupper() for t in node.targets])
     41 
     42         return False

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ipynb/fs/defs/__init__.py in <listcomp>(.0)
     38 
     39         if isinstance(node, ast.Assign):
---> 40             return all([t.id.isupper() for t in node.targets])
     41 
     42         return False

AttributeError: 'Tuple' object has no attribute 'id'

From: Andrew Davidson <An...@SantaCruzIntegration.com>
Date: Friday, April 14, 2017 at 2:19 PM
To: <bussonnie...@gmail.com>
Subject: Re: [jupyter] newbie trouble importing notebooks

Hi Matthias

I am a total newbie how do I report a bug? D

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CANJQusUDLjR83GqX1UtxqUoHO_GxkWoB3nzPWQMnnBCXPVF3ag%40mail.gmail.com.