AttributeError: 'module' object has no attribute 'tree'

[Thomas Evangelidis]

I have written some objects into a pickle file using:

    with bz2.BZ2File('dictionaries.pickle.bz2', 'wb') as f:
        cPickle.dump((dict1,
                      dict2,
                      multidict), f)

The last object is a multidimensional dictionary which I created using the following function:

    import collections
    def tree(): # function to create multidimensional dictionaries
        return collections.defaultdict(tree)

Now I want to run a script that loads the dictionaries from file "dictionaries.pickle.bz2" and runs in parallel some calculations using the scoop module. The scripts in simplified form looks like this:

    import bz2, cPickle
    from scoop import futures
    import collections
    def tree(): # function to create multidimensional dictionaries
        return collections.defaultdict(tree)
           
    def load_all_poses_scores(ConsScorTK_file):
       
        with bz2.BZ2File(ConsScorTK_file, "rb") as pickled_file:
            pickle_object = cPickle.load(pickled_file)
        dict1 = pickle_object[0]
        dict2 = pickle_object[1]
        multidict = pickle_object[3]
   
    load_all_poses_scores("dictionaries.pickle.bz2")
   
    def main():
        print  "Entered main()"
   
    if __name__ == "__main__":
       main()

If I try to execute the script serially I have no problem:

    python test.py

But when I execute it using multithreading with scoop I get the following error:

    python2.7 -m scoop --hosts localhost -n 2 test.py
    WARNING: Could not find system address(es):
    [Errno -2] Name or service not known
    [2016-10-14 02:19:10,057] launcher  INFO    SCOOP 0.7 2.0 on linux2 using Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)], API: 1013
    [2016-10-14 02:19:10,058] launcher  INFO    Deploying 2 worker(s) over 1 host(s).
    [2016-10-14 02:19:10,058] launcher  INFO    Worker d--istribution:
    [2016-10-14 02:19:10,058] launcher  INFO       localhost: 1 + origin
    WARNING: Could not find system address(es):
    [Errno -2] Name or service not known
    Launching 2 worker(s) using .
    WARNING: Could not find system address(es):
    [Errno -2] Name or service not known
    WARNING: Could not find system address(es):
    [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/runpy.py", line 174, in _run_module_as_main
        "__main__", fname, loader, pkg_name)
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/runpy.py", line 72, in _run_code
        exec code in run_globals
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/site-packages/scoop-0.7.2.0-py2.7.egg/scoop/bootstrap/__main__.py", line 298, in <module>
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/site-packages/scoop-0.7.2.0-py2.7.egg/scoop/bootstrap/__main__.py", line 92, in main
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/site-packages/scoop-0.7.2.0-py2.7.egg/scoop/bootstrap/__main__.py", line 252, in run
      File "/home/thomas/Programs/Anaconda2/lib/python2.7/site-packages/scoop-0.7.2.0-py2.7.egg/scoop/bootstrap/__main__.py", line 213, in setupEnvironment
      File "test.py", line 15, in <module>
        load_all_poses_scores("sample_activity_for_ROC.training_set.5xcrosval.best14SFs.allposes.pickle.bz2")
      File "test.py", line 10, in load_all_poses_scores
        pickle_object = cPickle.load(pickled_file)
    AttributeError: 'module' object has no attribute 'tree'
    Entered main()
    [2016-10-14 02:19:10,569] launcher  (127.0.0.1:48830) INFO    Root process is done.
    [2016-10-14 02:19:10,570] launcher  (127.0.0.1:48830) INFO    Finished cleaning spawned subprocesses.

The problem is that when loading the pickle.gz file it cannot find the definition of the function "tree()" I defined to make multidimensional dictionaries. Is this a case of circular dependency? Any ideas how to solve this problem?