AttributeError when running generate_stats.py example.

[Chris Linzy]

I have followed the instructions at https://github.com/openworm/movement_validation/blob/master/INSTALL.md and also copied the latest data folders from the associated Google drive  into the movement_validation/examples directory so that the folders are accessible to the example scripts. This later part was done after following initial errors generated by the generate_stats.py script that pointed to the 30m_wait data being missing. The generate_stats.py script now finds the data but I get the following output (ellipsis indicate many repeating lines truncated) and I am wondering if others have an idea of where I am going wrong:

Could not find a pickled version of the histogram managers so let's calculate from scratch and then pickle
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-08___15_42_48/483 AQ2947 on food R_2012_03_08__15_42_48___1___8_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-15___09_51_04/83 AQ2947 on food R_2012_03_15__09_51_04___1___2_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_2/2012-03-15___10_13_46/483 AQ2947 on food R_2012_03_15__10_13_46___2___3_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_3/2012-03-08___12_12_23/483 AQ2947 on food R_2012_03_08__12_12_23___3____features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_4/2012-02-09___12_13_00/483 AQ2947 on food R_2012_02_09__12_13___4___5_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_56_13/483 AQ2947 on food R_2012_03_08__12_56_13___5___4_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_59_13/483 AQ2947 on food R_2012_03_08__12_59_13___5___5_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-02-09___10_28_18/483 AQ2947 on food R_2012_02_09__10_28_18___6___1_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-08___13_12_48/483 AQ2947 on food R_2012_03_08__13_12_48___6___6_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-15___11_35_52/483 AQ2947 on food R_2012_03_15__11_35_52___6___7_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_7/2012-03-08___13_37_17/483 AQ2947 on food R_2012_03_08__13_37_17___7___7_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-08___15_22_34/483 AQ2947 on food R_2012_03_08__15_22_34___8___9_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-15___09_36_52/483 AQ2947 on food R_2012_03_15__09_36_52___8___1_features.mat
['//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-08___15_42_48/483 AQ2947 on food R_2012_03_08__15_42_48___1___8_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-15___09_51_04/83 AQ2947 on food R_2012_03_15__09_51_04___1___2_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_2/2012-03-15___10_13_46/483 AQ2947 on food R_2012_03_15__10_13_46___2___3_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_3/2012-03-08___12_12_23/483 AQ2947 on food R_2012_03_08__12_12_23___3____features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_4/2012-02-09___12_13_00/483 AQ2947 on food R_2012_02_09__12_13___4___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_56_13/483 AQ2947 on food R_2012_03_08__12_56_13___5___4_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_59_13/483 AQ2947 on food R_2012_03_08__12_59_13___5___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-02-09___10_28_18/483 AQ2947 on food R_2012_02_09__10_28_18___6___1_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-08___13_12_48/483 AQ2947 on food R_2012_03_08__13_12_48___6___6_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-15___11_35_52/483 AQ2947 on food R_2012_03_15__11_35_52___6___7_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_7/2012-03-08___13_37_17/483 AQ2947 on food R_2012_03_08__13_37_17___7___7_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-08___15_22_34/483 AQ2947 on food R_2012_03_08__15_22_34___8___9_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-15___09_36_52/483 AQ2947 on food R_2012_03_15__09_36_52___8___1_features.mat']
['//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-08___15_42_48/483 AQ2947 on food R_2012_03_08__15_42_48___1___8_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_1/2012-03-15___09_51_04/83 AQ2947 on food R_2012_03_15__09_51_04___1___2_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_2/2012-03-15___10_13_46/483 AQ2947 on food R_2012_03_15__10_13_46___2___3_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_3/2012-03-08___12_12_23/483 AQ2947 on food R_2012_03_08__12_12_23___3____features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_4/2012-02-09___12_13_00/483 AQ2947 on food R_2012_02_09__12_13___4___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_56_13/483 AQ2947 on food R_2012_03_08__12_56_13___5___4_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_5/2012-03-08___12_59_13/483 AQ2947 on food R_2012_03_08__12_59_13___5___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-02-09___10_28_18/483 AQ2947 on food R_2012_02_09__10_28_18___6___1_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-08___13_12_48/483 AQ2947 on food R_2012_03_08__13_12_48___6___6_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_6/2012-03-15___11_35_52/483 AQ2947 on food R_2012_03_15__11_35_52___6___7_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_7/2012-03-08___13_37_17/483 AQ2947 on food R_2012_03_08__13_37_17___7___7_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-08___15_22_34/483 AQ2947 on food R_2012_03_08__15_22_34___8___9_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/L/tracker_8/2012-03-15___09_36_52/483 AQ2947 on food R_2012_03_15__09_36_52___8___1_features.mat']
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_1/2012-02-09___10_25_36/483 AQ2947 on food L_2012_02_09__10_25_36___1___1_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_2/2012-02-09___12_13_27/483 AQ2947 on food L_2012_02_09__12_13_27___2___5_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_3/2012-02-09___11_48_00/483 AQ2947 on food L_2012_02_09__11_48___3___4_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_3/2012-03-15___10_34_49/483 AQ2947 on food L_2012_03_15__10_34_49___3___4_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_4/2012-03-08___12_32_38/438 AQ2947 on food L_2012_03_08__12_32_38___4___3_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_4/2012-03-15___10_54_39/483 AQ2947 on food L_2012_03_15__10_54_39___4___5_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_5/2012-02-09___11_19_46/483 AQ2947 on food L_2012_02_09__11_19_46__3_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_5/2012-03-15___11_17_52/483 AQ2947 on food L_2012_03_15__11_17_52___5___6_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_7/2012-02-09___12_17_56/483 AQ2947 on food L_2012_02_09__12_17_56___7___4_features.mat
//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_8/2012-02-09___10_57_38/483 (AQ2947) on food L_2012_02_09__10_57_38___8___2_features.mat
['//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_1/2012-02-09___10_25_36/483 AQ2947 on food L_2012_02_09__10_25_36___1___1_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_2/2012-02-09___12_13_27/483 AQ2947 on food L_2012_02_09__12_13_27___2___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_3/2012-02-09___11_48_00/483 AQ2947 on food L_2012_02_09__11_48___3___4_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_3/2012-03-15___10_34_49/483 AQ2947 on food L_2012_03_15__10_34_49___3___4_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_4/2012-03-08___12_32_38/438 AQ2947 on food L_2012_03_08__12_32_38___4___3_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_4/2012-03-15___10_54_39/483 AQ2947 on food L_2012_03_15__10_54_39___4___5_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_5/2012-02-09___11_19_46/483 AQ2947 on food L_2012_02_09__11_19_46__3_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_5/2012-03-15___11_17_52/483 AQ2947 on food L_2012_03_15__11_17_52___5___6_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_7/2012-02-09___12_17_56/483 AQ2947 on food L_2012_02_09__12_17_56___7___4_features.mat', '//Users/chris/Projects/OpenWorm/movement_validation/examples/30m_wait/R/tracker_8/2012-02-09___10_57_38/483 (AQ2947) on food L_2012_02_09__10_57_38___8___2_features.mat']
Number of feature files passed: 3
../movement_validation/statistics/histogram_manager.py:358: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if data == None or not isinstance(data, np.ndarray) or data.size == 0:
../movement_validation/statistics/specs.py:255: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if self.index != None and data != None:
In Histogram.merge_histograms... # of histograms to merge: 3
found a None video in the feature list
...
found a None video in the feature list
Number of feature files passed: 3
In Histogram.merge_histograms... # of histograms to merge: 3
found a None video in the feature list
...
found a None video in the feature list
found a None video in the feature list
Experiment has a total of 673 histograms
Traceback (most recent call last):
  File "generate_stats.py", line 182, in <module>
    main()
  File "generate_stats.py", line 39, in main
    stats = mv.StatisticsManager(experiment_histograms, control_histograms)
  File "../movement_validation/statistics/manager.py", line 99, in __init__
    t_statistics, p_values = stats.ttest_ind(exp_hists.mean_per_video,
AttributeError: 'HistogramManager' object has no attribute 'mean_per_video'

[Jim Hokanson]

Hi Chris,

  I am unable to replicate those errors. 

 One note, the path to those files should be discoverable via the user_config.py file in the movement_validation folder (a subfolder of the repo). I've looked at our install directions and although they specify that the user_config file needs to be created, it is lacking in details. I've added an issue to the repo regarding the need to update the installation to be much clearer.

  That being said, it doesn't explain your problem. I'd try fixing the above mentioned problem and updating the repo as Michael may have made changes that fix your problem. If you are still having issues then we can perhaps do an online debugging session together. 

To fix the above problem, you need to do the following:

Create a file called user_config.py in the movement_validation package. In my case the path is:

C:\D\repos\temp\movement_validation\movement_validation\user_config.py

                                repo                              package

Note that there are two movement_validation folders. The first folder is the repo and its name can vary. The second folder is the package and its name should not vary. 

In the file, add on the following:

EXAMPLE_DATA_PATH = r'C:\Users\RNEL\Google Drive\open_worm\example_data'

Replace this with the value for where you put the data.

In general we try and put things that should be in the user_config.py file in another file called user_config_example.txt which is in the same folder.

Let me know if you have any questions. Best of luck.

Jim

--
Visit us online at:
http://openworm.org
http://blog.openworm.org
http://github.com/openworm
http://twitter.com/openworm
https://plus.google.com/s/openworm
---
You received this message because you are subscribed to the Google Groups "OpenWorm-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openworm-discu...@googlegroups.com.
To post to this group, send email to openworm...@googlegroups.com.
Visit this group at http://groups.google.com/group/openworm-discuss.
To view this discussion on the web visit https://groups.google.com/d/msgid/openworm-discuss/f9a1c928-e9d4-415f-9410-804064491e0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.