Unable to install HDDM

[傅韋銘]

Hi,

I now have a problem to install HDDM in anaconda. I am now using the newest version of anaconda (python 3.8).

When I tried to install the hddm package, I consistently get the following error message:

"Solving environment: failed with initial frozen solve. Retrying with flexible solve."

And in the end.

Package conda-verify conflicts for:

conda-verify

conda-build -> conda-verifyThe following specifications were found to be incompatible with your CUDA driver:

  - feature:/win-64::__cuda==8.0=0

  - feature:|@/win-64::__cuda==8.0=0

Your installed CUDA driver is: 8.0

I have already tried to create a python 3.5 environment in conda first and then install hddm. I have also tried to downgrade my python version to python 2.7. Both methods did not work.

I am wondering if there is anyone can help me in this situation. I will grateful for the help, sincerely.

Wei-ming 

[hcp...@gmail.com]

Dear Wei-MIng,

I think using docker image would be better solution. Once you succuessfully installed docker, you can simply pull the image and run the image in docker. 

Mads had created one: https://hub.docker.com/r/madslupe/hddm

Based on that one, I created a new one with `ipyparallel` to support parallel processing: https://hub.docker.com/r/hcp4715/hddm. I also wrote a brief instruction about how to install and use the docker image, you can find it on the docker hub page.

Hope this helps.

Chuan-Peng

[傅韋銘]

Dear Chuan-Peng,

Thank you for replying. I have solved the problem by downgrade my environment to python 2.7 in anaconda. I am now able to install hddm and use it in jupyter notebook. Sadly, I am not able to get the sample data csv('./cavanagh_theta_nn.csv') in the tutorial. I am still working on it. Thank you again for your help.

Sincerely,

Wei-ming 

hcp...@gmail.com <hcp...@gmail.com> 於 2020年8月6日 週四 下午3:17寫道：

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/bade6de7-609b-494d-b544-7597de44c522n%40googlegroups.com.

[Mads Lund Pedersen]

great you got it working. see here for data: https://github.com/hddm-devs/hddm/blob/master/hddm/examples/cavanagh_theta_nn.csv

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/CANj0BWmGArxwF3u3vwZMPnDugE5M1yWYtCqT4TxNmk972q%3D3gw%40mail.gmail.com.

--

Best, 

Mads

[Jochem van Kempen]

Hi there,

I received a similar error as Wei-ming and was wondering whether it can be solved without using docker, as using Docker is not an option on my University PC.

Best,

Jochem

Op donderdag 6 augustus 2020 09:57:00 UTC+1 schreef Mads Lund Pedersen:

To unsubscribe from this group and stop receiving emails from it, send an email to hddm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/bade6de7-609b-494d-b544-7597de44c522n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "hddm-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hddm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/CANj0BWmGArxwF3u3vwZMPnDugE5M1yWYtCqT4TxNmk972q%3D3gw%40mail.gmail.com.

--

Best, 

Mads

[hcp...@gmail.com]

Hi, Jochem,

Can you use singularity (https://sylabs.io/singularity/) in your university PC? If yes, then you can easily also pull use the docker image in singularity.

Best

Chuan-Peng

[gabrielle sheehan]

Is the only way to get HDDM on python versions > 2.7 to use a docker now? I have been able to install packages in environment but during the tutorial I am just getting a lot of error messages. 

[Michael J Frank]

There is an even easier way using Google colab - we've been working with this for a workshop we are holding at Brown this week. (more announcements on exciting developments in allowing other models with collapsing bounds etc to come very soon). 

To try this approach, go here:

https://colab.research.google.com/

From there just do the following (in this case everything is installed on the notebook in the cloud, not on your local machine). 

!pip install pymc
!pip install kabuki
!pip install hddm

then you can upload files like your dataset to the cloud from your drive, like so 

from google.colab import files

uploaded = files.upload()

or mount your drive like:

drive.mount('/content/drive')

 

from then on you can use HDDM as usual  

M

Michael J Frank, PhD | Edgar L. Marston Professor

Director, Carney Center for Computational Brain Science 

Laboratory of Neural Computation and Cognition  

Brown University
website 

To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/8edcd98b-cc61-4bc5-a39c-25e8b987a490o%40googlegroups.com.

[gabri....@gmail.com]

Hi Michael, 

Thanks for the additional way to use the HDDM toolbox. Although there is less errors, I am still getting this one:  ]

m.plot_posterior_predictive(figsize=(14, 10))

--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-14-916850d95b91> in <module>() ----> 1 m.plot_posterior_predictive(figsize=(14, 10))

7 frames

/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py in _update_property(self, k, v) 1000 if not callable(func): 1001 raise AttributeError('{!r} object has no property {!r}' -> 1002 .format(type(self).__name__, k)) 1003 return func(v) 1004 AttributeError: 'Polygon' object has no property 'normed'

SEARCH STACK OVERFLOW

Any idea what is causing this one? I get it from using HDDM on jupyter as well. 

Best, G

[Mads Lund Pedersen]

You need to downgrade matplotlib in order to get the plotting to work. So you can do:

!pip install matplotlib==3.1.2

!pip install pymc

!pip install kabuki

!pip install hddm

You'll be prompted to restart runtime when you include the downgrade of matplotlib, which you should accept. And then you should be good to go.

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/10044c92-aee3-4ba1-8d81-d0356c0bc7cdn%40googlegroups.com.

--

Best, 

Mads

[Arkadiy Maksimovskiy]

Hi Michael,

Thank you for the suggestion to try Colab. It worked well overall, but for some reason it times out when running for too long (e.g., at 10,000 samples).

Here is an example:

# # Get samples

odd_hddm_model.sample(10000, burn=2000, thin=5, dbname='hddm_trace' + '.db', db='pickle')

My process timed out at 99% (almost there!):

[-----------------99%----------------- ] 9954 of 10000 complete in 43150.2 sec

I upgraded to Colab Pro and changed runtime to TPU, and still had this issue. I was wondering if you were able to get around this issue?

Best,

Arkadiy

[Michael J Frank]

I don't think I ever sampled quite that long in the colab versions but man that must have been frustrating to get to 99%!

there are some tricks to allow colab to keep active, I have seen some on the forums eg. https://stackoverflow.com/questions/54057011/google-colab-session-timeout

but search around for a few others if that doesn't work (and please report back!)

M 

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/04d5ef7e-e799-41d2-85dc-3957abf86ea3n%40googlegroups.com.

[Arkadiy Maksimovskiy]

Yes, I wish it stopped just a little bit earlier just for basic kindness. :)

Thank you for the suggestions! I will take a look and report back.

Best,

Arkadiy

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/CAGYaZ%2BfVp1CM1gOMKbc%3DJkW7akh-gxUgxdi45W%2Bztn7U8wzuoA%40mail.gmail.com.

[Michael J Frank]

 great. by the way you usually should not need 10K samples, unless you want a very smooth posterior eg for a paper, or if you have a lot of concerns about convergence. You should often see convergence with hundreds of samples and you may be better off running a few chains of a few hundred each.  But sorry if you already knew this and had other reasons!

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/CAJk4AZ6hfyOECcxnqx7WY%3DWS_qfvmUk7uPayg%3Db3L_4AbPVWsA%40mail.gmail.com.

[Dan Dillon]

Thanks Michael! And good to have your insight on the number of samples; I’m to blame there as I’ve been running 10K and tossing 5K as burn-in with the HDDM and told Arkadiy to do the same. Sounds like maybe I’ve been overdoing it.

Dan 

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/CAGYaZ%2BdPJBcR-s6o6es3k%3Dk5WnN_aZkHbGzT7xFqeC5KYuzVUg%40mail.gmail.com.

[Arkadiy Maksimovskiy]

Hi Michael,

Thank you for your input on samples!

Just an update on Google Colab timeout issue. I have not found anything of use and contacted the Colab team a few days ago but have not yet heard back. I had success running this model using a GPU processor on Colab (if I am not mistaken, that is what you initially suggested). That was a little bit faster than TPU (by 3,000 seconds) and it ran to the end and saved.

I am having a problem loading the saved traces and was wondering if you might have any tips for resolving this issue.

This is the initial command to run and save the traces:

# # Get samples

odd_hddm_model.sample(10000, burn=2000, thin=5, dbname='hddm_trace' + '.db', db='pickle')

# # Save model

odd_hddm_model.save('/content/drive//My Drive/Colab Notebooks/hddm_trace.db')

It saved successfully to my Google Drive, and I was able to load a CSV file from there to ensure that it is not an issue with the path. Also, I visually verified that the 'hddm_trace.db' file exists in my google drive (3 MB). However, loading the traces does not work, please see below for my attempts. I have done this systematically and numbered my attempts, to see if there is a mistake in my loading command- which I do not suspect there is. I would love to hear if you might have any suggestions.

# Load the model traces - Attempt 1

odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/hddm_trace.db')

Error 1:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-26-499a807a2e44> in <module>() 1 # Load the model traces - Attempt 1 ----> 2 odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/hddm_trace.db')

4 frames

/usr/local/lib/python3.6/dist-packages/pymc/database/pickle.py in load(filename) 80 Return a Database instance. 81 """ ---> 82 file = open(filename, 'rb') 83 container = std_pickle.load(file) 84 file.close() FileNotFoundError: [Errno 2] No such file or directory: 'hddm_trace.db'

# Load the model traces - Attempt 2

odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/hddm_trace')

Error 2:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-27-875a6d2c3ac3> in <module>() 1 # Load the model traces - Attempt 2 ----> 2 odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/hddm_trace') /usr/local/lib/python3.6/dist-packages/kabuki/utils.py in load(fname) 21 22 """ ---> 23 model = pickle.load(open(fname, 'rb')) 24 return model 25 FileNotFoundError: [Errno 2] No such file or directory: '/content/drive//My Drive/Colab Notebooks/hddm_trace'

# Load the model traces - Attempt 3

odd_hddm1_model = hddm.load('hddm_trace')

Error 3:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-28-94e811c27bcb> in <module>() 1 # Load the model traces - Attempt 3 ----> 2 odd_hddm1_model = hddm.load('hddm_trace') /usr/local/lib/python3.6/dist-packages/kabuki/utils.py in load(fname) 21 22 """ ---> 23 model = pickle.load(open(fname, 'rb')) 24 return model 25 FileNotFoundError: [Errno 2] No such file or directory: 'hddm_trace'

# Load the model traces - Attempt 4

odd_hddm1_model = hddm.load('hddm_trace.db')

Error 4:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-29-73dea1c0c0bb> in <module>() 1 # Load the model traces - Attempt 4 ----> 2 odd_hddm1_model = hddm.load('hddm_trace.db') /usr/local/lib/python3.6/dist-packages/kabuki/utils.py in load(fname) 21 22 """ ---> 23 model = pickle.load(open(fname, 'rb')) 24 return model 25 FileNotFoundError: [Errno 2] No such file or directory: 'hddm_trace.db'

# Load the model traces - Attempt 5

odd_hddm1_model = hddm.load('odd_hddm_model')

Error 5:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-30-871b74c2d815> in <module>() 1 # Load the model traces - Attempt 5 ----> 2 odd_hddm1_model = hddm.load('odd_hddm_model') /usr/local/lib/python3.6/dist-packages/kabuki/utils.py in load(fname) 21 22 """ ---> 23 model = pickle.load(open(fname, 'rb')) 24 return model 25 FileNotFoundError: [Errno 2] No such file or directory: 'odd_hddm_model'

# Load the model traces - Attempt 6

odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/odd_hddm_model.db')

Error 6:

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-32-3cee46f4cbbf> in <module>() 1 # Load the model traces - Attempt 6 ----> 2 odd_hddm1_model = hddm.load('/content/drive//My Drive/Colab Notebooks/odd_hddm_model.db') /usr/local/lib/python3.6/dist-packages/kabuki/utils.py in load(fname) 21 22 """ ---> 23 model = pickle.load(open(fname, 'rb')) 24 return model 25 FileNotFoundError: [Errno 2] No such file or directory: '/content/drive//My Drive/Colab Notebooks/odd_hddm_model.db'

Any input you might have would be greatly appreciated.

Thank you for your time and help with this,

Arkadiy

[Alexander Fengler]

Hi Arkadiy,

the saver was not behaving properly with the HDDMnn() after some refactoring, sorry about that.

For HDDMnn() class this should be fixed now. Please let me know if you still have problems with that.

Will work this out for the HDDMnnRegressor(), and HDDMnnStimcoding() asap.

Best,

Alex

[Alexander Fengler]

Realized this thread did not refer to the HDDMnn() class.

Ignore previous message, does prob not help you in any way :).

 Got fooled by mentions of TPU (which I am surprised would help with basic HDDM at all?!).

Best,

Alex

[Michael J Frank]

Hi Arkadiy,

I think the issue is that you were saving the model to traces.db - the traces are separate from the entire model construction.

Try this:

odd_hddm_model.sample(10000, burn=2000, thin=5, dbname='hddm_trace.db', db='pickle')

odd_hddm_model.save('odd_hddm_model')

odd_hddm_model2 = hddm.load('odd_hddm_model')

you should then be able to e.g. plot_posteriors from the loaded model etc

To view this discussion on the web visit https://groups.google.com/d/msgid/hddm-users/cd498813-4f57-4282-96c7-b0d1a088c209n%40googlegroups.com.

[Arkadiy Maksimovskiy]

Makes sense! Thank you!!