MemoryError Template Matching

[DAVID HERREROS]

Good evening,

I am a Scipion developer at the National Center for Biotechnology (CSIC) in Spain. Currently, I am trying to integrate template matching for tomography in Scipion. I have started with Eman version 2.3 as I was more familiar with it, but now I would like to move to Eman 2.31.

In order to test if everything has been implemented correctly, I am using a dummy dataset. When I run the test for template matching, the execution finishes with the following error:

00624:   600/600 finished.

00625:   (534427, 3)

00626:   Traceback (most recent call last):

00627:     File "/home/davidherreros/scipion3/software/em/eman-2.31/bin/e2spt_tempmatch.py", line 245, in <module>

00628:       main()

00629:     File "/home/davidherreros/scipion3/software/em/eman-2.31/bin/e2spt_tempmatch.py", line 168, in main

00630:       dst=scidist.cdist(pts, pts)+(np.eye(len(pts))*dthr*100)

00631:     File "/home/davidherreros/scipion3/software/em/eman-2.31/lib/python2.7/site-packages/scipy/spatial/distance.py", line 2602, in cdist

00632:       dm = np.empty((mA, mB), dtype=np.double)

00633:   MemoryError

Apprently, this error appears because numpy attemps to create a matrix of large dimensions, raising the exception. All the steps prior to the error seem to have run correclty.

I would thank you if you could give me a hint about how to solve this issue. I am at your disposal if you need any further information.

Thank you in advance for your support and I am looking forward to your answer.

[Steve Ludtke]

Hi David,

while this list is fine, this is probably more of a question for the normal EMAN2 list, which is monitored by a larger group. If memory serves, the template matching program did have an issue with large memory requirements in 2.31, which was fixed later in the fall. Our focus then switched to getting a Python 3 release out. While we had basically everything done as planned, by Jan-Feb, COVID and coordination with our German collaborators has delayed a formal Python 3 release. I'm still hoping that we will get that done in the near future.

Anyway, if you are doing any development work at all, you should absolutely be working with the current source-based release, since the Python2 -> Python3 transition is likely to have a significant impact, and there will be no further development work on the Python2 version. I would strongly encourage following these instructions:

http://eman2.org/COMPILE_EMAN2_ANACONDA

to get a source-based install. It takes barely more effort than the binary install. While this isn't an official release yet, all of the major functionality is working, and the tomography pipeline has had very significant improvements and bugfixes since the last release.  I suspect this will solve your current problem as well.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 

Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)

Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

--
You received this message because you are subscribed to the Google Groups "EMAN2 and SPARX Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2-develope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2-developers/26a5d78b-ce4c-463a-806e-dcad49c76acfo%40googlegroups.com.

[DAVID HERREROS]

Good morning Steve,

First, I would like to thank you for you answer and I also want to apologize for the delay on my response, but I wanted to test everything before writing you back.

I have installed the new version of Eman and checked that it works fine inside Scipion. The error of template matching I was having is no longer appearing so, as you suggest, I will continue working with the new version.

Again, thank you so much for all your help.

Kind regards

To unsubscribe from this group and stop receiving emails from it, send an email to eman2-de...@googlegroups.com.