Error during particle extraction using GUI

[Gabriella Collu]

Dear All,

I am encountering this issue while extracting 3d particles using the GUI. I am using EMAN2.91. I reconstructed the tomograms using EMAN2.91 too.

 12/30 finished.IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/scisoft/x86-64/eman/eman2.9_sphire1.4_sparx_bullseye/bin/e2spt_extract.py", line 987, in <module>

    main()

  File "/scisoft/x86-64/eman/eman2.9_sphire1.4_sparx_bullseye/bin/e2spt_extract.py", line 145, in main

    do_extraction(a, options)

  File "/scisoft/x86-64/eman/eman2.9_sphire1.4_sparx_bullseye/bin/e2spt_extract.py", line 522, in do_extraction

    threed["class_ptcl_idxs"]=pjids

SystemError: <Boost.Python.function object at 0x5597af57b700> returned a result with an error set

Thanks for any help.

Best,

Gabriella

[Muyuan Chen]

From the output it seems that some particles have already been extracted without problems? If that is the case I would guess it is caused by one particle at a coordinate that the program does not expect. For example outside the boundaries of the tomogram. Double check the particle locations visually and see if there is anything wrong. Also upgrading to the continuous build may help as I have implemented some error handling after 2.91 that might help.

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/57e927cb-244d-4ab7-9b66-3e7557f579f8n%40googlegroups.com.

[Gabriella Collu]

Thank you for your response. Yes it seems that it goes through and stops at a certain point. But, when typing e2iminfo.py, I get this and it says 0 particles. 

ali_20220127_particles_00.hdf    31 images in HDF5 format (swap)        256 x 256 x 256

31 total images

representing 0 particles

I am a new user of EMAN2 so I am not sure if this is the way it should be.

[Steve Ludtke]

"representing 0 particles" is related to class-averages not the particles themselves. If you have an image created by averaging other images then it would be 1 image representing 10 particles. ie - ignore the line about representing particles

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/b920e970-e3a3-4dde-9148-76bd3be8d1c8n%40googlegroups.com.

[Gabriella Collu]

Thank you. I have removed that tomogram from the list and now indeed the extraction has run till the end. However, the initial model generation does not proceed. It stops at iteration 1 without progress.

[Muyuan Chen]

You will need to be more specific on that. What is the command you run and what is the output from command line? Is there any files being generated and what do they look like?

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/0c6cdb3a-d35d-4b95-be67-7e8fc8698e61n%40googlegroups.com.

[Gabriella Collu]

I am launching the process from the GUI and I have attached the command in this message.

This is what appears on the terminal after launching:

NOT Writing notes, ppid=-2

Found particles with following labels:  particles_00
e2proclst.py particles3d/ali_20220127_001_corrected__particles_00.hdf particles3d/ali_20220127_007_corrected__particles_00.hdf particles3d/ali_20220127_009_corrected__particles_00.hdf particles3d/ali_20220127_010_corrected__particles_00.hdf particles3d/ali_20220127_011_corrected__particles_00.hdf --create sets/particles_00.lst
Processing 31 images in particles3d/ali_20220127_001_corrected__particles_00.hdf
Processing 30 images in particles3d/ali_20220127_007_corrected__particles_00.hdf
Processing 23 images in particles3d/ali_20220127_009_corrected__particles_00.hdf
Processing 20 images in particles3d/ali_20220127_010_corrected__particles_00.hdf
Processing 44 images in particles3d/ali_20220127_011_corrected__particles_00.hdf
148 particles in sets/particles_00.lst
Done
NOT Writing notes, ppid=-2
Writing in sptsgd_00..
Making random references...
####################
Iteration 1: 

##

After this, it does not proceed further and the files that get generated are

0_spt_params.json  

output.hdf 

particle_parms_01.json  

ref.hdf

[Steve Ludtke]

It produced 2 hash marks in iteration 1, are you sure it isn't just taking a long time?  

We need to see the command you executed to understand what might be going wrong (command tab in project manager or the appropriate line from the .eman2log.txt file). 

What is the box size of your particles?

148 is quite a lot of particles to use for the old initial model generator

generally we suggest moving to a current EMAN2 snapshot version and using the new tools being developed for the next release:

https://blake.bcm.edu/emanwiki/EMAN2/e2tomo_new

we have been gradually updating the standard tutorial to use the new tools as well:

https://blake.bcm.edu/emanwiki/EMAN2/e2TomoSmall

the newer initial model and refinement tools are significantly faster and generally produce at least slightly improved results.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/67456cc7-65ef-4db2-8e74-ddde40973ddcn%40googlegroups.com.
<Capture.PNG>

[Gabriella Collu]

I do not think it is taking long time, after one day it is still at the same point, two hash marks and same files.

My box size is 256.

This is the command from the projectmanager command tab:

e2spt_sgd.py sets/particles_00.lst --sym=c6 --filterto=0.02 --fourier --batchsize=12 --learnrate=0.1 --niter=5 --nbatch=10 --shrink=4

[Muyuan Chen]

If you kill the job, what error message does it throw out? Also check how much memory it occupies when running.

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/91cabfb2-36c3-4649-ae8a-d68bdf2fc130n%40googlegroups.com.

[Gabriella Collu]

No error message! 

[Muyuan Chen]

I do not know anything that could cause this issue, and without an error message it is difficult to figure it out. I would recommend repeat this a few time and see if it does the same every time;  test the program on a different dataset to see whether it is the problem with the program or with the particles here; or upgrade to the latest version and try the e2spt_sgd_new.py program.

Sorry I can't be of more help here...

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/dbc8b001-95c9-4a3d-a3bc-19f6a15cbcf1n%40googlegroups.com.