segclassreconstruct runs slow on cluster

[Andrea Nans]

Hello,

I'm just working through the TMV tutorial and using version .84-1470. For the symmetry search, I have limited it to only 150 symmetry pairs but this still takes over 41 hours on 24 CPUs. segrefine3dgrid and segmentrefine3d complete in a much more reasonable amount of time. Any ideas as to why segclassreconstuct would take such a amount of time? Our /tmp directory has 1TB and there doesn't seem to be any problems with memory.

Thanks,

Andrea

[Carsten Sachse]

Dear Andrea,

I assume you are using the exact same parameters as outlined in the tutorial except for the smaller grid. Are you also using the binned class with a pixel size of 2.124 Angstrom? As detailed in the tutorial we recorded the time it took at our cluster for a much larger grid. So I am surprised to hear that but we can run a test on our cluster next week to make sure that the values in the tutorial are still valid.

Best wishes,

Carsten

[Andrea Nans]

I just checked my parameters from Segmentclass and I used a binning option of 2, so I think the averages.hdf file that I used as input for Segclassreconstruct is binned by 2. I also specify a pixel size of 2.124 in segclassreconstruct.

Andrea

[Carsten Sachse]

Dear Andrea,

Sorry for the delayed response. We looked into the issue. Indeed, the reported numbers in the tutorial are not up to date. There were some code changes to improve the accuracy of the volume symmetrization step at the expense of computing time. Therefore, we will update the tutorial such that it correctly reflects the computing time.

Generally, there are several parameters that will influence the time needed for SEGCLASSRECONSTRUCT:

1. Size of the grid to be tested

2. 'Percent of image reconstruction size'. The larger the volume size to be reconstructed the more expensive. For cases like TMV you can easily decrease this to for example 30%.

Explanation: the image size is directly related to your symmetry parameters, e.g. if you have a small rise as in TMV 1.4 Å and are reconstructing a 400 Å volume, you will have to include 284 (400/1.4) image slices in the reconstruction. Followed by another volume symmetrization that is more expensive with larger volumes. The tutorial case of TMV is quite extreme as the high redundancy requires the symmetrization of many copies. However, this will result in very good coverage of your angles and therefore very good reconstructions and accurate reprojections.

I hope this helps and thanks for pointing out the inconsistency.

Best wishes,

Carsten

On Friday, January 29, 2016 at 10:27:02 AM UTC+1, Andrea Nans wrote:

[Andrea Nans]

Hi Carsten,

Many thanks for the response and the advice. It's good to know that the software is running normally on our end, and I'm sure we won't need to use such extreme parameters for future samples.

Best,
Andrea Nans
Rosenthal Group
Francis Crick Institute