SEGMENTCLASS/sx_kmeans error

[Carsten Sachse]

Dear Venkata,

In order to make k-means using SPARX work - we have found the following useful:

1. Try to decrease the number of classes as sx_kmeans.py cannot deal with empty clusters.
2. Try to decrease the number of processors as in the sx_kmeans.py mpi version the number of processors are taken as number of trails and that may lead to failure of sx_kmeans.py

Best wishes,


Carsten


On Monday, January 6, 2014 10:45:48 PM UTC+1, Carsten Sachse wrote:

On Monday, January 6, 2014 10:45:48 PM UTC+1, Carsten Sachse wrote:

Dear Venkata,
On 5 Jan 2014, at 17:09, venkata prasad dandey wrote:

3. I was able to install the software in our cluster (Sungrid engine) and RHEL6. But I have to explicitly do this:

When I tried to submit job for Segment Class, it gives this error in report.log

      export LD_PRELOAD=/opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
INFO:CPU0:startlog:progress state: 1 %  [>                                        ]
INFO:CPU0:classify:

INFO:CPU0:classify:progress state: 10 %  [====>                                    ]
INFO:CPU0:sx_kmeans:

INFO:dandey:/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/openmpi/bin/mpirun -v --hostfile nodefile_4663.dat -np 64 /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/bin/springenv /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py PYD_K2_stack_perturb.hdf sxk_means00 rectmask.hdf --K=12 --maxit=500 --rand_seed=-1 --crit=D --MPI
logged on Thu, 02 Jan 2014 20:10:15
INFO:CPU0:sx_kmeans:springenv /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py PYD_K2_stack_perturb.hdf sxk_means00 rectmask.hdf --K=12 --maxit=500 --rand_seed=-1 --crit=D --MPI
logged on Thu, 02 Jan 2014 20:12:01
Traceback (most recent call last):
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/bin/segmentclass", line 125, in <module>
    sys.exit(spring.segment2d.segmentclass.main())
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 732, in main
    stack.classify()
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 700, in classify
    avgstack, varstack = self.sx_kmeans(aligned_stack, self.maskfile, self.noclasses)
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 451, in sx_kmeans
    external_kmeans_run.check_expected_output_file(program_to_be_launched, avgstack)
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/csinfrastr/csproductivity.py", line 520, in check_expected_output_file
    raise IOError(error_message)
IOError: /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py did not finish successfully. The output file sxk_means00/averages.hdf was not found, please check logfile of /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py for details.
 

It is some how trying to get the files form tmp folders and unable to get it. how to solve this issue?

This looks like SPARX' k-means failed. Please check the logfile_xxx of sxk_means.py in the SEGMENTCLASS folder. The stderr error message could also be illuminating.

If it is a consistent library error you should deploy those libraries to your nodes. Nevertheless, it can also be a failed k-means run caused by the input parameters - I can provide you with more details if needed.

Best wishes,

Carsten

Dear Venkata,
On 5 Jan 2014, at 17:09, venkata prasad dandey wrote:

3. I was able to install the software in our cluster (Sungrid engine) and RHEL6. But I have to explicitly do this:

When I tried to submit job for Segment Class, it gives this error in report.log

      export LD_PRELOAD=/opt/intel/Compiler/11.1/072/lib/intel64/libimf.so
INFO:CPU0:startlog:progress state: 1 %  [>                                        ]
INFO:CPU0:classify:

INFO:CPU0:classify:progress state: 10 %  [====>                                    ]
INFO:CPU0:sx_kmeans:

INFO:dandey:/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/openmpi/bin/mpirun -v --hostfile nodefile_4663.dat -np 64 /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/bin/springenv /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py PYD_K2_stack_perturb.hdf sxk_means00 rectmask.hdf --K=12 --maxit=500 --rand_seed=-1 --crit=D --MPI
logged on Thu, 02 Jan 2014 20:10:15
INFO:CPU0:sx_kmeans:springenv /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py PYD_K2_stack_perturb.hdf sxk_means00 rectmask.hdf --K=12 --maxit=500 --rand_seed=-1 --crit=D --MPI
logged on Thu, 02 Jan 2014 20:12:01
Traceback (most recent call last):
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/bin/segmentclass", line 125, in <module>
    sys.exit(spring.segment2d.segmentclass.main())
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 732, in main
    stack.classify()
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 700, in classify
    avgstack, varstack = self.sx_kmeans(aligned_stack, self.maskfile, self.noclasses)
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/segment2d/segmentclass.py", line 451, in sx_kmeans
    external_kmeans_run.check_expected_output_file(program_to_be_launched, avgstack)
  File "/home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/lib/python2.7/site-packages/emspring-0.80dev-py2.7.egg/spring/csinfrastr/csproductivity.py", line 520, in check_expected_output_file
    raise IOError(error_message)
IOError: /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py did not finish successfully. The output file sxk_means00/averages.hdf was not found, please check logfile of /home/phil2/dandey/Sofwares/csachse-emspring-e5d6cdf38e56/parts/EMAN2/bin/sxk_means.py for details.
 

It is some how trying to get the files form tmp folders and unable to get it. how to solve this issue?

This looks like SPARX' k-means failed. Please check the logfile_xxx of sxk_means.py in the SEGMENTCLASS folder. The stderr error message could also be illuminating.

If it is a consistent library error you should deploy those libraries to your nodes. Nevertheless, it can also be a failed k-means run caused by the input parameters - I can provide you with more details if needed.

Best wishes,

Carsten

[Venkata Prasad Dandey]

Dear Carsten,

Thanks for the feedback. Yes! it worked after decreasing the number of classes and number of processors. But it is really slow for stack of 7000 segments.

regards,

venkat 

[Carsten Sachse]

Dear Venkata,

Glad to hear that it worked. If you are not happy about the speed you can turn on the internal binning to 2 or 3 depending on your pixel size, e.g. if your pixels size is 1.2 Å binning of 3 still gives you a resolution of 2 x 3 x 1.2 Å.

Best wishes,

Carsten 

[tanmay...@gmail.com]

Hi Venkata,
Just to add to what Carsten mentioned - another option could be making the size of the extracted boxes smaller.
Tanmay