goodparticles*.star not containing all the particles

[matthew....@monash.edu]

Hey mate,

So WARP works amazingly well, except the output goodparticles*.star or allparticles*.star is missing lots of data. There are "particles".star files for the extracted particles in the ./particles directory, but this is not reflected in the output in the root directory? Even when I stop and start the processing in WARP it doesn't update the missing particles, even though it has written an extracted particle stack.

Is there something simple I'm missing here? 

Also, thanks for this amazing bit of software!

cheers

matt B

[Dimitry Tegunov]

Hi Matt,

we've experienced this behavior on very rare occasions – rare enough for me not to be able to figure out the cause :-) I think it would be easier to just add an automatic repair step. 

In the meantime, you can use the 'Export Particles' dialog (button in the overview tab) with the corresponding per-micrograph STARs from the ./Matching folder, and the output set to 'Only write STAR' to recreate goodparticles.star, or allparticles.star if you check 'Include items outside of filter ranges'. This will only work if you haven't altered the per-micrograph STARs after the particles were extracted. If you have, you will have to re-extract the particles using the same dialog.

Cheers,

Dimitry

[matthew....@monash.edu]

Dimitry,

All good, worked perfectly when you did as you suggested. I'm just reprocessing some of our older data now to see how WARP goes. So far, its excellent. I'll report back when its done calculating.

cheers

matt B

[Matthew Belousoff]

Hey Dimitry,

Just thought I'd share the results with you. With minimal intervention on WARP, and using your uploaded masked neural network for particle picking. On a data set we have previously collected it refined to 3.0 ang in under 6 hours, from uncorrected movies to final structure. A more careful analysis of our data set got it to 2.8 ang, so you have done a marvelous job with your software to get it to such a state where the refined particle stack out of WARP is of comparable quality. To be honest the maps looks almost identical compared to our 'particle polished' and carefully curated particle stack from RELION.

Great job mate! We will definitely be using your software into the future, and thanks for your great contribution here!

cheers

matt B

--

Dr. Matthew Belousoff
Biomedicine Discovery Institute &
Department of Microbiology,
Monash University
AUSTRALIA

http://lithgow-lab.med.monash.edu.au/index.php/research  
NHMRC Program in Cellular Microbiology (cellularmicrobiologyprogram.org.au)

Mail: c/-
Monash University
Department of Microbiology
23 Innovation Walk,
Clayton Campus
Melbourne 3800

[Dimitry Tegunov]

Very happy to hear that, thanks!

Re: original issue: Upcoming v1.0.5 implements an automatic repair mechanism for allparticles.star among other improvements.

Cheers,

Dimitry

[Amédée des Georges]

Hello Dimitry, 

I have a very large dataset that was processed almost entirely without problems (11k mics) but now everytime I want to restart it gets stuck at the step "repairing metadata" . A few files are causing this, but I can't find them. Any tips on how to fix this? 

On another piece of the dataset, it restarts fine but then the wheel does not turn and I need to reboot the computer to get out of it. Does not even quit with the task manager. These issues have cost me more than 2 weeks, which is unfortunate. 

Best,

Amedee