Manually Picking Particles

[Kyle Messina]

Hello,

I was able to get around the issues I was previously having but the particle picker still does not recognize my particle that well.  Fortunately, my particles have enough contrast that I can manually pick them from the dozen or so tomograms I have at the moment.  The Warp tutorial does not have a guide for manually picking, so I was wondering if it was possible to manually pick particles within Warp or if there are other recommended programs for picking particles.  I'm fine with either so any suggestions and tips are appreciated!

Thank you in advance!

Kyle

[Digvijay Singh]

The WARP family comes with a picking software called cube

You may find its executable in your Windows' /ProgramFiles/WARP/

You can load up any reconstructions from WARP into cube and then using left clicks, you can manually annotate your particles, which can then be imported into .star which can be directly used in WARP for generating subtomograms. 

HTH & cheers,

Digvijay

[Kyle Messina]

Hi Digvijay,

Alister actually told me about Cube in another post and that worked great.  So happy that its also posted here for anyone else who needs a quick solution. 

For anyone else, I'm now running into issues after exporting the subtomograms and trying to use relion_reconstruct to make sure the particle positions are correct as suggested in the tutorial.  The error I'm getting is (x and y are swaped out for the actual folder names):

 "relion_reconstruct --i /mnt/c/x/y/subtomo_extract/Position_2_8.40Apx.star --o /mnt/c/x/y/subtomo_check/Position_2_check --3d_rot --ctf
 

+ Taking angpix from the first optics group: 2.1
 + Back-projecting all images ...
000/??? sec ~~(,_,">                                                          [oo]in: ./src/reconstructor.cpp, line 490
ERROR:
ERROR: cannot find rlnCtfImage for 3D CTF correction!"

My guess is that the issue may be related to running Relion through Ubuntu.  Initially this command would not run since it was explicitly looking for a file named "../imod/particleseries/Position_1/Position_1_0000000_2.10A.mrcs" under _rlnImageName in the input file.  Once I changed the file path in the _rlnImageName column of the input star file from "../" to the full file path, including a "/mnt/c/", the command worked until it hit the above error.  We thought using Ubuntu may save us time but if it is known to cause issues like this, it may be worth it to swap to either something else or compile (?) Relion for Windows.  Any help and advice is appreciated since I'm still getting a handle on how the computational backend/file systems work. Thank you!

[Zwei]

Hello Again!

So it looks like the volume option was not checked for an unexpected reason.  I only realized volume wasn't selected after looking at the tutorial image and saw it had a full blue circle on the volume option and not a thin blue outline like mine had.  I included two screenshots since it's the only visual glitch I've had with Warp but could was resolved after a quick restart. I'm not sure what could have caused it but thankfully had an easy fix.

Unfortunately, I ran into a new error with the volume option enabled. 

The command prompt error is:

Received "TomoExportParticles", with 4 arguments, for GPU #0, 0 MB free
System.Exception: cudaErrorIllegalAddress
   at Warp.GPU.CheckGPUExceptions() in D:\Dev\warp2\WarpLib\GPU.cs:line 1175
   at Warp.TiltSeries.ReconstructSubtomos(ProcessingOptionsTomoSubReconstruction options, float3[] positions, float3[] angles) in D:\Dev\warp2\WarpLib\TiltSeries.cs:line 2130
   at WarpWorker.Program.Main(String[] args) in D:\Dev\warp2\WarpWorker\Program.cs:line 238

While the Warp error log is:

System.Exception: Couldn't export the particles!
   at Warp.WorkerWrapper.TomoExportParticles(String path, ProcessingOptionsTomoSubReconstruction options, float3[] coordinates, float3[] angles) in D:\Dev\warp2\WarpLib\WorkerWrapper.cs:line 197
   at Warp.Controls.DialogTomoParticleExport.<>c__DisplayClass14_5.<ButtonWrite_OnClick>b__23(TiltSeries series, Int32 gpuID) in D:\Dev\warp2\Warp\Controls\TaskDialogs\Tomo\DialogTomoParticleExport.xaml.cs:line 457
   at Warp.Tools.Helper.<>c__DisplayClass43_1`1.<ForEachGPU>b__0() in D:\Dev\warp2\WarpLib\Tools\Helper.cs:line 525
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
   at Warp.WorkerWrapper.TomoExportParticles(String path, ProcessingOptionsTomoSubReconstruction options, float3[] coordinates, float3[] angles) in D:\Dev\warp2\WarpLib\WorkerWrapper.cs:line 197
   at Warp.Controls.DialogTomoParticleExport.<>c__DisplayClass14_5.<ButtonWrite_OnClick>b__23(TiltSeries series, Int32 gpuID) in D:\Dev\warp2\Warp\Controls\TaskDialogs\Tomo\DialogTomoParticleExport.xaml.cs:line 457
   at Warp.Tools.Helper.<>c__DisplayClass43_1`1.<ForEachGPU>b__0() in D:\Dev\warp2\WarpLib\Tools\Helper.cs:line 525
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Quick look on the forums says it may be related to GPU or memory issues? There's only a handful of posts with "cudaErrorIllegalAddress" and only one one of them was related to subtomo export.  Any ideas/suggestions for me to troubleshoot this?

Also since I realized it may be relevant/cause issues, we had to shuffle some files around at some point due to storage issues on this PC.  While everything should be back in the right spots, something did get changed in the shuffle and Warp saw the tilt stacks produced by IMOD as not CTFcorrected/preprocessed (.tomostar menu).  Without thinking about it I did rerun the preprocessing without redoing the reconstructions since they were all finished.  Could this cause any issues later on downstream?  I can redo the reconstructions, and adjust the particle picking (since the coordinates probably won't change that much)on the redone reconstructions if that may resolve any potential issues.

From,

Kyle

On Wed, May 8, 2024 at 1:40 AM Alister Burt <alist...@gmail.com> wrote:

Hi Kyle,

Do you have ‘rlnCtfImage’ entries in the particle star file that is written out? If not, you might have written out 2D images rather than 3D volumes

Cheers,

Alister

Sent from mobile - apologies for brevity

On May 7, 2024, at 23:19, Kyle Messina <kmessi...@gmail.com> wrote:

Hi Digvijay,

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/ec024758-3c6a-4a19-9c2d-7d1c6ebd57bdn%40googlegroups.com.

[Alister Burt]

Hi Kyle,

Do you have ‘rlnCtfImage’ entries in the particle star file that is written out? If not, you might have written out 2D images rather than 3D volumes

Cheers,

Alister

Sent from mobile - apologies for brevity

On May 7, 2024, at 23:19, Kyle Messina <kmessi...@gmail.com> wrote:

Hi Digvijay,

[Zwei]

It's been a few months since we collected this dataset but we definitely did use a very high defocus (my notes say -15) for contrast since the virus had migrated to a thicker part of the cell and contrast was quite poor without it.  And as much as we would like it to be 4.2, for this go around we're not expecting it to be that high.  So reprocessing the micrographs with a bigger bin value is something I can do to get around this potential issue. 

Also glad you confirmed about turning off 'pre-rotate.'  I figured I should but wasn't entirely sure. 

Any idea why it may be trying to extract 4k particles from a tomogram I did not process with Cube (or knowingly process)? I tried to figure out where it was reading in coordinates from but I'm stumped. 

Thanks again!

Kyle

On Wed, May 8, 2024 at 11:13 AM Alister Burt <alist...@gmail.com> wrote:

I'm surprised you're running into memory issues during export on a 16GB card - is your defocus exceptionally high? The box size used for extraction internally is dynamically determined as the minimum box size which will avoid CTF aliasing, this can lead to extremely large box sizes if defocus is high and pixel size is small.

2.1 A/px is quite small but would ideally work, if you're not expecting to get to 4.2 you could try a larger pixel size

Don't hit 'pre-rotate' sub volumes - you will lose track of the initial orientation after running RELION refinements and M won't work - that option was added for debugging and should be removed

Cheers,

Alister 

On Wed, 8 May 2024 at 16:36, Zwei <kmessi...@gmail.com> wrote:

Just noticed that the initial line literally says 0 MB free so I'm guessing it is a memory issue!  Taking a second look at the command prompt it goes from having 15 GB free to 0 after exporting 4551 particles from Position_1.  The issue is that I did not do any particle picking in Position 1 with Cube and my total particles from Cube over 15 tomograms is ~2,000. Any idea if I may have done something odd/wrong with the file pathing?

Received "LoadGainRef", with 5 arguments, for GPU #0, 15253 MB free
Loaded gain reference and defect map: 11520, 8184, 1, normal, real, 1 A/px, ID = -1, False, False, False,
0.931

Received "TomoExportParticles", with 4 arguments, for GPU #0, 15253 MB free
Exported 4551 particles for C:\Tomo_Datasets\2023_10_12_CVB3_Hela_Kyle_Tomo_Movies\imod\Position_1.tomostar
30.221

Received "TomoExportParticles", with 4 arguments, for GPU #0, 0 MB free
System.Exception: cudaErrorIllegalAddress
   at Warp.GPU.CheckGPUExceptions() in D:\Dev\warp2\WarpLib\GPU.cs:line 1175
   at Warp.TiltSeries.ReconstructSubtomos(ProcessingOptionsTomoSubReconstruction options, float3[] positions, float3[] angles) in D:\Dev\warp2\WarpLib\TiltSeries.cs:line 2130
   at WarpWorker.Program.Main(String[] args) in D:\Dev\warp2\WarpWorker\Program.cs:line 238

[Alister Burt]

I'm surprised you're running into memory issues during export on a 16GB card - is your defocus exceptionally high? The box size used for extraction internally is dynamically determined as the minimum box size which will avoid CTF aliasing, this can lead to extremely large box sizes if defocus is high and pixel size is small.

2.1 A/px is quite small but would ideally work, if you're not expecting to get to 4.2 you could try a larger pixel size

Don't hit 'pre-rotate' sub volumes - you will lose track of the initial orientation after running RELION refinements and M won't work - that option was added for debugging and should be removed

Cheers,

Alister 

On Wed, 8 May 2024 at 16:36, Zwei <kmessi...@gmail.com> wrote:

Just noticed that the initial line literally says 0 MB free so I'm guessing it is a memory issue!  Taking a second look at the command prompt it goes from having 15 GB free to 0 after exporting 4551 particles from Position_1.  The issue is that I did not do any particle picking in Position 1 with Cube and my total particles from Cube over 15 tomograms is ~2,000. Any idea if I may have done something odd/wrong with the file pathing?

Received "LoadGainRef", with 5 arguments, for GPU #0, 15253 MB free
Loaded gain reference and defect map: 11520, 8184, 1, normal, real, 1 A/px, ID = -1, False, False, False,
0.931

Received "TomoExportParticles", with 4 arguments, for GPU #0, 15253 MB free
Exported 4551 particles for C:\Tomo_Datasets\2023_10_12_CVB3_Hela_Kyle_Tomo_Movies\imod\Position_1.tomostar
30.221

Received "TomoExportParticles", with 4 arguments, for GPU #0, 0 MB free
System.Exception: cudaErrorIllegalAddress
   at Warp.GPU.CheckGPUExceptions() in D:\Dev\warp2\WarpLib\GPU.cs:line 1175
   at Warp.TiltSeries.ReconstructSubtomos(ProcessingOptionsTomoSubReconstruction options, float3[] positions, float3[] angles) in D:\Dev\warp2\WarpLib\TiltSeries.cs:line 2130
   at WarpWorker.Program.Main(String[] args) in D:\Dev\warp2\WarpWorker\Program.cs:line 238

On Wed, May 8, 2024 at 10:09 AM Zwei <kmessi...@gmail.com> wrote:

[Zwei]

Just noticed that the initial line literally says 0 MB free so I'm guessing it is a memory issue!  Taking a second look at the command prompt it goes from having 15 GB free to 0 after exporting 4551 particles from Position_1.  The issue is that I did not do any particle picking in Position 1 with Cube and my total particles from Cube over 15 tomograms is ~2,000. Any idea if I may have done something odd/wrong with the file pathing?

Received "LoadGainRef", with 5 arguments, for GPU #0, 15253 MB free
Loaded gain reference and defect map: 11520, 8184, 1, normal, real, 1 A/px, ID = -1, False, False, False,
0.931

Received "TomoExportParticles", with 4 arguments, for GPU #0, 15253 MB free
Exported 4551 particles for C:\Tomo_Datasets\2023_10_12_CVB3_Hela_Kyle_Tomo_Movies\imod\Position_1.tomostar
30.221

Received "TomoExportParticles", with 4 arguments, for GPU #0, 0 MB free
System.Exception: cudaErrorIllegalAddress
   at Warp.GPU.CheckGPUExceptions() in D:\Dev\warp2\WarpLib\GPU.cs:line 1175
   at Warp.TiltSeries.ReconstructSubtomos(ProcessingOptionsTomoSubReconstruction options, float3[] positions, float3[] angles) in D:\Dev\warp2\WarpLib\TiltSeries.cs:line 2130
   at WarpWorker.Program.Main(String[] args) in D:\Dev\warp2\WarpWorker\Program.cs:line 238

On Wed, May 8, 2024 at 10:09 AM Zwei <kmessi...@gmail.com> wrote:

[Alister Burt]

Any idea why it may be trying to extract 4k particles from a tomogram I did not process with Cube (or knowingly process)? I tried to figure out where it was reading in coordinates from but I'm stumped. 

The method that writes particles out takes per-frame coordinates, the number you're seeing is likely n_particles * n_frames :)