Error during M core Refinement

Reika Watanabe

unread,

Dec 23, 2024, 4:08:49 PM12/23/24

to Warp

Hello

I have encountered an error during M refinement.

By following a tutrial with my data, it worked until test refinement (iteration 0).

When I started the first refinement iteration, I got the following error message (highlighted in red).

Is this something to do one bad tilt series data? or something to do with my setting?

Thank you very much for your kind attention and happy holidays!

Reika

MTools create_population \

> --directory /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M \
> --name NC3A
Running command create_population with:
directory = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M
name = NC3A

Population created: /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population
(warp) [rwatanabe@c110144 NP_delta601_all]$ MTools create_source \
> --name NC3A \
> --population /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population \
> --processing_settings /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries.setting \
>
Running command create_source with:
population = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population
processing_settings = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries.setting
name = NC3A
nframes =
files =

Looking for data... found 67 files
Loading metadata... 67/67

Adding 67 items.
Calculating data hashes... 67/67
Committing initial version... Done
Data source created: /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/NC3A.source
(warp) [rwatanabe@c110144 NP_delta601_all]$ MTools create_species \
> --population /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population \
> --name EBOVNC \
> --diameter 240 \
> --temporal_samples 1 \
> --half1 /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_half1_class001_unfil.mrc \
> --half2 /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_half2_class001_unfil.mrc \
> --mask /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/mask_3apx.mrc \
> --particles_relion /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_data.star \
> --angpix_resample 2.14885 \
> --lowpass 11
Running command create_species with:
population = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population
name = EBOVNC
diameter = 240
sym = C1
helical_units = 1
helical_twist = 0
helical_rise = 0
helical_height = 0
temporal_samples = 1
half1 = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_half1_class001_unfil.mrc
half2 = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_half2_class001_unfil.mrc
mask = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/mask_3apx.mrc
angpix =
angpix_resample = 2.14885
lowpass = 11
particles_relion = /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/relion/Refine3D/job018/run_data.star
particles_m =
angpix_coords =
angpix_shifts =
ignore_unmatched = False

Reading maps... Done
--angpix not specified, using 3.0000 A/px from half-map.
Resampling maps to 2.1488 A/px... Done
Padding or cropping half-maps to 2x molecule diameter... Done
Padding or cropping mask to 2x molecule diameter... Done
Processing half-maps... Done
Parsing particle table... Done
Calculating resolution and training denoiser model...
4/5: Training denoising: Adjusting the number of iterations to 1500 to match bat5/5: Applying denoising
Calculating particle statistics... Done
Committing results... Done
Species created: 'EBOVNC' (856f94d8-322f-4d02-a858-b774f7a66bdb), /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/species/EBOVNC_856f94d8/EBOVNC.species
To check if everything went alright, it's best to run M once with all refinements turned off and see if the new maps resemble your input.
(warp) [rwatanabe@c110144 NP_delta601_all]$ MCore \
> --population /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population \
> --iter 0
Loading population... Done
Creating directories... Done
Spawning workers... Done
Preparing for refinement – this will take a few minutes per species
Preparing refinement requisites...
1/1
Performing refinement
Preparing population for data source NC3A...Done
Loading gain reference for NC3A... Done
Refining all series in data source...
67/67
Commiting changes in NC3A...Done
Saving intermediate refinement results for NC3A...Done
Finishing refinement
Gathering intermediate results, then reconstructing and filtering...
EBOVNC: 10.10 Å
1/1
(warp) [rwatanabe@c110144 NP_delta601_all]$ MCore \
> --population /mnt/staging/rwatanabe/DataProcess/Warp_tool/NP_delta601_all/warp_tiltseries/M/NC3A.population \
> --refine_imagewarp 6x4 \
> --refine_particles \
> --ctf_defocus \
> --ctf_defocusexhaustive \
> --perdevice_refine 4
Loading population... Done
Creating directories... Done
Spawning workers... Done
Preparing for refinement – this will take a few minutes per species
Preparing refinement requisites...
1/1
Performing refinement
Preparing population for data source NC3A...Done
Loading gain reference for NC3A... Done
Refining all series in data source...
54/67Unhandled exception. System.Exception: BadRequest: System.Net.Http.HttpConnectionResponseContent
at Warp.WorkerWrapper.SendCommand(NamedSerializableObject command) in /usr/share/miniconda/envs/package-build/conda-bld/warp_1732560643671/work/WarpLib/WorkerWrapper.cs:line 231
at Warp.WorkerWrapper.MPARefine(String path, String workingDirectory, ProcessingOptionsMPARefine options, DataSource source) in /usr/share/miniconda/envs/package-build/conda-bld/warp_1732560643671/work/WarpLib/WorkerWrapper.cs:line 458
at MCore.MCore.<>c__DisplayClass16_4.<DoProcessing>b__4(Int32 ifile, Int32 threadID) in /usr/share/miniconda/envs/package-build/conda-bld/warp_1732560643671/work/MCore/MCore.cs:line 473
at Warp.Tools.Helper.<>c__DisplayClass51_0.<ForCPU>b__0(Object id) in /usr/share/miniconda/envs/package-build/conda-bld/warp_1732560643671/work/WarpLib/Tools/Helper.cs:line 719
Aborted (core dumped)

Reika Watanabe

unread,

Dec 23, 2024, 4:48:26 PM12/23/24

to Warp

Here is detail info:

I had processed with Relion5/particle series and final 3d refinement gave 9.3 A (3A pixel size).

Moving to M, I used 2.14885 A (original pixel size).

The WARP VERSION

# Name Version Build Channel

warp 2.0.0dev31 py311_0 warpem

Thank you very much!

Reika

Alister Burt

unread,

Dec 23, 2024, 9:15:31 PM12/23/24

to Reika Watanabe, Warp

Hi Reika,

It looks like the worker crashed, probably out of memory as your pixel size for refinement is small - try again with a larger pixel size and see if that helps!

Cheers and happy holidays,

Alister

Sent from mobile - apologies for brevity

On Dec 23, 2024, at 13:48, Reika Watanabe <reika.wa...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/warp-em/124700ff-7422-4453-9ad9-d27c6ca8fddbn%40googlegroups.com.

Reika Watanabe

unread,

Dec 25, 2024, 11:49:23 AM12/25/24

to Alister Burt, Warp

Thank you very much for your timely reply, Alister, as always!

By removing

--perdevice_refine 4

from the any refinement steps solved the issue.

I have not tried such as 2, but, the refinement steps done with reasonable time (20-30 min so).

I have 55k particles from 67 tilt series with 240 diameter particles.

With 2 GPU (RTX3090), it worked well.

Thanks a lot for helping us tremendously! Happy holidays for everyone!

Reika

REIKA WATANABE CASTILLON

E-mail: reika.wa...@gmail.com
Address: 3211 CaminitoEastbluff unit 12, La Jolla 92037, CA, USA
Mobile:1(858)242-6090

Website: https://www.linkedin.com/pub/reika-watanabe/69/6ab/7b8

Huy Bui

unread,

Feb 13, 2025, 9:00:48 AM2/13/25

to Warp

Hi Alister,

I have a similar problem here with a 4090 card. Pixel size 2.12, box size 378 pixel. When working with 10000 particles, M run no problem and get to 7 Angstrom. When I ran with 100,000 particles, running the --iter 0 to reconstruct get to 5.7 Angstrom. Then every step of Mcore would crash when getting to "Refining all series in data source...".

How can I know my card is enough for M?

Best,

Huy

Reply all

Reply to author

Forward