White Spots Created in Warp, but not RELION5

[Luke]

Hello all, 

When processing a series of tomograms in warp, I'm observing some artificial bright white spots shown here:

The white spots spread along the rotation axis as the tilt increases.

When I process in RELION5 using the same gain, reconstruction dimensions, and dose parameters the white dots are not present.  I have also used both a .gain and tif2mrc converted gain thinking that could be the issue. 

Does anyone have any idea what might be different between the two reconstructions?  

[Huy Bui]

Hi Luke,
You can look at the stack from Warp & Relion if you see very bright pixels there (or even the motion corrected micrographs). There are certain times that our camera produces a lot of hot pixels that cannot be corrected using the gain reference and might/might not be addressed using IMOD Xray removal.

So, at least you can conclude whether Relion corrects for those hot pixels better.

Best,
H

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/warp-em/22476e4b-cbc3-4546-81c3-670f6477ed0an%40googlegroups.com.

[Marten Chaillet]

Hi Luke,

Those hot pixels don't get corrected by the gain file. I think IMOD and RELION do some automatic thresholding to filter out these pixels while Warp does not. The ringing you see around them happens because of the CTF correction during reconstruction. In the averaged frames they should visible as single pixels. 

I had the same issue and managed to correct it by creating a defects file myself (I probably have some notes on it somewhere but need to search). But to get it working I had to make a bug fix and make a custom installation of WarpTools. And my PR has not been merged yet: https://github.com/warpem/warp/pull/45 .

If I find the notes I will post them :)

Cheers,
Marten

Op dinsdag 7 april 2026 om 20:06:03 UTC+2 schreef huy...@gmail.com:

[Marten Chaillet]

I found it: https://gist.github.com/McHaillet/dfddd32eff547198c5d6e091b9e2ac32 

I think masking above 8 standard deviations is what IMOD does by default.

Op woensdag 8 april 2026 om 15:51:33 UTC+2 schreef Marten Chaillet:

[Luke]

Hello Marten and Huy, 

Thank you both for pointing this out.  Specifically thank you Marten for sending me your script, I'll give that a shot and report back. 

--Luke 

[Warp Bot]

Hi Luke,

Huy and Marten are correct -- this is a hot pixel issue. To add some detail from the codebase:

In the WarpTools CLI, the automatic X-ray pixel correction (GPU.Xray) that the old Warp GUI used is disabled. Hot pixel removal only happens if you provide a defects file explicitly. RELION, by contrast, does its own automatic outlier thresholding during preprocessing, which is why you don't see the spots there.

To fix this in WarpTools, you can supply a defects file when creating your settings:

WarpTools create_settings [...] --defects_path path/to/defects.mrc

The defects file should be an MRC image the same size as your frames, with non-zero values marking bad pixels. Marten's script (linked above) that identifies hot pixels by thresholding at several standard deviations above the mean is a good approach for generating one.

Note that a bug in defects file handling was fixed in PR #65 (https://github.com/warpem/warp/pull/65), which has been merged into the current version.

— Warp Bot
THIS IS AN AUTOMATED MESSAGE GENERATED BY AN LLM. IT MAY OR MAY NOT SOLVE YOUR PROBLEM. IF YOU'D LIKE TO SPEAK TO A HUMAN, SAY SO IN YOUR MESSAGE.

--

[teg...@gmail.com]

I'll add that this should be ideally solved via gain reference. Outlier removal existed in earlier versions, but using a threshold that works well for SPA data (like 8) generated too many false positives in much noisier tilt series frames. I think I set it to 20 sigmas at some point. I don't have hands-on experience with old CCD data, but my impression was that direct detectors were much less susceptible to the assumed source of non-persistent hot pixels in images: X-rays. And since every persistent hot pixel should be captured by the gain reference, I just disabled outlier detection completely.

Cheers,

Dimitry