Tilt series deselected with change_selection still processed

[Spencer Rothfuss]

Hello! I'm having trouble with tilt series deselection using WarpTools change_selection (docs).

After running fs_motion_and_ctf and ts_import, I can see that several of my tilt series aren't worth processing (ice contamination, bad tracking, grid bars blocking the view, etc.) so I run:

WarpTools change_selection --settings warp_tiltseries.settings --deselect --input_data tomostar/Position_1_4.tomostar

For each excluded tilt series I can confirm success by reading the XML (emphasis added):

grep UnselectManual warp_tiltseries/*.xml

warp_tiltseries/Position_1_2.xml:<TiltSeries DataDirectory="/data/warp/tomostar" AreAnglesInverted="False" PlaneNormal="0, 0, 0" LevelAngleX="0" LevelAngleY="0" Bfactor="0" Weight="1" MagnificationCorrection="1, 0, 0, 1" UnselectFilter="False" UnselectManual="" CTFResolutionEstimate="0">
warp_tiltseries/Position_1_4.xml:<TiltSeries DataDirectory="/data/warp/tomostar" AreAnglesInverted="False" PlaneNormal="0, 0, 0" LevelAngleX="0" LevelAngleY="0" Bfactor="0" Weight="1" MagnificationCorrection="1, 0, 0, 1" UnselectFilter="False" UnselectManual="True" CTFResolutionEstimate="0">

However, when I run later analyses all positions are processed even the "unselected" ones. For example, I ran ts_ctf and power spectra were generated for all positions:

ls -l warp_tiltseries/powerspectrum/
total 172M
-rw-rw-r--. 1 Spencer 25M Apr  4 17:15 Position_1_2.mrc
-rw-rw-r--. 1 Spencer 25M Apr  4 17:15 Position_1_4.mrc

(This is true for other commands also, for example ts_aretomo also processes all positions.) Am I misusing or misunderstanding change_selection?

Thanks!

---

FWIW, I'm running WarpTools 2.0.0 from SBGrid on Rocky 8.10.

[Hamidreza Rahmani]

Hi,

I am not sure but I believe the change selection is for images and it is hard to trust. 

I would just deselt the xml files in the warp_tiltseries/ directly for those bad tilt series.

Best,

Hamid

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/warp-em/4a5ced95-3cbc-422e-890c-84ce5a2fd8a3n%40googlegroups.com.

[Spencer Rothfuss]

Hmm. Okay. I've been using --input_data include.txt with a list of all tilt series except the ones I want to exclude but that's cumbersome. I'll try deleting XMLs.

What about excluding only certain tilts? For example, if I have 49 images in a tilt series and only the highest tilt is blocked/bad, can I exclude that and reconstruct a tomogram with the remaining 48 tilts? change_selection claims to work on warp_frameseries.settings also but now I'm dubious.

Thanks,

Spencer

[Hamidreza Rahmani]

For that I recommend editing the UseTilt section of the xml file.

I wrote a code to do it for etomo that you might find useful:

GitHub - hamid13r/warp_remove_skipped_views: Removing etomo skipped views from Warp xml files to improve the tomogram reconstruction. · GitHub https://share.google/6BFuW0XUqtpOXcLSo

[Spencer Rothfuss]

Thanks, Hamid. I already found your repo and tried that approach but got errors because the numbers of tilt images differed from the number of lines in the tomostar. If UseTilt=False still works for you, I can try to reproduce and share that error.

It seems like change_selection needs a code update or to be removed from the website. Make me a PR, Warp Bot! 😜

Spencer

[Hamidreza Rahmani]

That would be great, thank you. It is pretty messy and I havent found a way to make it robust. I will try to fix it if you send me the issue, I am also not very proficient and this is great practice for me.

Best,

Hamid

[Warp Bot]

Hi Spencer,

You're right, this is a bug. I traced through the code and can confirm: change_selection correctly sets UnselectManual="True" in the tilt series XML, but no downstream commands (ts_ctf, ts_aretomo, etc.) check that flag before processing. The root cause is in BaseOptions.cs -- all files matching the input pattern are loaded into InputSeries without filtering by UnselectManual. The processing loop then iterates over every item unconditionally.

I've filed a GitHub issue to track the fix.

For excluding entire tilt series, your --input_data include.txt workaround is the most reliable approach right now. Deleting the XMLs from warp_tiltseries/ as Hamid suggested also works.

For excluding individual tilts within a series: change_selection DOES work at the frame series level. You can deselect bad frames using --settings warp_frameseries.settings, then re-run ts_import. It checks UnselectManual on each frame series and excludes deselected frames from the assembled tilt series (ImportTiltseries.cs, around line 287). This is the most reliable approach for individual tilt exclusion.

The UseTilt field in the tilt series XML is the other mechanism. Each entry corresponds to one tilt, and setting it to False should exclude that tilt. However, be aware that excluded tilts may still be weighted at 0.0001 rather than truly zero in some reconstruction paths. For reliable exclusion, re-importing with the bad frames deselected at the frame series level is the safer route. The mismatch errors you saw with Hamid's script are likely because the number of UseTilt entries must match the number of tilts listed in MoviePath within the same XML.

-- Warp Bot
THIS IS AN AUTOMATED MESSAGE GENERATED BY AN LLM. IT MAY OR MAY NOT SOLVE YOUR PROBLEM. IF YOU'D LIKE TO SPEAK TO A HUMAN, SAY SO IN YOUR MESSAGE.

To view this discussion visit https://groups.google.com/d/msgid/warp-em/emc9eafbb4-d2ea-478f-b582-3970dc1b1ca5%40gmail.com.

[Spencer Rothfuss]

Thanks Warp Bot/Dimitry. It sounds like there's a couple layers here. Removing an entire tilt series is not supported (Warp Bot opened #462 for that). Removing a frame series (aka "a tilt") can be done in two ways:

1. Set UnselectManual="True" in the headmatter of warp_frameseries/*_Fractions.xml and re-run import (recommended by Warp Bot)
2. Set the boolean list <UseTilt>True...</UseTilt> in the body of warp_tiltseries/*.xml (used by Hamid's script)

#1 appears to work for me (at very least the .tomostar is changed and no errors with reconstruction) repeating ts_import was the step I was missing. However, when I try #2 by manually setting some to False and then run WarpTools ts_aretomo, I get an error:

Exception details:
System.Exception: 49 active tilts in series, but 39 lines in /.../warp_tiltseries/tiltstack/Position_1_5/Position_1_5_aligned_Imod/Position_1_5.xf

Do you get errors when editing <UseTilt>, Hamid?

[Spencer Rothfuss]

Thanks Warp Bot/Dimitry. There's a couple layers here. Removing a tilt series is not supported (other than by deleting warp_tiltseries/*.xml, Warp Bot opened an issue for that). Removing a frame series (aka "a tilt") can be done in two ways:

1. Set UnselectManual="True" in the headmatter of warp_frameseries/*_Fractions.xml and rerun import (recommended by Warp Bot)
2. Modify the boolean list <UseTilt>True ... </UseTilt> in warp_tiltseries/*.xml (used by Hamid's script)

#1 appears to work fine for me (at very least it updates the .tomostar and reconstructs without error), rerunning ts_import was the step I was missing. #2 does not, when I manually edit <UseTilt> and run ts_aretomo I get an error:

Failed to process /.../warp_tiltseries/Position_1_5.tomostar

...

Exception details:
System.Exception: 49 active tilts in series, but 39 lines in /.../warp_tiltseries/tiltstack/Position_1_5/Position_1_5_aligned_Imod/Position_1_5.xf

Can you edit <UseTilt> without error, Hamid?

On Sunday, April 12, 2026 at 9:10:27 PM UTC-5 Warp Bot wrote:

[Warp Bot]

Hi Spencer,

Glad approach #1 is working for you.

The error you see with approach #2 (manually editing UseTilt) is a separate bug. Here is what happens step by step:

1. ts_aretomo first calls StackTilts, which correctly reads your UseTilt entries and builds a stack with only the 39 tilts marked True.
2. AreTomo aligns those 39 tilts and produces a 39-line .xf file.
3. ts_aretomo then calls ImportAlignments to read the results back in. But ImportAlignments starts by resetting UseTilt to all True (TiltSeries.ImportAlignments.cs, line 17), making NValid = 49.
4. It checks whether the .xf has 49 lines. It has 39. Crash.

The only mechanism ImportAlignments has for learning about excluded tilts is a "_cutviews0.rawtlt" file (an eTomo convention listing excluded tilt angles). AreTomo does not produce this file for your manually excluded tilts, so ImportAlignments has no idea the stack was smaller.

In short, StackTilts respects your UseTilt edits but ImportAlignments does not. They disagree and the process crashes. I have filed a separate issue for this bug.

For now, approach #1 (deselecting bad frames in warp_frameseries via change_selection and re-running ts_import) is the reliable way to exclude individual tilts.

-- Warp Bot
THIS IS AN AUTOMATED MESSAGE GENERATED BY AN LLM. IT MAY OR MAY NOT SOLVE YOUR PROBLEM. IF YOU'D LIKE TO SPEAK TO A HUMAN, SAY SO IN YOUR MESSAGE.

To view this discussion visit https://groups.google.com/d/msgid/warp-em/e8a316f8-fb5e-46de-abca-22c6b8b95971n%40googlegroups.com.