applying selection filters

[Juha Huiskonen]

Hi,

I am plotting histograms after frame alignment and CTF fitting:

WarpTools filter_quality --settings warp_frameseries.settings --histograms

Then I am removing some tilts with this command:

WarpTools filter_quality --settings warp_frameseries.settings --defocus 2.4 --resolution 1 10 --motion 0 13  --output warp_frameseries_filtered.txt

However, the removed tilts are not updated in the settings file.  I would expect them to be removed when I run the histogram plotting again.  How do I exclude the bad tilts before the next step?

Cheers,

Juha

[Alister Burt]

Hi Juha,

The settings file does not store selection state, you can pass your warp_frameseries_filtered.txt file as --input_data to any other WarpTools program, including change_selection, if you want your updated selection to be stored in the frame/tilt series metadata.

Cheers,

Alister

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/403c40c6-b620-4835-8a9c-f202fe9fff64n%40googlegroups.com.

[Alister Burt]

Hi Juha,

This makes sense to me, I want to be able to see the full distribution when deciding on filters...

Cheers,

Alister

On Mon, 21 Oct 2024 at 22:28, Juha Huiskonen <juha.hu...@gmail.com> wrote:

Hi Alister,

Thanks, I managed to do this but it seems that when histograms are
plotted the flag in the XML file is ignored and the stats are plotted
for all particles.

Best wishes,
Juha

[Alister Burt]

Hi Juha,

I chatted with Dimitry about this today - his perspective is that the selections in WarpTools only determine which movies should be included in a tilt-series and don't affect whether items are processed by individual programs. The selection mechanisms are a holdover from Warp v1... this didn't match my expectations (or the subset selection documentation that I may need to change) and I would be interested to know if there are any exceptions to this in WarpTools. 

As a consequence, the 'approved' way to achieve what you want is to specify --input_data in your call to WarpTools filter_quality.

Cheers,

Alister

On Thu, 24 Oct 2024 at 01:09, Juha Huiskonen <juha.hu...@gmail.com> wrote:

Hi Alister,

For plotting histograms, it would be nice to have an option to show
only selected (or unselected) to check that the filters have worked
(and the effect of one filter on other parameters).

Regarding the use of filters and how they are taken into account in
the following data processing steps, I have another question.  If I
remove one tilt series with this command

WarpTools change_selection \
--settings warp_tiltseries.settings \
--deselect \
--input_data tomostar/lamella7_ts_008.tomostar

I would expect this to be removed from all next processing steps.
However, when I check for example handedness, it is still being used.
I understand that the selection is indicated in the XML files.  But
what processing steps actually consider this flag?

Cheers,
Juha

[Alister Burt]

opened an issue to track: https://github.com/warpem/warp/issues/280

[Juha Huiskonen]

Hi Alister,

For plotting histograms, it would be nice to have an option to show
only selected (or unselected) to check that the filters have worked
(and the effect of one filter on other parameters).

Regarding the use of filters and how they are taken into account in
the following data processing steps, I have another question. If I
remove one tilt series with this command

WarpTools change_selection \
--settings warp_tiltseries.settings \
--deselect \
--input_data tomostar/lamella7_ts_008.tomostar

I would expect this to be removed from all next processing steps.
However, when I check for example handedness, it is still being used.
I understand that the selection is indicated in the XML files. But
what processing steps actually consider this flag?

Cheers,
Juha