Aligning gridsquares to atlas on small squares (shift to marker offset)

[Willem Noteborn]

Hey Jonathan,

Recently we've noticed a pattern in smartscope crashes on one of our systems when the software tries to realign to relative small squares. It seems that there is no correction applied where a stored shift (shift to marker offset) between the atlas magnification and search/view is in place and needed. Could this be true and if so could it be implemented that this shift is applied everytime atlas collection is completed so that centering of even tiny squares gets more robust?

Best,

Willem

[Jonathan Bouvette]

Hi Willem,

You're correct, the shift to marker correction is not applied in scripts happening outside of the navigator in SerialEM. I actually have fields in the Detector object for that kind of correction (Atlas to search offset x and y),  never finished implementing it since the recentering took care of most of it.

Just to clarify something about the small squares: Is it that the square ends up fully or almost fully ousitde of the image or is it there visible and not recentered?

I have an idea on how to set this up. Let me test a few things.

Best,

Jonathan

--
You received this message because you are subscribed to the Google Groups "SmartScope" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smartscope+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smartscope/27f4eb0e-ee96-4924-996d-675e40fc1b4dn%40googlegroups.com.

[Willem Noteborn]

Hey Jonathan,

Typically it ends up with a very small piece of the thick edge of a square (the dark outer ring part you would normally not even consider collecting anything on) in the very bottom corner of the the gridsquare image. Basically not even with a single foilhole identifiable by eye. I guess it won't be enough for the algorithm to catch onto.

Best,

Willem

Op dinsdag 3 oktober 2023 om 15:48:14 UTC+2 schreef jonathan...@gmail.com:

[Jonathan Bouvette]

Hi Willem,

I've built in the offset behavior that you can start testing. You may add values to the detector item for the stage offset in X and Y in the admin portal or have smartscope estimate the values from the recentering events that occured on previous runs.

The command to estimate the offset is: 

./smartscope.sh run get_atlas_to_search_offset DETECTOR_NAME

where DETECTOR_NAME is the name of the detector object from the admin portal.

The script will query all acquired squares and calculate mean and standard deviation of the shift. At the end of the calculation, it will prompt for a y/n answer to save the values in the database.

This new feature along with the skip ability is in the 0.9.2-rc.4 docker image.

Best,

Jonathan

[Willem Noteborn]

Hey Jonathan,

Lovely, I'll try both and see if there is a big difference!

Best,

Willem

Op zaterdag 7 oktober 2023 om 18:21:44 UTC+2 schreef jonathan...@gmail.com:

[Willem Noteborn]

Hey Jonathan,

I've finally managed to get some time to test again (now on 0.9.2!) but I can't seem to get the offsets to work. I'm still one square over most of the time. What are the units for the offsets? Are they in microns?

Best,

Willem

Op zaterdag 7 oktober 2023 om 19:32:15 UTC+2 schreef Willem Noteborn:

[Jonathan Bouvette]

Hi Willem,

Yes, the units are in microns. This procedure will work if we're trying to resolve the shift between the magnifications at the optics level. This shift is not dependent on stage position. The effect can usually be seen if centering on a feature with the flu screen at the square mag and then just lowering the magnification with the control pad. There is usually a shift that increases as the mag decreases (i.e. your feature will not be centered anymore).

Did you try to run the command to estimate the shifts or did you try to calculate it from navigator points? If the shift is worse, maybe try inverting the sign in the detector object in the admin portal.

Other issues that would affect the square targeting can be related to the rotation angle and the pixel size calibration in LM. The effect for this type of issue is usually less when the target is centered on an atlas tile vs at the edge.

Best,

Jonathan

To view this discussion on the web visit https://groups.google.com/d/msgid/smartscope/80c090ab-a087-4547-bc31-8c1bd61ac679n%40googlegroups.com.

[Willem Noteborn]

Hey Jonathan,

Yes, I've tried the estimation command. That's where the initial confusion started in my mind. The values were drastically different from "my own" manual shift to marker. The command works lovely, but the issue arises that when the calculated shift is a shift that still sends you to the neighboring wrong square, you are correcting wrongly. So basically a good option for the shift was an incorrect square but that did give a good fit. After using the manual shift again, things worked out fine.

The hole finding and centering now works beautifully!

Unfortunately, now the next issue comes to hand after some extensive testing: if a small thick ice square is found, then it seems that the ptolemey hole finder cant properly find any holes (I guess), throws an error and ends the session. Is there any way to circumvent that?

Attached an example atlas and the error text.

Still very good progress though in my opinion!

Best,

Willem

Op donderdag 7 december 2023 om 19:57:53 UTC+1 schreef jonathan...@gmail.com:

[Jonathan Bouvette]

Hi Willem,

Thanks again for the valuable feedback. I will look into the function that calculates the shift as there may be an issues with the calculation.

As for the hole finding crashing. This is a new bug during the hole selection process it seems. I will look into it and try to fix this issue.

Although that doesn't solve the annoyances of a crashed run, there is a now a way to Skip an activate target that got stuck in an error by clicking it and selecting Skip in the action dropdown before restarting the run.

Best,

Jonathan

[Willem Noteborn]

Hey Jonathan,

I think the shift issue is instrument dependent and your calculations are sound. Most likely our LM TEM mode alignments could use some love. In EFTEM it is not an issue and things are aligning well, also with your functions. We only have a bit of a wonky K2 on the test system. Therefore, our Falcon3EC is a bit more of a robust test beast. So it only seems to fail when the offsets between the chosen mags in the specific imaging modes are quite off. Should in my opinion not be an issue that your software would have to be able to correct for. Just a bit of context! ;)

Best,

Willem

Op dinsdag 12 december 2023 om 17:03:52 UTC+1 schreef jonathan...@gmail.com:

[Jonathan Bouvette]

Hi Willem,

I just pushed a new docker image to fix the crash when no holes are found and some other minor things `./smartscope.sh update latest` to update.

Best,

Jonathan