Create projects with source type 1 "File-based image stack"

[youmin Wu]

Hi all,

I ran into some problems when I tried to create some projects with source type 1 "File-based image stack" with several zoom-levels.

I have 2 projects, one project has 1 stack with 1 stack mirror, the other has 1 stack with multiple stack mirrors.

The problem is that the browser displayed incompletely in the 2nd project with zoom_level=0;

The images has 2 zoom-levels, they are organized like this:

The 1st project's config is under below:

When I chose zoom-level=0, it displayed four images 0_0_0.jpg, 0_1_0.jpg, 1_0_0.jpg and 1_1_0.jpg under below:

The weird thing happened in the 2nd project.

The config of 2nd project is under below:

When I entered the 2nd project, chose the stack mirror with z-index=6 and zoom-level=0, it just displayed one image 0_0_0.jpg like this:

Anyone can help me to figure out what's wrong with my settings?

Any help would be greatly appreciated.

Thanks a lot in advance!

[Tom Kazimiers]

Hi Youmin,

On Wed, Sep 04, 2019 at 04:21:00AM -0700, youmin Wu wrote:
>I ran into some problems when I tried to create some projects with
>source type 1 "File-based image stack" with several zoom-levels.
>
>I have 2 projects, one project has 1 stack with 1 stack mirror, the other
>has 1 stack with multiple stack mirrors.
>
>The problem is that the browser displayed incompletely in the 2nd project
>with zoom_level=0;

It looks like the documentation should explain this better, but if I
understand your current setup correctly, you confuse zoom levels with
stack mirrors.

Stack mirrors are intended to hold all the information to access the
actual image data for a particular stack. If you will, the stack stores
"what" and the stack mirrors store "where" and "how". Each stack mirror
of a stack is expected to provide access to the exact same image data
set, possibly hosted somewhere else or with a different tile source
type. Stack mirrors are mainly there to make it possible to host the
image data for a stack at different places and users or the front-end
can choose whichever mirror is physically closer or faster accessible
and then select this mirror to lower latency when loading image data.

Zoom levels are *not* expressed through stack mirrors. Zoom levels are a
property of a stack model and is expressed through the downsample
factors setting for each stack. All mirrors of a stack are expected to
provide the same number of zoom levels. By default CATMAID (downsample
factors setting: CATMAID default) will assume a scale pyramid where each
zoom level above zoom level zero is half the size of the previous in the
X and Y dimensions, ignoring Z. Assuming a dataset of 1000x1000x1000 px
in zoom level 0, the CATMAID default downsampling would assume
500x500x1000 px in zoom level 1 and 250x250x1000 px in zoom level 2, and
so on. CATMAID keeps on scaling down until either the X and Y dimension
is below 1024px, which can of course be adjusted if wanted.

The reason Z isn't scaled by default is that a common use case in
CATMAID is to browse through individual sections of anisotropic image
data. If you work with isotropic image data, you might want to scale Z
as well, which can be achieved through custom downsample factors.

If you have an anisotropic dataset, I would suggest to just keep the
CATMAID default downsample factors (which would typically mean, CATMAID
will generate the zoom levels for you by "halfing" the X and Y
dimensions until one of the dimensions hits a lower limit, resulting in
an explicitly stored downsample pyramid stored for the stack).

We really should add some documentation on how this works.

For CATMAID to find and use the zoom levels of your dataset, you don't
have to do anything else. And like said above, all mirrors are expected
to have the same data and hence the same number of zoom levels. The tile
source type determines how the different zoom levels are actually
accessed.

>The images has 2 zoom-levels, they are organized like this:
>[image: 1.png]

Yes, for tile source type 1, tiles are organized with a folder for each
Z coordinate (you are in folder 0) and each file follows the pattern
<row>_<col>_<zoomlevel>.<fileextension>. In your case zoom level 0 has
four tiles, and zoom level 1 has one tile.

The documentation has more details on this [2].

>The 1st project's config is under below:
>[image: 2.png][image: 3.png]

That looks correct to me for the tiles you show above. The downsampling
factors setting says there is one additional zoom level (0 and 1 in
total, i.e. original dimensions and zoom level 1 with half the X/Y
dimensions).

>When I chose zoom-level=0, it displayed four images 0_0_0.jpg, 0_1_0.jpg,
>1_0_0.jpg and 1_1_0.jpg under below:
>[image: 4.png]

Does it also display zoom level 1 correct for this stack?

>The weird thing happened in the 2nd project.
>The config of 2nd project is under below:
>[image: 2.png][image: 6.png]

The stack mirror setup doesn't seem right. It didn't sound as if you
wanted to have seven copies of the stack mirror. The downsampling is set
up like with your first stack, i.e. CATMAID expects one extra zoom level
besides zoom level zero, which is 512x512x1. That extra zoom level would
be expected to be 256x256x1 px with the current setup.

If the different stack mirrors in your current setup represent different
zoom levels, this would mean there seven different ones, where zoom
level 0 wouldn't be scaled and the six other "mirrors" (which they
aren't, if they are zoom levels), scaled 50% in X and Y with each level.
To let CATMAID know about this, select "Factor-2 downsampling scale
pyramid" with 7 scale levels. CATMAID should figure out the access to
the individual zoom levels on its own. Assuming you don't have any actual
mirrors, you only want to have a single stack mirror entry, pointing to
your image data. This image data is assumed to contain all zoom levels.

The "position" field of the stack mirrors only tells the front-end in
which order stack mirrors should be tried. In case one isn't accessible,
the next one in this order is tried.

>When I entered the 2nd project, chose the stack mirror with z-index=6
>and zoom-level=0, it just displayed one image 0_0_0.jpg like this:
>[image: 7.png]

It doesn't look like you actually have 6 Z sections in your dataset. The
"z-index" slider is in fact the Z coordinate of your stack, i.e. 0 is
the first slice, 1 the second and so one. According to your above setup,
you only have one Z section though (Z dimension of stack setup).
Therefore, I believe the front-end doesn't actually accept the "6" you
put into the z-index field and it remains at the first (and only) Z
section. It should probably display all four tiles though. If you make
the above changes (i.e. only one mirror, 7 scale levels for the factor-2
downsampling in X and Y), does it still behave like that?

I hope this could bring some clarity on what stacks and stack mirrors
are supposed to be and how zoom levels are handled. We are certainly
lacking some documentation to make these points clearer.

Best,
Tom

[1] https://catmaid.readthedocs.io/en/stable/importing_data.html#importing-project-and-stack-information
[2] https://catmaid.readthedocs.io/en/stable/tile_sources.html

[youmin Wu]

Hi Tom,

Thank you very much for your detailed explanation! Yes, you are right I misunderstand the usage of stack mirror.

And I tried to set up the stack mirror following your instructions, it worked well. Thanks again!