Overlay two 3D-arrays in ImageView

[Alex No]

Hello everyone! 

I am relatively new to pyqtgraph and recently I have been using it for a GUI that visualizes medical image data. What you can do is drag and drop files into an ImageView, visualize for example a CT image and scroll through the CT-volume (since pyqt graph nicely detects the 3d array and adds the third axis as "time" axis which in this case allows to scroll through the volume). 

I also have image masks of the same dimension as the original image (also 3d numpy arrays) and I want to overlay image and mask and ideally adapt the transparency of both. Currently I am doing this by calculating a completely new array which combines both image mask and image (I normalize both and then create a weighted sum of both arrays). This works but has some disadvantages, for example I cannot color mask and image separately. 

My Question is now: do you know another way how I can overlay both 3D-arrays (mask + image) as separate layers while keeping the possibility to scroll through the volume, such that both mask and image change when scrolling through the volume (see images)? I tried around with the addItem method defined in the ViewBox class, but without any success. 

I would really appreciate some help and ideas. 

Kind regards and thanks in advance

Alex

Image at certain z-value of volume:

Image after scrolling through volume at different z-value:

[Patrick]

Hi,

I'm not totally sure I understand the question, but if you want to just mask off regions in the image you could simply merge the mask information to the alpha channel of the image. ImageItem pixel data (for a single image frame) can be four dimensional (R, G, B, A), where A is the alpha/transparency. Set the A channel according to your mask (0 = transparent, 255 = opaque). For the time series data you'll just push that dimension along one. You'll need to play with the various numpy slicing/concatenation methods to merge the data correctly.

As a very simple example, if you look at the ImageView example, you can insert/change the following and the images will have that alpha/mask info. This is a poor example since every pixel has the same alpha=128, but it should give you the idea:

dataAlpha = 128*np.ones_like(dataRed)

data = np.concatenate(

    (dataRed[:, :, :, np.newaxis], dataGrn[:, :, :, np.newaxis], dataBlu[:, :, :, np.newaxis], dataAlpha[:, :, :, np.newaxis]), axis=3

)

Patrick

[Alex No]

Hi Patrick,

thanks for your answer. If I understand your idea correctly this would just allow me to change the alpha value of the image regions which correspond to the mask. 

But I'd like to have mask and CT image as two separate images overlayed on top of each other. This way I would have two levels, which I could control individually, but I am unsure if that's even possible in pyqtgraph. So basically I would like to add more than one image item (3d arrays) to the image view which are overlayed and both change when using the "volume" slider. What works for example is adding other kinds of items like a grid item which does not change when using the slider. 

Kind regards 

Alex

[Martin Chase]

Hey Alex,

You can have multiple ImageItems inside a single ViewBox ( via multiple calls to `vb.addItem(img)` ), and as long as you set appropriate zValues and alpha values, one of the images can be semi-transparent over the other. The app I work on[1] uses this feature, if you want to see it in use.

Good luck,

 - Martin (he/him)

1 - https://github.com/acq4/acq4/blob/main/acq4/util/imaging/imaging_ctrl.py#L252

--
You received this message because you are subscribed to the Google Groups "pyqtgraph" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyqtgraph+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/8299b9a7-92ee-4f5e-be2a-6546b0ff27f0n%40googlegroups.com.

[Alex No]

Hi Martin! Thank you, I will check it out :)