16bit video from PointGrey camera

[Eirinn Mackay]

Hi all,
Has anyone had success getting 16bit video from a PointGrey camera into Bonsai? I've put the driver into "RAW 16" mode and FlyCap says it's getting 16bit frames, but Bonsai's FlyCapture node still reports depth=U8.

Cheers,
Eirinn

[Gonçalo Lopes]

Hi Eirinn,

You are correct, I did not account for that case. I attached a pre-release version with this possibility but google groups rejected because of the large size (you probably got it in your normal mail though). Please give it a try.

There are two ways in which it works with Raw16 images:

 1) if ColorProcessing is set to None, or there is no bayer tile format specified, the output will be a single-channel (bayer-mono) 16-bit image

 2) if you have ColorProcessing set to Default, the output will be a BGR 16-bit color image

This basically follows the pattern for Raw8 (which does work). My only doubt is whether 2) makes sense, because there is a 3rd possibility: you might want to actually output an 8-bit color image even if you have Raw16: the rationale is that resolving the BayerMask with a Raw16 gives better color precision even if your color is ultimately 8-bit (because of numerical rounding precision), which means you should still get nicer 8-bit color and get to save some space.

This may be relevant since a 16-bit BGR color image actually takes 6x more space than the equivalent grayscale Mono8 and most video encoders cannot even handle 16-bit video anyway...

So... I am slightly inclined at this point to resolve the color processing for Raw16 as an 8-bit video for the "debayered" case. Any thoughts? Can you tell me a bit more about what you were trying to get out of the Raw16?

Hope this helps.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/ca54dfc1-7d6d-45d3-a9ba-9f5172154d07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eirinn Mackay]

Hi Goncalo,
Thanks a lot for providing the updated node. I'm not sure it's working properly though... when colorProcessing is set to default, the resulting image contains two small versions of the video (side by side), and blank space underneath. The colours also seem to be a bit off.
Changing the camera mode to Mono 16 works: Bonsai reports U16, 1 channel, and the image looks fine. It's just this Raw 16 that doesn't work. It knows it's U16, 3 channels, but seems like the de-bayering isn't working, perhaps.

My goal here is to do some pre-processing (background subtraction) in 16-bit colourspace before rescaling to 8-bit for saving. Though actually - can I even use the BackgroundSubtraction node with 16-bit BGR images?
Cheers,
Eirinn

On Saturday, 10 March 2018 10:58:27 UTC, goncaloclopes wrote:

Hi Eirinn,

You are correct, I did not account for that case. I attached a pre-release version with this possibility but google groups rejected because of the large size (you probably got it in your normal mail though). Please give it a try.

There are two ways in which it works with Raw16 images:

 1) if ColorProcessing is set to None, or there is no bayer tile format specified, the output will be a single-channel (bayer-mono) 16-bit image

 2) if you have ColorProcessing set to Default, the output will be a BGR 16-bit color image

This basically follows the pattern for Raw8 (which does work). My only doubt is whether 2) makes sense, because there is a 3rd possibility: you might want to actually output an 8-bit color image even if you have Raw16: the rationale is that resolving the BayerMask with a Raw16 gives better color precision even if your color is ultimately 8-bit (because of numerical rounding precision), which means you should still get nicer 8-bit color and get to save some space.

This may be relevant since a 16-bit BGR color image actually takes 6x more space than the equivalent grayscale Mono8 and most video encoders cannot even handle 16-bit video anyway...

So... I am slightly inclined at this point to resolve the color processing for Raw16 as an 8-bit video for the "debayered" case. Any thoughts? Can you tell me a bit more about what you were trying to get out of the Raw16?

Hope this helps.

On 8 March 2018 at 06:46, Eirinn Mackay <eirinn...@gmail.com> wrote:

Hi all,
Has anyone had success getting 16bit video from a PointGrey camera into Bonsai? I've put the driver into "RAW 16" mode and FlyCap says it's getting 16bit frames, but Bonsai's FlyCapture node still reports depth=U8.

Cheers,
Eirinn

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

[Gonçalo Lopes]

Hi Eirinn,

Again the file size restrictions, you should have got it in normal mail. Let me know how it goes!

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/6ed43b32-75f7-4c60-9359-3e4eeab15c07%40googlegroups.com.

[Eirinn Mackay]

Hi Goncalo,
I don't see a difference with this new version - same side-by-side thing as before.
Cheers,
Eirinn

[Gonçalo Lopes]

Hmmm, it's possible then that the memory layout of these color images is not the same as OpenCV... there may be some padding for alignment, etc.

Let me see if I can grab hold of a camera that supports Raw16 so I can try the hardware directly.

Can you tell me the exact model you are using?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/26e52584-cc7e-46dc-a88a-2871b664fc36%40googlegroups.com.

[Eirinn Mackay]

Sure it's a Chameleon3 CM3-U3-13Y3C.

[Alexandra Tran Van Minh]

Hi Goncalo,

I am experiencing a similar problem : I am using a Grasshopper3, set in Monochrome 12-bit mode, at present reported as depth=U8 in Bonsai in the FlyCapture node. I was wondering if there's a way to also give a go to the pre-release version? My understanding from this thread is that in the monochrome case this may be already working well.

thanks,

Alex

On Saturday, 10 March 2018 10:58:27 UTC, goncaloclopes wrote:

Hi Eirinn,

You are correct, I did not account for that case. I attached a pre-release version with this possibility but google groups rejected because of the large size (you probably got it in your normal mail though). Please give it a try.

There are two ways in which it works with Raw16 images:

 1) if ColorProcessing is set to None, or there is no bayer tile format specified, the output will be a single-channel (bayer-mono) 16-bit image

 2) if you have ColorProcessing set to Default, the output will be a BGR 16-bit color image

This basically follows the pattern for Raw8 (which does work). My only doubt is whether 2) makes sense, because there is a 3rd possibility: you might want to actually output an 8-bit color image even if you have Raw16: the rationale is that resolving the BayerMask with a Raw16 gives better color precision even if your color is ultimately 8-bit (because of numerical rounding precision), which means you should still get nicer 8-bit color and get to save some space.

This may be relevant since a 16-bit BGR color image actually takes 6x more space than the equivalent grayscale Mono8 and most video encoders cannot even handle 16-bit video anyway...

So... I am slightly inclined at this point to resolve the color processing for Raw16 as an 8-bit video for the "debayered" case. Any thoughts? Can you tell me a bit more about what you were trying to get out of the Raw16?

Hope this helps.

On 8 March 2018 at 06:46, Eirinn Mackay <eirinn...@gmail.com> wrote:

Hi all,
Has anyone had success getting 16bit video from a PointGrey camera into Bonsai? I've put the driver into "RAW 16" mode and FlyCap says it's getting 16bit frames, but Bonsai's FlyCapture node still reports depth=U8.

Cheers,
Eirinn

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

[Gonçalo Lopes]

Hi Alex,

That would be yet another case: the mode I upgraded for Eirinn was Raw16 or Mono16. There is actually no support for Mono12 on OpenCV, they always require byte-aligned images in order to guarantee image processing efficiency.

One option is to upscale the bit depth to 16-bit, but that will generate videos with double the size. Also, there are currently few codecs that can even encode 16-bit videos. Why do you need the extra bit-depth? Is it for any specific kind of real-time signal processing, or for raw video encoding?

P.S.: @Eirinn, I didn't get a chance yet to try the code on a Chameleon3, so I don't know why the color 16-bit conversion doesn't work yet. I may only get a chance to do it next week, but will let you know as soon as I have more info.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/9cf29f7a-cdbc-4d3e-914d-71473a532fea%40googlegroups.com.

[Alexandra Tran Van Minh]

Hi Goncalo,

indeed I just tried the camera in mode Mono16 and indeed that shows as expected as U16 in Bonsai.
Regarding bit depth: I am looking for very small signal changes, at the moment the Matlab solution I am using (and trying to upgrade to Bonsai) involves doing processing and frame averaging before saving a collection of 16bit tiffs instead of the full 16bit video. Is there a way to do something similar through ImageWriter? And what would be an appropriate image format? I have tried it and just give a .tif extension to the saved file, but I can't open the resulting image so this is probably not the right way to do it...

thanks a lot!

Alex

[Gonçalo Lopes]

Hi Alex,

We actually learned how to generate TIFF stacks (saved individually) using VideoWriter in another question:
https://groups.google.com/forum/#!topic/bonsai-users/rPvhmDidsGo

Basically you can set the codec to uncompressed format and specify a pattern for saving images, and you can feed it any image sequence, including 16-bit.

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/e154bb24-c7a9-4070-ba87-636b3313c25f%40googlegroups.com.

[Alexandra Tran Van Minh]

thanks Goncalo,

the tiff saving is very useful! this works perfectly with the 16bit images, indeed.
cheers

Alex