image visualizer

[Johannes Larsch]

Hi,

I am new to Bonsai. I want to try it out for fish tracking and visual stimulation.

I am using an IDS uEye USB3 camera (UI337xCP-NIR_4102798564) and I can capture a video stream to an avi file using VideoCaptureDevice or CameraCapture linked to video writer. Applying transforms like thresholding also works. 

For some reason trying to hook lmageVisualizer into the CameraCapture I get an unhandled exeption error:

************** Exception Text **************

System.NullReferenceException: Object reference not set to an instance of an object.

   at Bonsai.Vision.Design.IplImageVisualizer.Unload()

   at Bonsai.Design.VisualizerDialogLauncher.<InitializeComponents>b__5(Object param0, FormClosedEventArgs param1)

   at System.Windows.Forms.FormClosedEventHandler.Invoke(Object sender, FormClosedEventArgs e)

   at System.Windows.Forms.Form.OnFormClosed(FormClosedEventArgs e)

   at System.Windows.Forms.Form.WmClose(Message& m)

   at System.Windows.Forms.Form.WndProc(Message& m)

   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)

   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)

   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

I am probably missing something very basic? 

Also, is there a newbie-type tutorial or example files on how to get started with simple video processing , e.g. thresholding and tracking of one (or multiple) object?

how do I properly configure CaptureProperties of CameraCapture

are there some sort of step by step instructions or example workflows how to extract features from an image and write to an output text? (tracking)

and how to configure the video writer? E.g. how to select codecs, compression rates etc.

any help is greatly appreciated

Johannes

[goncaloclopes]

Hi Johannes,

Welcome to the forums! I'm sorry for the long post, turns out there were lots to discuss here. Feel free to create additional questions for a more in-depth discussion of individual points.

Crash Report

If possible, can you attach an example workflow file or a screenshot of the workflow you are using?

Also, can you provide some specs of the computer you are using to run this? More specifically, does it have a dedicated graphics card or is it an integrated Intel HD graphics?

Finally, does it only happen on the CameraCapture? Can you use image visualizers on other nodes and/or the VideoCaptureDevice node?

Tutorials/Getting Started

Regarding examples, you can maybe start with the introductory crash course video at:

https://www.youtube.com/watch?v=srcqJXd6Vz4

I think it has most of the things you will need.

You can also find some examples in the Starter Pack package and also in the wiki. More specifically, you can check:

https://bitbucket.org/horizongir/bonsai/wiki/FAQ#!where-can-i-find-examples-of-bonsai-at-work

https://bitbucket.org/horizongir/bonsai/wiki/Tutorials (under construction)

For a more in-depth discussion about the framework internals there is always the paper:

http://journal.frontiersin.org/article/10.3389/fninf.2015.00007/abstract

Video Codecs

Regarding video codec configuration, VideoWriter does not currently support configuring compression rates. You can change the codec using the FourCC property. The list of FourCC codes is available at:

http://fourcc.org/codecs.php

However, not all of them are supported.

For raw video storage, you can use "DIB " (make sure to include the whitespace at the end)

In order to have more precise control over how the video is encoded, I recommend using "ffmpeg" command line. You can find it some windows binaries at:

http://ffmpeg.zeranoe.com/builds/

Then what you can do is use the ImageWriter node to pipe frames in real-time to ffmpeg.

If you would like to know more about this last option, would you mind asking a new question in the forums? It will get a bit involved, and this answer is long enough as it is :-)

Thanks,

Gonçalo

[Johannes Larsch]

Hi, thanks for the fast response!

attached is the workflow that works for me as long as I don't use the imageVisualizer.

The imageVisualizer produces errors on all nodes on my system. I have tried the video capture device node and also the two transform nodes in the example file (background subtraction and color conversion). All nodes work fine if I just save the output to a video file.

I am using a windows 7 64bit computer. Nvidia quadro fx 3500 graphics board.

It turns out I am getting the same error when I run any of your vision examples (see screenshot). Perhaps I have a more fundamental problem with my installation?

johannes

[goncaloclopes]

Hmmm, this looks like a problem initializing the OpenGL drivers. Are you running the latest graphics card drivers? If not, could you update them at Nvidia website?

From what I can see, that graphics card should run at least OpenGL 3.0 no problem...

[Johannes Larsch]

yes, updating nvidia drivers fixed the issue!

thanks a lot!

[Konstantin Hartmann]

This is a very helpful suggestion.

I had a very similar error when using visualizers to get a live feed of any video. Error message was "Object reference not set to an instance of an object." and it showed the same withe frame with red X. Updating the graphic card drivers solved that! Thank you!