Save video with tracking overlaid

[Sarah Anne Tennant]

Hi,

I've just started using Bonsai to track the movement of a mouse in an open field. So far I've got a workflow that superimposes the trajectory onto the cameracapture visualizer but im unsure how to save the video with the trajectory superimposed. Is this possible?

Ive uploaded an image of the workflow - I can upload the actual file if necessary.

Thank you,

Sarah

[goncaloclopes]

Hi Sarah and welcome to the forums!

Nice to hear you got Bonsai to work for you. I guess the answer to your question will depend slightly on the goal of the video.

1) If your goal is to show a small clip of Bonsai in action (for illustration/presentation purposes), then the easiest way to do this is actually to use the ScreenCaptureStream node (included in the Bonsai.Video package). This node will capture regions of your screen (full-screen by default) and you can save that video stream to a file. This will capture everything that is happening on-screen, including any visualizations you might want to display and is usually fast enough for viewing purposes (~30 Hz at least).

2) If on the other hand you really want the visualization to be a data output from your workflow (e.g. if you want to make sure to include all processed frames, etc), then you will have to draw the trajectory as part of the data processing workflow. Attached you can find a workflow to do this, which is the following:

Basically this is a normal tracking workflow but at the end you extract the centroid of the largest object and Zip it with the image where you want to draw the trajectory. Then I wrote a piece of drawing code in a PythonTransform like so:

import clr
clr.AddReference("OpenCV.Net")
from OpenCV.Net import *

# Accumulators for keeping history
trajectory = None
prevpoint = None
color = Scalar.Rgb(255,0,0)
thickness = 3

@returns(IplImage)
def process(value):
  global trajectory, prevpoint
  centroid = value.Item1
  image = value.Item2

  # Initialize trajectory image accumulator if needed
  if trajectory is None:
    trajectory = IplImage(image.Size, image.Depth, 3)
    trajectory.SetZero()

  # Draw point or line depending on whether there is a past history
  if prevpoint is None:
    CV.Circle(trajectory,Point(centroid),thickness,color,-1)
  else:
    CV.Line(trajectory,Point(prevpoint),Point(centroid),color,thickness)

  # Draw trajectory on top of image
  output = image.Clone()
  mask = IplImage(image.Size, image.Depth, 1)
  CV.CvtColor(trajectory,mask,ColorConversion.Bgr2Gray)
  CV.Threshold(mask,mask,0,255,ThresholdTypes.Binary)
  CV.Copy(trajectory,output,mask)

  # Update history for next drawing
  prevpoint = centroid
  return output

# Reset accumulators
def unload():
  global trajectory, prevpoint
  trajectory = None
  prevpoint = None

What this basically does is accumulate a trajectory mask over frames and then renders that colored mask on top of the image. It's true that it would be useful to have this provided as a built-in node given that it has been asked many times before, so I will definitely consider packing it into a node for the next release. For now, this is the usable workaround.

Hope this helps.

Best,

[Sarah Anne Tennant]

Thank you for your informative and swift reply!! I tried the workflow attached and it worked beautifully first time!! I couldn't be happier! Thank you so much, I just edited it to include a csv reader and timestamp and it works perfectly for my application!

Really beautiful bit of software you've built here, im very grateful.

Thank you again for your help :)

Sarah

 

[Evgeny Resnik]

Hi Goncalo,

The PythonTransform you suggested to overlay a cumulative animal track with the video stream works nicely with a Source.Image (+ extracted Centroid, of course) as a source, but generates an error message when i feed an output (also IplImage) of Crop instead of the full frame Source.Image.
Why does it happen?

Thanks in advance!
Best,
Evgeny

[Gonçalo Lopes]

Interesting. What is the error? Can you pinpoint which line causes it (by commenting each of the CV commands in turn)?

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/b7521080-f26d-4f76-9bfe-2e62ad228138%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Evgeny Resnik]

Hi Gonsalo,

This string generates an error: CV.Copy(trajectory,output,mask)
"Assertion failed: src.channels() == dist.channels()in function cvCopy, copy.cpp(576)"

best,
Evgeny

[Gonçalo Lopes]

Hmmm, I think the issue here is that your input image is grayscale, while the original code was designed for color. You can see this in the way the trajectory image is initialized:

trajectory = IplImage(image.Size, image.Depth, 3)

Probably you still want the final output to be a color image (since trajectory overlays work better in color), so I would suggest converting the output to a color image, instead of simply cloning it. Basically, you need to replace this line:

output = image.Clone()

with this:

output = IplImage(image.Size, image.Depth, 3)
CV.CvtColor(image,output,ColorConversion.Gray2Bgr)

That should make it work.

--

You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/dbb7c99d-ff65-4b7b-b6cc-126621e05f15%40googlegroups.com.

[Evgeny Resnik]

Hi Goncalo,

You were right, the problem was because of the grayscaled image fed to the PythonTransform.
Thanks a lot!

[Natalia Guadalupe Díaz Gil]

Hi, 

I am new in Bonsai and I want to do something similar but only with the trajectory, without the overlaid. In order words, I want to extract the trajectory of the moving object. Is this possible? 

Thanks in advance :D

[bruno...@neuro.fchampalimaud.org]

Sure, if you just want to save a video with the trajectory of the tracked point you can use a similar strategy to the one described here:

https://groups.google.com/u/1/g/bonsai-users/c/e1VAC0AP_lc/m/fiOP1bugAAAJ

I am also attaching the workflow. You just need to make sure the your tracking pipeline outputs a X,Y coordinate.

cheers,

B

[Natalia Guadalupe Díaz Gil]

Thanks a lot! :D