Simple question about text to video

[Pedro Ferreira]

Hey there!

I am using the FileCapture node to read videos, and then, after a whole bunch of analysing, I get some text results in the form of integers. I would like these results to show up in the video. Is it possible to have text coming from another node going showing in the video?

Thank you all!

[Gonçalo Lopes]

Hi Pedro,

I am planning to include a draw text operator for the next release:

https://bitbucket.org/horizongir/bonsai/issues/598/add-drawtext-operator-to-vision-package

In the meantime, it is possible to do this using a custom Python transform. There was a previous question about this, that you can find here.

Hope this helps!

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/51f98fb3-d2c9-4a88-a167-ac5d5fd15a57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pedro Ferreira]

Thank you, that did the trick!

But it created another issue!

For some reason, the video that is being played back from the PythonTransform node, now with the text displaying, is slowed down quite a lot. You have any idea why?

On Saturday, 7 October 2017 11:11:19 UTC+1, goncaloclopes wrote:

Hi Pedro,

I am planning to include a draw text operator for the next release:

https://bitbucket.org/horizongir/bonsai/issues/598/add-drawtext-operator-to-vision-package

In the meantime, it is possible to do this using a custom Python transform. There was a previous question about this, that you can find here.

Hope this helps!

On 7 October 2017 at 00:41, Pedro Ferreira <pedro.bio...@gmail.com> wrote:

Hey there!

I am using the FileCapture node to read videos, and then, after a whole bunch of analysing, I get some text results in the form of integers. I would like these results to show up in the video. Is it possible to have text coming from another node going showing in the video?

Thank you all!

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

[Gonçalo Lopes]

Hi Pedro,

We need more details to understand what the problem is. Here are some factors to think about:

 1) How large are your images?

 2) How fast is your frame rate?

 3) By how much was the actual slowdown? 2x, 5x, 10x?

 4) How many times are you calling your draw text script? Did you draw all your text in a single script, or did you call the script multiple times, one for each text you want to draw?

I am asking all of this is because of an important detail that is included in the script which may easily go unnoticed. In order to preserve the ability to safely share images between branches, the Python script first Clones the original image before drawing anything on top.

You might ask: why do this at all?

This behavior is important in general when modifying input images, because if the image source goes to a different branch, you might be writing on memory that is not your own and scramble the processing of other nodes that are operating in parallel. However, if you know for a fact that you are not sharing this buffer with anyone else, you could remove the call to the Clone method.

In theory, however, this should not greatly affect your processing unless you were dealing with color images @ very large resolutions (> fullHD color images) or if before you were playing them at a fast frame rate.

Hope this helps. Otherwise, please provide more details and/or share the exact workflow you are using.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/5e649f99-7447-4515-b0d4-54cce89ff2c4%40googlegroups.com.

[Pedro Ferreira]

Hey Gonçalo!
Thank you for the extensive response.

So, answering your questions:

1) My videos are about 30Mb each;

2) Frame rate is 30fps

3) I'd say about 5x-10x, but can't be sure.

4) I'm only calling it once, for one line of text.

Here goes my workflow:

Here goes the script I'm using:

import clr

clr.AddReference("OpenCV.Net")

from OpenCV.Net import *

font = Font(2)

@returns(IplImage)

def process(value):

  b1 = value.Item1.Item1

  b2 = value.Item1.Item2

  fr = value.Item1.Item3

  video = value.Item2

  img = video.Clone()

  Text = str(b1) + ',' + str(b2) + ',' + str(fr)

  CV.PutText(img, Text, Point(0,30), font, Scalar.Rgb(255,255,0))

  return img

Hope this info is enough for you to give me an answer!

Best regards,

[Gonçalo Lopes]

Hi Pedro,

Regarding 1), I meant the resolution of each frame, and not the total size of the video. Video decoding and processing speed depends mostly on frame resolution. Also, is the video color or grayscale?

Just to confirm, in your current workflow, if you simply remove the Text To Vid node at the end (without removing anything else), do you see a 5-10x performance increase in playback?

What happens if you replace the line img = video.Clone() with just img = video?

Finally, what is the reason for the SkipUntil node before the last Zip?

Hope this gives some clues into the situation.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/e13b86f7-f5c3-4712-813b-96d5cace2574%40googlegroups.com.

[Pedro Ferreira]

Hey there! I just now noticed that for some reason my reply hadnt been posted, hence the delay.

The video resolution is 640x480, color.

To clarify, the video being slowed down is the one coming from the Text to Vid node, with the text, not the one from FileCapture node. FileCapture node and Crop node are playing the video at regular speed, but the Text to Vid node is giving a considerably slower video, although the text that is being displayed matches the frame of the FileCapture node video. 

I tried replacing the line you mentioned, no change.

The SkipUnitl node was something I added on a hunch, and it actually sped up the video a bit, although it is still considerably slower (5x-10x) than the FileCapture node video. If I remove it, it gets even slower. 

Any ideas?

[Gonçalo Lopes]

Hi Pedro,

Can you attach the workflow you are using so it is easier to reproduce this behavior?

I'm still not convinced that the drawing is to blame. As a final test, can you change the python script to:

import clr

clr.AddReference("OpenCV.Net")

from OpenCV.Net import *

font = Font(2)

@returns(IplImage)

def process(value):

  return value.Item2

Basically, this is a script that just returns each video frame unaltered. Can you confirm whether this still slows down the video? If it does, then the bottleneck is somewhere else, and not in the text drawing node.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/f5a7209f-72ea-45d5-b0d1-bd031defe140%40googlegroups.com.

[Gonçalo Lopes]

Thanks for the workflow and the heads-up. If the playback is still slow, then I would say the bottleneck is not drawing to the video. There is quite a lot being done before the actual drawing, so there is probably some node that is slowing things down significantly.

My suggestion now is to start deleting operators systematically (starting from the end) and see at what point do you get a fast playback. For example, if you delete all the nodes after SortBinaryRegions, do you still get a slow playback? If you do, then the bottleneck is before that, etc. This makes more sense to try on your computer since you have your own computational resources and data to benchmark the processing.

Let me know once you have pinned down the culprit, and I can then try to help you understand what is going on.

On 14 October 2017 at 17:39, Pedro Ferreira <pedro.bio...@gmail.com> wrote:

Here is the workflow.

Even after that change, video from the Text to Vid node is still slowed down for some reason.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/f5a7209f-72ea-45d5-b0d1-bd031defe140%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/f166c6dc-4425-41d1-b8b3-b2e7bde8c341%40googlegroups.com.

[Pedro Ferreira]

I managed to solve it!

It was actually quite simple, my bad on not noticing it. Due to my workflow, I was only feeding data to the Text to Vid node every 5 frames of the video being played in the FileCapture node. I branched it off a bit earlier and changed the code a bit so it feeds NaN values in between every 5 frames, instead of nothing, and now it is working at proper speed!

Thank you for your help!