Source.Orientation

[Armando Remondes]

I´ve been using this function to record head direction of a behaving animal.
As far as I could tell this function records direction of the long axis regardless of front and back, the resulting angles varying only within the half-circle. Please let me know if this is incorrect.
Is there a version of this function that takes front and back into account, thus reporting polarized direction? How could I edit it for my purposes? Is there another function I could use for this?
Thanks so much!!
Miguel

[Adam Kampff]

Hi Miguel,

   Here is the C# code for a node that determines the "heading direction" from the orientation and shape of a binary object. In this case, the "heading" is the direction along the orientation axis that points to the side of the object containing the most foreground pixels. It returns an angle from 0 to 2pi (by replacing the value of "orientation" for the input CC object). This would work well for an "arrow" shape, but it also does a good job tracking zebrafish swim direction. However, it may be that your animal has a less arrow-like shape, in which case, other criteria will be necessary.

You will have to package it into a custom C# Bonsai Transform, but there is a good tutorial on the Bonsai website covering how to do this.

Note: This might be easier to prototype as a Python Transform. Does anyone have the code?

Cheers,

Adam

p.s. a Tutorial on distributing custom NuGet packages would be super. :)

The code as text:

-----------------------

using Bonsai;
using Bonsai.Vision;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;
using System.Text;

using System.ComponentModel;
using System.Runtime.InteropServices;
using OpenCV.Net;

// Input and Output Bonsai Types
using TSource = Bonsai.Vision.ConnectedComponent;
using TResult = Bonsai.Vision.ConnectedComponent;

namespace Bonsai.ARK.Vision
{
    // Change the "orientation" of the input CC to be the directional "heading".
    // - - - Note: It is still called "orientation" in the CC Object
    // - Along the orientation axis, the portion of the CC that has more pixels (e.g. an arrow) defines the heading.
    // - - Result is returned in radians:
    //      ::   0 deg (0 radians) is right
    //      ::  90 deg(pi/2 radians) is down
    //      :: 180 deg (pi radians) is left
    //      :: 270 deg (3pi/2 radians) is up
    //       : (+ is CW, - is CCW)

    public class HeadingFromOrientation : Transform<TSource, TResult>
    {
        // Useless Local Def
        float pi = (float) System.Math.PI;

        // Constructor (set defaults)
        public HeadingFromOrientation()
        {
        }

        // Event Processing
        public override IObservable<TResult> Process(IObservable<TResult> source)
        {
            return source.Select(input =>
            {
                if (input.Contour != null)
                {
                    float boxW = input.Contour.Rect.Width;
                    float boxH = input.Contour.Rect.Height;
                    float boxX = input.Contour.Rect.X;
                    float boxY = input.Contour.Rect.Y;
                    float X = boxX + (boxW / 2.0f) - input.Centroid.X;
                    float Y = boxY + (boxH / 2.0f) - input.Centroid.Y;

                    double OrientRad = input.Orientation;
                    // Orientation: 0 is right, -pi/2 is up, pi/2 is down
                    // Heading: 0 rad is right, left is pi, down is pi/2, up is 3pi/2 (+ is CW, - is CCW)
                    if ((OrientRad >= 0.0) && (OrientRad < pi/4.0f))
                    {
                        if (X < 0.0f)
                        {
                            input.Orientation = OrientRad;
                        }
                        else
                        {
                            input.Orientation = OrientRad + pi;
                        }
                    }
                    else if ((OrientRad >= pi/4.0f) && (OrientRad <= pi/2.0f))
                    {
                        if (Y < 0.0f)
                        {
                            input.Orientation = OrientRad;
                        }
                        else
                        {
                            input.Orientation = OrientRad + pi;
                        }
                    }
                    else if ((OrientRad < 0.0) && (OrientRad > -1.0*pi/4.0f))
                    {
                        if (X < 0.0f)
                        {
                            input.Orientation = OrientRad + (2 * pi);
                        }
                        else
                        {
                            input.Orientation = OrientRad + pi;
                        }
                    }
                    else if ((OrientRad <= -1.0*pi/4.0f) && (OrientRad >= -1.0*pi / 2.0f))
                    {
                        if (Y > 0.0f)
                        {
                            input.Orientation = OrientRad + (2 * pi);
                        }
                        else
                        {
                            input.Orientation = OrientRad + pi;
                        }
                    }
                }
                return input;
            });

[goncaloclopes]

@Adam:

Thanks for the quick reply :-)

We really should start packing these useful nodes into a Bonsai.Fish package to include in the Bonsai.Neuro repository.

@Miguel:

You're right about this limitation. Computing the full orientation requires tracking the "head" and the "tail" of the object over time. Unfortunately there is no general solution to this, which is why I didn't include anything in the Vision package. However, there are a number of domain-specific solutions as Adam already hinted at with the "arrow" shape solution.

Another very easy naive solution is to use the two points that come out of "BinaryRegionExtremes" and keep a memory. You start out by assuming an arbitrary "head" and "tail" and then on every frame you decide your new "head" and "tail" are the points that are closest to the previous memory. This will work if you assume the object does not flip 180º in a single frame interval (which is for the most part reasonable).

Here's the code for this solution ready to paste into a PythonTransform (I can also translate Adam's code, but I had this one handy):

import clr
clr.AddReference("OpenCV.Net")
from System import Tuple, Math, Single
from OpenCV.Net import Point2f

head = None
tail = None

def distancesquare(pt1,pt2):
  dx = (pt2.X - pt1.X)
  dy = (pt2.Y - pt1.Y)
  return dx * dx + dy * dy

def angleline(pt1,pt2):
  dx = (pt2.X - pt1.X)
  dy = (pt2.Y - pt1.Y)
  return Math.Atan2(dy, dx)

@returns(Single)
def process(value):
  global head, tail
  pt1 = value.Item1
  pt2 = value.Item2
  if pt1.X == Single(float.NaN):
    head = None
    tail = None
    return float.NaN
  else:
    if head is None or distancesquare(pt1, head) < distancesquare(pt1, tail):
      head = pt1
      tail = pt2
    else:
      head = pt2
      tail = pt1
    return angleline(head,tail)

It takes as input a Tuple of two points. You can get this by using BinaryRegionExtremes, which uses various strategies to compute the most extreme points of an object's shape. I would recommend trying out "MajorAxis" first and see how it goes.

If anyone has even more solutions to chip in, would be happy to hear them and maybe we can package polished versions of some of them in the main distribution.

Hope this helps,

G