using fishmidline script to get head/centroid coordinates

[Molly Clark]

Hello, 

I'm trying to use the fishmidline script to get the head and centroid coordinates from my tracked videos. The code seems to be doing what I want, however the coordinates don't match the coordinates from my trajectories.csv files.

I'll attach my code at the end here, I'm not sure if this is my user error or if the coordinates are different than the video file? If there is a way to match these up it would be very helpful!

Thanks,

Molly

[idtracker.ai]

Hi Molly,

Your approach is almost correct. The spline obtained in fishmidline is computed in the Blob's bounding box (bbox) coordinate system. If you want to transform these into full frame coordinates (to compare with the idtrackerai output trajectories) you have to do the correction you are already doing. You have to add the bottom value of the bbox to the X values and the left value of the bbox to the Y values. But the Blob.bbox_in_frame_coordinates is structured like [(bottom, left), (top, right)]. So when adjusting the head and the centroid values, make sure to add blob.bbox_in_frame_coordinates[0][0] to the X values and blob.bbox_in_frame_coordinates[0][1] to the Y values (note that I'm using different indices from the ones you are using).

Apart from that, the centroids we provide in our output trajectories are not the middle point of the splines (like you are computing) but the center of mass of the blob's contour. You can access this value with the property Blob.centroid.

I hope this helps and tell me if you need more information,

Jordi

PS: Next time send your code as text so that everyone can check it more easily :)

--
You received this message because you are subscribed to the Google Groups "idtracker.ai users group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to idtrackerai_us...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/idtrackerai_users/634f82bc-78d4-4bfc-82c3-fdc9c4562ec4n%40googlegroups.com.

[Molly Clark]

Hi Jordi,

Thank you for the quick response, this was very helpful! I have adjusted my code and will paste it below in case it is helpful for anyone else. I decided that using the midpoint of the spline is best for my purposes for now, and will compare later with the overall centroid (Blob.centroid), which I am assuming is the value given in the idtrackerai trajectories output? 

Best wishes,

Molly

import csv

import numpy as np

from idtrackerai import ListOfBlobs

from scipy.interpolate import splev

from fishmidline import get_spline

import pandas as pd

frame_sections = pd.read_csv("frame_sections.csv")

# info about the frames

frame_sections_group1 = frame_sections[frame_sections["group_ID"] == 2]

# split into sections

list_of_blobs = ListOfBlobs.load(r"path_002\list_of_blobs.pickle")

# load the blobs

# Create an empty list to store the data

results = []

# the number of points on the midline

n_points = 10

# Loop through the frames

for idx, row in frame_sections_group1.iterrows():

    frame_start = row["frame_start"]

    frame_end = row["frame_end"]

    for frame_number in range(frame_start, frame_end + 1):

        blobs_in_frame = list_of_blobs.blobs_in_video[frame_number]

        for identity in range(1, 11):

            matching_blobs = [blob for blob in blobs_in_frame if identity in blob.final_identities]

            if matching_blobs:

                blob = matching_blobs[0]

                spline_params = get_spline(blob)

                head = splev(0, spline_params)

                centroid = splev(0.5, spline_params)

                head_adjusted = (head[0] + blob.bbox_in_frame_coordinates[0][0],

                                 head[1] + blob.bbox_in_frame_coordinates[0][1])

                centroid_adjusted = (centroid[0] + blob.bbox_in_frame_coordinates[0][0],

                                     centroid[1] + blob.bbox_in_frame_coordinates[0][1])

                delta_x = head_adjusted[0] - centroid_adjusted[0]

                delta_y = head_adjusted[1] - centroid_adjusted[1]

                angle_from_centroid_to_head = np.arctan2(delta_y, delta_x)

                angle_from_centroid_to_head_deg = np.degrees(angle_from_centroid_to_head)

            else:

                head_adjusted = (np.nan, np.nan)

                centroid_adjusted = (np.nan, np.nan)

                angle_from_centroid_to_head_deg = np.nan

            results.append([

                frame_number,

                identity,

                head_adjusted[0],

                head_adjusted[1],

                centroid_adjusted[0],

                centroid_adjusted[1],

                angle_from_centroid_to_head_deg

            ])

# Save the results to a CSV file

csv_filename = "fish_midline_data.csv"

with open(csv_filename, mode='w', newline='') as file:

    writer = csv.writer(file)

    # Write the header row

    writer.writerow(['frame', 'identity', 'head_x', 'head_y', 'centroid_x', 'centroid_y', 'angle_from_centroid_to_head_deg'])

    # Write the data

    writer.writerows(results)

print(f"Data saved to {csv_filename}")

note: code runs through select frames, which are saved in the frame_sections.csv, and over each identity which in my case are 1-10.

[idtracker.ai]

Hi Molly,

Yes, the trajectories outputted by idtrackerai follow the blob's centroids (except when solving crossings).

Jordi

To view this discussion, visit https://groups.google.com/d/msgid/idtrackerai_users/5a3c003b-63fc-4458-92c3-3da3b44ab214n%40googlegroups.com.