Duplicate records in downloaded data

85 views
Skip to first unread message

WSDOT Wildcat

unread,
Oct 20, 2024, 1:24:08 PM10/20/24
to Wildlife Insights
Hi, 

I'm noticing that many records are duplicated in my downloaded data. For instance, one detection of 16 elk shows up as two lines in the data, but are exactly the same (have the same sequence ID). This of course will lead to incorrect data summaries by location. Has anyone else experienced this or know how to correct the issue?

Thanks,
Glen
dataerror.png

Nicolas

unread,
Oct 21, 2024, 1:22:11 AM10/21/24
to Wildlife Insights
Hey Glen,

if you know some coding with Python, you could write a little script to delete the duplicates.
e.g.: You could use the libary "pandas" to load and delete the duplicates by adressing the column "start_time".

import pandas as pd

# Load the CSV file into a DataFrame
df = pd.read_csv('data.csv')

# Remove duplicate rows based on the 'start_time' column
df = df.drop_duplicates(subset='start_time')

# Save the cleaned DataFrame back to a CSV file
df.to_csv('cleaned_data.csv', index=False)

print("Duplicates removed and cleaned data saved to 'cleaned_data.csv'.")

Does this help?

Kind regards,
Nico

Wildlife Insights

unread,
Oct 22, 2024, 9:08:07 AM10/22/24
to Wildlife Insights
Hi All,
Glen thank you for the report, this is a known bug that we are working to fix to remove the duplicate records from project downloads.

Best,
Leah
Reply all
Reply to author
Forward
0 new messages