Video metadata extraction and Duplicate removal

[Rajashree Sain]

I have videos from a camera trap dataset that i have tagged in digiKam. I have used camtrapR to extract metadata for pictures from the same dataset before and got it in csv format. But it is not working for same for. Tried running this code below-

1. exifTagNames(fileName = "E:/from crucial x6/2017/block-2/Teendhara/25-5/New folder vm72/V__00018.M4V")
2. exifTagNames(fileName = "E:/from crucial x6/2018/cam-trap/BLOCK 2/aarey-18/vipmay2018/16-05-2018/MFDC8585.AVI")
3. recordtable2017<-recordTable(inDir="E:/from crucial x6/2017/block 1/",

                             IDfrom="metadata",
                             metadataSpeciesTag="species",
                             video = list(file_formats=c("mp4","m4v","avi"),
                                          dateTimeTag="QuickTime:CreateDate",
                                          db_directory="E:/digiKam database",
                                          db_filename= "digikam4.db"))

But many videos were eliminated as duplicates.

The warning messages are as follows-

timeZone is not specified. Assuming UTC Warning: 1-tara-devi-temple : no readable date/time information. Skipping Warning: 14-dokra-dokra-karnala : no readable date/time information. Skipping Warning: 33-upper ridge line : removed 3 images out of 32 because of missing species metadata tag 33-upper ridge line : 29 images 26 duplicates removed |== | 11% Warning: 34-ductline : no readable date/time information. Skipping 36-patona-pada : 2 images 1 duplicates removed |==== | 18% Warning: 37-bhendinala-start-patona : no readable date/time information. Skipping Warning: 38-bhendi-nala : no readable date/time information. Skipping Warning: 39-Mhais Bodan : no readable date/time information. Skipping 40-manpada-machan : 1 images 0 duplicates removed |====== | 32% 41- Munda Dongri : 1 images 0 duplicates removed |======= | 36% Warning: 42-sandip-house : no readable date/time information. Skipping Warning: 43-Chena up : no readable date/time information. Skipping Warning: 44- Khairacha-maal : no readable date/time information. Skipping Warning: 45-bailkadi-pada : no readable date/time information. Skipping Warning: 46- Nagla-tower-line : no readable date/time information. Skipping Warning: 47-Nagla-plot water : no readable date/time information. Skipping Warning: 48- other trail kaman : no readable date/time information. Skipping 49-Den : 86 images 85 duplicates removed |============= | 64% 6-manpada : 2 images 0 duplicates removed |============== | 68% Warning: 7-ovale-pankhanda : no readable date/time information. Skipping Warning: 8-chenamango-orchard : no readable date/time information. Skipping Warning: Golden-swan-transect : no readable date/time information. Skipping Warning: Humayun-bandhara-4x4 : no readable date/time information. Skipping Warning: julie-mulund-checked : no readable date/time information. Skipping Warning: just-images : no readable date/time information. Skipping Warning: leopards : no readable date/time information. Skipping Mama-bhanja : 6 images 4 duplicates removed |=================== | 96% Warning: people-working : no readable date/time information. Skipping There were 21 warnings (use warnings() to see them)

1. Can someone please help me fix this?
2. Also how does camtrapR read metadata and classify duplicate? 

Also do I need to put anymore information?

[Juergen Niedballa]

I see you're having trouble extracting metadata from your camera trap videos using camtrapR. Let's address your issues one by one.

Date/Time Extraction from Videos

The primary issue seems to be that camtrapR is unable to read date/time information from many of your video files. This is a common challenge with videos because unlike JPG images, video files don't have a standardized metadata tag for storing date/time information.

When working with videos in camtrapR, you need to:

1. First verify which date/time tag is available in your video files, e.g. using exifTagNames()
2. Then specify that tag in the dateTimeTag parameter of the video argument

Looking at your error messages, many of your directories show: "no readable date/time information. Skipping"

Here's how to troubleshoot this:

1. Check available metadata tags in your videos

Try examining a single video file that's working and one that's not:

# For a working video exifTagNames(fileName = "path/to/working/video.mp4") 

# For a non-working video exifTagNames(fileName = "path/to/problematic/video.avi")

Check for date/time related tags in the output. Common video date/time tags include:

• "QuickTime:CreateDate"
• "QuickTime:MediaCreateDate"
• "QuickTime:MediaModifyDate"

2. Test with a different date/time tag

If you find a different tag available in your problematic videos, try using that instead.

Regarding how camtrapR identifies duplicates:

The function recordTable() considers records as duplicates if they:

1. Are from the same station
2. Show the same species
3. Were taken at exactly the same time
4. Are from the same camera (if camera ID is specified)

Additionally, there's temporal filtering through the parameter minDeltaTime (in minutes), which lets you define what constitutes an independent record. By default, it's set to 0, meaning all records are returned. If you set it to, for example, 60, then records of the same species at the same station within 60 minutes of each other would be considered non-independent (but that is not the same as duplicates).

To keep all records without removing any "duplicates", try:

recordtable2017 <- recordTable(
  inDir = "E:/from crucial x6/2017/block 1/",

  IDfrom = "metadata",
  metadataSpeciesTag = "species",

  timeZone = "Asia/Kolkata",
  minDeltaTime = 0,
  removeDuplicateRecords = FALSE,  # Don't remove records at exactly the same time
  video = list(
    file_formats = c("mp4", "m4v", "avi"),

    dateTimeTag = "QuickTime:CreateDate",
    db_directory = "E:/digiKam database",
    db_filename = "digikam4.db"
  )
)

Additional Suggestions:

1. Specify your time zone: Add the timeZone parameter to avoid the UTC warning
2. Check digiKam database path: Ensure your database path and filename are correct
3. Check if the videos are in your digiKam database: Make sure the videos you're trying to process are actually included in the digiKam database
4. Try with a smaller subset first: Test with a directory that has fewer videos to troubleshoot more easily

Best regards,

Jürgen

[Rajashree Sain]

Thank you so much. It worked!