A comprehensive framework for handling location error in animal tracking data

[Christen Fleming]

Hi Folks,

We have a comprehensive manuscript in review that gives a fairly broad overview of location error, including solutions for many problems and a bunch of demonstrations. The pre-print is up on bioRχiv, if anyone might benefit from this:

https://www.biorxiv.org/content/10.1101/2020.06.12.130195v1

These features have been in ctmm for a little while now, as the manuscript took some time to compile, but the manuscript will likely answer more questions than a help file or vignette.

Best,

Chris

[christin...@gmail.com]

Thanks for sharing Chris, this manuscript is a great help. 

I have a couple questions about the error model selection procedures used in the paper: 

In some of the examples, you note removing gross outliers from the calibration data before estimating error models. How exactly did you determine which points constituted a "gross outlier"? In my calibration data (mort events from deployed collars) there are quite a lot of bad points in some cases, not just 1 or 2.

It looks like you've tested some error models where you treat time to fix as a continuous variable instead of the default ctmm behavior of creating in-time and timeout classes. How can we override the default behavior to test such a model? Should I just name the GPS.time.to.fix column as GPS.DOP to force it to treat it as a continuous value? Is there another way?

Many thanks,

Christina

[Christen Fleming]

Hi Christina,

I took an initial guess at the outliers using an initial guess at the RMS UERE (and error model), then obtained a conditionally "good" error model and validated that I would have made the same choice of outliers. I was only removing the very crazy numbers, so there was never any inconsistency.

If you have a lot of outliers, with no clear separation between the good data and the outliers, and you can find no good error model that can pull those outliers inwards (when looking at the residuals) or, equivalently, to expand their error circles towards the truth, then really what you would like is a heavy tailed error distribution, like a bi-variate t-distribution. Unfortunately, I do not have this implemented in ctmm yet. (I don't think it's implemented in any continuous-time R package yet.) This is on our agenda, but I have to get this paper through review first.

Using TTF as a continuous variable was just something abnormal that popped up with that one model GPS device, when exploring the residuals versus different information provided by the device. And, yes, I just made a fake HDOP column using true HDOP * TTF, so that the fake HDOP column would be loaded as a real HDOP. I haven't streamlined this process better yet because going in my understanding was that this was not supposed to be necessary. DOPs are just supposed to work.

Best,

Chris

[christin...@gmail.com]

Thanks for clarifying Chris,

After applying an initial guess at a good error model - did you visually pick out the bad outliers or use some cutoff based on metrics like the speed and core deviation?

 

Right now I'm having a hard time deciding whether to proceed with some outlier cutoff, because I have more than just one or two bad points. I think a heavy tailed distribution would be most appropriate, but a majority of the data do have reasonable residuals after fitting what seems to be the most appropriate error model. There are some case when outliers make up a larger portion of the data due to the habitat in which the animal died (the collar with the most outliers was under a 15 ft rocky overhang). 

Without removing outliers, the best model only gets me down to a Z^2 of 13.4 and I've tried about every combination of available error metrics I can think of. But if I fit the best error model, then remove calibration data greater than the 1.5*IQR + third quantile of speed or core deviation for each fix class within each test collar, I can get down to a Z^2 of 2.9. 

Do you think this would be a good approach, even if for some collars, this outlier filter removed 6, or 12% of the data? These were the worst cases where the habitat was unusual and likely caused substantial bounce. And would you recommend a second round of checking for outliers after this first filter? I notice that even with the low Z^2, there are still some, but much fewer outliers in the calibration data.

If you're in need of some calibration data that might be useful toward developing a heavy tailed model, I'd be happy to share these with you.

Many thanks,

Christina   

[Christen Fleming]

Hi Christina,

I remember that you were having bad multi-pathing issues from the terrain. Unfortunately, I will not be getting around to implementing heavy tailed error distributions very soon, and I don't know, off-hand, any easy-to-use R movement/timeseries packages that will model t-distribution error. I've seen a lot of people use Template Model Builder for this (e.g., https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2664.12909), but that is a lot of effort and requires rolling your own model.

I can look into what the appropriate extreme value distribution is and what the exact thresholds would be at different levels. Tossing out outliers is, in my opinion, something you do when the number of outlier is small, and so their ability to inform a heavy-tailed error distribution is weak... and so its better to toss them out than to have a poorly performing error model with poorly resolved tail parameters. But if you don't have any good options, then I won't stop you. And, in general, I would recommend iterating until consistency.

Best,

Chris

[christin...@gmail.com]

Really appreciate your thoughts! 

I'll take a look at template model builder and see whether it's within my abilities to fit a t-distributed error model that way. I'd much rather fit a model that matches the patterns in the data if possible, so it's worth a try. Thanks for the link to an example. 

[Genevieve Finerty]

Hi Chris, 

I'm just working through the error calibration process following the paper for 3 Vectronics collars (2 GPS Plus, and one Vertex Plus). The calibration datasets are 29 days long (for the first GPS Plus collar) and 2-days for the second two. I had a couple of things that I wanted to run past you to see if you had any insight into them. The various calibration datasets only had some of the three classes of fixes (GPS-2D, GPS-3D, and validated GPS-3D) present -- except the 29-day one, which had all three -- but the RMS UERE value for val. GPS-3D (which they all had) was pretty similar (1.29 - 1.32) for all three collars, which seems reassuring. But for the collars that also had the other two fix types, the RMS UERE for GPS-3D and GPS-2D were consistently smaller than the validated GPS-3D -- which surprised me, as I had been working on the assumption that val. GPS-3D was the highest quality fix type. This made me wonder if I am either interpreting high/low RMS UERE values erroneously or if I am interpreting the quality of fix types wrong....

Second, the whole process works as I would expect for the two GPS Plus collars, and I get calibrated data I can plot and work through the outlier process and so on -- but for the Vertex plus collar, when I plot the calibration data I can't see any ellipses (everything else from there seems to work fine) -- I wondered if this had something to do with it not having the error column that other vectronics collars have? As I can't see any other differences between it and the GPS Plus collars? I saw in the paper you work with the error output directly, rather than the ambiguous DOP values, but I couldn't see a way to directly specify this or not... Looking in the telemetry objects, the variables stored appear to be dop and class (fix type) so I'm not sure why the error column would come in to it, but was just a guess!

Thanks!

Gen

[Christen Fleming]

Hi Genevieve,

Regarding the fix classes, you should expect validated 3D is better than 3D is better than 2D. For some devices, their DOP values are all on the same scale and these classes are just notifying you that the location estimate is derived from some threshold number of satellites. In that case, you can safely remove this column before importing, so that everything is calibrated on the same scale. However, for some devices (like Telonics), the classes are also notifying you that the location estimate algorithm is changing and they should be on different scales, and so the column should be kept for importing/calibration. In that case, I would compare something like RMS[UERE] * median(DOP) rather than straight RMS[UERE] values, to make sure that things look okay.

Try plotting with error=FALSE or zooming out. If the realized errors are tight compared to the size of the error circles, then they might be encircling the plot.

uere(DATA) <- uere.fit(CALIBRATION) takes the DOP and class information and produces an error variance column.

Best,

Chris

[Genevieve Finerty]

Hi Chris, 

Thanks for that! Have added a couple of comments below.

I am constantly amazed by the effort you put into answering our queries on here -- so just to say it is really appreciated!

On Wednesday, 2 September 2020 at 20:34:18 UTC+1 chris.h...@gmail.com wrote:

Hi Genevieve,

Regarding the fix classes, you should expect validated 3D is better than 3D is better than 2D. Good to know I haven't totallly misunderstood that one! For some devices, their DOP values are all on the same scale and these classes are just notifying you that the location estimate is derived from some threshold number of satellites. In that case, you can safely remove this column before importing, so that everything is calibrated on the same scale. However, for some devices (like Telonics), the classes are also notifying you that the location estimate algorithm is changing and they should be on different scales, and so the column should be kept for importing/calibration. In that case, I would compare something like RMS[UERE] * median(DOP) rather than straight RMS[UERE] values, to make sure that things look okay. Ok, great, got it -- I didn't think about the fact that median DOP might vary between the classes -- when comparing RMS[UERE] * median(DOP) for the fix types, validated 3D has a lower measurement error (around 2.86) than 3D (around 8.6). 2D still has a very low score, but it also accounts for a very small proportion of the total fixes (0.005-0.006 for both the calibration and deployed dataset) so I'm not too worried about that.

Try plotting with error=FALSE or zooming out. If the realized errors are tight compared to the size of the error circles, then they might be encircling the plot. Expanding x/ylim of the plot did the trick, thank you, was driving me crazy where they had gone... 

uere(DATA) <- uere.fit(CALIBRATION) takes the DOP and class information and produces an error variance column. Yup - I got that. What I was referring to is a column produced directly by Vectronics collars called 3D error, which appears to be in metres. It looked like this was what had been selected as the top error model in the paper for this collar type, but maybe I'm confusing something! The newer (Vertex) collars don't seem to have this anyway.

Best,

Chris

[Christen Fleming]

Hi Genevieve,

I haven't seen "3D error" as a column name, so as.telemetry isn't coded to recognize that. This stuff is unfortunately not very standardized. For the moment, you would want to import that column with a name like "HDOP" (and deleting the real DOP value) and check that it imports and performs better than the provided DOP values with model selection (as in the error vignette). If that column is working better than the DOP values, then I can add its name to my code and it will import by default with that model collar from then on. "3D error" is the exact column name?

Best,

Chris

[Genevieve Finerty]

The exact name the collar spits out is "3D_Error [m]" -- the collar manual describes it as "the difference [m] between the real position and transmitted position". I'm pretty sure that's what's used in the pre-print here,

   

I'll try it out on our collars and let you know how it goes!

Gen

[Genevieve Finerty]

Replacing the dop values with the collar generated "error" values, I found that "error" was the top model selected for the Vectronic GPS Plus collars. Vectronics described the value as "The 3D Error is a mathematical value from the satellite geometry, which indicates on how many meters the position can be wrong" and noted that for later models (e.g. Vertex Plus) they have chosen not to provide this information any more.
 

[Genevieve Finerty]

Two more questions....

(1) If a homoskedastic error model is selected, moving forward, am I right in thinking you would just replace the DOP values in the actual movement data with 1s so they're all the same?

(2) If the comparison of joint vs individual models returns individual models, indicating that there are differences between two collars of the same model/manufacturer -- ideally you would then calibrate. each collar individually -- but where this isn't retrospectively possible, for the collars without calibration datasets would it make sense to use the joint estimate as the best (average) knowledge you have?

[Christen Fleming]

Thanks for checking that, 3D error should get imported automatically after the next push to GitHub.

1. If the homoskedastic error model is selected, and you really wanted that model, then you could either remove the DOP columns or replace them with 1s. However, the problem with the homoskedastic error model is that it's going to assume that all of the calibration and tracking data are all collected in the same kind of environment, with respect to satellite reception. In some settings (open sea/air, grassland), this can be acceptable, but in other cases it won't be.

2. If the the difference is big, then you would want to calibrate individually. This is partially why I constructed a goodness-of-fit statistic, because a large AIC difference could just be from having a lot of data. After I get this manuscript published, in a future paper I want to show how you can take multiple individual calibrations and construct a "prior" for an out-of-sample collar, and propagate the uncertainty forward. Priors like that are kind of a middle ground between the old way of doing things (simultaneous estimation) and what I'm pushing as more ideal (full calibration). But I haven't coded this up yet though, so for the moment you would want to use the joint calibration and perhaps check the sensitivity of the results if there is a lot of variation among the individual collars.

Best,

Chris

[Genevieve Finerty]

Cool, that makes a lot of sense -- the reduced Z squared values are actually pretty similar,  with the difference mostly in AICc comparison, and once I run error-informed speed/distance calculations, there is little to no biologically-relevant difference between the joint and individually parameterised models. Looking forward to hearing more about the prior approach some point in the future!

All the best,
Gen