IsoriX calibration

[Scott Clem]

Hello all,

I emailed this to Dr. Courtiol but I am also posting it here.

I have been using IsoriX to perform isotopic assignments of migratory hover flies in North America. We just got the reviews back on our manuscript, and one comment is that we need to present uncertainty rasters in the results and that these should integrate uncertainty from both the calibration equation and the precipitation isoscape. This seems easy enough for the precipitation ("mean_predVar" raster), but I am uncertain about the calibration given that we used a different method from the workflow. I generated a calibration equation by rearing flies on three different waters spiked with increasing levels of deuterium and then applying the calibration equation (see the attached graph) to the raw GNIP precipitation data. So, I did not use the 'calibfit' function. With the way that we are running it, is this still incorporating uncertainty from the calibration? If not, do you have any considerations?

Thank you,

Scott Clem

USDA NIFA Postdoctoral Fellow

University of Georgia

Department of Entomology
Personal Website

[Alexandre Courtiol]

Dear Scott,

Welcome to this google group!

I really like the approach you took in the paper and performing a calibration using heavy water in the lab is an interesting idea whenever possible since calibration is often what severely limit the accuracy of assignments.

Before to turn to calibration, I wanted to confirm that indeed using the raster mean_predVar is the right choice for showing the uncertainty of the precipitation raster.

For calibration, when performing that step outside IsoriX 2 issues generally emerge:
- the uncertainty (variances but also covariances) of the precipitation isoscape is not accounted for during the calibration step, but that is not an issue in your case because you did the calibration experimentally.
-  the uncertainty of the calibration will not be accounted for during the assignment.

This second point is the thing that is more problematic here.

As you can see in our code, we compute the variance of the assignment test (https://github.com/courtiol/IsoriX/blob/1254559567c7ebe93138035eac163047b5954010/IsoriX/R/isofind.R#L200-L213) as a function of the calibration uncertainty (itselfed measured here: https://github.com/courtiol/IsoriX/blob/1254559567c7ebe93138035eac163047b5954010/IsoriX/R/calibfit.R#L233-L243).

I thus see 3 possibilities:

- option 1: not accounting for the variation in calibration uncertainty

- option 2: accounting for the variation in calibration uncertainty crudely

- option 3: accounting for the variation in calibration uncertainty finely

For option 1, I would document the uncertainty in your regression slope (e.g. 95% CI on the regression slope) and if small enough I would argue that it is low enough that we assumed that it should not strongly impact the results of the assignment.

It is hard to know if there is or not a departure from that assumption without exploring option 2 or 3...

Option 2 would be to bootstrap data (resampling with replacement) and do "calibration + rescaling + assignment" many times and then combine the assignment rasters obtained and recompute p-values.

That is a rather brute force way, but that seems easy enough to do and could satisfy a reviewer.

If you don't know how to but could prepare a mini-scrip starting from the data and leading to the assignment I could try to show you exactly how to (the week after next, since next week I will be at ESEB2022, a conference about evolutionary biology).

Option 3 would be that we do the maths properly and add external calibration as a core functionality in IsoriX.

I was always against this option since usually people request that so as to use calibration based on sedentary organisms, which would thus imply that there result depend on a different isoscape as the one used in IsoriX and thus creates unknown biases and uncertainty.

But, you won me over, since you external calibration is one that does not rely on an isoscape.

Others will be happy, since I am sure they will hijack this to use external calibration performed differently.

I am happy to implement option 3 within the next couple of months, provided I get a little help on the stats side from another developer (we will discuss this next week).

I cannot promise though that it will be done before your deadline.

If it turns out easy enough and if we find the time soon and can get to it the week after next, then I would suggest we don't bother with option 2.

I hope this helps,

Best,

Alex

[Scott Clem]

Hi Alex,

Thank you very much for the reply and I apologize for the delayed response. I've been in the field and unavailable. Hope you're enjoying the conference.

I was able to use the least squares method to calculate slope and standard error (uncertainty) of my data as 0.5 +/- 0.04, yielding a percent uncertainty of 7.5. My 95% confidence interval of the slope is 0.41 to 0.58. Thoughts? Do you think this is low enough to argue option 1?

I wish I had realized all this earlier, and I would definitely pursue option 3. Seeing as I need to resubmit soon I may go with option 1 (preferred) or 2, and if they insist I will wait for option 3. Regardless, I will likely be doing isotope assignments in the future using these same data, so I think it would be useful to incorporate that component into the package. As you mention, it may also be useful to others.

Thanks,

Scott

[Alexandre Courtiol]

Hi Scott,

## Good news:

With my colleague François Rousset, we have tried out option 3 yesterday.

It seems to be technically doable with minimal change in the code of IsoriX.

We coded the hard part already and the rest is just wrappers around that.

From the user perspective the only change will be a new argument in calibfit() which I will call "method".

This argument will have to be set to either "wild" (or "field"?), "lab" or "desk" to perform alternative types of calibration depending on how calibration data have been generated.

The value "wild" is to do what IsoriX already does, "lab" will be for your case, and "desk" will be for handling the case where the calibration fit is done elsewhere and only the slope and intercept is known.

That way, it will be easy to "fix" your paper: the calibration will be done with your data in IsoriX directly and all the figures will account for the uncertainty in your calibration.

It will also be easy to describe in your paper (just add in the methods you used the setting method = "lab" in calibfit()).

## Bad news:

We can nail this by mid September but not before the end of this month.

This is because we need to triple check the draft of our code for the revised calibration procedure, write the code (easy), write the documentation (always the most painful part), update spaMM (developed by FR) on CRAN since new extractors need to be added to this package IsoriX heavily relies on, and then update IsoriX (on GitHub or CRAN).

All of this is relatively straightforward but because we (and CRAN) also have other commitments, it must take a few days.

## Conclusion:

I would request a small extension for the revisions.

Ask end of September and we will do our best to make it possible for you to surprise them by resubmitting earlier.

That should not be no big deal for everyone but you.

Just explain the situation to your PI and the editor (i.e. the reviewer is correct,  and for addressing the comment small modifications to the software you used are needed; the developers are working on it).

Do have everything ready but that.

What do you think?

Best,

Alex

[Scott Clem]

Hi Alex,

This is mostly good news! I'm glad it doesn't seem like it will be that huge of a code change. I emailed the editor to request an extension, so hopefully that gets granted and won't be a problem. I will keep you updated.

Thank you,

Scott

[Scott Clem]

Hi Alex,

Just letting you know they granted my extension to the end of September. Please let me know when you have things ready. I very much appreciate your help!

Thank you,

Scott

[Scott Clem]

Dear Alex,

They have extended the due date for my resubmission to the end of September. Please keep me updated on this and let me know if you need anything.

Thank you!

Scott

On Tuesday, August 23, 2022 at 2:58:35 AM UTC-4 Scott Clem wrote:

[Alexandre Courtiol]

Hi Scott,

On the IsoriX side, everything is already ready for you to use.

The changes have been made under the development version of IsoriX, which I will keep updating until the official release of IsoriX 0.9 planned in mid/end September.

You can already read the new documentation here:  https://github.com/courtiol/IsoriX/blob/master/IsoriX/R/calibfit.R

I will explain to you how to install the development version only once a development version of spaMM will be on the web too.

Without the new version of spaMM, the new version of IsoriX won't work.

The release of a devel version of spaMM should happen very soon, but meanwhile you have to wait a little.

We will not forget you :-)

Alex

[Scott Clem]

Excellent, thank you! Yikes, I apologize, I realize I accidentally sent you the same message twice.

[Alexandre Courtiol]

Dear Scott,

Everything is now ready for you to try.

Step 1: Download the new version of spaMM from this link https://www.dropbox.com/s/2wke6958vss1tff/spaMM_3.12.47.tar.gz?dl=0 (do not extract, just copy as is on your hard drive, ideally in your working directory)

Step 2: Install spaMM and the new devel version of IsoriX by running the following R code in your R console:

if (!requireNamespace("remotes", quietly = TRUE)) install.packages("remotes")
remotes::install_local("spaMM_3.12.47.tar.gz") ## adjust the path if the file is not in your working directory
remotes::install_github("courtiol/IsoriX/IsoriX")

Step 3: run your workflow as before but do perform the calibration within IsoriX using calibfit(..., method = "lab").

This new calibration method will use your measurements for the environment instead of fetching them from the fitted isoscape.

For this you need to format your calibration dataset in a specific way.

I wrote the documentation for that, so please do check ?calibfit and try to follow the instructions.

Do let me know if the instructions are unclear.

Note 1: I have not updated the bookdown (working on it), but I did update all relevant help files embedded in the package.

Note 2: other things have slightly changed too. The main thing being that getelev() now rely on another package to download an elevation raster (so do check ?getelev if you use that function).

Please do let me know if you find any glitches anywhere, this devel version of IsoriX should correspond to what will be released officially in September under the version number 0.9 (so you can cite that version number in your paper).

The only things I am planning to add would be fixes to small bugs, if any are to be discovered.

All the best,

Alex