DSM Model Producing Distorted Predictions Despite Clean Inputs and Adequate Sample Size

[leob...@gmail.com]

Hi all,

I’m fitting a simple spatial model using the dsm() function in R and encountering persistently distorted results, even though the input data appears clean and complete.

Context:

• I'm modeling aerial survey data for an animal species using dsm(count ~ s(x, y), ...) with method = "REML".

• The detection function is a standard hazard-rate model (ds(..., key = "hr", adjustment = NULL)), which fits well and shows no obvious issues.

• My segment data (mysegdata) includes:

  • Effort (line length) in kilometers,

  • Coordinates (X, Y) in EPSG:3310 (California Albers),

  • Covariates such as elevation and vegetation metrics(though I haven't even gotten to modeling covariates yet).

• Observation data includes ~170 detections with all required fields (object, Sample.Label, distance, size, etc.).

• The prediction grid uses 500m x 500m cells, with matching CRS and reasonable covariate values.

Issue:
Even with a basic model like this:

dsm_xy <- dsm(count ~ s(X, Y), ddf.obj = hr_null,
              segment.data = mysegdata,
              observation.data = obs_filtered,
              method = "REML")

the output is clearly off:

• Linear predictor values are heavily skewed (e.g., from –3000 to 0),

• Residuals are tightly clustered around zero,

• Spatial prediction surfaces are nonsensical.

I’ve confirmed that:

• There are no missing or extreme values in effort or covariates,

• Coordinate and projection systems are consistent across all inputs (,

• Observation and segment tables are correctly linked.

Would appreciate any guidance on what might be going wrong or where else to look.

I've also attached a plot of my transect segment midpoints, my observations, and my prediction grid. Everything looks good when I plot, but then when I try to use predict (and vis.gam()), the output is distorted (output also attached).

Thanks very much,

Brian

[leob...@gmail.com]

Sorry, noticed the prediction plot did not show up.  Here it is. 

[Eric Rexstad]

Morning Brian

I haven't look through all your details as yet, but your plot of segment locations and detections raises a question. You have labelled the axes of your ggplot as if the coordinates are in meters. However the tick marks for the axes are in degrees of latitude and longitude. This suggests to me that you believe things are happening in one coordinate space, while the software thinks something else.

That's where I'd go digging to sort out the problem.

---

From: distance...@googlegroups.com <distance...@googlegroups.com> on behalf of leob...@gmail.com <leob...@gmail.com>
Sent: 06 June 2025 03:36
To: distance-sampling <distance...@googlegroups.com>
Subject: [distance-sampling] DSM Model Producing Distorted Predictions Despite Clean Inputs and Adequate Sample Size

 

--
You received this message because you are subscribed to the Google Groups "distance-sampling" group.
To unsubscribe from this group and stop receiving emails from it, send an email to distance-sampl...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/distance-sampling/9ba7ada8-c7f8-4d82-b832-d41a053a2e20n%40googlegroups.com.

[Brian Leo]

Hi Eric and all,

While cleaning up my data to send out for review, I discovered the issue. It turned out that I was using my unsegmented "Sample.Label" data that was used in detection function model fitting for the Sample.Labels of the segmented transects used for DSM analysis.  This was probably the reason for the densities being bunched up in the northern portion of the plot, because all of the detections were being lumped into the sample labels with lower values.

Thanks all,

Brian