Isotonic regression

[Virgilio Gómez Rubio]

Hi,

I have been searching how to do an isotonic regression with INLA and it seems that the simplest approach is to use the 'clinear' latent effect and force the coefficient to be positive. However, is there any other approach that I should be aware of? I have tried to Google this but I am afraid that I have been unsuccessful. Perhaps with rgeneric and/or inlabru?

Best,

Virgilio

[Finn Lindgren]

You’d have to spell out some more details for me to know exactly what the model you want is, but if it just involves constraints on the coefficient, then yes, clinear is an option. In inlabru you can also do things like

  covariate * exp(beta)

in the eta predictor, where beta is a latent Gaussian variable. Which option is to be preferred depends on the details...

Finn

--
You received this message because you are subscribed to the Google Groups "R-inla discussion group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to r-inla-discussion...@googlegroups.com.
To post to this group, send email to r-inla-disc...@googlegroups.com.
Visit this group at https://groups.google.com/group/r-inla-discussion-group.
For more options, visit https://groups.google.com/d/optout.

[Virgilio Gómez Rubio]

Hi Finn,

Many thanks for your reply. Our model is a spatial model for a point a point pattern and the covariate is distance to a pollution source. So, the linear predictor is a SPDE plus a covariate with the distance to the pollution source and, hence, we want the coefficient to be either negative or zero so that it can assess whether the intensity of the underlying point process decreases with distance to the pollution source. So, actually, we want the coefficient to be negative and not positive.

Another interesting approach would be to model this using splines, imposing an overall decreasing trend on the distance. But I guess that this would be difficult to handle with INLA. So, we'll try with 'clinear' for now, with range (-Inf, 0). Although this case does not seem to be covered according to the manual page of 'clinear'.

Best,

Virgilio

2018-04-09 12:47 GMT+02:00 Finn Lindgren <finn.l...@gmail.com>:

You’d have to spell out some more details for me to know exactly what the model you want is, but if it just involves constraints on the coefficient, then yes, clinear is an option. In inlabru you can also do things like

  covariate * exp(beta)

in the eta predictor, where beta is a latent Gaussian variable. Which option is to be preferred depends on the details...

Finn

On 9 Apr 2018, at 11:17, Virgilio Gómez Rubio <becario...@gmail.com> wrote:

Hi,

I have been searching how to do an isotonic regression with INLA and it seems that the simplest approach is to use the 'clinear' latent effect and force the coefficient to be positive. However, is there any other approach that I should be aware of? I have tried to Google this but I am afraid that I have been unsuccessful. Perhaps with rgeneric and/or inlabru?

Best,

Virgilio

--
You received this message because you are subscribed to the Google Groups "R-inla discussion group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to r-inla-discussion-group+unsub...@googlegroups.com.
To post to this group, send email to r-inla-discussion-group@googlegroups.com.

[Helpdesk]

Hi

isotonic regression is estimation with ordering constraints, like that
a spline is always increasing (zero included). so that would not work
in general since it would impose an ordering constraints on some
Gaussian. you can trick it with clinear, moving that part up to the
hyperpar, but only in small cases....

H

On Mon, 2018-04-09 at 11:47 +0100, Finn Lindgren wrote:
> You’d have to spell out some more details for me to know exactly what
> the model you want is, but if it just involves constraints on the
> coefficient, then yes, clinear is an option. In inlabru you can also
> do things like
> covariate * exp(beta)
> in the eta predictor, where beta is a latent Gaussian variable. Which
> option is to be preferred depends on the details...
>
> Finn
>

> On 9 Apr 2018, at 11:17, Virgilio Gómez Rubio <becarioprecario@gmail.

> com> wrote:
>
> > Hi,
> >
> > I have been searching how to do an isotonic regression with INLA
> > and it seems that the simplest approach is to use the 'clinear'
> > latent effect and force the coefficient to be positive. However, is
> > there any other approach that I should be aware of? I have tried to
> > Google this but I am afraid that I have been unsuccessful. Perhaps
> > with rgeneric and/or inlabru?
> >
> > Best,
> >
> > Virgilio

--

Håvard Rue
Helpdesk
he...@r-inla.org

[Finn Lindgren]

I see. Well, if you're doing an LGCP likelihood, I would recommend
inlabru anyway, since that can construct that automatically.
You'll need to have a look at the latest Workshop slides&practicals
for more information and examples.

We've used clinear to construct monotonic effects, but it's a bit
fiddly; a general spline like model (spde on a 1D mesh) is easier.
For your negative covariate effect, you could use something like this
(see practicals for the spatial covariate technique):

mesh1 <- inla.mesh.1d(seq(cov_mini, cov_maxi, length=knots), degree=2,
boundary="free")
mesh2 <- inla.mesh.2d(...)
spde1 <- inla.spde2.pcmatern(mesh1, ...)
spde2 <- inla.spde2.pcmatern(mesh2, ...)
components <- ~ field(map = coordinates, model=spde2) + effect(map =
fcovariate(x, y), model=spde1) + Intercept
formula <- coordinates ~ field - exp(effect) + Intercept

You want a fixed long range for a spline-like model (and remove the
Intercept or try constr=TRUE in inla.spde2.pcmatern for spde1).
Unfortunately, last time I tried to fix one of the pcmatern paramters
INLA gave an error (have not had time to check where the error was).

Allowing the effect to be exactly zero is difficult, as it rules out
using a simple transformation. In theory, you're allowed to use a
function that actually goes to zero, as opposed to exp(), but I won't
guarantee convergence by the current inlabru iteration method...

For an effect like "distance from something" it would make sense to
pin down the spde1 model at one end instead of an integrate-to-zero
constraint. use
boundary=c("dirichlet", "free") to pin it down at the left endpoint (cov_mini).

Finn

On 9 April 2018 at 12:05, Virgilio Gómez Rubio

>> email to r-inla-discussion...@googlegroups.com.

>> To post to this group, send email to

>> r-inla-disc...@googlegroups.com.

>> Visit this group at
>> https://groups.google.com/group/r-inla-discussion-group.
>> For more options, visit https://groups.google.com/d/optout.
>
>

--
Finn Lindgren
email: finn.l...@gmail.com

[Finn Lindgren]

On 9 April 2018 at 12:28, Helpdesk <he...@r-inla.org> wrote:
> isotonic regression is estimation with ordering constraints, like that
> a spline is always increasing (zero included). so that would not work
> in general since it would impose an ordering constraints on some
> Gaussian. you can trick it with clinear, moving that part up to the
> hyperpar, but only in small cases....

Ah, yes, this is exactly what we used clinear for originally.

Since INLA treats clinear as a hyperparameter, the inlabru
approximation is cheaper (but less accurate).
One would need construct the spline basis more explicitly, with one
effect per monotonic basis function, and do
basis1*exp(effect1)+basis2*exp(effect2)+...
This is tedious but possible.

There is also a third option: Not imposing the monotonicity
constraint, but hoping that the data imposes it.
If necessary, one can use posterior samples (inla.posterior.sample()
or inlabru::generate() and filter out non-monotonic samples;
this is the same thing as truncating and renormalising a non-constrained prior.
This approach has the added benefit of not presuming too much, and
also to see if the data is informative enough to impose the
constraint.

Finn

[Finn Lindgren]

Just one more note on using clinear for monotonic spline coefficients:
Since each coefficient will be independent in the prior, this will only give you the highly knot-dependent kind of spline, and not the more wellbehaved continuous penalty splines (that are the modes of spde posteriors).

Finn

[Virgilio Gómez Rubio]

Hi,

Many thanks again for the answers. I like the idea of using a 1D SPDE!! 

Best,

Virgilio

[Virgilio Gómez Rubio]

Hi,

2018-04-09 13:28 GMT+02:00 Helpdesk <he...@r-inla.org>:

Hi

isotonic regression is estimation with ordering constraints, like that
a spline is always increasing (zero included). so that would not work
in general since it would impose an ordering constraints on some
Gaussian. you can trick it with clinear, moving that part up to the
hyperpar,  but only in small cases....

Thanks. I see this is a complex issue. We'll try 'clinear' and 1D SPDE for now and keep thinking about a more general approach for isotonic regression with INLA.

Best,

Virgilio

 

[Haavard Rue]

I’m not at all convinced that any good solution will exists…

[Tim Meehan]

Hey Virgilio. Did you ever figure out a reasonable solution for a monotonic smoother? If so, is there any example code out there? Thanks! Tim

[Håvard Rue]

the issue with the isotonic smoothing thing, is that its impose constraints

x1 <= x2 <= x3 <= ...

which cannot be handled in the latent process, as the joint is not Gaussian. You
might say its proportional with a Gaussian, as long as the constraint is
satisfied, which is true, but that does not help in general. it only helps in
the case where the data say that the isotonic assumptions is true, as then the
constraints are not needed and the isotonic assumption is not needed either. its
most needed when there is prior-data conflict, which is a sign of concern (for
me).

its possible to do this as hyperparameters, which we can rewrite the isotonic
into a sequence of all positive variables. in that case, it can be prototyped
using 'rgeneric'


to be honest, I think isotonic regression is a little 'constructed' problem,
which I do not worry much about

Best
Havard

> --
> You received this message because you are subscribed to the Google Groups "R-
> inla discussion group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to r-inla-discussion...@googlegroups.com.

> To view this discussion, visit
> https://groups.google.com/d/msgid/r-inla-discussion-group/2374a4a6-4080-41bf-90d0-448e1391d9c7n%40googlegroups.com
> .

--
Håvard Rue
hr...@r-inla.org

[Finn Lindgren]

To add to Håvard's comment,

we did try this (almost 10 years ago I think) for a distance sampling
detection model, where we wanted the detection probability model to
decrease monotonically from 1 to zero by doing
exp(-\sum_k B_k(x) * \beta_k) where the B_k are monotonically
increasing and the \beta_k are "clinear" models via INLA::f(). The
"clinear" model was added by Haavard for this special purpose, and
allows a "positive linear effect" to be implemented.
However, we ended up giving up on this approach, and instead used an
non-monotonic 2nd order random walk model constrained to zero at
distance zero. When the data was clear about the detection
probability, this resulted in monotonic models, except in cases where
the monotonic assumption would have been wrong, or such a small
probability of non-monotonicity that those could be eliminated from
the posterior samples instead of -pre-eliminating them in the model
itself. So we got the "best of both worlds"; useful estimates, and a
way to detect non-monotonicity.

Finn

> You received this message because you are subscribed to the Google Groups "R-inla discussion group" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to r-inla-discussion...@googlegroups.com.

> To view this discussion, visit https://groups.google.com/d/msgid/r-inla-discussion-group/81d8cbcec9f62825481b3d25dcbe2ac4b3819c3f.camel%40r-inla.org.

[Tim Meehan]

Hey Håvard and Finn,

Thanks for the ideas!  Håvard, I wish I could not bother with a monotonic smooth. 

But unfortunately I am working with a variable that should only have a positive effect, 

there is very often an asymptote, the effect is not very consistent, the data is really 

noisy, and the response is nbinom distributed. I am trying to get a nonlinear asymptotic 

model component to work in inlabru (first choice), but the characteristics I mentioned 

above make it hard to clamp down on initial values and get reliable convergence. 

Finn, I'd like to try an rw2 constrained to (0, 0). Can either of you point me to an example 

of how that is done in inla() or bru()? 

I had never thought of ditching posterior samples of negative effects. Interesting idea. 

I might try it down the line if other solutions don't work. Same goes for the clinear approach. 

Thankfully I don't need many "knots", so I might try that too if I get desperate.

Best,

Tim

[Håvard Rue]

maybe you could just implement this in 'rgeneric', which should be pretty
straight forward to do

[INLA help]

splinefun has monotone option and you can control the knots using rgeneric 

-- 

Håvard Rue 

hr...@r-inla.org

To view this discussion, visit https://groups.google.com/d/msgid/r-inla-discussion-group/8c94b3b7b35431291cdf7abf3eca8985d4863d3a.camel%40r-inla.org.

[Tim Meehan]

Thanks,  Håvard! I'll give that a try.