cut revisited

[Bill Gillespie]

Dear Andrew et al,

I beg your indulgence for posting a topic that is not strictly a Stan issue. I am posting it here because it follows up on an earlier thread (https://groups.google.com/forum/#!msg/stan-users/ruxPAhBPI4A/Aj-BvWGOlEQJ). In addition I am considering the use of an approach to "cut" in an analysis using Stan.

During the previous thread Andrew Gelman stated: "The reason why I (and others) perceive "cut" as bad news is because when you do "cut," you are no longer doing Bayesian inference.  To put it another way, if it's really true that you don't want inference going from A to B, I don't think you want it going from B to A either. Joint distributions go both ways."

I accept that this is true for formal fully Bayesian inference, but I also think that prevention of "Bayesian feedback" is a useful heuristic in some practical contexts. David Lunn et al (Pharmacokinet Pharmacodyn (2009) 36:19–38) attempt to make that case for PKPD modeling. In that context there are usually 2 categories of dependent variable data: PK and PD, typically plasma drug concentrations and one or more measures of drug effects. In a typical PKPD model the PD outcomes are assumed to be a consequence of the PK (among other covariates). 

My rationale for using cut in PKPD modeling is the same (I think) as that expressed in Lunn et al. Bayesian PKPD modeling is an excellent formalism for dealing with uncertainty in the model parameters. However our inferences are also conditioned on the mathematical structure of our model. There is also uncertainty in the form of our model and we usually do not account for that in any formal way. In PKPD modeling there are usually 2 (or more) linked models: one for PK outcomes and one for PD outcomes where it is usually assumed that the PD outcomes are affected by the PK (but not the reverse though there are exceptions). In many cases the uncertainty in the PD submodel structure is substantially greater than that for the PK model structure. This is due in part to differences in our state of knowledge about the underlying processes that cause the observed magnitudes in PK and PD outcomes. In addition there is often interest in making inferences about both PK and PD. Given such intentions, the disparity in model uncertainty and the assumed causal path, I argue it is undesirable for the larger potential PD model misspecification to impact inferences regarding PK.

I am interested in hearing other viewpoints on questions like:

• Do you agree that the disparity in submodel uncertainty is an issue to be addressed in some way in the data analysis? 
  
• Is the notion of "cutting feedback" from the PD data to the PK model a reasonable heuristic approach for addressing that issue, and, if not, what are the alternatives? 
  
• Couldn't "cut" in the PKPD application be interpreted as a Bayesian analysis in which the PK model predictions are considered covariate data rather than a component of the prior? This is an attempt to argue for treating "cut" as multiple imputation of data.

So the bottom line is that I am trying to make the case for "cut", or to provoke someone into setting me straight.

I should also mention that I am aware of Martyn Plummer's convincing criticism of the approach to "cut" used in BUGS (http://www.pages.drexel.edu/~mwl25/mcmski/slides/mp_cuts.pdf), so I am not arguing for that approach. Instead I plan to apply the multiple imputation approach he suggests.

Thanks,

Bill

[Bob Carpenter]

On Feb 27, 2014, at 7:31 PM, Bill Gillespie <bi...@metrumrg.com> wrote:

> Dear Andrew et al,
>
> I beg your indulgence for posting a topic that is not strictly a Stan issue. I am posting it here because it follows up on an earlier thread (https://groups.google.com/forum/#!msg/stan-users/ruxPAhBPI4A/Aj-BvWGOlEQJ). In addition I am considering the use of an approach to "cut" in an analysis using Stan.

We can't resist general stats discussion, either. Maybe
Andrew will take this up on his blog as it might be interesting
to get feedback from a wider audience and because it's connected
to the way mi uses an approach that feels just like a BUGS cut
model.

Maybe phrasing it this way will make him change his mind, but
I'd guess he'd say that he's using mi as a heuristic and he'd
do the full joint model if he could formulate the computations.
(In fact, his postdoc Yajuan (Sophie) Si did just this with non-parametric
Bayesian approaches to categorical missing data in her Duke Ph.D. thesis and
I believe is extending the work to continuous data with Andrew.)

>
> During the previous thread Andrew Gelman stated: "The reason why I (and others) perceive "cut" as bad news is because when you do "cut," you are no longer doing Bayesian inference. To put it another way, if it's really true that you don't want inference going from A to B, I don't think you want it going from B to A either. Joint distributions go both ways."
>
> I accept that this is true for formal fully Bayesian inference, but I also think that prevention of "Bayesian feedback" is a useful heuristic in some practical contexts. David Lunn et al (Pharmacokinet Pharmacodyn (2009) 36:19–38) attempt to make that case for PKPD modeling. In that context there are usually 2 categories of dependent variable data: PK and PD, typically plasma drug concentrations and one or more measures of drug effects. In a typical PKPD model the PD outcomes are assumed to be a consequence of the PK (among other covariates).
>
> My rationale for using cut in PKPD modeling is the same (I think) as that expressed in Lunn et al. Bayesian PKPD modeling is an excellent formalism for dealing with uncertainty in the model parameters.

David Lunn and Martyn Plummer both spoke on this topic at a session during
the MCMski conference. I didn't follow all the details, and alas, I was
the only one of the Stan team in that session. But they might share their
slides with you if you sent them e-mail.

> However our inferences are also conditioned on the mathematical structure of our model. There is also uncertainty in the form of our model and we usually do not account for that in any formal way. In PKPD modeling there are usually 2 (or more) linked models: one for PK outcomes and one for PD outcomes where it is usually assumed that the PD outcomes are affected by the PK (but not the reverse though there are exceptions). In many cases the uncertainty in the PD submodel structure is substantially greater than that for the PK model structure. This is due in part to differences in our state of knowledge about the underlying processes that cause the observed magnitudes in PK and PD outcomes. In addition there is often interest in making inferences about both PK and PD. Given such intentions, the disparity in model uncertainty and the assumed causal path, I argue it is undesirable for the larger potential PD model misspecification to impact inferences regarding PK.
>
> I am interested in hearing other viewpoints on questions like:
> • Do you agree that the disparity in submodel uncertainty is an issue to be addressed in some way in the data analysis?

Absolutely!

> • Is the notion of "cutting feedback" from the PD data to the PK model a reasonable heuristic approach for addressing that issue, and, if not, what are the alternatives?

The amount of certainty coming from the PK model can
be addressed as a prior on the variances in the usual
Bayesian way. This makes the joint inference consistent.
If you really are that certain about the PK estimates, then
they should have a small posterior variance and perhaps be
defined in a very narrow range of plausible support. If they
don't, then you don't have as much certainty as you think they
do.

I think we're all a bit confused as to why those in PK/PD
think these inferences should only go one way.


> • Couldn't "cut" in the PKPD application be interpreted as a Bayesian analysis in which the PK model predictions are considered covariate data rather than a component of the prior? This is an attempt to argue for treating "cut" as multiple imputation of data.

If you do this, the result is not Bayesian inference on
the data being imputed. There's also no way to implement it
in Stan or JAGS.

This is a good point. Andrew's approach to multiple imputation is
not fully Bayesian in that it's not doing inference over a joint
probability model.

In your context, suppose we have two models:

PK: p(phi,y1) params: phi, data: y1

PD: p(theta,y2|phi) params: theta, data y2

Then doing multiple imputation on the PK posterior p(phi|y1)
by drawing a sample phi^1,...,phi^M and
and then plugging that into the PD model to compute

p(theta) =?= INT p(theta|y2,phi) p(phi|y1) d.phi

=approx= SUM_m p(theta | y2, phi^m)

will not yield the same inferences as the joint PK/PD model

PK/PD: p(phi,theta,y1,y2) = p(theta,y2|phi) * p(phi,y1)

specifically because of the way phi is "cut".

Now I'm curious as to what Andrew will say about multiple imputation,
because it suffers exactly the same defect w.r.t. full Bayes.

Or what other PK/PD modelers will say, because we've heard this
request before in exactly this context ("we trust our PK more
than our PD so want to cut the PK inferences from the PD inferences").

For instance, what happens when you do the joint inference? Do you
get results that are completely implausible? Maybe that says that
the PD model is just not right rather than that you shouldn't be doing
joint inference.

- Bob

> So the bottom line is that I am trying to make the case for "cut", or to provoke someone into setting me straight.
>
> I should also mention that I am aware of Martyn Plummer's convincing criticism of the approach to "cut" used in BUGS (http://www.pages.drexel.edu/~mwl25/mcmski/slides/mp_cuts.pdf), so I am not arguing for that approach. Instead I plan to apply the multiple imputation approach he suggests.
>
> Thanks,
> Bill
>

> --
> You received this message because you are subscribed to the Google Groups "stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.