Q test of significance of the trend of the data

[Cosine]

Hi:

Sometimes we would like to demonstrate that the data has some particular type of trend, e.g., monotone increase or decrease. How do we demonstrate that this trend has statistical significance?

Thank you,

[David Jones]

As a first step, you need to think about the context of whatever data
you have. In particular, you need to consider whether there is temporal
correlation/dependence present in addtition to whatever trend you might
postulate. Also, you should think about whether a possible "trend"
should also include a possible change in variability instead-of or in
addition-to a change in location.

In the simplest case, you can just take a model-free approach whereby
you return to first-principles. Specifically: (a) find a numerical
measure of how much trend there is; (b) find the null distribution of
your numerical measure by evaluating the same numerical measure for
random permutations of the original data. It will be clear how the
assumptions needed for the validity of this relate to my first
paragraph.

In the more general case, you could resort to the usual thing of
building a full probabilistic model and testing via maximum likelihood.

[Rich Ulrich]

On Fri, 6 Jan 2023 14:17:07 -0000 (UTC), "David Jones"
<dajhawk18xx@@nowhere.com> wrote:

>Cosine wrote:
>
>> Hi:
>>
>> Sometimes we would like to demonstrate that the data has some
>> particular type of trend, e.g., monotone increase or decrease. How do
>> we demonstrate that this trend has statistical significance?
>>
>> Thank you,
>
>As a first step, you need to think about the context of whatever data
>you have. In particular, you need to consider whether there is temporal
>correlation/dependence present in addtition to whatever trend you might
>postulate.

Since we don't know at all what you are doing, and since the many
variations of "time series" offer many pitfalls, I would have stopped
right there. In order to ask:
What are you measuring?

We likely can give you advice relevant to a particular problem
(and its pitfalls). "Statistically significant" says, "not by
chance". It does not always imply "interesting"; stupid,
obvious confounding relationships are more common for time-series
than for other data.

If your trends are across something one-dimensional other than time,
the problems are apt to be fewer.

> Also, you should think about whether a possible "trend"
>should also include a possible change in variability instead-of or in
>addition-to a change in location.
>
>In the simplest case, you can just take a model-free approach whereby
>you return to first-principles. Specifically: (a) find a numerical
>measure of how much trend there is; (b) find the null distribution of
>your numerical measure by evaluating the same numerical measure for
>random permutations of the original data. It will be clear how the
>assumptions needed for the validity of this relate to my first
>paragraph.
>
>In the more general case, you could resort to the usual thing of
>building a full probabilistic model and testing via maximum likelihood.

--
Rich Ulrich

[Cosine]

Say we have five groups of subjects, and each receives different concentrations of medicine, from low to high.

At the endpoint, we measure the diameters of the lesion of each subject and calculate the mean diameter of each group.

We expect a monotone decrease trend of the mean diameters of the groups. But how do we demonstrate the significance?

[David Jones]

As part of the first step in significance testing, you need to have a
null hypothesis as well as an alternative hypothesis. There are two
obvious but distinct possibilities for one aspect of what might be
going on: in one the null hypothesis has an unspecified but varying
pattern, to be compared to a monotone pattern, while in the other the
null hypothesis has constant value, to be compared with a monotone
pattern.

[Rich Ulrich]

On Sun, 8 Jan 2023 04:36:55 -0000 (UTC), "David Jones"
<dajh...@nowherel.com> wrote:

>Cosine wrote:
>
>> Say we have five groups of subjects, and each receives different
>> concentrations of medicine, from low to high.
>>
>> At the endpoint, we measure the diameters of the lesion of each
>> subject and calculate the mean diameter of each group.
>>
>> We expect a monotone decrease trend of the mean diameters of the
>> groups. But how do we demonstrate the significance?
>
>As part of the first step in significance testing, you need to have a
>null hypothesis as well as an alternative hypothesis.

Or - you can have a situation where you want to provide a
precise assessment, where basic "significance" is assumed, and
readily established by any test.

Having 5 concentrations, without a Zero comparison, implies
that the questions (hypotheses) concern whether the lowest
dose (concentration) has much effect, or if there is continued
gain from increasing dose by each step.

A overall test:
Assuming that the doses here are judged (by the PI) to be
(in the relevant sense) equal intervals, a simple correlation
will show that increasing dose /matters/. This will be HIGHLY
significant, you hope.

(Also, the outcome should probably take into account the size
of the original lesion. Log of the Pre/Post ratio might be natural,
if lesions don't decrease to 0.)

If I had data like these, I would want to plot the Pre vs. Post
for the 5 doses, and figure out from the picture what there is
to describe. A strong linear trend of efficicay across dose (log
concentration) with tiny contributions from the nonlinear ANOVA
components would be the outcome most convenient to describe.

> There are two
>obvious but distinct possibilities for one aspect of what might be
>going on: in one the null hypothesis has an unspecified but varying
>pattern, to be compared to a monotone pattern, while in the other the
>null hypothesis has constant value, to be compared with a monotone
>pattern.

--
Rich Ulrich

[David Jones]

Rich Ulrich wrote:

> On Sun, 8 Jan 2023 04:36:55 -0000 (UTC), "David Jones"
> <dajh...@nowherel.com> wrote:
>
> > Cosine wrote:
> >
> >> Say we have five groups of subjects, and each receives different
> >> concentrations of medicine, from low to high.
> >>
> >> At the endpoint, we measure the diameters of the lesion of each
> >> subject and calculate the mean diameter of each group.
> >>
> >> We expect a monotone decrease trend of the mean diameters of the
> >> groups. But how do we demonstrate the significance?
> >
> > As part of the first step in significance testing, you need to have
> > a null hypothesis as well as an alternative hypothesis.
>
> Or - you can have a situation where you want to provide a
> precise assessment, where basic "significance" is assumed, and
> readily established by any test.
>
> Having 5 concentrations, without a Zero comparison, implies
> that the questions (hypotheses) concern whether the lowest
> dose (concentration) has much effect, or if there is continued
> gain from increasing dose by each step.
>
> A overall test:
> Assuming that the doses here are judged (by the PI) to be
> (in the relevant sense) equal intervals, a simple correlation

> will show that increasing dose matters. This will be HIGHLY

> significant, you hope.
>
> (Also, the outcome should probably take into account the size
> of the original lesion. Log of the Pre/Post ratio might be natural,
> if lesions don't decrease to 0.)
>
> If I had data like these, I would want to plot the Pre vs. Post
> for the 5 doses, and figure out from the picture what there is
> to describe. A strong linear trend of efficicay across dose (log
> concentration) with tiny contributions from the nonlinear ANOVA
> components would be the outcome most convenient to describe.
>
>
> > There are two
> > obvious but distinct possibilities for one aspect of what might be
> > going on: in one the null hypothesis has an unspecified but varying
> > pattern, to be compared to a monotone pattern, while in the other
> > the null hypothesis has constant value, to be compared with a
> > monotone pattern.

The OP has been very unclear, so there seems also to be at least one
other possibility, where the null hypothesis is that there is a
monotone pattern, with the alternative hypothesis (that which one is
looking evidence might be happening) is that there is a change in
direction of the pattern as the dosage increases (but possibly just one
turning point).

[Rich Ulrich]

On Sun, 8 Jan 2023 10:48:46 -0000 (UTC), "David Jones"

Concerning alternative hypotheses: The OP's example might have been
made up, but I've seen real instances where the PI did not consider,
What do I REALLY want to show? Will my numbers be able to show it?

Oh -'randomization' is necessary if one wants the easier conclusions
of a 'randomized trial' (compared to observational reports). If size
of lesion varies a lot, it could be worth stratifying the
randomization.

"Monotonic increase in response" is not as interesting as the actual
degree of improvement. Or: Has someone argued that 'high' will be
bad?

If there is special concern about the end-points, it could be
worthwhile to use larger Ns at the ends. (Is a no-dose condition
non-informative? or well-known as having No-change?)

Also, 'statistical power' is the reason that two-group studies are
by far the most common. Trying to reach a firm conclusion about
whether every two groups (dose) differ, out of five groups, when
the dose-differences are small ... would require a larger N than
anyone ordinarily justifies.

--
Rich Ulrich