Single value indicating whether curved distribution treds up or down

[Christoph Ruehlemann]

Hi All,

I'm working with data with lots of internal variation; an example vector is this:

 data <- c(1, 1.5, 2, 1.7, 2, 2.1, 3, 5, 9, 15)




My interest is to find out whether despite its internal variation the distribution  *overall* trends upward or downward; also I need a *single* value indicating that trend for the whole distribution.




I've been trying out the following methods:




1. mean derivative:

    mean(diff(data, differences = 1))

2. mean second derivative:

    mean(diff(data, differences = 2))

3. polynomial regression:

     data <- c(1, 1.5, 2, 0, 2, 2.1, 3, 5, 9, 10)
    pos <- c(1:10)
    mod <- lm(data ~ pos + I(pos^2)) # I for 'as is'
    summary(mod)

I'm not sure whether any of these methods is going in the right direction and if so, which one is. If the polynomial regression is the way forward, which of the coeffcients indicates the slope? Also are there other methods available? Which ones?

Best wishes and many thanks in advance

Christoph










--

Albert-Ludwigs-Universität Freiburg

Projekt-Leiter DFG-Forschungsprojekt "Multimodale Turn-Abschlusssignale"

(https://gepris.dfg.de/gepris/projekt/497779797)

Homepage: http://paul.igl.uni-freiburg.de/ruehlemann/?Home

ἰχθύς

[Stefan Th. Gries]

I think there's lots of ways to go about this, but I also think you need to define "overall trend" before one can even think about this:

1) Does this curve

par(mfrow=c(1,2))
plot(qwe <- c(3.5,2.25,3,4,3,2,seq(2, 3.4, 0.1)), type="b"); grid()

# note: mean(diff(qwe)) is <0 and mean(diff(qwe, 1, 2)) is also >0

- have an overall up trend (because it goes up across a longer 'distance'/range of values on the x-axis)?

- have an overall downwards trend
-- because its y-value at x's max is a bit smaller (3.4) than its y-value at x's min (3.5))?
-- because its y-value at x's max is not greater than the max of all y-values (4)?

(Obviously, an analogous question would arise if the max of qwe was more shifted to the right.)

2) In addition, in the above case one might 'feel' it goes up because on the right it's so close to where it started on the left so there's a 'lot of going up' again. But what about this one?

plot(asd <- c(3.5,2.25,3,4,3,2,seq(2, 2.25, 0.017857)), type="b"); grid()

# note: mean(diff(asd)) is also <0 and mean(diff(asd, 1, 2)) is also >0

Does this curve have

- an overall up trend (because it goes up across a longer 'distance'/range of values on the x-axis)? In
- an overall downwards trend (because its y-value at x's max is a lot smaller (2.25) than its y-value at x's min (3.5))?

So, what roles do the endpoints of y for min(x) and max(x) play, what role(s) does it play for how long y goes up and how much for how long on the x-axis?

I'm not implying I would have solutions for whatever answers you'd give -- all I'm saying is you might not be able to develop an answer without having some sort of operationalizable criteria to apply to such questions.