stirring chocolate pudding backwards: identifying coordinates
Andreas
The following function samples sets of coordinates from a multi-dimensional simplex:
sampleSimplex[sampleSize_, dimensions_] := Normalize[#, Total](*)*)& /@ RandomReal[ExponentialDistribution[2], {sampleSize, dimensions}]
Each set of coordinates (sub-lists in the output) totals 1. The following graphs the coordinates so you can
see what the code does:
Graphics3D[Point /@ sampleSimplex[1000, 3, 0, 1], ImageSize -> 250, ViewPoint -> {Pi, Pi/2, 2}]
To keep things simple let's generate 4 sets of coordinates, each with 3 dimensions:
samples = sampleSimplex[4, 3]
{
{0.375153,0.412505,0.212342},
{0.470678,0.360788,0.168533},
{0.0510186,0.828575,0.120407},
{0.24481,0.00851303,0.746677}
}
Again note: each sub-list totals 1. Also in an actual application I would take many more samples from the simplex.
Next, I generate time series data for just 3 intervals with the same dimensionality as the above:
values = RandomReal[{.08, 1.2}, {3, 3}]
{
{0.518072,0.490701,0.813364},
{0.404083,0.356724,0.362498},
{0.507292,0.436016,0.247148},
}
The next function calculates a weighted average of all the sets of coordinates, with the weights in proportion to the corresponding time series values. Note: each sub-list of values corresponds to a single time interval whereas the following function will average the values of the entire set of sub-lists in the list "samples" at each time interval.
weightedAverage[values_, samples_] := Mean[(t = #; 1 + Inner[Times, t, # - 1, Plus] & /@ values) & /@ samples]
weightedAverages = weightedAverage[values, samples]
{
0.599181
0.372042
0.397434
}
A bunch of people of this board have helped me get this far. So far so good. Many thanks to all of you.
Next I need to determine 2 things. I need to identify the specific coordinates associated with:
1. the weighted average value at each interval and
2. the highest value at each interval.
To help clarify this (I hope), I think of the triangular surface depicted in the Graphics3D[] above and imagine a value axis extending perpendicular to it. This represents a single time interval. I picture that the
weightedAverage[] function calculates a value for each point on the surface hovering above the samples triangle. It then averages them. So one can imagine a second surface hovering above the first representing the values associated with each point of the first.
So, again I need to identify the point associated with the highest value at each interval (this could only come
from the highest values point from the list, "samples") and similarly the point associated with the weighted average value (this might or might not be a point within the list, "samples" because the weighted average value might fall at a point between the samples).
Not certain how to approach a solution. From the functional programming constructs I've used so far it seems a little like stirring chocolate pudding backwards and hoping to get back to the original ingredients.
Any help much appreciated, I hope everyone hasn't disappeared for the summer.
Thanks,
A
Ray Koopman
{.375153, .412505, .212342},
{.470678, .360788, .168533},
{.0510186, .828575, .120407},
{.24481, .00851303, .746677}};
values = {
{.518072, .490701, .813364},
{.404083, .356724, .362498},
{.507292, .436016, .247148}};
values.Mean[samples]
{0.599181, 0.372042, 0.397434}
which are the same as your weightedAverages.
The answer to your second question is that the point associated
with the weighted average value is always the average sample point.
The answer to your first question -- the point asssociated with
the highest value at each interval -- is a little more complicated.
We have
values.Transpose@samples
{{0.569484, 0.557963, 0.530949, 0.738327},
{0.375717, 0.379988, 0.359836, 0.372629},
{0.422651, 0.437733, 0.416912, 0.312442}}
so the indices of the maximizing points are given by
Flatten[Ordering[samples.#,-1]& /@ values]
{4, 2, 2}
On Jun 25, 4:13 am, Andreas <aa...@ix.netcom.com> wrote:
> Some background to start, which I'll follow with two questions:
>
> The following function samples sets of coordinates from a multi-dimension=
al simplex:
>
> sampleSimplex[sampleSize_, dimensions_] := Normalize[#, Total](*)*)=
& /@ RandomReal[ExponentialDistribution[2], {sampleSize, dimensions}]
>
> Each set of coordinates (sub-lists in the output) totals 1. The followin=
g graphs the coordinates so you can
> see what the code does:
>
> Graphics3D[Point /@ sampleSimplex[1000, 3, 0, 1], ImageSize -> 250, V=
iewPoint -> {Pi, Pi/2, 2}]
>
> To keep things simple let's generate 4 sets of coordinates, each with 3 d=
imensions:
>
> samples = sampleSimplex[4, 3]
>
> {
> {0.375153,0.412505,0.212342},
> {0.470678,0.360788,0.168533},
> {0.0510186,0.828575,0.120407},
> {0.24481,0.00851303,0.746677}
> }
>
> Again note: each sub-list totals 1. Also in an actual application I woul=
d take many more samples from the simplex.
>
> Next, I generate time series data for just 3 intervals with the same dime=
nsionality as the above:
>
> values = RandomReal[{.08, 1.2}, {3, 3}]
>
> {
> {0.518072,0.490701,0.813364},
> {0.404083,0.356724,0.362498},
> {0.507292,0.436016,0.247148},
> }
>
> The next function calculates a weighted average of all the sets of coordi=
nates, with the weights in proportion to the corresponding time series valu=
es. Note: each sub-list of values corresponds to a single time interval wh=
ereas the following function will average the values of the entire set of s=
ub-lists in the list "samples" at each time interval.
>
> weightedAverage[values_, samples_] := Mean[(t = #; 1 + Inner[Time=
s, t, # - 1, Plus] & /@ values) & /@ samples]
>
> weightedAverages = weightedAverage[values, samples]
>
> {
> 0.599181
> 0.372042
> 0.397434
> }
>
> A bunch of people of this board have helped me get this far. So far so g=
ood. Many thanks to all of you.
>
> Next I need to determine 2 things. I need to identify the specific coord=
inates associated with:
>
> 1. the weighted average value at each interval and
> 2. the highest value at each interval.
>
> To help clarify this (I hope), I think of the triangular surface depicted=
in the Graphics3D[] above and imagine a value axis extending perpendicular=
to it. This represents a single time interval. I picture that the
>
> weightedAverage[] function calculates a value for each point on the surfa=
ce hovering above the samples triangle. It then averages them. So one can=
imagine a second surface hovering above the first representing the values =
associated with each point of the first.
>
> So, again I need to identify the point associated with the highest value =
at each interval (this could only come
> from the highest values point from the list, "samples") and similarly the=
point associated with the weighted average value (this might or might not =
be a point within the list, "samples" because the weighted average value mi=
ght fall at a point between the samples).
>
> Not certain how to approach a solution. From the functional programming =
constructs I've used so far it seems a little like stirring chocolate puddi=
ng backwards and hoping to get back to the original ingredients.
>
> Any help much appreciated, I hope everyone hasn't disappeared for the sum=
mer.
>
> Thanks,
> A
Andreas
Thx for the input. I've started working through your solution for my first question -- the point associated with the highest value at each interval -- and it makes sense to me, so far. I may have further questions about it as I push my way through it, but it looks spot-on.
I do have questions about your answer to my second question, probably because I may have miss-stated what I want to do.
Using 3 coordinates as in my original example, as the sample size increases the Mean[samples] approaches {.333..., .333..., .333...}. This makes sense as the Mean of the indexes of the population from which I sample would give us the same list of coordinates.
But this point may or may not correspond to the value of what I need to know.
I want to find the coordinates at each time increment of the weighted average of all samples with the weights in proportion to their performance or "values" in the time series data. Wouldn't this point vary, over time, across the plane from which I select the samples (especially as in its projected application I'll have values for thousands of time increments and they may have far greater variance than my simple example)? Also, it seems like it wouldn't have to correspond to any specific point in the list of samples.
I have a penchant for making things more complicated than necessary, but I think what I want differs from the Mean[samples]. Any insight much appreciated.
A
Ray Koopman
Dimensions[samples] = {p,d} = {# of points, # of dimensions}. If
each sample point has a value associated with it at each of t time
intervals, then Dimensions[values] should be either {p,t} or {t,p}.
Assume it's {t,p}, with each row corresponding to a time interval.
Then values.samples/(Tr/@values) will give a matrix whose dimensions
are {t,d}, with each row containsing the weighted average coordinates
for a time interval.
Andreas
I can't get values.samples/(Tr /@ values) to work. I get messages like the following:
Dot::dotsh: "Tensors {{...}} and {{..}} have incompatible shapes.
I'll try to think through what you're doing over the holiday and try to come up with something. In the mean time any ideas much appreciated.
Best,
A
P.S. your values.Mean[samples] runs blazingly fast, even on very large data sets. Makes my previous effort look clunky. I always learn something from your posts. Thx. -- A
Ray Koopman
> I want to find the coordinates at each time increment of the
> weighted average of all samples with the weights in proportion to
> their performance or "values" in the time series data. Wouldn't this
> point vary, over time, across the plane from which I select the
> samples (especially as in its projected application I'll have values
> for thousands of time increments and they may have far greater
> variance than my simple example)? Also, it seems like it wouldn't
> have to correspond to any specific point in the list of samples.
On Jul 2, 4:15 am, Ray Koopman <koo...@sfu.ca> wrote:
> Dimensions[samples] = {p,d} = {# of points, # of dimensions}. If
> each sample point has a value associated with it at each of t time
> intervals, then Dimensions[values] should be either {p,t} or {t,p}.
> Assume it's {t,p}, with each row corresponding to a time interval.
> Then values.samples/(Tr/@values) will give a matrix whose dimensions
> are {t,d}, with each row containsing the weighted average coordinates
> for a time interval.
On Jul 4, 3:38 am, Andreas <aa...@ix.netcom.com> wrote:
> I can't get values.samples/(Tr /@ values) to work.
> I get messages like the following:
>
> Dot::dotsh: "Tensors {{...}} and {{..}} have incompatible shapes.
If the analysis in my July 2 post is correct, and if you're trying to
analyze the data from your initial post, then the 'values' matrix is
the wrong size. Its dimensions are {3,3}, but they should be {3,4} or
{4,3}. Actually, things would be clearer if you used distinct values
for p, d, and t. For instance:
p = 5 sample points, d = 3 dimensions, t = 4 time intervals.
samples
{{0.35212, 0.195537, 0.452343},
{0.259074, 0.481887, 0.259039},
{0.778938, 0.00217905, 0.218883},
{0.056913, 0.266178, 0.676909},
{0.166826, 0.0516333, 0.781541}}
values
{{1.07072, 0.975643, 0.881083, 0.962515, 1.55834},
{0.411462, 0.200276, 0.823472, 1.59278, 1.31966},
{1.24984, 0.837737, 1.26199, 0.920523, 1.4315},
{1.23532, 1.81471, 0.63395, 0.0366297, 0.251216}}
values.samples/(Tr/@values)
{{0.299331, 0.186865, 0.513804},
{0.264282, 0.154305, 0.581413},
{0.338738, 0.170088, 0.491175},
{0.363291, 0.287056, 0.349653}}