How to scale this expression

[Luca]

Hello everyone,

I have a bit of an issue regarding scaling of an expression. So, the
scenario is as follows.

I have a confidence value that can be associated with the solution
given by an optimization routine and it is as follows:

C = exp(-A)/(exp(-A) + exp(-B))

where A, B and C are some energy values returned by the optimization
routine and C represents the confidence or the probability assigned to
the solution A. Also, B is always greater than A.

After some simple manipulation, the expression becomes:

C = 1.0 / (1 + exp(A-B))

Now, in the beginning my issue was that the values A, B and C were
usually quite large (in tens of thousands). So this expression was
giving values of 0.5 when A and B were very close and when the
difference was something a bit larger (in absolute numbers), then the
expression would basically become 1.

So, I realized I needed to do some normalization and the first thing I
tried was divide everything by A. So, now the expression becomes:

C = 1.0 / (1 + exp(1-B/A))

Now typically B/A is something from 1 to 1.01. So, now I have a
similar problem: As exp(1-B/A) will basically be 0.5.

So, what I would like to do is introduce some scaling, normalization
on this expression that would help me basically capture the changes in
my data range. I would be grateful for any suggestions that anyone
might have.

Thanks,
Luca

[William Elliot]

On Fri, 10 Feb 2012, Luca wrote:

> I have a confidence value that can be associated with the solution
> given by an optimization routine and it is as follows:
>
> C = exp(-A)/(exp(-A) + exp(-B))
>
> where A, B and C are some energy values returned by the optimization
> routine and C represents the confidence or the probability assigned to
> the solution A. Also, B is always greater than A.
>
> After some simple manipulation, the expression becomes:
>
> C = 1.0 / (1 + exp(A-B))
>

c = 1/(1 + e^(a - b))

> Now, in the beginning my issue was that the values A, B and C were
> usually quite large (in tens of thousands). So this expression was
> giving values of 0.5 when A and B were very close and when the
> difference was something a bit larger (in absolute numbers), then the
> expression would basically become 1.

No, it would basicaly be 0 or 1.

If a and b are very close, then e^(a-b) is close to 1 and c close to 1/2.
If a << b, then a - b is very negative and c is close to 1.
If b <= a, then a - b is very positive and c is close to 0.
(x << y notation for x is much smaller than y, y is much larger than x)

> So, I realized I needed to do some normalization and the first thing I
> tried was divide everything by A. So, now the expression becomes:
>
> C = 1.0 / (1 + exp(1-B/A))
>

1/(1 + e^(a-b)) does not equal 1/(1 + exp(1 - b/a))
which has different behavior for b << a.

c = 1/(1 + (exp(1 - b/a))^a) provided 0 < 1 - b/a.

> Now typically B/A is something from 1 to 1.01. So, now I have a
> similar problem: As exp(1-B/A) will basically be 0.5.
>
> So, what I would like to do is introduce some scaling, normalization
> on this expression that would help me basically capture the changes in
> my data range. I would be grateful for any suggestions that anyone
> might have.
>

c = e^(-a) / (e^(-a) + e^(-b)) = 1/(1 + e^a e^(-b))
= e^b / (e^a + e^b) = 1/(1 + e^a / e^b)

Use the ratio of e^a to e^b, it'll be more accurate.

-- ??
1/c = 1 + e^a / e^b
-log c = log 1/c = sum(n=1,oo) ((-1)^(n+1) (e^a / e^b)^n / n)

log c = sum(n=1,oo) ((-1)^n (e^a / e^b)^n / n)

----

[Luca]

Thanks for the reply!

>
> c = e^(-a) / (e^(-a) + e^(-b)) = 1/(1 + e^a e^(-b))
> = e^b / (e^a + e^b) = 1/(1 + e^a / e^b)
>
> Use the ratio of e^a to e^b, it'll be more accurate.

That is the problem. This ratio is very hard to evaluate as A and B
are quite large...

Just to highlight, let us assume that my confidence values C is as
follows:

C = 1 / (1 + e^(1-B/A)) B >= A So the value in the exponential is 0 or
negative.

Typically, the ratio B/A is from 1 to 1.01. What I would like to do is
scale it in such a way that when B/A is very close to 1, we get values
still close to 0.5 but as we start to diverge the exponential gets
closer to 0. In my work, I can find out a priori what the maximum
values for A and B are going to be.

Many thanks,

Luca

[Luca]

Just to elaborate a bit on the problem:

So, to reiterate

C= 1 / (1 + e^(B/A))

Typically, B/A ranges from 0.01 to 1.01. What I would like to do is
have some sort of scaling so when B/A is "close" to 1, then C is close
to 0.5 but when it starts to diverge, then C starts to get closer to 0
(when B/A is less than 1) and closer to 1 when B/A is greater than 1.
However, I do not want a linear scale as I want to exaggerate the
differences.

I can now a priori what the maximum B and A values will be. I was
wondering what suitable function can I use. One thing that comes to
mind is to actually use the exponential:

So,

C = 1 / (1 + e^(e^(factor)*B/A)).

However, I am not sure how I can derive this "factor" term in a
suitable way from B and A values that would make sense.

Thanks,

Luca

[William Elliot]

On Sat, 11 Feb 2012, Luca wrote:

> > c = e^(-a) / (e^(-a) + e^(-b)) = 1/(1 + e^a e^(-b))
> > = e^b / (e^a + e^b) = 1/(1 + e^a / e^b)
> >
> > Use the ratio of e^a to e^b, it'll be more accurate.
>
> That is the problem. This ratio is very hard to evaluate as A and B
> are quite large...

e^b / e^a = e^(b-a) or for smaller numbers
log e^b/e^a = log e^b - log e^a = b - a.

> Just to highlight, let us assume that my confidence values C is as
> follows:

> C = 1 / (1 + e^(1-B/A)) B >= A So the value in the exponential is 0 or
> negative.
>
> Typically, the ratio B/A is from 1 to 1.01. What I would like to do is
> scale it in such a way that when B/A is very close to 1, we get values
> still close to 0.5 but as we start to diverge the exponential gets
> closer to 0. In my work, I can find out a priori what the maximum
> values for A and B are going to be.

No, for that new and different c(b/a),
c(1) = 1/2 and c(r) -> 1 as r -> oo.

[William Elliot]

On Sat, 11 Feb 2012, Luca wrote:

> Just to elaborate a bit on the problem:
>
> So, to reiterate
>
> C= 1 / (1 + e^(B/A))
>

That's not reiterating; it's introducing yet another third and different
c(a/b).

> Typically, B/A ranges from 0.01 to 1.01. What I would like to do is
> have some sort of scaling so when B/A is "close" to 1, then C is close
> to 0.5 but when it starts to diverge, then C starts to get closer to 0
> (when B/A is less than 1) and closer to 1 when B/A is greater than 1.
> However, I do not want a linear scale as I want to exaggerate the
> differences.
>

Let f(x) = 1/(1 + x) and try using a few terms of the Taylor series
f(x + r) = f(r) + f'(r)x + f"(r)x^2 / 2! + ...

where r is in the middle of the range of e^(b/a), perhaps r = 2,
so we can have |x| < 1

Determine the radius of convergence of the series
and how fast it converges.

[Tim Little]

On 2012-02-11, Luca <luca.pa...@gmail.com> wrote:
> Typically, B/A ranges from 0.01 to 1.01. What I would like to do is
> have some sort of scaling so when B/A is "close" to 1, then C is
> close to 0.5 but when it starts to diverge, then C starts to get
> closer to 0 (when B/A is less than 1) and closer to 1 when B/A is
> greater than 1. However, I do not want a linear scale as I want to
> exaggerate the differences.

It sounds like f(x) = x^n / (x^n + 1) would do for some choice of n,
where x = B/A. Larger values of n will "stretch" values near 1 more.

This function also satisfies a symmetry condition, such that

f(B/A) = 1 - f(A/B).

I don't know if you wanted that or not.


--
Tim