[SciPy-User] characteristic functions of probability distributions

[josef...@gmail.com]

The characteristic function is just the (continuous) fourier transform
of the probability density function.

I tried to use fft and ifft to convert between the characteristic
function and the density function but I don't manage to get the units
or discretization correctly. Does anyone have an example script for
any distribution. Right now it's mostly a theoretical exercise, but
there are some interesting applications in finance.

Second related question, since I'm not good with complex numbers.

scipy.integrate.quad of a complex function returns the absolute value.
Is there a numerical integration function in scipy that returns the
complex integral or do I have to integrate the real and imaginary
parts separately?

Thanks,

Josef
_______________________________________________
SciPy-User mailing list
SciPy...@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user

[nicky van foreest]

Hi Josef,

2009/11/2 <josef...@gmail.com>:

> The characteristic function is just the (continuous) fourier transform
> of the probability density function.
>
> I tried to use fft and ifft to convert between the characteristic
> function and the density function but I don't manage to get the units
> or discretization correctly. Does anyone have an example script for
> any distribution. Right now it's mostly a theoretical exercise, but
> there are some interesting applications in finance.

There is an inversion formula used to invert the characteristic
function to the distribution function (the density should follow
easily then), see e.g. Chung (or any other book on graduate
probability). I don't know about its numerical properties though. The
formula is used to prove the central limit theorem. I also recall that
Ward Whitt (see his homepage) used Fourier theory to invert Laplace
transforms. He was also concerned with numerical properties, so this
might be the best place to look for. He also uses the inversion
formula, and refers to Feller.

>
> Second related question, since I'm not good with complex numbers.
>
> scipy.integrate.quad of a complex function returns the absolute value.
> Is there a numerical integration function in scipy that returns the
> complex integral or do I have to integrate the real and imaginary
> parts separately?

You want to compute \int_w^z f(t) dt? When f is analytic (i.e.,
satisfies the Cauchy Riemann equations) this integral is path
independent. Otherwise the path from w to z is of importance. You
might like the book Visual Complex Analysis by Needham for intuition.

bye

Nicky

[David Goldsmith]

On Mon, Nov 2, 2009 at 12:51 PM, nicky van foreest <vanfo...@gmail.com> wrote:

Hi Josef,

> Second related question, since I'm not good with complex numbers.

>
> scipy.integrate.quad of a complex function returns the absolute value.
> Is there a numerical integration function in scipy that returns the
> complex integral or do I have to integrate the real and imaginary
> parts separately?

You want to compute \int_w^z f(t) dt? When f is analytic (i.e.,
satisfies the Cauchy Riemann equations) this integral is path
independent. Otherwise the path from w to z is of importance. You
might like the book Visual Complex Analysis by Needham for intuition.

Furthermore, if f is analytic in an (open) region R homotopic to an (open) disc, then the integral (an integer number of times) around *any* _closed_ path wholly in R is identically equal to zero; there's a similar statement (though the end value is a multiple of 2ipi) if f has only poles of finite order in R.  (Indeed, these properties should be used to unit test any numerical complex path integration routine.)  Are any of your paths closed?

DG

[Anne Archibald]

2009/11/5 David Goldsmith <d.l.go...@gmail.com>:

> On Mon, Nov 2, 2009 at 12:51 PM, nicky van foreest <vanfo...@gmail.com>
> wrote:
>>
>> Hi Josef,
>> > Second related question, since I'm not good with complex numbers.
>> >
>> > scipy.integrate.quad of a complex function returns the absolute value.
>> > Is there a numerical integration function in scipy that returns the
>> > complex integral or do I have to integrate the real and imaginary
>> > parts separately?
>>
>> You want to compute \int_w^z f(t) dt? When f is analytic (i.e.,
>> satisfies the Cauchy Riemann equations) this integral is path
>> independent. Otherwise the path from w to z is of importance. You
>> might like the book Visual Complex Analysis by Needham for intuition.
>
> Furthermore, if f is analytic in an (open) region R homotopic to an (open)
> disc, then the integral (an integer number of times) around *any* _closed_
> path wholly in R is identically equal to zero; there's a similar statement
> (though the end value is a multiple of 2ipi) if f has only poles of finite
> order in R.  (Indeed, these properties should be used to unit test any
> numerical complex path integration routine.)  Are any of your paths closed?

This may well be a red herring. It happens fairly often (to me at
least) that I want to integrate or otherwise manipulate a function
whose values are complex but whose independent variable is real.

Such a function can arise by substituting a path into an analytic
function, but there are potentially many other ways to get such a
thing - for example you might choose to represent some random function
R -> R2 as R -> C instead. Even if it's obtained by feeding a path
into some function from C -> C, it happens very often that that
function isn't analytic - say it involves an absolute value, or
involves the complex conjugate.

There are definitely situations in which all the clever machinery of
analytic functions can be applied to integration problems (or for that
matter, contour integration may be the best way available to evaluate
some complex function), but there are also plenty of situations where
what you want is just a real function whose values happen to be
complex numbers. (Or vectors of length n for that matter.) But I don't
think that any of the adaptive quadrature gizmos can handle such a
case, so you might be stuck integrating the real and imaginary parts
separately.

If you *are* in a situation where you're dealing with an analytic
function, then as long as you're well away from its poles and your
path is nice enough, you may find that it's very well approximated by
a polynomial of high degree, which will let you use Gaussian
quadrature, which can very easily work with complex-valued functions.
The Romberg integration might even work unmodified.

Anne

[josef...@gmail.com]

Sorry for not coming back to this earlier,

Thanks Nicky, I looked at some papers by Ward Whitt and they look
interesting but much more than what I want to chew on right now. There
is more background, that I would have to read, than I have time right
now for this. I finally added "matlab" to my google searches, and I
think I found some references that use discretization and fft more
directly.

The integration problem should be pretty "nice", just a continuous
fourier transform and the inverse

http://en.wikipedia.org/wiki/Characteristic_function_%28probability_theory%29#Definition
http://en.wikipedia.org/wiki/Characteristic_function_%28probability_theory%29#Inversion_formulas

For many distributions there is an explicit formula for both the
density and the characteristic function, e.g. normal
http://en.wikipedia.org/wiki/Normal_distribution#Characteristic_function

For some distributions only the characteristic functions has a closed
form expression, and the pdf or cdf has to be recovered numerically,
and I would have liked to have a generic method to go between the two.

I don't think I ever needed a path integral in my life, and I'm pretty
much a newbie to complex numbers, so parts of your explanations are
still quite a bit over my head. I think, I will come back to this
after I looked more at the examples where the estimation of a
statistical model or of a distribution is done in terms of the
characteristic function instead of the density.

The immediate example that I had tried, was (integration from -large
number to +large number)

integral exp(i t x)dF(x) = integrate.quad(exp(itx)*f(x))

or do I have to do

integral exp(i t x)dF(x) = integrate.quad(real(exp(itx)*f(x))) + j *
integrate.quad(imag(exp(itx)*f(x)))

or is there another way?
The solution/integral might be either real or complex.

Thanks,

Josef

[nicky van foreest]

Hi Joseph,

> Thanks Nicky, I looked at some papers by Ward Whitt and they look
> interesting but much more than what I want to chew on right now.

I understand. I wish I had the time to study these papers in more detail.

> I don't think I ever needed a path integral in my life,

That must be a most undesirable state of affairs, :-)

> integral exp(i t x)dF(x)  = integrate.quad(real(exp(itx)*f(x))) + j *
> integrate.quad(imag(exp(itx)*f(x)))
> or is there another way?

Perhaps you recall that Re(exp(ix)) = cos(x), and Im(exp(ix) =
sin(x). Hence, you might try simply:

integrate.quad(cos(tx)f(x)) + i integrate.quad(sin(tx) f(x))

(untested code though..)

I have my doubt about the stability of these integrations, although I
am by no means an expert on this. Suppose that t is big. Then cos(tx)
varies rapidly in comparison to f(x) as a function of x. Then you are
adding lots of negative and possitive numbers of roughly the same
size... This must result in bogus.

Perhaps it is better not to include any "generic" code to tranform the
characteristic function into a density, unless the methods work
reasonably well.

bye

Nicky