Tips and tricks for functional tests on chaotic systems

[blais...@gmail.com]

Dear all,

I hope you are well.

It is me again pestering you all with questions or looking for advice :).

We are currently working on our DEM simulation engine using deal.II particles features. DEM lead to very chaotic systems (with positive Lyapunov exponents, like in MD), which means that slight discrepancies in floating point numbers can lead to exponentially different results. You can imagine for example that if a particle is to fall exactly on top of another, a slight difference in round-off error can lead to the particle sliding to the right or to the left of the other particle. Consequently, very small differences accumulate and lead to drastically different results.

Right now, we are testing everything using numdiff within a ctest framework identical to deal.II. However, since we are comparing text files with particles positions and velocities, the tests end up being extremely fragile because they depend on the compiler version and MPI library being used (I guess?).

I was wondering if any of you had experience on what would be the best way to write functional tests that test the full code in the context of systems which show highly chaotic behavior like this? Right now we try to test for very small time, thus ensuring that differences don't have the time to propagate, but this is becoming more and more fragile and sometimes tests will crash on a peculiar machine, yet work on 95% of the other ones (such as our github actions).

Thanks and I wish you all great holidays!

Bruno

[Wolfgang Bangerth]

Hi Bruno,

> We are currently working on our DEM simulation engine using deal.II particles
> features. DEM lead to very chaotic systems (with positive Lyapunov exponents,
> like in MD), which means that slight discrepancies in floating point numbers
> can lead to exponentially different results. You can imagine for example that
> if a particle is to fall exactly on top of another, a slight difference in
> round-off error can lead to the particle sliding to the right or to the left
> of the other particle. Consequently, very small differences accumulate and
> lead to drastically different results.
>
> Right now, we are testing everything using numdiff within a ctest framework
> identical to deal.II. However, since we are comparing text files with
> particles positions and velocities, the tests end up being extremely fragile
> because they depend on the compiler version and MPI library being used (I guess?).
>
> I was wondering if any of you had experience on what would be the best way to
> write functional tests that test the full code in the context of systems which
> show highly chaotic behavior like this? Right now we try to test for very
> small time, thus ensuring that differences don't have the time to propagate,
> but this is becoming more and more fragile and sometimes tests will crash on a
> peculiar machine, yet work on 95% of the other ones (such as our github actions).

Interesting :-)

I think that conceptually, you probably do want to test certain aspects of
your code in this deterministic way. For example, if you want to check the
correctness of the particle trajectory integrator, you can do that with a
small number of particles whose trajectories stay well away from chaotic points.

But there are of course also other aspects that really are chaotic and that
you also want to test. In those cases, you need to identify the statistical
properties that *do* behave deterministically. For example, the motion of
individual stars in globular star clusters is likely chaotic, but the motion
of the center of mass, or the evolution of the angular momentum and total
energy is not. These are things you can compute and output and they should be
comparable among compilers and platforms. If you have enough particles, you
could also consider things such as kernel density estimates of particle
densities, momentum densities, etc.

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[blais...@gmail.com]

Hi Wolfgang,

Thank you for the suggestion. I did not think about it, but testing statistical properties (center of mass, average velocity, total kinetic energy) seems like a very sound idea!

If you want to see an animation of DEM done using the deal.II particle library, you can find one on the following youtube link:

https://www.youtube.com/watch?v=jPrxQ3KqNcI&feature=youtu.be

This is just an illustration, but it actually scales to millions of particles at the moment.

[Wolfgang Bangerth]

On 12/28/20 10:54 PM, blais...@gmail.com wrote:
>
> If you want to see an animation of DEM done using the deal.II particle
> library, you can find one on the following youtube link:
> https://www.youtube.com/watch?v=jPrxQ3KqNcI&feature=youtu.be

Nice -- that just made it to the deal.II Facebook account :-)

Cheers

[blais...@gmail.com]

There is a Facebook deal.II account? Now I know what to follow :)!

Cheers!
Bruno