Q about orientation average for large simulation
Junum
large random orientations.
I have enough computational resources, but a problem is that the
allowed wall time on a cluster is 24 hour which is not enough to
finish jobs.
This cluster has tons of CPUs, but another problem is CPU limitation
in ADDA.
My target particle has 288 X 124 X 144 dimension.
I think that maximum efficiency is achieved by setting # of CPU equal
to number of Z dimension, e.g., it is 144 in this case.
Use of CPU more than 144 does not improve computing time.
My avg_params.dat has Jmax=5, 4, and 3 for alpha, beta, and gamma,
respectively, thus it is 4352 orientation average.
This simulation exceeds allowed wall time (i.e., 24 hour), so I am
considering separate simulations.
Two options are in my mind.
1. Separate one large simulation as single orientation simulations.
In this case total number of separate simulations are 4352 jobs.
Each simulation has different orientation with -orient option.
A question in this option is about an orientation average over
independent results.
With 4352 results, is a linear orientation average ok?
Or is scattering cross section weighting or other technique necessary?
My interst is muller matrix, Qext, Qabs, Qsct, and g.
2. Separate one simulation as less small simulations.
For example, change avg_params.dat as Jmax=4, 2, and 2 for alpha,
beta, and gamma, respectively, thus it is 320 orientation average and
repeats this 14 times, 320 orientations X 14 = 4480 orientations.
A probelm for this option is that there in no way to change particle
orientation or direction of incident light for orientation average in
ADDA, I need to generate a particle with different orientation each
time as an input.
If default particle shape is same (i.e., orientation) respect to
reference frame, these 14 simulations would be identical, right?
Since target particle is not available in ADDA particle library, I
wrote a new code to generate my target particle.
In summary, which one woulde be more efficient for large orientation
average?
Is an average over independent results valid?
Is there any spefic technique required for average (e.g., any
weighting.. )?
If you have any suggestion for my simulations, please let me know.
Thank you very much.
Sincerely,
Jun
baptiste
A convenient scheme for orientation averaging is to use quasi-Monte
Carlo (QMC) integration: you just need to add up the results without
worrying about weights; you can even refine a previous calculation by
adding more integration points taking into account the previous ones.
Maxim recently pointed out this article in a discussion,
A. Penttil and K. Lumme, Optimal cubature on the sphere and other
orientation averaging schemes, J. Quant. Spectrosc.
Radiat. Transfer 112, 1741-1746 (2011). doi: 10.1016/j.jqsrt.
2011.02.001
where the QMC scheme is described and its efficiency compared to other
quadratures.
Typically, you could do something like this,
int ll=0;
for(ll=0; ll<NQMC; ll++){ // loop over integration points
phi = QMC(ll, 1)*2*pi, psi = asin(2*QMC(ll, 0) - 1);
Qext += extinction(phi, psi);
}
where QMC is an array of quasi-random integration points, e.g. a
Halton sequence,
for 10 points:
0.5000 0.33333333
0.2500 0.66666667
0.7500 0.11111111
0.1250 0.44444444
0.6250 0.77777778
0.3750 0.22222222
0.8750 0.55555556
0.0625 0.88888889
0.5625 0.03703704
0.3125 0.37037037
for 12 points: all the above, plus
0.8125 0.70370370
0.1875 0.14814815
and so on.
In your case each iteration would be an independent calculation
specified with either -orient (particle) or -prop (incident beam).
HTH,
baptiste
Antti
alpha angle is fast, so average over that in one simulation, and
combine these simulations afterwards without any weights. You will
need to use own avg_params.dat -file for each simulation. Put e.g.
alpha:
min=0
max=360
Jmin=5
Jmax=5
eps=0
equiv=true
periodic=true
beta:
min=*beta*
max=*beta*
Jmin=0
Jmax=0
eps=1e-3
equiv=false
periodic=false
gamma:
min=*gamma*
max=*gamma
Jmin=0
Jmax=0
eps=1e-3
equiv=false
periodic=false
where *beta* and *gamma* are angles selected by the cubature (QMC, for
example). In this way you do not need to create several copies of the
particle shape in different orientations.
[Hope I remembered the correct use of Jmin=0 and Jmax=0, in any case,
the idea is to 'average' over only one value in beta and gamma]
Best,
Antti
Junum
Thank you very much, baptiste and Antti.
You guys save my life!
Antti, your paper is really useful.
baptiste, thank you for letting me that paper.
Sincerely,
Jun
Maxim Yurkin
The original question was answered by Baptiste and Antti, I only have a few comments for general information.
1) Concerning parallel efficiency of ADDA. Indeed, using number of CPU cores larger than n_z (144) will not help (See
section 5 "System Requirements" of the manual). However, total required resources (measured as, e.g., wall time times
number of cores) somewhat decrease with decreasing number of cores. In other words, the latter improves parallel
efficiency (see Section 11.3 "Parallel performance" of the manual). So if you can divide orientation average into many
parts, so that wall time of a single run is no more an issue, it is overall more efficiently to use as small number of
cores per single run as possible. The limit is usually determined by the required memory.
Another way to improve parallel efficiency is to generate your particle so, that its largest dimension is along z-axis.
Unfortunately, this is not possible for predefined shape, but is perfectly possible for shapes read from file. It is
just a matter of modifying your shape generator a little bit.
2) Concerning the averaging. General rule is that all intensity-based quantities should be averaged (with weights
specified by a particular cubature if needed). So this applies to the Mueller matrix and cross sections (Cext, etc.). If
the efficiencies (Qext, etc.) are defined by dividing by the cross section of the volume-equivalent sphere (as is done
in ADDA), then they can also be directly averaged. The alternative definition is to divide by the area of geometrical
shadow, which depends on orientation. Then one should first average Cext, etc. and divide it by average area of
geometrical shadow. Finally, the asymmetry parameter can not be averaged, but it should be computed from the averaged
Mueller matrix. Moreover, computing asymmetry parameter for each single orientation can be time consuming and, hence, is
not recommended.
When you need to average only cross section, it is more convenient to use the single orientation runs (with "-orient
...", as described by Baptiste). Then the mean of Cext (or Cabs) from files CrossSec-X and CrossSec-Y will be equal to
the result averaged over the angle alpha (see the method of Antti). Although the method of Antti can be also used
efficiently in this case (by adding "-scat_matr none", which will effectively cancel the averaging over alpha inside
ADDA), it is probably more difficult to automate.
If you need to average the Mueller matrix (in particular, to compute g) the method of Antti, which includes fast
averaging over alpha, should be more appropriate.
3) Concerning the orientation averaging scheme - Romberg integration - that is used in ADDA. It's main advantage is
adaptability (using internal error estimate) and potentially rapid convergence with increasing number of orientations
(when it is already large enough) - see Section 11.5 "Romberg integration" of the manual. The main drawback is lack of
flexibility, which comes partly from the high-order integration methods and partly from deep rooting of the averaging
inside the ADDA code. The latter may potentially allow additional optimization - see
http://code.google.com/p/a-dda/issues/detail?id=54 , but it is far from being implemented.
The structure of the file avg_params.dat is also not completely flexible. So combining results from two ADDA runs with
different sets of orientation angles is problematic. One can separate angular intervals (given by min=... and max=...)
between the runs and decrease Jmax correspondingly (by 1 for 2 runs, by 2 for 4 runs, etc.). However, these scheme will
lose all the benefits of Romberg integration, and will be similar to a simple averaging on a regular grid.
For now using an external averaging (e.g., by a script), as discussed by Baptiste and Antti, is definitely a good option
for many applications. Hopefully, some of this functionality will be implemented in ADDA as well, see
http://code.google.com/p/a-dda/issues/detail?id=40 .
Finally, the problem of limited wall time available on computer clusters can be potentially solved with the checkpoint
system, implemented in ADDA - see Section 11.4 "Checkpoints" of the manual. Unfortunately, this system currently works
only for single orientation and single incident polarization - see http://code.google.com/p/a-dda/issues/detail?id=9 .
The latter link also provides possible workarounds.
Maxim.
Junum
Thank you very much for your answer, Maxim.
I have one more question.
Now what I am trying to do is to compare performance of ADDA internal
orientation average and that of external orientation average using QMC
as suggested by Baptiste and Antti.
For 4352 orientation average, I modified avg_params.dat file as
alpha:
min=0
max=360
Jmin=2
eps=0
equiv=true
periodic=true
beta:
max=180
Jmin=2
Jmax=4
eps=1e-3
equiv=false
periodic=false
gamma:
min=0
max=360
Jmin=2
Jmax=3
eps=1e-3
equiv=true
periodic=true
Thus, I expect this setting will give (2^5)*(2^4 + 1)*(2^3) = 32*17*8
= 4352 orientations.
Result log file says
Particle orientation - averaged
alpha: from 0 to 360 in 32 steps
beta: from 0 to 180 in (up to) 17 steps (equally spaced in cosine
values)
gamma: from 0 to 360 in (up to) 8 steps
ORIENTATION STEP beta=180 gamma=0
ORIENTATION STEP beta=0 gamma=0
ORIENTATION STEP beta=90 gamma=0
ORIENTATION STEP beta=90 gamma=180
ORIENTATION STEP beta=90 gamma=90
ORIENTATION STEP beta=90 gamma=270
ORIENTATION STEP beta=90 gamma=45
ORIENTATION STEP beta=90 gamma=135
ORIENTATION STEP beta=90 gamma=225
ORIENTATION STEP beta=90 gamma=315
ORIENTATION STEP beta=120 gamma=0
ORIENTATION STEP beta=120 gamma=180
ORIENTATION STEP beta=120 gamma=90
ORIENTATION STEP beta=120 gamma=270
ORIENTATION STEP beta=120 gamma=45
ORIENTATION STEP beta=120 gamma=135
ORIENTATION STEP beta=120 gamma=225
ORIENTATION STEP beta=120 gamma=315
ORIENTATION STEP beta=60 gamma=0
ORIENTATION STEP beta=60 gamma=180
...
...
...
My question is why beta=180 and 0 have only one gamma angle of 0.
Other beta angles have 8 gamma angles of 0, 180, 90, 270, 45, 135,
225, and 315, respectively.
Is it related to Romberg integration?
Could you answer my question?
Thank you.
Sincerely,
Jun
\
Maxim Yurkin
whole alpha range is also performed). This is similar to angles theta and phi in spherical coordinate system - see also
pictures in Antti's paper.
Originally, ADDA used completely uniform grid, but then we removed considering redundant gamma values for polar values
of beta. This just saves some simulation time without any effect on the result.
Maxim.
Claudia M
Maxim Yurkin
Maxim.
--
You received this message because you are subscribed to the Google Groups "ADDA questions and answers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adda-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/adda-discuss/4c6d236a-c174-419b-aefa-d75f9f42300dn%40googlegroups.com.