Convergence problem for High Index Contrast system

[Liberman, Vladimir - 1006 - MITLL]

I am trying to do a granule calculation with the following command

adda -size 1 -dpl 2000 -lambda 4 -granul 0.5 0.005 -m 1.4627 6.54E-04 4.2 0

 

The problem  I am having is that the convergence is extremely slow and the reported convergence parameter seem to asymptote at ~1e-3 even after >200 iterations.

 

Vlad Liberman

 

Staff  Member, Active Optical Systems Group
MIT Lincoln Laboratory               E-mail: vl...@ll.mit.edu
Room S3-357                             Phone: 781 981 3706

                                                Cell: 339 368 9811
244 Wood St.                           
Lexington, MA 02421-6426

 

[Maxim Yurkin]

Dear Vlad,

I suggest to try the advanced DDA formulations, like FCD or IGT. Read Sections 10.1 and 10.2 of the manual. Specific additions to the command lines are:

-pol fcd -int fcd

or

-pol igt_so -int igt 3

These options are known to make wonders for small high-contrast very-weakly-absorbing homogeneous particles. See

Yurkin M.A. Performance of iterative solvers in the discrete dipole approximation, URSI Commission B International Symposium on Electromagnetic Theory (EMTS 2016), 14–18 August 2016, Espoo, Finland, pp. 488–491. (paper, slides)

Hopefully, they will help for granulated particles as well.

Maxim.

P.S. This answer has been forwarded to your e-mail address for your convenience. However, if you want to continue the discussion please reply to the group's e-mail. You are also advised to check the corresponding discussion thread at http://groups.google.com/group/adda-discuss and/or subscribe to it to receive automatic notifications, since other answers/comments will not be necessarily forwarded to your e-mail.

18.10.2021 20:30, Liberman, Vladimir - 1006 - MITLL пишет:

--
You received this message because you are subscribed to the Google Groups "ADDA questions and answers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to adda-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/adda-discuss/PH2P110MB08905392955E12D135B86194E1BC9%40PH2P110MB0890.NAMP110.PROD.OUTLOOK.COM.

[Liberman, Vladimir - 1006 - MITLL]

Partially good news. Both formulations converge much faster than the default method. However, the give very different SCATTERING cross section results for otherwise the same parameters.

 

IGT formulation reports

Cext       = 0.4089851739

Cabs      = 0.0007480331024

 

While FCD formulation reports

Cext       = 0.480852603

Cabs      = 0.0007178603914

 

Vlad Liberman

 

Staff  Member, Active Optical Systems Group
MIT Lincoln Laboratory               E-mail: vl...@ll.mit.edu
Room S3-357                             Phone: 781 981 3706

                                                Cell: 339 368 9811
244 Wood St.                           
Lexington, MA 02421-6426

 

[Maxim Yurkin]

Here are a few ideas:

- since you are using randomly placed granules, the result is not deterministic. So if you run ADDA several times with the same command line you will get some distribution of results. Naturally, we expect the corresponding standard deviation to be not large, but it strongly depends on the specific parameters of particle and granules. So a more fair comparison is to run both FCD and IGT 10 times each and compare the distributions.

- in principle, comparing two formulations for a fixed discretization is hard, since it is not a priori known, which one is more accurate (and can be used as a reference). So a more rigorous way is to study the convergence of both formulations with refining discretization. As I understand, your problem already uses a lot of dipoles, so you may try to come up with some simpler test problem first. The general idea is to achieve good convergence (result changes very little with further refining discretization) with both formulations. At least the difference between them (in terms of distribution of values) should be much smaller than now. Then both of them can be used as a reference with estimated uncertainty equal to this difference. Then one can ask a question, which of two formulations has better accuracy for fixed coarser discretization (by comparing the results to this reference).

- more detailed and computationally efficient accuracy tests can be performed if the granule placement is fixed. Then repetition of simulations is not needed, while the values can be compared directly (and their convergence compared) instead of distributions. For that you can use -store_grans in a specific DDA run to produce the reference shape. However, unfortunately, ADDA can not directly generate shape from this file. But that should be easy using one of the options described in https://github.com/adda-team/adda/wiki/FAQ#how-to-simulate-light-scattering-by-a-particle-which-shape-can-not-be-described-by-any-of-the-adda-predefined-shapes . You can write a dedicated script in your favorite language, use Hyperfun toolchain, or even use calltarget routine from DDSCAT (since it have a collection of spheres built-in option). Additional complication is that you need to repeat the shape generation for different discretization levels to study the convergence.

- overall, the FCD is designed for homogeneous particles, or at least when large homogeneous chunks (domains) are involved. That is exactly the opposite to your case of a lot of small granules. So I would expect (as a wild speculation) the IGT to be more accurate. However, your case is additionally complicated by the fact that granule diameter is only 2.5 larger than the dipoles, so one cannot hope to well reproduce each granule. Rather one can hope for averaging to reduce the total error. Thus, the use of FCD vs. IGT may also effectively correspond to different flavors of this averaging (FCD would involve some type of spectral filtering) - not clear which one will be more accurate at the end.

- to conclude, I suggest to simplify the problem a little bit, so you can play with all the above options using reasonable computational time. For instance, try to use '-dpl 200 -granul 0.5 0.05' and then increase dpl keeping the granule size fixed.

Maxim.

19.10.2021 18:37, Liberman, Vladimir - 1006 - MITLL пишет:

[Liberman, Vladimir - 1006 - MITLL]

Thanks for your detailed reply. I studied it carefully and I have a few follow on questions.

1.       You say : “ your case is additionally complicated by the fact that granule diameter is only 2.5 larger than the dipoles, so one cannot hope to well reproduce each granule”  

  I was under the impression that DDA could do some local interpolation since it is a semi-analytical method.   Is that  what the “-jagged” option is supposed to do? If so, I am not understanding the workflow (workaround?) for using the “jagged” option with the granule generator.

2.       For a deterministic shape construction, rather than using auxiliary packages, couldn’t I just use a Matlab script and generate a lattice set and write it to file? And then, couldn’t I just use ‘-read’ option on that file?

3.       For debugging the shape construction, what is the recommended current way of visualizing the constructed geometry “just as ADDA would see it”. There was a package LiteBil mentioned on the Wiki, but that seems old. Is that still the best/only way?

Vlad Liberman

 

Staff  Member, Active Optical Systems Group
MIT Lincoln Laboratory               E-mail: vl...@ll.mit.edu
Room S3-357                             Phone: 781 981 3706

                                                Cell: 339 368 9811
244 Wood St.                           
Lexington, MA 02421-6426

 

To view this discussion on the web visit https://groups.google.com/d/msgid/adda-discuss/fbf3b1bd-5a0a-d3a9-2647-5488d5595ec1%40gmail.com.

[Maxim Yurkin]

Thanks for your detailed reply. I studied it carefully and I have a few follow on questions.

1.       You say : “ your case is additionally complicated by the fact that granule diameter is only 2.5 larger than the dipoles, so one cannot hope to well reproduce each granule”  

  I was under the impression that DDA could do some local interpolation since it is a semi-analytical method.   Is that  what the “-jagged” option is supposed to do? If so, I am not understanding the workflow (workaround?) for using the “jagged” option with the granule generator.

That is a very deep question. By itself, the DDA is the stupidest (lowest order) method with maximum staircasing effect. The dipole (cube) is either fully in or fully out of the particle, i.e. no smoothing or interpolation is employed (although see https://github.com/adda-team/adda/issues/12 related to the weighted discretization). Thus, the DDA is best suitable (or at least best understood, see below) for particles with relatively large domains (exactly as I wrote before about the FCD formulation). More specifically, only in this case the DDA is known to be a numerically exact method. If we fix the scatterer (even the complicated one with a lot of small domains) and then refine discretization, the DDA will converge to the exact solution. And in doing so it will definitely reach the regime of "many dipoles per each domain", i.e. when staircasing at the boundaries is small. And this argument holds even if the refractive index varies smoothly inside each domain, i.e. the discontinuities of the refractive index are fixed, while the values at specific dipoles may slightly vary with refining discretization. Moreover, the convergence may be significantly different for various DDA formulations, but the limiting value should be the same.

That is exactly the regime, where you can rigorously investigate the DDA accuracy and refining discretization (generating multiple shape files) is the straightforward way for that. -jagged option is not recommended. While it is more convenient, as you use a single shape file and refine it automatically by ADDA itself, it reduces only the discretization effect, but not the shape (staircasing) error. I.e. it also does not include any interpolation. Even if you decide to try it, you still need a shape files to avoid randomness of granule placement at each ADDA run (as I explained previously).

However, there is another possible mode of the DDA, that can be called "effective-medium" one. Suppose, each dipole in the DDA has a random polarizability chosen from some distribution (or chosen deterministically based on some rule) such that most of the neighboring dipoles have largely different polarizability. In this way, one effectively samples the random distribution of refractive index (discontinuous everywhere), which invalidates the above convergence results for the DDA. Still, one expects that a medium consisting of a lot of (random) dipoles should have some properties of that of the homogeneous medium. Here one may expect some effective interpolation arising from the large number of dipoles. But this correspondence is the most direct when Clausius-Mossotti (CM) polarizability is used, corresponding to point dipoles. By contrast, modern formulations of the DDA (FCD and IGT) are designed specifically for finite-size dipoles, i.e. they try to model a tightly packed collection of cubes, which is very different from the picture of effective medium. Thus, I foresee that each DDA formulation may lead to different limiting values (of effective refractive index and other optical properties) when the number of dipoles is increased. In other words, here the choice of the DDA formulations becomes critical for interpretation of the results (in addition to numerical performance).

And the intermediate case (of a few dipoles per granule diameter) is probably the most complicated in terms of interpretation of the results, i.e. how to choose a proper reference result. Still, you can steer it towards one of the two limiting cases described above depending on your goals.

All this discussion becomes even more complicated with regard to your specific problem. First, you seem to be testing some effective-medium approximation (based on the scatterer). Then you probably want to use the DDA as a reference, but then it should not be too similar to the effective-medium approach itself. Second, you're probably not that interested in the comparison for a single realization of your random scatterer, but rather for a large set of such scatterers. Then the detailed error control for each scatterer may be an overkill - still, it seems a viable strategy to tune the DDA parameters.

2.       For a deterministic shape construction, rather than using auxiliary packages, couldn’t I just use a Matlab script and generate a lattice set and write it to file? And then, couldn’t I just use ‘-read’ option on that file?

Yes, that is a perfectly viable option. And that is what I meant by "write a dedicated script in your favorite language".

3.       For debugging the shape construction, what is the recommended current way of visualizing the constructed geometry “just as ADDA would see it”. There was a package LiteBil mentioned on the Wiki, but that seems old. Is that still the best/only way?

Indeed, LiteBil is old (and looks outdated), but other researchers and I use it almost every day and I haven't heard of any significant limitations. And it is very efficient, which becomes important with a huge number of dipoles, compared to, e.g., some simple visualizations you can make in Matlab, Mathematica, etc.

However, you should get yourself familiar with LiteBil by pressing all possible buttons inside, which affect the final visualization (like change the box to be visualized and show caps).

Maxim.

To view this discussion on the web visit https://groups.google.com/d/msgid/adda-discuss/PH2P110MB089087E9233D90EECB92B3F8E1BE9%40PH2P110MB0890.NAMP110.PROD.OUTLOOK.COM.

[Liberman, Vladimir - 1006 - MITLL]

A couple more simpler questions

1.       Cross section x and Cross section Y yield very similar results, at least for larger fill fractions that I have tested. Is it possible to just calculate one of the above quantities and not the other to save on computation time, short of exiting the program prematurely?

2.       Sometimes, ADDA reports that it will use the algorithm “For small granules.” However, sometimes, after a small parameter tweak, it reports that it is using an algorithm for “large granules.” What does that mean?

To view this discussion on the web visit https://groups.google.com/d/msgid/adda-discuss/d8ee1d22-060d-c741-9fa1-f78765b0d10c%40gmail.com.

[Maxim Yurkin]

Indeed, those are simpler questions.

A couple more simpler questions

1.       Cross section x and Cross section Y yield very similar results, at least for larger fill fractions that I have tested. Is it possible to just calculate one of the above quantities and not the other to save on computation time, short of exiting the program prematurely?

Try '-sym enf', which tells ADDA to treat the particle as symmetric with respect to 90-degree rotation around the propagation direction.

2.       Sometimes, ADDA reports that it will use the algorithm “For small granules.” However, sometimes, after a small parameter tweak, it reports that it is using an algorithm for “large granules.” What does that mean?

That's the internals of the granule placement. Brief comments are given here - https://github.com/adda-team/adda/blob/6c1931a76e78b7fae9c4555a7a6ba54893f6b8e6/src/make_particle.c#L514-L524 . You can then look through this function for definition of sm_gr, which is used to distinguish the two cases. Unfortunately, I do not remember exactly, how the threshold is defined.

Maxim.