FABM performance update
jo...@bolding-bruggeman.com
Hi all,
Over the past months we’ve worked to further optimize FABM for large domains, in particular for hydrodynamic models that use spatial masks (NEMO, MOM, FVCOM, HYCOM). The result of this effort (the “packtest” family of branches) has now been merged into master. If you are on any of the packtest branches, please move to master and pull. The changes are fully backward compatible (*) and have received extensive testing.
Two key changes were made: (1) for packing/unpacking data (=removal and insertion of masked values), we no longer use Fortran’s pack/unpack intrinsics, but custom logic that exploits the fact that the same operation is done on many variables. (2) the scratch space that is used to exchange information between host and biogeochemistry is now allocated once, globally, rather than again on every call (allocation is cheap, but by doing it once we improve caching).
FABM now also includes a “host emulator” that can be compiled stand-alone for the purpose of unit testing, profiling and performance testing. This tool (make target “test_host”) has been used to assess the impact of the recent changes on all hosts FABM supports out of the box:
Host Relative change in runtime (%)
0d -61 %
brom -9 %
fvcom -39 %
getm -44 %
glm -16 %
gotm -15 %
hycom -48 %
mom4p0d -54 %
mom4p1 -44 %
mom5 -40 %
nemo -41 %
python -61 %
schism -14 %
Before raising your expectations too much: the above gains will not be realized in real world applications! First because FABM’s contribution to the total runtime of the a 2D/3D simulation is typically minor, and second because the emulator is a small program that benefits more from CPU caching (change #2) than a large host. The numbers also depend on the size of domain and the BGC model (here: msi/ergom1). Thus, while I’m pretty sure the changes will never cause performance degradation, you’re mileage may vary, and I’m keen to hear what impact on runtime you experience.
Cheers,
Jorn
(*) The changes do not affect any of FABM’s APIs, but they do switch from per-call to global allocation of some data structures. That will break multithreaded (OpenMP) use of FABM, e.g., where an API is called in parallel from different threads that each operate on different parts of the spatial domain. This will be addressed in a later update (where the host can send the data structures along with each call). If you believe you are affected by this, let me know. I’m currently not aware of any host needing this.
Karsten Bolding
--
You received this message because you are subscribed to the Google Groups "FABM-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fabm-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.