Simulation 2023: Enabling massively parallel, ad-hoc exploration of the design space for simulation models.- Reviewer Jacob Barhak

[Jacob Barhak]

Title: Enabling massively parallel, ad-hoc exploration of the design space for simulation models.

Venue: Simulation: Transactions of the Society for Modeling and Simulation International

Authors:
Rossetti, Manuel; University of Arkansas Fayetteville, Industrial Engineering
Gibson, Andrew; University of Arkansas Fayetteville, Industrial Engineering


How would you describe the research quality of the paper?
 Intermediate

Is the English satisfactory?
 Yes

Do you feel that the paper contributes to the state-of-the-art of modeling and simulation? If so, please summarize your reasons in the box below (a 500-character minimum is suggested):

Yes, the authors are on to a good idea that will allow scientists to use the availability of computing power to improve their research and automate tasks that are many times considered too slow.
Unfortunately scientists in many fields are not yet capable of using this technologies even though lambda are about a decade old by now. So the authors are doing a good service. Hopefully in a decade papers like these will not be necessary, yet for now - the authors are doing a good service. They also explain why those services are needed - note that this reviewer holds patents that benefit from parallelization and therefore biased towards accepting this paper - the editor should note that conflict of interest, since the reviewer has a lot to gain from acceleration of parallelization of simulations while his patents are active....

However, this conflict does not take from the goo work the authors did - they actually educated me on some aspects serverless users need to take into account and provided  some simple examples to illustrate how to build such a serverless simulation.

They also provided some experimental results and even discussed and summarized limitations So the paper is good to go even as is, yet I was wondering about some things that can improve the paper if the authors would take the time for that.



Provide comments and suggestions you have for the author(s). A minimum of 1000 characters is suggested.

The editors allowed me to provide my comments to the paper in an un blind  fashion and make those public - My name is Jacob Barhak and I was using HPC for a long time for simulations.

A centralized controller that communicates with the functions and serves them their parameters needs to have several elements such as:
- ability to communicate with the the functions to feed them parameters which may create a bandwidth problem at some point - it would be nice if this is explored in more details since this may become a bottleneck - I know that communication becomes a bottleneck even on a small cluster - so serveries technology may be more vulnerable - if authors can improve discussion on this, it would make it a better paper.

- with many parallel computations it is not uncommon to have some functions fail - it happens occasionally - so if the authors discuss possible mechanisms to cope with this, it will improve the paper.

- some functions produce a lot of data, especially during long simulations - short responses are not common in many simulations and the authors may want to address how to cope with large amounts of data generated and what issues they create. This many even create communication issues and even cost increases. If the authors discuss this it would make a better paper.

Yet even without changes, the paper if good to go - more scientists should use those techniques and the authors are helping this happen - and to remind you I am biased. 

[Jacob Barhak]

A second review round results in the following:

Title: Enabling massively parallel, ad-hoc exploration of the design space for simulation models.

Venue: Simulation: Transactions of the Society for Modeling and Simulation International

Authors:
Rossetti, Manuel; University of Arkansas Fayetteville, Industrial Engineering
Gibson, Andrew; University of Arkansas Fayetteville, Industrial Engineering


How would you describe the research quality of the paper?
 Intermediate

Is the English satisfactory?
 Yes

Do you feel that the paper contributes to the state-of-the-art of modeling and simulation?

This paper could have been accepted last round - I urge the editor to accept it in its current form - other than proofs - I do not understand why so many changes were necessary -  the paper was good before for publication.

Look, AWS Lambda has been around for about a decade give or take, so whatever the authors do, will not be very novel. However, what they are doing is important since scientists still do not use such capabilities on a regular basis - here is where the paper importance is. One example is the other comments that the authors had to address showing that other scientists need to know more about this..

The sooner this paper gets accepted, the faster this knowledge gets propagated in the scientific community.

To be clear this is a strong accept.  

Recommendation

Accept With No Changes