Snapshots Module Notes

[EricS]

Hello all,

I am interested in learning how to calculate converged turn over frequencies for surface processes (adsorption, desorption, reaction).  It seems like the most sensible place to start is with the "MyFirstSnapshots" example.  Unfortunately, this example is a bit confusing.  How correct are my notes below?

The build file make sense and is very normal.  The only new feature is the process entry tof_count parameter of add_process.  Apparently this accepts a dictionary.  In this specific case, there is only one entry in this dictionary, the key is the same as the parent process name, and the dictionary value is 1.  Does the key need to the be same as the process name.  I assume the value is a weighting factor that is normally set to 1.  What is the practical value of using a dictionary here?

Moving into the local_smart directory I see 5 groups:

1.) a README

2.) a kmc_settings.py and corresponding pyc

3.) a pickle folder

4.) a group of files with a root "runfile"

5.) a group of files with a root "MyFirstSnapshots"

First, why do we have both 4 and 5?

Second, we can summarize the README:

Snapshots must import snapshots.py (has 4 important functions) and snapshots_globals.py (gives access but only modify "write_output," "parameters_of_interest," and "simulation_name").

The four files are create_headers(), do_snapshots_time(), do_snapshots(), and create_log()

Why is there a do_snapshots_time and a do_snapshots?  Practically speaking, when would you want to use one over the other?

create_log output can be disabled by the global write_output.  Why would you do this?

What is the practical value of these log files?  The runfile.py has a comment that one might want to run snapshots but turn off writing.  Why?  Is this to recording data that has been relaxed or propagated to the steady state?

From the runfile.py, it seems the most basic use of snapshots can be accomplished by setting the parameters_of_interest global, running create_headers, running do_shapshots, and running create_log().  The comments offer dump_config() between snapshots if desired: what does the dump config have that the snapshot doesn't?  The comments suggest that it may be useful to access the follow arrays/lists between snapshots:

sg.occ_header_array

sg.TOF_data_list

sg.occ_data_list

sg.last_snapshot_outputs

sg.snapshot_output_headers

isn't this data printed to the csv file?  Is the idea that you can monitor the progression of the snapshots to know when to terminate the loop?

An alternate runfile demonstrates that you can inspect specific lattice sites between shapshots and alter them with the put command.  It prints sg.model.lattice.get_species(<loc>) which returns an index (0 for the first species entry in the xml, 1 for the second etc.)  Using put you can mutate what is at this site.

Another alternate runfile demonstrate that you can export a global configuration as a meshgrid matrix and analyze it with a radius parameter and the function get_local_configurations but the example is broken and won't run.

Best,

Eric

[Aditya "Ashi" Savara]

Q0: A TOF key can be any string. One can consider it a "tag" and that tag can be applied to more than one process even. https://kmcos.readthedocs.io/en/latest/reference/index.html

it does not have to be related to the process name, but there is an acceleration feature (the throttling module, SQERTSS algorithm which you can look up) which -- if you later wanted to use -- probably requires that convention. You could look at the MyFirstThrottling example.Of course, this can be changed later since that would just be a matter of renaming before building.

Q1: why do we have both 4 and 5?   <-- runfiles are files that can be run, the other ones (if they are the files I think you are referring to) are outputs from a run.

Q2: do_snaphots does a certain number of steps between snapshots.  do_snapshots_time tries to achieve a certain amount of time (seconds of simulation time) per snapshot -- but one has to put an upper limit for the number of steps (like 1 million or 10 million or whatever) and it will stop if that limit is reached so that it does not run forever.  The readme should really be updated to reflect this.

Q3:  I don't think it only disables that particular output. But in general, if running millions of simulations, one may not want the standard output cluttering up the hard drive, and may only want some specific output.

Q4: dump config dumps (writes) the configuration to a file.  I don't think a snapshot normally does so.

Q5: " Is the idea [of checking some variables] that you can monitor the progression of the [simulation]..." Yes. Termination is not the only thing a person may want to do, but that is one of them.

Q6: Broken example: Hmm...I had been editing the meshgrid related functions in recent months. The feature is pretty new. I'll probably have to debug it next month.  Even if that example doesn't work, get_local_configurations should work. Info about it in the file here, since it is in active development: https://github.com/kmcos/kmcos/blob/master/kmcos/run/__init__.py

[Aditya Ashi Savara]

The snapshots module is like a wrapper around kmcos. I don't think anyone has tried to use it with the acf module (which is more like within kmcos). But it should work to use the snapshots module with the acf module, I think. If someone wanted to use both together I would just run the same number of steps within a snapshot versus normal steps and then check.

 

On Friday, February 17, 2023, Aditya "Ashi" Savara <adityasa...@u.northwestern.edu> wrote:

Q0: A TOF key can be any string. One can consider it a "tag" and that tag can be applied to more than one process even. https://kmcos.readthedocs.io/en/latest/reference/index.html

it does not have to be related to the process name, but there is an acceleration feature (the throttling module, SQERTSS algorithm which you can look up) which -- if you later wanted to use -- probably requires that convention. You could look at the MyFirstThrottling example.Of course, this can be changed later since that would just be a matter of renaming before building.

Q1: why do we have both 4 any 5? <-- runfiles are files that can be run, the other ones (if they are the files I think you are referring to) are outputs from a run.

--

You received this message because you are subscribed to a topic in the Google Groups "kmcos-users" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/kmcos-users/eneB_bkERZs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to kmcos-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kmcos-users/6afd022c-8f0d-4d92-947e-f297e06aaf2en%40googlegroups.com.

[Aditya Ashi Savara]

Also, the TOF feature is core kmcos and can be accessed without the snapshots module. The snapshots module just makes accessing it more convenient. 

[EricS]

Thank you for the rapid reply!

The occupations output by this snapshot machinery is interesting.  They report fractional population for each binding site in the unit cell.  

I have some immobile adsorbates in a system I am interested in.   I would love to know the nearest neighbor population separately from the population everywhere else.  Because they are few in number and immobile, I know the lattice coordinates I am interested (e.g., (1,1,2, <site1>)) in so I could extract the population at those sites as illustrated in your “snapshots/runfile_put” example.  Perhaps there is another way based on relative position from adsorbate location?

are the snapshot occupation numbers consistent with numbers obtained by direct analysis of the config extracted immediately after a snapshot? (e.g. if there are 1200 binding sites and the last snapshot yields a pop of 0.01 that is 12 adsorbates.  If i then calc the adsorbate population near my immobile sites using the post-snapshot config and determine the number is 3, can I conclude a total of 9 adsorbates everywhere else?) 

Any guidance/explanation in this direction is appreciated.

Best,

Eric 

[EricS]

I may be overestimating what data snapshots collect in the specified time/step interval.  It seems like they only report properties at the end of each interval.  Am I correct?

Best,

Eric

[kmcos-users]

1) There are two kinds of TOFs in kmcos: the "TOF data" is an average observed TOF over the snapshot. The "TOF int" is an analytical TOF based on that moment in time at the end of the snapshot. This was documented somewhere. I think in the original kmos paper.  It seems I need to add some documentation about this. I have added an issues card to add this into intro2kmcos.

2) The occupations (concentrations) per snapshot are at the end of the snapshot, not averages. Getting an average for each site across a simulation would slow down kmc too much. But if one makes their snapshots small enough for their purposes, one can average across the snapshots to get a correct average.

3) What you're describing wanting is, I think, the same as what is accomplished by the feature I added most recently. Try using the feature get_local_configurations here: https://github.com/kmcos/kmcos/blob/master/kmcos/run/__init__.py 

If you need to get that for each kmc step (like I do for the problem I am working on) the kmc simulation will become slwo because it means taking 1 step snapshots and running that command after each kmc_step, and also it will export large amounts of data. Or you could set the export to false and then remove the parts of the tile list you want, for your problem.  I believe this statement is correct: if there are 1200 binding sites and the last snapshot yields a pop of 0.01 that is 12 adsorbates.  If i then calc the adsorbate population near my immobile sites using the post-snapshot config and determine the number is 3,  ....[  I can conclude a total of 9 adsorbates everywhere else]

[Aditya "Ashi" Savara]

get_local_configurations is (currently) mainly intended to be used for looking at and/or parsing the outputs written to the files it makes.  My point is you should look at the files it makes. If I remember correctly there is a minor bug where the first image may not come out, but I think I fixed that. Anyway, just try calling it more than once , and if you see any bugs where the first image or text file doesn't come out or something like that, let me know.

I should warn you that right now this only works correctly (or rather, only works in an easy to interpret way) for simple cubic lattices.  Trying to extend it to other ones is what my next task was.  Maybe that is something you can help do. It just requries python programming and some thought. I would have to find my notes etc. which probably has to wait until I get back from India, which is a March 1st flight so let's say around March 4th is probalby the earliest.

On Friday, February 17, 2023 at 10:14:04 PM UTC-5 kmcos-users wrote:

1) There are two kinds of TOFs in kmcos: the "TOF data" is an average observed TOF over the snapshot. The "TOF int" is an analytical TOF based on that moment in time at the end of the snapshot. This was documented somewhere. I think in the original kmos paper.  It seems I need to add some documentation about this. I have added an issues card to add this into intro2kmcos.

2) The occupations (concentrations) per snapshot are at the end of the snapshot, not averages. Getting an average for each site across a simulation would slow down kmc too much. But if one makes their snapshots small enough for their purposes, one can average across the snapshots to get a correct average.

3) What you're describing wanting is, I think, the same as what is accomplished by the feature I added most recently. Try using the feature get_local_configurations here: https://github.com/kmcos/kmcos/blob/master/kmcos/run/__init__.py 

If you need to get that for each kmc step (like I do for the problem I am working on) the kmc simulation will become slwo because it means taking 1 step snapshots and running that command after each kmc_step, and also it will export large amounts of data. Or you could set the export to false and then remove the parts of the tile list you want, for your problem.  I believe this statement is correct: if there are 1200 binding sites and the last snapshot yields a pop of 0.01 that is 12 adsorbates.  If i then calc the adsorbate population near my immobile sites using the post-snapshot config and determine the number is 3,  ....[  I can conclude a total of 9 adsorbates everywhere else]

On Friday, February 17, 2023 at 8:03:03 PM UTC-5 EricS wrote:

I may be overestimating what data snapshots collect in the specified time/step interval.  It seems like they only report properties at the end of each interval.  Am I correct?

Best,

Eric

On Friday, February 17, 2023 at 2:18:54 PM UTC-8 EricS wrote:

Thank you for the rapid reply!

The occupations output by this snapshot machinery is interesting.  They report fractional population for each binding site in the unit cell.  

I have some immobile adsorbates in a system I am interested in.   I would love to know the nearest neighbor population separately from the population everywhere else.  Because they are few in number and immobile, I know the lattice coordinates I am interested (e.g., (1,1,2, <site1>)) in so I could extract the population at those sites as illustrated in your “snapshots/runfile_put” example.  Perhaps there is another way based on relative position from adsorbate location?

are the snapshot occupation numbers consistent with numbers obtained by direct analysis of the config extracted immediately after a snapshot? (e.g. if there are 1200 binding sites and the last snapshot yields a pop of 0.01 that is 12 adsorbates.  If i then calc the adsorbate population near my immobile sites using the post-snapshot config and determine the number is 3, can I conclude a total of 9 adsorbates everywhere else?) 

Any guidance/explanation in this direction is appreciated.

Best,

Eric 

On Friday, February 17, 2023 at 1:36:06 AM UTC-8 Aditya "Ashi" Savara wrote:

Also, the TOF feature is core kmcos and can be accessed without the snapshots module. The snapshots module just makes accessing it more convenient. 

On Friday, February 17, 2023, Aditya Ashi Savara <adityasa...@u.northwestern.edu> wrote:

The snapshots module is like a wrapper around kmcos. I don't think anyone has tried to use it with the acf module (which is more like within kmcos). But it should work to use the snapshots module with the acf module, I think. If someone wanted to use both together I would just run the same number of steps within a snapshot versus normal steps and then check.

 On Friday, February 17, 2023, Aditya "Ashi" Savara <adityasa...@u.northwestern.edu> wrote:Q0: A TOF key can be any string. One can consider it a "tag" and that tag can be applied to more than one process even. https://kmcos.readthedocs.io/en/latest/reference/index.htmlit does not have to be related to the process name, but there is an acceleration feature (the throttling module, SQERTSS algorithm which you can look up) which -- if you later wanted to use -- probably requires that convention. You could look at the MyFirstThrottling example.Of course, this can be changed later since that would just be a matter of renaming before building.Q1: why do we have both 4 any 5? <-- runfiles are files that can be run, the other ones (if they are the files I think you are referring to) are outputs from a run.Q2: do_snaphots does a certain number of steps between snapshots. do_snapshots_time tries to achieve a certain amount of time (seconds of simulation time) per snapshot -- but one has to put an upper limit for the number of steps (like 1 million or 10 million or whatever) and it will stop if that limit is reached so that it does not run forever. The readme should really be updated to reflect this.Q3: I don't think it only disables that particular output. But in general, if running millions of simulations, one may not want the standard output cluttering up the hard drive, and may only want some specific output.Q4: dump config dumps (writes) the configuration to a file. I don't think a snapshot normally does so.Q5: " Is the idea [of checking some variables] that you can monitor the progression of the [simulation]..." Yes. Termination is not the only thing a person may want to do, but that is one of them.Q6: Broken example: Hmm...I had been editing the meshgrid related functions in recent months. The feature is pretty new. I'll probably have to debug it next month. Even if that example doesn't work, get_local_configurations should work. Info about it in the file here, since it is in active development: https://github.com/kmcos/kmcos/blob/master/kmcos/run/__init__.py

On Friday, February 17, 2023 at 12:16:42 AM UTC-5 EricS wrote:Hello all,
I am interested in learning how to calculate converged turn over frequencies for surface processes (adsorption, desorption, reaction). It seems like the most sensible place to start is with the "MyFirstSnapshots" example. Unfortunately, this example is a bit confusing. How correct are my notes below?
The build file make sense and is very normal. The only new feature is the process entry tof_count parameter of add_process. Apparently this accepts a dictionary. In this specific case, there is only one entry in this dictionary, the key is the same as the parent process name, and the dictionary value is 1. Does the key need to the be same as the process name. I assume the value is a weighting factor that is normally set to 1. What is the practical value of using a dictionary here?

Moving into the local_smart directory I see 5 groups:1.) a README2.) a kmc_settings.py and corresponding pyc3.) a pickle folder4.) a group of files with a root "runfile"5.) a group of files with a root "MyFirstSnapshots"

First, why do we have both 4 and 5?

Second, we can summarize the README:Snapshots must import snapshots.py (has 4 important functions) and snapshots_globals.py (gives access but only modify "write_output," "parameters_of_interest," and "simulation_name").The four files are create_headers(), do_snapshots_time(), do_snapshots(), and create_log()

Why is there a do_snapshots_time and a do_snapshots? Practically speaking, when would you want to use one over the other?

create_log output can be disabled by the global write_output. Why would you do this?What is the practical value of these log files? The runfile.py has a comment that one might want to run snapshots but turn off writing. Why? Is this to recording data that has been relaxed or propagated to the steady state?
From the runfile.py, it seems the most basic use of snapshots can be accomplished by setting the parameters_of_interest global, running create_headers, running do_shapshots, and running create_log(). The comments offer dump_config() between snapshots if desired: what does the dump config have that the snapshot doesn't? The comments suggest that it may be useful to access the follow arrays/lists between snapshots:sg.occ_header_arraysg.TOF_data_listsg.occ_data_listsg.last_snapshot_outputssg.snapshot_output_headersisn't this data printed to the csv file? Is the idea that you can monitor the progression of the snapshots to know when to terminate the loop?

An alternate runfile demonstrates that you can inspect specific lattice sites between shapshots and alter them with the put command. It prints sg.model.lattice.get_species(<loc>) which returns an index (0 for the first species entry in the xml, 1 for the second etc.) Using put you can mutate what is at this site.
Another alternate runfile demonstrate that you can export a global configuration as a meshgrid matrix and analyze it with a radius parameter and the function get_local_configurations but the example is broken and won't run.

Best,Eric--You received this message because you are subscribed to a topic in the Google Groups "kmcos-users" group.To unsubscribe from this topic, visit https://groups.google.com/d/topic/kmcos-users/eneB_bkERZs/unsubscribe.To unsubscribe from this group and all its topics, send an email to kmcos-users...@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/kmcos-users/6afd022c-8f0d-4d92-947e-f297e06aaf2en%40googlegroups.com.

[Aditya Ashi Savara]

I have thought about it more, and getting an average across each site probably would not slow down the simulation too much, especially if we made a seperate do_steps command for that scenario.  It would not be a bad idea to add such a feature, but since a person can usually just use small enough snapshots (in the most extreme case, snapshots of 1 step), I suspect there will always be higher priority things.

To view this discussion on the web visit https://groups.google.com/d/msgid/kmcos-users/882a4164-3e6f-47cf-9c23-6073d40c0fddn%40googlegroups.com.

[EricS]

Interesting to note that it might be possible but i agree that it might not be the highest priority item.  Lots to say but I will start with a note that the examples promote the syntax do_snaphots(sps, n_snapshots) but the function in kmos/run/snapshots.py reverses these positional arguments.

-Eric

[EricS]

I have a strong understanding of snapshots now but I believe I am running into issues related to fast diffusion and slow reactions and want to try this throttling you mentioned in this thread.  As suggested, I renamed the tof counters to match the process name with the F#p0/R#p0 convention and have compiled the model.  The example runfile demonstrates how to load and save parameters for the throttling and snapshot machinery.  Do i need to manually prepare these parameter files the first time (23 for shapshots, 59 for throttle)?

[Aditya Savara]

I don't understand what your numbers of 23 and 59 refer to. The build file must have all kmc parameters. The throttling module parameters will then be set from the runfile.

---

From: kmcos...@googlegroups.com <kmcos...@googlegroups.com> on behalf of EricS <eric...@gmail.com>
Sent: Tuesday, February 28, 2023 5:08:20 PM
To: kmcos-users <kmcos...@googlegroups.com>
Subject: [kmcos-users] Re: Snapshots Module Notes

 

--
You received this message because you are subscribed to the Google Groups "kmcos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kmcos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kmcos-users/e2010894-ff71-40ff-9b26-bdd2baed2946n%40googlegroups.com.

[EricS]

There are 23 and 59 parameters that need to be set according to the txt files loaded in the example file.  Can you provide a commented version of these files explaining what each mean and how to best set them?

-Eric

[EricS]

I assume the recommended way to set these parameters is by editing the txt files as this is what is shown in the example runfile.  I will do my best to set as many as possible but it is a bit daunting.  Is there a smaller subset of parameters that are crucial for basic submission?  What is the syntax for setting these variables from the runfile?

[Aditya Ashi Savara]

I probably made the example that you are referring to, but I don't remember making an example that relied on txt files.

As far as I can tell,  there are no links or files attached to your email.  The email should have sufficient information for the question to be answered.

[EricS]

Excellent point.  I am discussing the throttling example.  Here are links to the relevant files.

https://github.com/kmcos/kmcos/blob/master/examples/MyFirstThrottling/MyFirstThrottling_local_smart/runfile.py

Lines 19-25 load parameters from the following files.

https://github.com/kmcos/kmcos/blob/master/examples/MyFirstThrottling/MyFirstThrottling_local_smart/test_reaction_snapshots_parameters.txt

https://github.com/kmcos/kmcos/blob/master/examples/MyFirstThrottling/MyFirstThrottling_local_smart/test_reaction_throttling_parameters.txt

What am I asking: This runfile doesn’t help a user unfamiliar with this feature prepare throttling for their system.  It is a bare-bones runfile that shows how to run throttling once it is already setup.  If there is minimal subset of parameters that must be set from the runfile can the syntax be demonstrated.  If it is safest to specify every parameter in the txt files, can there be a commented version to explain how to prepare each line?

Very excited to try this feature and grow my familiarity with kmcos.

Best,

Eric

On Tuesday, February 28, 2023 at 10:44:47 AM UTC-8 EricS wrote:

[Aditya Ashi Savara]

There is a probably simple to follow throttling module example with species A,B,C,D,E,F. 

My recollection is that it is described in the supporting information of the SQERTSS paper (Danielson, Sutton, Hin, Savara).

I believe it is in the throttling subdirectory of the unit tests directory. 

(I just boarded a flight so you may want to check at your convenience) 

[EricS]

Thanks for the guidance.  I will look into it.

[Aditya Ashi Savara]

I took a look at that throttling module example you linked.

 It seems I should make that example more clear

I believe you can delete all those eic eil load save lines and the example should still work. It does not look to me like the loading feature was even used in that example.

Those txt files are internal logs, not meant for a user to edit, but they are intended for users to look at if desired. A person could even make their extensions .py or .log, the extension is arbitrary.