How the seed is managed in BatchRunner Replications

[Yilin Huang]

Hi all,

is there any information about how the seed is managed in BatchRunner with different replications? For example, if I want to run one scenario with say 30 replications. Just for the sake of reproducibility, I want to define my own seeds. How to do that?

I defined the scenario numbers and seeds as variable_parameters, but got rared results.

See the screenshot attached. The top is the output data frame I see in the debugger. Below is the print of the same data frame in the terminal. The header is flipped??? I am speechless.

[Yilin Huang]

To clarify, replications = iterations in batch run

[Majd Al-shihabi]

Hi Yilin - How did you pass the seed to the model?

You pass the seed as an initialization parameter to the model instance. For example, if we are using the MoneyModel in the introductory tutorial,

model = MoneyModel(50, 10, 10, seed=0)

So if you want to apply the same thing to the batch, you do something like this

fixed_params = {
    "width": 10,
    "height": 10,
    "seed": 0, # change this according to your needs
}

variable_params = {"N": range(10, 500, 10)}

# The variables parameters will be invoke along with the fixed parameters allowing for either or both to be honored.
batch_run = BatchRunner(
    MoneyModel,
    variable_params,
    fixed_params,
    iterations=5,
    max_steps=100,
    model_reporters={"Gini": compute_gini}
)

batch_run.run_all()

Does that make sense?

Majd

--
Project repos: github.com/projectmesa
---
You received this message because you are subscribed to the Google Groups "ProjectMesa" group.
To unsubscribe from this group and stop receiving emails from it, send an email to projectmesa...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectmesa/7b705c4b-e332-4fff-96ab-c7192f3614d4n%40googlegroups.com.

[Yilin Huang]

Hi Majd,

I did something like this: 

parameter_sweep = {'scenario': range(0, 9), 'seed': ['1234567', '123']}

num_iterations = 1

...

batch_run = BatchRunner(MyModel, variable_parameters=parameter_sweep,
                        display_progress=False, iterations=num_iterations, max_steps=num_steps,
                        agent_reporters=agent_reporter_dict)

With the above, the scenario/seed combinations (set as variable parameters) are actually configured as individual runs. That’s why num_iternations=1.

But I got strange results: 

You see when I print the seed/scenario settings, they are flipped. These were run with completely the same code. 

It is also possible that the variable_parameters shall be configured in another way. So actually I have two separate questions: 

1. Is it possible to configure variable_parameters with more than one parameter? How?

2. In batch run, is it possible to set/control the seeds of the iterations? How? 

Thank you very much in advance! 

Best,

Yilin

[Yilin Huang]

Hi Majd, hi all,

I think my first question is answered. There is a small bug in the batchrunner.py

def _prepare_report_table

where the index_cols is sometimes wrongly set. But the data content is correct, and the run settings were correct too, which were most important. 

Still, an answer to the second question will be very much appreciated! 

2. In batch run, is it possible to set/control the seeds of the iterations? How? 

Many thanks! 

Best,

Yilin

On 10 Mar 2021, at 11:10, Yilin Huang <huang....@gmail.com> wrote:

Hi Majd,

I did something like this: 

parameter_sweep = {'scenario': range(0, 9), 'seed': ['1234567', '123']}

num_iterations = 1

...

batch_run = BatchRunner(MyModel, variable_parameters=parameter_sweep,
                        display_progress=False, iterations=num_iterations, max_steps=num_steps,
                        agent_reporters=agent_reporter_dict)

With the above, the scenario/seed combinations (set as variable parameters) are actually configured as individual runs. That’s why num_iternations=1.

But I got strange results: 

<Screenshot 2021-03-10 at 10.48.05.png>

<Screenshot 2021-03-10 at 10.49.48.png>

[Majd Al-shihabi]

Hi Yilin - you might be running into the issue that is fixed in this pull request: https://github.com/projectmesa/mesa/pull/979

You can either edit the file directly in your code, or install mesa from that PR instead of the stable repository.

[Yilin Huang]

Thanks Majd. This is indeed is the issue mentioned :) 

Do you by chance also know, if I want to use the inbuilt iterations in the BatchRunner, is it possible to set the seeds of the iterations? 

 Many thanks. 

Best,

Yilin