****Averaging results over multiple runs with different seeds*****

[shahin]

Hi,

Le's say I have run my simulation 6 times with different seeds. Now I need to average over all of them. Is there a way in OMNeT to do that?

For examples I have a value called "end-to-end delay mean" which is generated for each simulation that I run, I need this end-to-end delay to be averaged over all my runs with different seeds. Of course I can use a calculator or input the data to an excel file for averaging but all these are time consuming when the number of runs increases. I need a fast solution. If anyone knows this please help.

Regards,

Shahin

[Rudolf Hornig]

OMNET does this automatically.

Just specify 

repeat=6

in your INI file and run it from the IDE specifying * as the run number (or use the opp_run_all script).

OMNeT++ will ensure that it runs six time. It WILL run each run with a DIFFERENT seed. All outputs from the different runs will be put into the result files. If you open the result files with the built in analytics tool, it will recognize that there were several repeats with the same attribute and it will average you results automatically.

Rudolf

--
Sent from the OMNeT++ mailing list. To configure your membership,
visit http://groups.google.com/group/omnetpp

[Shahin Shariat]

Dear Rudolf,

thanks for the reply to my problem

I have set the repeat parameter in my .ini file and run my simulation using the Cmdenv... and can see all the runs being done one by one...

The result folder contains the scalar and vector files for the number of runs i have set for repeat. When I click on them it creates a .anf file and opens the simulator analysis tool. For example I need the value of End-to-End delay and so go into Browse Data Tab and choose scalar from the top Tabs and from the third drop down menu choose the "endtoEndDelay:stats:mean" down there in the box I can see the end to end delay for all the number of runs that I had and they have different values which is not averaged. From this point onward, how can I see the average of all my runs?

regards,

shahin

[levent okcu]

Hi Shanin

you can apply many proceeses (avarage/divide/add etc.) over the
vectors on anf file but you can not do anything over scalars. You
should extract scalar data to csv or matlab file to process it. I
advice u to record ur scalar values in a vector to process on the anf
file using only omnetpp ide.

On 25 Mart, 16:31, Shahin Shariat <sh.shar...@gmail.com> wrote:
> Dear Rudolf,
>
> thanks for the reply to my problem
>
> I have set the repeat parameter in my .ini file and run my simulation using
> the Cmdenv... and can see all the runs being done one by one...
>
> The result folder contains the scalar and vector files for the number of
> runs i have set for repeat. When I click on them it creates a .anf file and
> opens the simulator analysis tool. For example I need the value of
> End-to-End delay and so go into Browse Data Tab and choose scalar from the
> top Tabs and from the third drop down menu choose the
> "endtoEndDelay:stats:mean" down there in the box I can see the end to end
> delay for all the number of runs that I had and they have different values
> which is not averaged. From this point onward, how can I see the average of
> all my runs?
>
> regards,
> shahin
>

> On Sun, Mar 25, 2012 at 10:39 AM, Rudolf Hornig <rudolf.hor...@gmail.com>wrote:
>
>
>
>
>
>
>
> > OMNET does this automatically.
>
> > Just specify
>
> > repeat=6
>
> > in your INI file and run it from the IDE specifying * as the run number
> > (or use the opp_run_all script).
>
> > OMNeT++ will ensure that it runs six time. It WILL run each run with a
> > DIFFERENT seed. All outputs from the different runs will be put into the
> > result files. If you open the result files with the built in analytics
> > tool, it will recognize that there were several repeats with the same
> > attribute and it will average you results automatically.
>
> > Rudolf
>

> > On Wed, Mar 21, 2012 at 2:57 PM, shahin <sh.shar...@gmail.com> wrote:
>
> >> Hi,
>
> >> Le's say I have run my simulation 6 times with different seeds. Now I
> >> need to average over all of them. Is there a way in OMNeT to do that?
> >> For examples I have a value called "end-to-end delay mean" which is
> >> generated for each simulation that I run, I need this end-to-end delay to
> >> be averaged over all my runs with different seeds. Of course I can use
> >> a calculator or input the data to an excel file for averaging but all these
> >> are time consuming when the number of runs increases. I need a fast
> >> solution. If anyone knows this please help.
>
> >> Regards,
> >> Shahin
>
> >> --
> >> Sent from the OMNeT++ mailing list. To configure your membership,

> >> visithttp://groups.google.com/group/omnetpp

[levent okcu]

Besides you should fix the bug about the omnetpp about data recording
http://dev.omnetpp.org/bugs/view_all_bug_page.php

On 25 Mart, 16:31, Shahin Shariat <sh.shar...@gmail.com> wrote:

> Dear Rudolf,
>
> thanks for the reply to my problem
>
> I have set the repeat parameter in my .ini file and run my simulation using
> the Cmdenv... and can see all the runs being done one by one...
>
> The result folder contains the scalar and vector files for the number of
> runs i have set for repeat. When I click on them it creates a .anf file and
> opens the simulator analysis tool. For example I need the value of
> End-to-End delay and so go into Browse Data Tab and choose scalar from the
> top Tabs and from the third drop down menu choose the
> "endtoEndDelay:stats:mean" down there in the box I can see the end to end
> delay for all the number of runs that I had and they have different values
> which is not averaged. From this point onward, how can I see the average of
> all my runs?
>
> regards,
> shahin
>

> On Sun, Mar 25, 2012 at 10:39 AM, Rudolf Hornig <rudolf.hor...@gmail.com>wrote:
>
>
>
>
>
>
>
> > OMNET does this automatically.
>
> > Just specify
>
> > repeat=6
>
> > in your INI file and run it from the IDE specifying * as the run number
> > (or use the opp_run_all script).
>
> > OMNeT++ will ensure that it runs six time. It WILL run each run with a
> > DIFFERENT seed. All outputs from the different runs will be put into the
> > result files. If you open the result files with the built in analytics
> > tool, it will recognize that there were several repeats with the same
> > attribute and it will average you results automatically.
>
> > Rudolf
>

> > On Wed, Mar 21, 2012 at 2:57 PM, shahin <sh.shar...@gmail.com> wrote:
>
> >> Hi,
>
> >> Le's say I have run my simulation 6 times with different seeds. Now I
> >> need to average over all of them. Is there a way in OMNeT to do that?
> >> For examples I have a value called "end-to-end delay mean" which is
> >> generated for each simulation that I run, I need this end-to-end delay to
> >> be averaged over all my runs with different seeds. Of course I can use
> >> a calculator or input the data to an excel file for averaging but all these
> >> are time consuming when the number of runs increases. I need a fast
> >> solution. If anyone knows this please help.
>
> >> Regards,
> >> Shahin
>
> >> --
> >> Sent from the OMNeT++ mailing list. To configure your membership,

> >> visithttp://groups.google.com/group/omnetpp

[N.omar]

This is exactly what i search for it !! 

I have 30 different seed and 4 different speed

It is too consuming time when i compute the avearage of throughput of each speed with 30 seed !!! 

I need to do this by Omnet , and I have tried the following 

repeat = 5

seed-set =  ${repetition}

but it needs to choose each time manually the run number 

[tanktoo]

I think I said it at another place. If you run from console you need the "-r" option with the runnumbers!

For example: "-r 0..19" will run 20 simulations.
In Eclipse you could set this in the run configuration.