SST 2.2.1 + gem5 for MPI program

[Ben Payne]

Hello,

I'm interested in running an MPI program. I don't care whether it's on a multi-core single node or multiple nodes with a router (which one is easier to run currently?).

Is there any documentation on how to set up either of the two above options?
I'd be running a very simple "hello world" type program as a statically compiled binary in system emulation mode.

I tried running a serial program on the supplied "dualCore-exampleM5.xml" but got the following error:

[bpayne@virtualcentos xml]$ pwd
/home/bpayne/sst/scratch/src/sst-simulator/sst/elements/M5/xml
[bpayne@virtualcentos xml]$ /home/bpayne/sst/local/sst-2.2.0/bin/sst.x --sdl-file=/home/bpayne/sst/scratch/sst-simulator-read-only/sst/elements/M5/xml/dualCore-exampleM5.xml --lib-path=/home/bpayne/sst/local/sst-2.2.0/lib/sst
Opening element library O3Cpu failed: /home/bpayne/sst/local/sst-2.2.0/lib/sst/libO3Cpu.so: cannot open shared object file: No such file or directory
0:Factory::CreateComponent():127:ABORT: can't find requested component O3Cpu.O3Cpu.


In a separate attempt at using MPI, I am able to get the supplied "pingpong_bench" to run on the host. However, I'm not clear on how to use sst.x with an MPI program.

[bpayne@virtualcentos bin]$ pwd
/home/bpayne/sst/local/sst-2.2.0/bin
[bpayne@virtualcentos bin]$ mpirun -np 2 pingpong_bench
# Ping-pong benchmark
# ------------------
# Command line "pingpong_bench"
#
# Running experiments of length 0 through 1310720 increasing increments
# Exchanging data between nodes 0 and 1 of, 2 nodes
# Requested precision is 1.000%
#
# Length                  Latency
# in bytes            in micro seconds
#            minimum         mean       median      maximum           sd       precision  trials
        0      0.125        0.551        0.493        4.524        0.268        0.010000    9108
        8      0.189        0.673        0.614        7.153        0.325        0.010000    8978
       16      0.203        0.670        0.618        6.544        0.284        0.010000    6928

[Ben Payne]

In gem5, the way I can boot a dual-core full system environment is

build/ARM/gem5.opt configs/example/fs.py -n 2

And for syscall emulation mode, I would run

build/ARM/gem5.opt configs/example/se.py -n 2 -c hello.lex

What is the equivalent configuration in SST?

Thank you,


Ben

[Simon Hammond]

SST does not let you run full system mode operating systems in the
current way we have our components implemented. Do you want to do full
system for a specific reasons? We can pretty much get threads working
in system emulation if thats of interest?

S.
--
Si Hammond

[Ben Payne]

Hello,

Which is easier to implement for you, one node per core and multiple nodes connected via a router, or one node with multiple cores?

Kindly,

Ben

PS: I'll be back on this project Wednesday

[Simon Hammond]

You can implement one node with multiple cores using threads today. Let me know if you're interested in this.

Multiple nodes is harder because you need a customised MPI installation.

S.

--
--
Si Hammond

[Ben Payne]

Yes, I'd like to be able to use MPI with multiple cores. 

[amina belhaj messaoud]

Le vendredi 21 décembre 2012 00:29:36 UTC+1, Si Hammond a écrit :
> SST does not let you run full system mode operating systems in the
>
> current way we have our components implemented. Do you want to do full
>
> system for a specific reasons? We can pretty much get threads working
>
> in system emulation if thats of interest?
>

how can we use threads with multiple core architecture ?
i tried the dualCore-exampleM5.xml as a configuration file inside the example.xml file , M5_EXE is : ~/scratch/sst-simulator-read-only/sst/elements/m5C/app/pthread_test/test , this is the binary file after the compilation
when i execute this : ~/scratch/sst-simulatorreadonly/sst/elements/m5C/app/pthread_test$ ./test 2
Ihave this :
numThreads 2
Creation Thread: 0
Creation Thread: 1
join Thread: 0
join Thread: 1
test done

my $ARG1 is 2

but I had that error :
/home/local/belhaj/local/bin/sst.x --sdl-file=example.xml --lib-path=/home/local/belhaj/local/lib/sst
WARNING: Building component "system" with no links assigned.
M5:configFile `dualCore-exampleM5.xml`
default system
default process
M5:registering exit `m5.nid0.core0.workload`
warn: cache block_size=64 hit_latency=1000
warn: cache block_size=64 hit_latency=1000
default system
default process
M5:registering exit `m5.nid0.core1.workload`
warn: cache block_size=64 hit_latency=1000
warn: cache block_size=64 hit_latency=1000
warn: bus width 64, clock 1000
warn: cache block_size=64 hit_latency=10000
warn: bus width 64, clock 1000
warn: cache block_size=64 hit_latency=10000
warn: bus width 64, clock 1000
M5:registering exit
0:Exit::refInc():61: component in construction increments exit multiple times.
warn: instruction 'fnstcw_Mw' unimplemented
warn: instruction 'fldcw_Mw' unimplemented
panic: Page-Fault: cpu=0, PC=0x406e12, Tried to read unmapped address 0.
@ cycle 10953000
[invoke:build/X86_SE/arch/x86/faults.cc, line 293]
Memory Usage: 4379944 KBytes
[cimepe39:13839] *** Process received signal ***
[cimepe39:13839] Signal: Aborted (6)
[cimepe39:13839] Signal code: (-6)
[cimepe39:13839] [ 0] /lib/libpthread.so.0(+0xeff0) [0x7f6ec188bff0]
[cimepe39:13839] [ 1] /lib/libc.so.6(gsignal+0x35) [0x7f6ec154d1b5]
[cimepe39:13839] [ 2] /lib/libc.so.6(abort+0x180) [0x7f6ec154ffc0]
[cimepe39:13839] [ 3] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_Z14__exit_messagePKciS0_S0_iS0_N7VarArgs8ArgumentIN2cp5PrintEEES5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_+0x8f5) [0x7f6ebc9c2675]
[cimepe39:13839] [ 4] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN6X86ISA9PageFault6invokeEP13ThreadContext14RefCountingPtrI10StaticInstE+0x32a) [0x7f6ebc25e7fa]
[cimepe39:13839] [ 5] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN9FullO3CPUI9O3CPUImplE4trapE14RefCountingPtrI9FaultBaseEsS2_I10StaticInstE+0x38) [0x7f6ebc0c7c18]
[cimepe39:13839] [ 6] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE10commitHeadER14RefCountingPtrI13BaseO3DynInstIS0_EEj+0x1141) [0x7f6ebc0b32e1]
[cimepe39:13839] [ 7] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE11commitInstsEv+0xe48) [0x7f6ebc0ba198]
[cimepe39:13839] [ 8] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE6commitEv+0x1f63) [0x7f6ebc0bea73]
[cimepe39:13839] [ 9] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE4tickEv+0x168) [0x7f6ebc0bf918]
[cimepe39:13839] [10] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN9FullO3CPUI9O3CPUImplE4tickEv+0x4af) [0x7f6ebc0ee37f]
[cimepe39:13839] [11] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN10EventQueue10serviceOneEv+0x34) [0x7f6ebca0b234]
[cimepe39:13839] [12] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_Z8simulatel+0x6bd) [0x7f6ebca573bd]
[cimepe39:13839] [13] /home/local/belhaj/local/lib/sst/libm5C.so(_ZN2M55clockEm+0xe7) [0x7f6ebd118607]
[cimepe39:13839] [14] /home/local/belhaj/local/bin/sst.x(_ZN3SST5Clock7executeEv+0x55) [0x784cd5]
[cimepe39:13839] [15] /home/local/belhaj/local/bin/sst.x(_ZN3SST10Simulation3RunEv+0x124) [0x83a474]
[cimepe39:13839] [16] /home/local/belhaj/local/bin/sst.x(main+0xb6b) [0x81a2eb]
[cimepe39:13839] [17] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f6ec1539c8d]
[cimepe39:13839] [18] /home/local/belhaj/local/bin/sst.x() [0x77acb9]
[cimepe39:13839] *** End of error message ***
Aborted


that would be kind of you if you help me with this

[Simon Hammond]

OK sorry for the late reply on this one, I've been on vacation enjoying spring in the USA :).

So the first question is how did you compile the pthread test? What pthread library did you use, SST cannot use the native pthread library directly since it requires signals to be supported for some operations. As SST-GEM5 does not have an operating system the use of signals isn't supported. With that in mind we have taken the M5-threads library from GEM5 and then modified it to work with SST, this overloads many pthread operations with things the simulator can understand.

Let me know on the above and we can talk you through getting threads to work if needed.


--
Simon Hammond
Scalable Computer Architectures
Sandia National Laboratories, NM, USA

> --
> You received this message because you are subscribed to the Google Groups "Structural Simulation Toolkit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to SST-simulato...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[amina belhaj messaoud]

in fact I made a make in the pthread_test directory so that the binary file test is created , and then i wanted to execute it on the architecture that is configured in the example .xml( exampleM5.xml) 

Unfortunately I have always this error : 

~/scratch/src/sst-2.3.0/sst/elements/M5/xml$ /home/local/belhaj/local/bin/sst.x --sdl-file=example.xml --lib-path=/home/local/belhaj/local/lib/sst

WARNING: Building component "system" with no links assigned.

M5:configFile `exampleM5.xml`

default system

default process

M5:registering exit `m5.nid0.cpu0.workload`

warn: cache block_size=64 hit_latency=1000

warn: cache block_size=64 hit_latency=1000

warn: cache block_size=64 hit_latency=10000

warn: bus width 64, clock 1000

warn: bus width 64, clock 1000

warn: cache block_size=64 hit_latency=10000

warn: bus width 64, clock 1000

M5:registering exit

0:Exit::refInc():61: component in construction increments exit multiple times.

warn: instruction 'fnstcw_Mw' unimplemented

warn: instruction 'fldcw_Mw' unimplemented

environ[0] RT_RANK=0

environ[1] 2

environ[2] (null)

panic: Page-Fault: cpu=0, PC=0x432564, Tried to read unmapped address 0x21.

 @ cycle 16678000

[invoke:build/X86_SE/arch/x86/faults.cc, line 293]

Memory Usage: 4866816 KBytes

[cimepe39:03641] *** Process received signal ***

[cimepe39:03641] Signal: Aborted (6)

[cimepe39:03641] Signal code:  (-6)

[cimepe39:03641] [ 0] /lib/libpthread.so.0(+0xeff0) [0x7f1cf40d9ff0]

[cimepe39:03641] [ 1] /lib/libc.so.6(gsignal+0x35) [0x7f1cf3d9b1b5]

[cimepe39:03641] [ 2] /lib/libc.so.6(abort+0x180) [0x7f1cf3d9dfc0]

[cimepe39:03641] [ 3] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_Z14__exit_messagePKciS0_S0_iS0_N7VarArgs8ArgumentIN2cp5PrintEEES5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_+0x8f5) [0x7f1cef210675]

[cimepe39:03641] [ 4] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN6X86ISA9PageFault6invokeEP13ThreadContext14RefCountingPtrI10StaticInstE+0x32a) [0x7f1ceeaac7fa]

[cimepe39:03641] [ 5] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN9FullO3CPUI9O3CPUImplE4trapE14RefCountingPtrI9FaultBaseEsS2_I10StaticInstE+0x38) [0x7f1cee915c18]

[cimepe39:03641] [ 6] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE10commitHeadER14RefCountingPtrI13BaseO3DynInstIS0_EEj+0x1141) [0x7f1cee9012e1]

[cimepe39:03641] [ 7] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE11commitInstsEv+0xe48) [0x7f1cee908198]

[cimepe39:03641] [ 8] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE6commitEv+0x1f63) [0x7f1cee90ca73]

[cimepe39:03641] [ 9] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN13DefaultCommitI9O3CPUImplE4tickEv+0x168) [0x7f1cee90d918]

[cimepe39:03641] [10] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN9FullO3CPUI9O3CPUImplE4tickEv+0x4af) [0x7f1cee93c37f]

[cimepe39:03641] [11] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_ZN10EventQueue10serviceOneEv+0x34) [0x7f1cef259234]

[cimepe39:03641] [12] /home/local/belhaj/scratch/src/sst-gem5-2.3.0/build/X86_SE/libgem5_opt.so(_Z8simulatel+0x6bd) [0x7f1cef2a53bd]

[cimepe39:03641] [13] /home/local/belhaj/local/lib/sst/libm5C.so(_ZN2M55clockEm+0xe7) [0x7f1cef966607]

[cimepe39:03641] [14] /home/local/belhaj/local/bin/sst.x(_ZN3SST5Clock7executeEv+0x55) [0x784cd5]

[cimepe39:03641] [15] /home/local/belhaj/local/bin/sst.x(_ZN3SST10Simulation3RunEv+0x124) [0x83a474]

[cimepe39:03641] [16] /home/local/belhaj/local/bin/sst.x(main+0xb6b) [0x81a2eb]

[cimepe39:03641] [17] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f1cf3d87c8d]

[cimepe39:03641] [18] /home/local/belhaj/local/bin/sst.x() [0x77acb9]

[cimepe39:03641] *** End of error message ***

Aborted

In fact I want to make it done on one cpu , and then my objective is to map each thread on one cpu for example in order to gain  parlellism 

thank you in advance 

You received this message because you are subscribed to a topic in the Google Groups "Structural Simulation Toolkit" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/SST-simulator/H49yh0WZm9s/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to SST-simulato...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Bel hadj Messaoud Amina
Engineering-Student at the National School for Computer Science (ENSI) - Tunisia

Microsoft Student Partner 2012 
Ex Vice_President at ENSI Junior Entreprise 2011-2012

tel: +216 24 363 189
       0033781372311

[amina belhaj messaoud]

hello Ben ,

it looks that you had the same objectif as me now . in fast i want to make build a 2D mesh architecture of Nocs and run an application on it .

Could you tell me , since you are more experimented than me ,shall I use ful system mode or system Emulation mode .

[Simon Hammond]

You cannot use full-system mode with SST and GEM5, at least it is not actively tested and supported.

We can run threaded codes on GEM5 without too many problems provided they are written using pthreads and do not require signals. MPI is something we are working on supporting (there are proof of concept methods to enable this but these are not supported as part of the SST release).

--
Simon Hammond
Scalable Computer Architectures
Sandia National Laboratories, NM, USA

[Weilong Cui]

Hi, I've been working with sst+gem5+dramsim2 recently and I have the exact same problem as you described with pthread_test/test.
Did you ever figure it out? Or is there someone who ca help?
Thanks a lot!

Weilong

[Hammond, Simon David (-EXP)]

Hi Weilong,

Can you tell me how you compiled the pthread_test code?

S.

--
Simon Hammond
Scalable Computer Architectures (CSRI/146, 01422)

[Weilong Cui]

Hi Si,

Thanks, I just use the makefile in /sst-3.0.0/sst/elements/m5C/pthread_test/ , type `make` in that dir and the executable test is generated.

And also, I tried to modified the LDFLAGS in the makefile to point to the pthread.o in the same dir when linking, but it still doesn`t work.

I run test with the given example.xml with nothing changed, the error message is the same:

panic: Page-Fault: cpu=0, PC=0x4074e5, Tried to read unmapped address 0.
@ cycle 10710000

It seems to be gem5 problem? Maybe something wrong with the configuration?

BTW, is there someplace a more detailed tutorial or manual that talks about the configuration xml, not the format which has been shown clearly in the wiki, but the exact meaning of the params.
For example, I used the genXml in the trunk to generate a xml with 2 cores for gem5, but they still only look for M5_EXE as the workloads by default, how can I tell the 2 cores to look for 2 different programs and execute them in parallel? No communication between the 2 processes, I just wanted to stress the memory systems in this way.

Weilong

[Hammond, Simon David (-EXP)]

So the pthreads work is a bit patchy right now, we will be sorting this
out ready for a future release. If you want to run threaded code on SST
with GEM5 then you should do the following steps:

(1) Download SST-GEM5 and SST from the SVN/Mercurial repositories, v3.0.0
may work but there have been some fixed to support some applications in
threaded mode.
(2) Download the SST group's recent commit to m5threads
(http://repo.gem5.org/m5threads/), in particular download the pthread.c,
compile this to a pthread.o
(3) Compile your application as normal but link to our pthread.o

We do not support multiple programs in a simulation as our focus is on
parallel (threaded/MPI) runs for our workloads. If this was of interest
you could raise an issue describing the scenario you would like to
simulate and we can see what support we can add in future releases.

Thanks!

[Weilong Cui]

Thanks! I`ll try it out later.