How to port a reinforcement learning algorithm (maddpg) on top of gem5?

54 views
Skip to first unread message

gogineni kailashnath

unread,
Jul 23, 2021, 12:59:34 AM7/23/21
to gem5-...@gem5.org, gem5-g...@googlegroups.com
I'm trying to port maddpg on top of gem5
Here's my process:
First I've downloaded the maddpg from open ai:https://github.com/openai/maddpg (I've also installed all the dependencies for maddpg on the multi-particle environment)
In the next step, I used the following command to train the maddpg o top of gem5 for a simple policy.
Command: sudo ./build/X86/gem5.opt configs/example/se.py --cpu-type=DerivO3CPU --caches --mem-type=DDR4_2400_8x8 --mem-size=8GB --cmd=./train -options="--scenario simple"
When I tried executing this command, I encountered the following output:
sudo ./build/X86/gem5.opt configs/example/se.py --cpu-type=DerivO3CPU --caches --mem-type=DDR4_2400_8x8 --mem-size=8GB --cmd=./train -options="--scenario simple"
gem5 Simulator System.  http://gem5.org
gem5 is copyrighted software; use the --copyright option for details.
gem5 version 21.0.0.0
gem5 compiled Jun 15 2021 10:46:56
gem5 started Jul 23 2021 00:43:08
gem5 executing on osboxes, pid 7057
command line: ./build/X86/gem5.opt configs/example/se.py --cpu-type=DerivO3CPU --caches --mem-type=DDR4_2400_8x8 --mem-size=8GB --cmd=./train '-options=--scenario simple'
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
warn: membus.master is deprecated. `master` is now called `mem_side_ports`
warn: membus.master is deprecated. `master` is now called `mem_side_ports`
warn: membus.slave is deprecated. `slave` is now called `cpu_side_ports`
Global frequency set at 1000000000000 ticks per second
warn: DRAM device capacity (16384 Mbytes) does not match the address range assigned (8192 Mbytes)
0: system.remote_gdb: listening for remote gdb on port 7000
** REAL SIMULATION **
info: Entering event queue @ 0.  Starting simulation...
warn: ignoring syscall access(...)
warn: ignoring syscall access(...)
warn: ignoring syscall access(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall access(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall access(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall access(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall mprotect(...)
warn: ignoring syscall set_robust_list(...)
warn: ignoring syscall rt_sigaction(...)
      (further warnings will be suppressed)
warn: ignoring syscall rt_sigprocmask(...)
      (further warnings will be suppressed)
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
info: Increasing stack size by one page.
warn: MOVNTDQ: Ignoring non-temporal hint, modeling as cacheable!
warn: ignoring syscall prctl(...)
Exiting @ tick 346243000 because exiting with last active thread context
Simulated exit code not 0! Exit code is 1

I used the pyinstaller to convert the train.py file into a binary executable. Can someone explain to me the reason, why the training is not happening? or any suggestions to get the result?

References:

  1. gem5: https://www.gem5.org/documentation/general_docs/building
  2. maddpg algorithm: https://github.com/openai/maddpg

Reply all
Reply to author
Forward
0 new messages