How to run DQN Program?

[孙林杰]

When I run Google DQN program, the following error:

 ../torch/bin/luajit: ./initenv.lua:58: module 'cutorch' not found:

no field package.preload['cutorch']

no file './cutorch.lua'

no file '/home/slj/Documents/DeepMind-Atari-Deep-Q-Learner-master/torch/share/luajit-2.0.3/cutorch.lua'

no file '/usr/local/share/lua/5.1/cutorch.lua'

no file '/usr/local/share/lua/5.1/cutorch/init.lua'

no file '/home/slj/Documents/DeepMind-Atari-Deep-Q-Learner-master/torch/share/lua/5.1/cutorch.lua'

no file '/home/slj/Documents/DeepMind-Atari-Deep-Q-Learner-master/torch/share/lua/5.1/cutorch/init.lua'

no file './cutorch.so'

no file '/usr/local/lib/lua/5.1/cutorch.so'

no file '/home/slj/Documents/DeepMind-Atari-Deep-Q-Learner-master/torch/lib/lua/5.1/cutorch.so'

no file '/usr/local/lib/lua/5.1/loadall.so'

stack traceback:

[C]: in function 'require'

./initenv.lua:58: in function 'torchSetup'

./initenv.lua:112: in function 'setup'

train_agent.lua:52: in main chunk

[C]: at 0x00406260

Seeking help, thank you!

[Tambet Matiisen]

You are missing the cutorch package. Just follow the instructions in install_dependencies.sh. The easiest path is to do it on Ubuntu.

  Tambet

--
You received this message because you are subscribed to the Google Groups "Deep Q-Learning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deep-q-learni...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/758f620a-2444-4a8f-b8b3-6ce61b761c62%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[孙林杰]

Dear sir:

      There are still some mistakes with the program after installing the cutorch package. I wonder that if you have the DQN program which can be run successfully? Would you mind to send me the program if you have one. 

 

      Million thanks for your help.

     Best Regards.

     Sun Linjie

在 2015年4月28日星期二 UTC+8下午2:55:05，Tambet Matiisen写道：

[Tambet Matiisen]

If you have missing packages, then me sending you the code is not going to help. What errors are you getting?

Regards,
Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/f703bf45-6a24-438e-98a7-04cf385c5844%40googlegroups.com.

[haitao wang]

fig 1

Actually, I can not install  cutorch. When I use this command  

ht@ht:~/deepmind/torch/bin$ luarocks install cutorch

The terminal  returns such errors in fig1.  It means that cutorch can't be installed automatically and I am finding methods to install it manually. If you have methods to solve this problem. Please give me some advice.  Thanks very much!

Haitao

[haitao wang]

I have fixed that problem. It is about the permission in linux and 'sudo' should be used.

On Thursday, April 30, 2015 at 3:47:37 PM UTC+8, Tambet Matiisen wrote:

[Tambet Matiisen]

You should run install_dependencies.sh included with the Deepmind code. It installs latest versions of Torch and required packages locally, so you don't need to worry about sudo.

Getting Deepmind code working with default Ubuntu Lua packages is possible, but probably too much trouble if you just want to try it out. This repo might be helpful, if you insist system-wide install: https://github.com/torch/ezinstall

  Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/f481c62e-7384-4c8a-81c6-7ff180f21428%40googlegroups.com.

[haitao wang]

fig2

Unlukily, although I run install_dependencies.sh one problem still come out like in fig 2. I know it may require qt.so. But I am not very sure how to fix it. Maybe you can give some advice. Thanks so much!

Haitao

[Tambet Matiisen]

I suspect you are using DeepMind code from kuz's GitHub repo https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner. This is modified to use qlua for game screen visualization. For that you need to additionally install:

sudo apt-get install libqt4-dev
$PREFIX/bin/luarocks install qttorch
$PREFIX/bin/luarocks install qtlua

$PREFIX is the place where install_dependencies.sh installed your lua/torch copy.

  Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/fb81feb0-fd18-4a65-9b38-92f4bbf6b67c%40googlegroups.com.

[haitao wang]

Yes, you are right. And thanks for your advice. I have solve this problem. Now, after running the code I get a file named "DQN3_0_1_breakout_FULL_Y.t7". Do you know how I can deal with this file or what it can present to us?

regards,

Haitao

[孙林杰]

can you give me your DQN program which can be run successfully ?Thank you very much.
在 2015年5月5日星期二 UTC+8下午2:00:49，haitao wang写道：

[haitao wang]

I download from here https://github.com/rahular/deepmind-dqn

[孙林杰]

Excuse me，When I run Google DQN program, the following error：

    

 ../torch/bin/luajit: ./initenv.lua:65: attempt to index global 'cutorch' (a nil value)

stack traceback:

./initenv.lua:65: in function 'torchSetup'

./initenv.lua:112: in function 'setup'

train_agent.lua:52: in main chunk

[C]: at 0x00406260

can you give me some advice to solve this problem?Thank you very much!

在 2015年5月6日星期三 UTC+8上午8:29:04，haitao wang写道：

[haitao wang]

There exists answer in front of your question. You should install cutorch and cunn again because the independcies.sh file from deep mind does not do that for you.

Haitao

[Tambet Matiisen]

I added some scripts to my copy of DeepMind code:
 - plot_results <game> - plot history recorded in model file,
 - test_gpu <game> (or test_cpu <game>) - play one session in test mode and record screens in animated GIF. It also records actions, Q-values and rewards in CSV file.

The aim of the last script is to reproduce Extended Data Figure 2 (http://www.nature.com/nature/journal/v518/n7540/fig_tab/nature14236_SF2.html) from the article.

NB! If you trained your model with run_gpu, you must use test_gpu to test it, otherwise deserialization of the model fails.

NB2! Writing animated GIF needs Lua-GD package, install it with:
$PREFIX/bin/luarocks install luagd

You can find those scripts here:
https://github.com/tambetm/DeepMind-Atari-Deep-Q-Learner/

  Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/7f9c7324-7a0c-451a-88b0-d9240e3cce19%40googlegroups.com.

[haitao wang]

Thanks, so sorry to reply to you so late. I am just busy with some project. I just try to install the module 'luagd', but it did not work as you say.

 When I used that command you give ./luarocks install luagd. it returns errors as follow.

I also download the code in your github, and run the    install_dependencies.sh file. But it still didn't install luagd as you write in line 90 of that file.  Also, i try to compile the source code of lugd. but it is too annoying and I just failed. Maybe you have run into similar problems. 

Thanks for your answers!

[Tambet Matiisen]

Sorry, my bad. I had forgotten how I actually installed Lua-GD. First take the latest version from GitHub:

git clone https://github.com/ittner/lua-gd.git
cd lua-gd

Then use your favorite editor to change Lua binary location on line 32, for example
LUABIN=/home/yourname/DeepMind-Atari-Deep-Q-Learner/torch/bin/luajit

Then make and install using luarocks (from your torch installation):
/home/yourname/DeepMind-Atari-Deep-Q-Learner/torch/bin/luarocks make

I'll include this in install_dependencies.sh ASAP.

  Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/97b9f7a7-afe5-4938-a1ac-39bda9d07e3b%40googlegroups.com.

[Liu Yang]

I also come up with the same question while attempting to run Tambet's code originally at https://github.com/tambetm/DeepMind-Atari-Deep-Q-Learner/. I try 

./run_gpu breakout

and get the following problem just as you do. Fortunally, I am able to run the 'cpu' version like this in a Ubuntu server

./run_cpu breakout

but it doesnot seem to be better after 8 hours running of 340,000 steps. Could you please  tell me how you solve this question and how long it will possibly take to train the model.

Thank you so much!

Best regards,

Yang

在 2015年4月27日星期一 UTC+8上午9:30:29，孙林杰写道：

[Tambet Matiisen]

It seems you need nvcc in PATH, otherwise install_dependencies.sh won't install cutorch and cunn.

Make sure you included CUDA bin folder in PATH before running install_dependencies.sh!
http://www.r-tutor.com/gpu-computing/cuda-installation/cuda7.0-ubuntu

  Tambet

--

You received this message because you are subscribed to the Google Groups "Deep Q-Learning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deep-q-learni...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/12c77379-e5dc-4e7b-a7b7-f98494f0a85c%40googlegroups.com.

[4905...@qq.com]

Hello, I have encountered the same problem. I failed to install the module 'luagd', the same errors as yours. Have you fixed the problem ?

在 2015年6月3日星期三 UTC+8上午10:25:50，haitao wang写道：

[Tambet Matiisen]

Sorry for misinformation in my previous e-mail. Lua-gd must be installed from GitHub repository, newer install_dependencies.sh in my GitHub fork should work:
https://github.com/tambetm/DeepMind-Atari-Deep-Q-Learner
(see the Lua-GD section just before the end of install_dependencies.sh)

Regards,
Tambet

To view this discussion on the web visit https://groups.google.com/d/msgid/deep-q-learning/c59bfcd8-a819-4ff0-af3f-b1708dcd72d3%40googlegroups.com.

[Anthony Proschka]

Hi Tambet,

thanks for the fork. I have tried it out and it seems to work fine. Now I want to train my dqn without displaying the graphics so it will be able to learn faster. How can i achieve that?

- It's weird but in AleEnv.lua (alewrap) the variable "display" is already false

- I am running your version that uses qlua instead of luajit (if i try running with luajit i get the error "/home/ubuntu/torch/install/bin/luajit: /home/ubuntu/torch/install/share/lua/5.1/image/init.lua:1237: module 'qt' not found"

- If i just try to run the qlua version without x11 connection to my local machine i get the error:

"qlua: not loading module qtgui (running with -nographics)

qlua: qtwidget window functions will not be usable (running with -nographics)

qtwidget window functions will not be usable (running with -nographics)

qlua: not loading module qtuiloader (running with -nographics)

qlua: /home/ubuntu/torch/install/share/lua/5.1/image/init.lua:1321: attempt to index global 'qtuiloader' (a nil value)

stack traceback:

[C]: in function '__index'

/home/ubuntu/torch/install/share/lua/5.1/image/init.lua:1321: in function 'window'

/home/ubuntu/torch/install/share/lua/5.1/image/init.lua:1276: in function 'display'

train_agent.lua:98: in main chunk"

Thanks for your help,

Anthony

...

[Tambet Matiisen]

This visualization was hacked into the code later, just comment out line
98 in train_agent.lua:

win =image.display({image=screen, win=win})

I don't think it will yield in significant performance boost, but give
it a try. You might also want to replace qlua in run_gpu with luajit.
Oh, you already did it, good :).

If you have Maxwell architecture GPU, you might want to try my own
implementation in Python:
https://github.com/tambetm/simple_dqn

I recently wrote a blog post that summarizes what you can do with it:
http://neuro.cs.ut.ee/deep-reinforcement-learning-with-neon/

Tambet

[Anthony Proschka]

Hey Tambet,

i think i figured it out: i just commented out your " win = image.display({image=screen, win=win})" in train_agent.lua for the time being and will keep it training without graphics. I will let it run for a few hours and then enable graphics again.

Thanks,

Anthony

Am Freitag, 4. September 2015 12:22:29 UTC+2 schrieb Tambet Matiisen:

...

[Anthony Proschka]

Ah didnt see your message. thanks for the fast response!!! This is a lot of fun haha. I saw your python version as well. Will have a look at it.

Thanks and best,

Anthony

[Anthony Proschka]

Hi Tambet,

maybe another question. I trained the network now for a couple of hours but it doesn't seem to have learned much. I would like to use plot_results.lua to see what happened during the training period. But when i run it it doesnt throw any errors, but also doesn't plot the graph(s). Do you know how to go about this?

- I could maybe use X11?

- I also tried it via vncserver: I have a GUI open for the machine (i run on a aws ec2 instance with gpu) but it also there is no sign of the gnuplots...

Thanks and cheers from the German Odenwald,

Anthony

[Anthony Proschka]

Ahhh I again made it work by using the terminal emulator within the vncserver GUI :D Sorry for spamming you so much, i am such a newbie

Am Mittwoch, 23. Dezember 2015 12:01:08 UTC+1 schrieb Tambet Matiisen: