Use Google Colab to generate Lc0 training games for FREE

[R J]

Credit to @borg, @Harry Seldon, @fischerandom and others for putting this together.

1. Download the script file "lc0_v8.ipynb" attached to this post

2. (OPTIONAL) If you want the games you generate to show under your username, then locate line 132 of the script and replace the user and password parameters (--user 'Google Colab' --password googlecolab) and save the file.

3. Go to https://colab.research.google.com/ in your web browser.

4. Sign in with your google account or create one.

5. Click on the upload tab and upload the "lc0_v8.ipynb" script.

6. (OPTIONAL) If you want to generate games for a specific run, then click the dropdown box under "Choose a run to contribute" and select 1 for T60 or 2 for T58.

7. Click "Runtime" on the top menu bar and choose "Run all". 

8. You will now begin generating games as long as the browser tab is open. Check on the tab periodically as it will sometimes disconnect and prompt you to reconnect via a popup box.

If you need more guidance, post here or join the discord channel  - https://discord.gg/pKujYxD

[R J]

Another quick note:

If Google Colab assigns you a Tesla K80 GPU, you will generate games very slowly. If you want to try to get a more powerful GPU like a P100 or T4, go to the top menu bar and click Runtime->Manage sessions and click Terminate on your active session. Wait at least 2-5 minutes and then click on Runtime->Run all. You will now be assigned to a random new GPU. Sometimes it takes a few tries but you should be able to eventually get a P100 or T4 GPU.

.If you dont know how to check what GPU you are assigned, open the pic attached to this post that will show you where to look.

[OmenhoteppIV]

Easiest tutorial ever what we need is man power.

[Stefan Pohl]

The tutorial is great !!

But, I have the problem, that I cannot use it overnight, because on my notebook, Colab ends after 60 minutes each time...

[ilmar ambos]

where i find games

[R J]

On Tuesday, December 3, 2019 at 12:30:53 AM UTC-8, Stefan Pohl wrote:

The tutorial is great !!

But, I have the problem, that I cannot use it overnight, because on my notebook, Colab ends after 60 minutes each time...

 

There is a way to make it run for 12 hours without it disconnecting you after the first hour.

Start generating games like normal, then scroll to the very bottom cell and find the stop button on the left (shown in picture attached). Click the stop button, and it should turn into a play button now. Click the play button and it should now run for 12 hours.

[Tadeusz R]

On Tuesday, 3 December 2019 09:30:53 UTC+1, Stefan Pohl wrote:

The tutorial is great !!

But, I have the problem, that I cannot use it overnight, because on my notebook, Colab ends after 60 minutes each time...

 

I think it might be the case when your system logs off (or locks screen) after some time. Then browser looses focus and colab can check for this and disconnect.

When I only turn off my screen but keep my user logged, colab session is active much longer.

[R J]

On Tuesday, December 3, 2019 at 2:48:06 AM UTC-8, Tadeusz R wrote:

I think it might be the case when your system logs off (or locks screen) after some time. Then browser looses focus and colab can check for this and disconnect.

When I only turn off my screen but keep my user logged, colab session is active much longer.

Actually Colab normally disconnects most people after the first hour, even if their browser has focus. Check my response above for how to make it run 12 hours without interruption.

Obviously if your PC is going into hibernation, logging off, or going into sleep mode (i think) then that will disconnect you also.

[Stefan Pohl]

Great. I will try this immediately!

[mpeskett]

Thanks for sharing this tip, it seems to be working for me beyond the hour. I've been wanting to contribute for some time but don't personally have the hardware to do so, so I'm glad I can use this method! Thanks for sharing!

[Stefan Albrecht]

Thanks for sharing this. It works fine!

Is using google colab totally for free or is there a limitation after reaching it you will be charged?

Just wanted to know if I can use google colab carelessly.

Thanks, Stefan

[R J]

There are no charges. Use all you want. 

[Jesse Jordache]

Thanks for the tips, RJ.  They all work - especially getting Colab to run for hours (though not quite twelve).  I had stopped using Colab because the one hour deadline seemed too little to matter.  The T4 card, which I drew in the device lottery, is just a shade faster than my laptop, so it's more than doubled my game contribution.

[Jesse Jordache]

Ooh, I got a P100 after my first reset.  Lets see how it does....

Oh, this is between +50% or double the speed of the T4.  Don't feel like waiting around for the games per day number to settle on a figure, but it had hit 4000 before I stopped it to enable long running.  Thanks again!

It'd be nice if we could sticky this thread.

[Robert Pope]

Maybe I just haven't contributed in too long, but when I ran the script, I noticed the parameter --visits=10000. I thought training used 800 visits per move.

[Jesse Jordache]

It is - it's just no longer so simple to set it at 800.

The way kld_gain and visits interact give you that actual number of visits, which they try to keep close to 800.

[Krethys]

Thanks a lot for this incredibly easy guide. I didn't ever suspect this would be possible for free!

[Nonsio 28]

I think you can actually double the games generated. If you make a copy of the file and rename it, you can open 2 GPU sessions in google colab using only one google account and load the 2 scripts. I tried and it works

[MathAndy]

Yes, it worked two weeks ago, which was great to contribute games. But then suddenly it started to become very hard to get a GPU.
And even worse: in the last days, even if you get a GPU, and even after you restarted the last box, it didn't run longer than 30 min.

[Aubrey]

thx it works

2019/12/31 10:30:06 lc0_main.go:175: Completed 52 games in 24m43.089748894s time (3029 games/day)
2019/12/31 10:30:44 lc0_main.go:663: trainDir=/content/lc0/build/data-ecnoufrhdxpx
2019/12/31 10:30:45 lc0_main.go:175: Completed 53 games in 25m21.771930215s time (3009 games/day)
2019/12/31 10:30:52 lc0_main.go:663: trainDir=/content/lc0/build/data-ecnoufrhdxpx
2019/12/31 10:30:52 lc0_main.go:175: Completed 54 games in 25m28.68048106s time (3052 games/day)
2019/12/31 10:30:54 lc0_main.go:663: trainDir=/content/lc0/build/data-ecnoufrhdxpx
2019/12/31 10:30:54 lc0_main.go:175: Completed 55 games in 25m30.805238852s time (3104 games/day)
2019/12/31 10:30:55 lc0_main.go:663: trainDir=/content/lc0/build/data-ecnoufrhdxpx
2019/12/31 10:30:55 lc0_main.go:175: Completed 56 games in 25m31.916998937s time (3158 games/day)

[MathAndy]

Yes, it worked two weeks ago, which was great to contribute games. But then suddenly it started to become very hard to get a GPU.
And even worse: in the last days, even if you get a GPU, and even after you restarted the last box, it didn't run longer than 30 min.

It is still very hard, but interestingly, if you rarely get a GPU then it has always been a P100, which is fine. And today two of them ran more than 1 hour.

[MathAndy]

Now I have found a better way: you need a NEW gmail account, then you will get a P100 GPU and it also MAY run longer than 30 min.
Obviously, Google is evaluating what users do. If they use too many resources, they will get very limited time and resources. They also say it in the Help.
In early december it was so much better. They were more generous.
Can you confirm that, folks?

[Aubrey]

yes ,on new account works longer than 30 min

[Aubrey]

today i get faster GPU T4 20000 games/day P1000 3000

2020/01/02 16:18:09 lc0_main.go:663: trainDir=/content/lc0/build/data-zojyllnhvcgj
2020/01/02 16:18:09 lc0_main.go:175: Completed 59 games in 4m18.904148902s time (19689 games/day)
2020/01/02 16:18:13 lc0_main.go:663: trainDir=/content/lc0/build/data-zojyllnhvcgj
2020/01/02 16:18:13 lc0_main.go:175: Completed 60 games in 4m23.391850915s time (19681 games/day)

[Jesse Jordache]

That looks like you just pulled run 2.  Are you sure it's working in the main 32x280?

In my exp, T4s are a shade slower than P100s.  They bracket my 2060, which is about halfway between them.

[Aubrey]

Run is set to defaul stats from now

2020/01/02 18:26:03 lc0_main.go:175: Completed 1757 games in 2h12m13.492650621s time (19134 games/day)
2020/01/02 18:26:10 lc0_main.go:663: trainDir=/content/lc0/build/data-nspydqxyvppm
2020/01/02 18:26:10 lc0_main.go:175: Completed 1758 games in 2h12m20.443496119s time (19128 games/day)
2020/01/02 18:26:11 lc0_main.go:663: trainDir=/content/lc0/build/data-nspydqxyvppm
2020/01/02 18:26:11 lc0_main.go:175: Completed 1759 games in 2h12m21.545052041s time (19137 games/day)
2020/01/02 18:26:14 lc0_main.go:663: trainDir=/content/lc0/build/data-nspydqxyvppm
2020/01/02 18:26:14 lc0_main.go:175: Completed 1760 games in 2h12m24.46464599s time (19140 games/day)
2020/01/02 18:26:19 lc0_main.go:663: trainDir=/content/lc0/build/data-nspydqxyvppm
2020/01/02 18:26:19 lc0_main.go:175: Completed 1761 games in 2h12m28.946142314s time (19140 games/day)

[123]

R J:

Credit to @borg, @Harry Seldon, @fischerandom and others for putting this together.

1. Download the script file "lc0_v8.ipynb" attached to this post

2. (OPTIONAL) If you want the games you generate to show under your username, then locate line 132 of the script and replace the user and password parameters (--user 'Google Colab' --password googlecolab) and save the file.

3. Go to https://colab.research.google.com/ in your web browser.

4. Sign in with your google account or create one.

5. Click on the upload tab and upload the "lc0_v8.ipynb" script.

6. (OPTIONAL) If you want to generate games for a specific run, then click the dropdown box under "Choose a run to contribute" and select 1 for T60 or 2 for T58.

7. Click "Runtime" on the top menu bar and choose "Run all". 

8. You will now begin generating games as long as the browser tab is open. Check on the tab periodically as it will sometimes disconnect and prompt you to reconnect via a popup box.

If you need more guidance, post here or join the discord channel  - https://discord.gg/pKujYxD

Can I do more runs in the same time?

For example use two or three different browser?

Or open two or three google chrome browser, placed on the monitor next to each other and open colab tabs?

Do I need for each new colab another google account?

Colab works on my iPad Pro too but not on my iPhone. That's bad.

Is there also a simple script to run the LC0 client with my two RTX 2080 Ti GPUs?

[Benedetto Romano]

NO, Google telled crearly if LC0 project abuse from his Coolab servers then they BAN all LC0 users, so please do not abuse and use only the time that they decided to let us, and do not try to force the things.

Thanks.

[madchuckle p]

There is also a v9 version of the script with recent sub-user support (for example: Google Colab / YourUserName)

I think it can be downloaded from this link: https://cdn.discordapp.com/attachments/425419483016855563/661931494809534524/lc0_v9.ipynb

[MathAndy]

Generally, I agree. That is was I suspected. But where do you have read that? Please give a citation.

In their FAQ (https://research.google.com/colaboratory/faq.html), Google writes:

Why are hardware resources such as T4 GPUs not available to me?
The best available hardware is prioritized for users who use Colaboratory interactively rather than for long-running computations. Users who use Colaboratory for long-running computations may be temporarily restricted in the type of hardware made available to them, and/or the duration that the hardware can be used for. We encourage users with high computational needs to use Colaboratory’s UI with a local runtime.
Please note that using Colaboratory for cryptocurrency mining is disallowed entirely, and may result in being banned from using Colab altogether.

That does not sound so harsh. But presently, "may be temporarily restricted" is done quite harshly. There is a very small time window to get a GPU if you have used your account too much in the past.
Is anybody here that still gets a GPU for hours each day with his old account?
Presently, the time limit seems to be 10 hours (for a new account).

[R J]

Google eventually starts greatly limiting GPU usage for accounts that have been using colab for many hours. After weeks of using colab, my main account gets sessions cut off after 30 minutes and unable to get another one for 24hrs.

The only way around this is to create a new account.

[Jesse Jordache]

Or you can wait a bit and let it cool down.  Depends on how much work you want to put into it.

[M MUSTERMANN]

madchuckle p:

And what exactly is the difference between v8 and v9? 

[Roland del Rio]

Well, as usual it's written in the changelog.

"* 2020-01-01: Added optional user name"

[MathAndy]

@Benedetto Romano  Again: Where does Google say this? Please give a link or citation.

Am Freitag, 3. Januar 2020 12:16:48 UTC+1 schrieb Benedetto Romano:

[Benedetto Romano]

I readed this on discord group from an admin

[MathAndy]

Really? I can't find it on discord.

It seems that since about Jan 4 you hardly or even never get a GPU on Colab if you have used your account with a GPU for many hours.
Unfortunately the limit currently seems to be only about 30-40 hours. And maybe Google even notices that you run lc0.
The conclusion is that we can really be thankful for the past, having got many GPU hours from Google for free, but now
it is like months before again.

[Stefan Albrecht]

After heavily using colab, I had the same problems.

They dissappear after waiting a few days, not using colab.

Now I can use it with my old account every day about 4 to 6 hours.

But I wait at least 24h before I run colab again...