TCEC Season 16

2008 views
Skip to first unread message

Stephane Nicolet

unread,
Aug 23, 2019, 8:32:10 AM8/23/19
to FishCooking

Hi all,

We have received the following email from Anton Mihailov (TCEC)

> Premier Division of TCEC Season 16, where your engine participates, is going to start soon. 
> Last move of bonus gauntlet after League 1 Playoff  is deadline for updates.
> Please, send us your latest stable version and any additional information that you  would like the admin to know.
> Keep Aloril and Kan on CC.  They are top professionals and will keep the smooth running of TCEC as usual.
> For information about the season check it out at this link http://www.chessdom.com/tcec-season-16-information-and-participants/ .  All the best in S16!


Does anybody want to play the "optimizing flags for special tournaments versions" game and try to provide a fast binary? :-)


Stephane Nicolet

unread,
Aug 23, 2019, 9:43:31 AM8/23/19
to FishCooking

We have about four days to send the updated version,
so it be nice to have our binary before Monday night.

vas...@gmail.com

unread,
Aug 23, 2019, 3:02:54 PM8/23/19
to FishCooking


Friends, Are you curious about this: I know that I know a live person who plays your version 10 program. Would you like to make an official match in several games? For example, for his part, he offers $ 200,000, and so do you. The winner gets everything. You are exhibiting your latest version of the program and you are responsible for making sure everything is fair. If you are interested, you can write me here vas...@ukr.net

Kiran Panditrao

unread,
Aug 25, 2019, 3:26:58 AM8/25/19
to FishCooking
Hi,

Can you please elaborate what is "optimizing flags for special tournaments versions"

Also can you please nominate a candidate (exact commit) that will be playing?

Regards,
KP

mal

unread,
Aug 26, 2019, 2:10:19 AM8/26/19
to FishCooking
I think we usually just link to the latest abrok compile nowadays.

The only decisions needed are which version (if not simply "the latest") and what contempt value to use.

I think we have a few more days yet, the RR for the current league will finish later today, then there is about 48h for the playoffs. I don't know how long the bonus gauntlet is, but I imagine we have until Thursday evening or later to decide.


Stephane Nicolet

unread,
Aug 26, 2019, 4:41:46 AM8/26/19
to FishCooking

@Kiran: yes, I had you in mind when I said that :-)

@mal Thanks for the info. The best candidate at this time of writing is "Tweak Late Move Reduction at root",
but we usually wait till about 24 hours before the official deadline to give a last chance to good Elo-gaining
patches to bloom in the framework (and we usually launch the progression/regression tests for the submitted
versions right after the start of the TCEC stage, because they are quite ressource heavy...)

Alayan

unread,
Aug 26, 2019, 4:59:17 AM8/26/19
to FishCooking
If TCEC supports it, a compile with large pages memory could give some "free" speed-up compared to the abrok version, but are there any other significant compile-optimizations we may expect ? We can't have the compile done directly on the TCEC machine.

The main other variables besides the binary are hash size and contempt.

Hash size is typically 16GB for TCEC, but not only can nps suffer but SF's pruning behavior vary based on hash size, and it's hard to know if e.g. the 32GB in SuFi15 (only possible for the SuFi) actually helped at all. In any case, sticking to the usual 16GB should work ok.

Then, there is contempt. Which value is best will depend on who wins the playoff, so I'll wait on seeing who is going to promote before sending a value. If Stoof and Scorpio both makes DivP, there will be 4 NN engines in DivP, so I'd just stick to default. If Ethereal and Laser manages to get in, though, a slightly higher contempt would be more interesting.

In any case, SF should manage fine to promote to SuFi.

Ente

unread,
Aug 26, 2019, 6:38:31 AM8/26/19
to FishCooking
Large pages are not used in TCEC due to memory fragmentation which could lead to an unfair advantage.
Contempt:
I'd say we use default contempt during the premier division. In the final we might lower it to 12 so SF doesn't take 3-folds as of move 5 but also not be too zeal0us ^^ Personally I've used C=12 recently and it works like a charm against strong opponents. Though I don't have enough data and default should be fine as well though it is probably a bit weaker against Leela.

Михаил Чалый

unread,
Aug 26, 2019, 6:59:19 AM8/26/19
to FishCooking
From data I witnessed at LTC c=0 does signifficantly better vs leelas.
At 4c we had pretty funny bonus where contempt 100 stockfish played vs leela, stein, some other nns, alongside contempt 0.
Sf 100 lost to sf 0 by like 40 elo perf (pretty expected) but vs nns it lost with >100 elo difference (to leela and jhortos leela). Also it style changes drastically - it always goes for hyperclosed positions compared to 0 contempt.
So
1) contempt forces SF to play more positional. It's not good vs NNs, 0 contempt fish is more of a tactical player and goes for tactical positions - which is good because NNs are weak there;
2) it reflects in results of some bonuses and also in 1x1 matches (run by mike young, for example).
So I do firmly believe that contempt = 0 is the way to go for SuFi.
Message has been deleted

Fauzi Dabat

unread,
Aug 26, 2019, 7:20:58 AM8/26/19
to FishCooking
My opinion was that the optimal evaluation values are different for each different contempt value.
So, for example, the perfect pawn value for contempt = 0 can be 140, while for contempt = 24, it would be 128.
That was my idea when I created this test and this pull request: https://github.com/official-stockfish/Stockfish/pull/2177
That we can lower the contempt, and slowly, in the long run, change the optimal values of many evaluations or search weights.
In the process, we would improve stockfish strength (since my tests passed both stc and ltc, and we also reduce contempt)
But unfortunately, the idea was rejected.

Kiran Panditrao

unread,
Aug 26, 2019, 11:06:51 AM8/26/19
to FishCooking

Kiran Panditrao

unread,
Aug 26, 2019, 11:15:53 AM8/26/19
to FishCooking
An this is what my compile did compared with abrok
Sf vs abrok.png

ts.tom...@gmail.com

unread,
Aug 26, 2019, 6:05:17 PM8/26/19
to FishCooking


W dniu poniedziałek, 26 sierpnia 2019 10:59:17 UTC+2 użytkownik Alayan napisał:
We can't have the compile done directly on the TCEC machine.

-march=broadwell

Alayan

unread,
Aug 27, 2019, 3:44:02 PM8/27/19
to FishCooking
I sent Kiran's binary as SF's official submission to TCEC, so there won't be issues with the deadline. We can always send an updated binary if another elo gainer gets in before the deadline, though that doesn't seem likely.

As Stoofvlees and ScorpioNN seem to be poised to promote, I kept default contempt in the submission. If by the end of the playoffs, Eth and/or Laser get in, this may be revised.

Stephane Nicolet

unread,
Aug 27, 2019, 7:06:05 PM8/27/19
to FishCooking

Thanks for taking care of that! 
What are our pronostics for Premier Division?

mal

unread,
Aug 28, 2019, 4:40:23 AM8/28/19
to FishCooking
With 4 decent NN engines I would be tempted to use contempt 12 in premier. With recent improvements we should be comfortably better than the traditional engines, but the nn engines might give us a little trouble?
Using contempt 12 might give us some useful info on C12 performance against other engines and inform a decision for the superfinal (assuming we get there). If we then use contempt 0 in the superfinal we would have some real competition experience of both 12 and 0.

mal

unread,
Aug 28, 2019, 4:43:40 AM8/28/19
to FishCooking
I imagine we should reach the superfinal, but leela and I imagine Allie will be strong opponents. Maybe Stoofvlees too?
What do you think Alayant?

Alayan

unread,
Aug 28, 2019, 7:02:35 AM8/28/19
to FishCooking
C24 is slightly worse against Leela and Allie, but clearly better against K, H and KMCTS. Scorpio/Stoof aren't stronger than K/H based on available data, so while we lack tests against them, I'd wager C24 doesn't lose elo against them. Overall, in RR format with 100+ elo weaker engines, contempt still helps to get on top.

My prediction for the rankings would be :
1. SF 2. Leela 3. Allie ; with K and H in 4/5 ; and Stoof/KMC/Scorpio in 6 to 8.

With the small sample size, Allie has an outside chance of going into SuFi. Allie's repo has not been updated since June, so I doubt it has gained a lot. But it is more likely that this would take out Leela out of SuFi than SF. I'm very confident in SF going into SuFi.

For SuFi, assuming Leela as an opponent, the plan is to go with contempt=0, as it seems to do better in h2h based on the (limited) data available, not trying so hard to go in positional struggles.

anshul...@rediffmail.com

unread,
Aug 28, 2019, 12:08:58 PM8/28/19
to FishCooking

Yes based on Mark Young's recent testing we have found that C=0 is the RIGHT contempt setting use when playing LC0 head to head. SF with C=0 has done very well against LC0.

I think we shud go with default contempt in divP and if and once we reach SUFI then, C=0 shud be used.

apospa...@gmail.com

unread,
Aug 30, 2019, 3:40:29 PM8/30/19
to FishCooking
C12 should be about 3 elo better vs C0 in head to head but it is not verified.
For the SuFi I think its wise to first verify the difference with fixed game test and then decide because both C0 and C12 have merit:

C0 probably counters leela style more, plus it might be optimal if we are weaker than leela
C12 offers the best "objective" chess we got, plus contempt has shown by numerous tests to hyperscale with TC

For the Premier Div, higher contempt than default makes sense but its risky because to be worth it, it must produce vs lower opponents double the points than it forfeits vs SuFi contenders.
With 4NN's it could lead to disaster, so I'd just stick with the default which is high enough anyway, as previous seasons have shown.

tutr...@gmail.com

unread,
Aug 31, 2019, 11:46:36 AM8/31/19
to FishCooking
When we concluded C0 was the best, did we include negative contempt as well ?

lucasme...@gmail.com

unread,
Aug 31, 2019, 12:27:16 PM8/31/19
to FishCooking
It can be interesting to use negative C if we are down material or when we have a higher mobility than the opponent.

Against leela anything lesser than C24 is better

Alayan

unread,
Aug 31, 2019, 2:39:33 PM8/31/19
to FishCooking
Contempt merges "readiness to accept a draw" and "willingness to simplify by exchanging pieces".

The willingness to simplify that is optimal against Leela on a given setup might be in the "negative contempt" territory (starting from a position with most pieces in, this is likely non linear with the value of pieces on board). But the "readiness to accept a draw" from negative contempt is counterproductive, as it would make SF go for a draw even when it has the edge.

That's one of the main reason why negative contempt has received so little attention.

tun nay

unread,
Sep 1, 2019, 7:34:13 PM9/1/19
to FishCooking
Keeping C24 is already very dangerous in Div P. Jhorthos tests in discord showed that this leela improved more than Stockfish. ( Previous Sufi elo difference was -20 but it is around -30 now). That being said, it is already taking a high risk of unnecessarily losing 1 game out of 6 in Div P. This also carry similar risk vs Allie too.

guo....@gmail.com

unread,
Sep 1, 2019, 10:14:18 PM9/1/19
to FishCooking
I think contempt is no different than any other parameter in the code and should not be changed unless proven confidently by statistics otherwise.

vizve

unread,
Sep 10, 2019, 5:32:38 AM9/10/19
to FishCooking
It seems this "optimized" build is the reason SF lost the game.

Andries Johannes

unread,
Sep 13, 2019, 10:11:36 AM9/13/19
to FishCooking
Agree

mikeba...@gmail.com

unread,
Sep 13, 2019, 9:17:26 PM9/13/19
to FishCooking
I'd recommend using Build Tester to test binaries before submission. I'd set the bench command to use 43 treads, a fixed depth typically reached in TCEC and a cache as close to TCEC as the machine you are running allows. Then let it run a few days. It doesn't matter if the machine doesn't have that many cores as speed and strength are irrelevant. You only need to spot crashes. Alternatively you can set time long enough on the test machine to reach TCEC depth _per bench position._ This isn't perfect, but beats playing a few games IMHO. Bigger hardware will run more completed tests per unit of time.

My 2 cents.

Reply all
Reply to author
Forward
0 new messages