Replace placeholder "No Way!" card with a valid card and running a simulation.

[Jan Molčík]

I am currently working on my master thesis "AI in games with incomplete information" and decided to use the Hearthstone-AI Competition framework.

One of my approaches to remove the uncertainty is to guess which deck opponent plays by tracking his play history and once it reaches some predefined threshold I will start simulating his possible plays. Unfortunately I'm straggling to find a proper way to do so (i suppose).

I implemented a MCTS and tried to enhance it with this approach but things got too complicated so I decided to start with simpler agent. I took the winner of 2019 competition - the DynamicLookaheadAgent by Tom Heimbrodt. If I understand it correctly, the agent basically ends the simulation process when reaching certain depth or the state where it's his opponent's turn.

So I implemented some kind of opponent action estimation to go even deeper. The idea is to remove all unknown cards from opponent HandZone and add cards from the guessed opponent's deck. After some trials and errors I came up using this particular method

player.HandZone.Add(Entity.FromCard(in player, deckCard));

and after adding all the cards I simply do

player.Options()

to get the List<PlayerTask> of all possible tasks. This seems to be kind of working but my question is whether this is valid (or if I should never manually change the player's HandZone).

I tested it but it turns out that it performs poorly versus the original DynamicLookaheadAgent. One reason that came to my mind is that now I need to also somehow tweak the Score function to evaluate the game state from the point of view of the opponent (since I want to choose his best option so I can play against it). I tried to remove the condition to always take first player's perspective

private static int Score(POGame state, int playerId)

{

var p = state.CurrentPlayer.PlayerId == playerId ? state.CurrentPlayer : state.CurrentOpponent;

return new CustomScore { Controller = p }.Rate();

}

to simply

private static int Score(POGame state, int playerId)

{

return new CustomScore { Controller = state.CurrentPlayer }.Rate();

}

but it doesn't seem to work. Any ideas how I can achieve this?

If you want to see the full implementation I can share it here or you can take a look at my GitHub fork of the framework - TestAgent source code.

Looking forward to your answer.

Best regards,

Jan Molčík

[Alexander Dockhorn]

Dear Jan,

thank you for using our framework. I hope it will serve you well in writing your master's thesis.

Your approach with replacing the opponent's hand cards is definitely possible with the framework and is even the intended way of doing so. Other agents have done so in the past. If I remember correctly, both, the Predator MCTS by Max Frick, Ünal Akkaya and the Tyche MCTS agent by Kai Bornemann, have done so. You can find them in the Bot Downloads section on our website to see how they did it. When using the Options method for your opponent you should always be sure that the NoWay card is not included anymore. This is important since the framework will not be able to handle any NoWay card options.

The Dynamic Lookahead agent by Tom Heimbrodt has been a strong agent in last years competition. In case you want to compare if your agent performs better than this one, you should try to use the same search scheme and only add the replacement of cards. The dynamic lookahead agent features an evolutionary optimized scoring function. Nevertheless, it may be necessary to tweak it for your agent.

Instead of rating the state from your opponent's view you might just always rate it from the current player's view and minimize the score in case you applied an opponent's action. While I am surprised that your proposed solution does not work properly, this approach should definitely work. Furthermore, the alpha-beta pruning agent by David Fernandez Atienza and the one by Hans-Martin Wulfmeyer should both use any kind of scoring the opponent's moves.

Since I have also done some work on Partial Observability of cards, I will send you an e-mail with some further details.

Best regards,

Alexander