Trading RL algo overfitting

[Georgios Kourogiorgas]

Hi. I have created my own gym based env for intra day FX trading.

I have tried stable-baselines' PPO2 and DQN so far with MLP and LSTM policies. So far the best results in the training dataset is with PPO2 and LSTM. However, it seems to overfit because in the ttest set results are bad (almost random returns). I will test your library as well. My actions are buy and hold. So, long only. I tried training the same agent with various currencies, but nothing yet. I haveen't done any serious hyperparamter ttuning butt results in training set are amazing (hence overfitting).

Any ideas?

[Deep Learning Tensor]

Your question of overfitting falls into the general issue of simulation-to-reality gap.

You may be interested to know the ensemble strategy: https://github.com/AI4Finance-LLC/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020 and also: https://github.com/AI4Finance-LLC/FinRL/blob/master/FinRL_ensemble_stock_trading_ICAIF_2020.ipynb

A rolling window may also help somehow.

Georgios Kourogiorgas <g.kouro...@gmail.com> 于2021年4月18日周日 上午11:53写道：

Hi. I have created my own gym based env for intra day FX trading.

I have tried stable-baselines' PPO2 and DQN so far with MLP and LSTM policies. So far the best results in the training dataset is with PPO2 and LSTM. However, it seems to overfit because in the ttest set results are bad (almost random returns). I will test your library as well. My actions are buy and hold. So, long only. I tried training the same agent with various currencies, but nothing yet. I haveen't done any serious hyperparamter ttuning butt results in training set are amazing (hence overfitting).

Any ideas?

--
You received this message because you are subscribed to the Google Groups "AI4Finance" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ai4finance+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ai4finance/40921782-dd70-47a7-829f-cf8df3847786n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Georgios Kourogiorgas]

Thanks for the reply. I think cross-validation might help. Problem is I am currently doing hyperparameter tuning as well using ray. Can you suggest a way on how to try the same set of params to all k-folds in terms of code structure? A for loop? How is that parallelized?