Pages are loaded very slowly and questions about Redis & Postgres

[Guy Lichtinger]

Hi all,

I ran a pilot of an experiment with 12 participants playing at the same time. 

Unfortunately, all the pages were loaded very slowly for all the participants, even though everything works fine when I run the experiment by myself.

Is it likely to happen because I am using the free plan of Heroku Redis and Heroku Postgress? or is there apparently another explanation? 

More generally,  I want to run the real experiment with around 200 participants (let's say that up to 50 participants at the same time). So do you know which plans of Heroku Redis and Heroku Postgres are recommended for me?

In addition, I understood that I pay for Heroku Redis and Postgres only when they are activated. So how can I pause Redis and Postgres when I don't want to use them and activate them again when I want to use them again?

Thank you very much!
Guy

[Chris @ oTree]

See here for performance advice: https://otree.readthedocs.io/en/latest/server/heroku.html#server-performance

[Guy Lichtinger]

Thanks Chris!

[Guy Lichtinger]

Hi,

So I went through the documentation, and it was very helpful.

Anyway, I didn't see recommendations about which plans of dyno, Postgres and Redis I should use.

So I want to run an experiment with 400 participants, while up to 100 participants play simultaneously. 

Can you recommend which plans of dyno, Postgres and Redis to use?

Thank you very much!
Guy

[feb...@free.fr]

The free plan on Heroku is not causing pages to load slowly. I guess, you have a problem here that needs to be investigated but the problem could come from the pc the pages are loaded on as well. There are many components which could be responsible for this low performance (from software to hardware).
To configure better your HEROKU servers, you need to take extra care of the load peaks. We run experiments of about 1000 participant but, at most, we have 45 simultaneous participants which is about as much as you are planning to have.
The configuration for this is quite light in this case. We have this configuration which could fit for your needs but:
- you need to try and measure (see reco on the link above)
-We took extra care in coding to handle race condition properly (with locks on database).   

Dynos:
        Standard 1X level
        3 web dynos
        1 timeout worker
    PostgreSQL:
        Standard-0 level
    Redis:
        Premium 1 level
    Papertrail
        Fixa level
    Region       

       EU

But bots do not work to test with this config. (multiple dynos). read this post
https://groups.google.com/g/otree/c/SVbWwIzxhFI/m/f0DJfXBnBAAJ

So maybe, in your case, I would adivse 1 web Performance L or M rather thant the cluster.

But first fix your problem on the page.

[Guy Lichtinger]

Thank you very much!!!
On some pages, I am using "time.sleep" to have a break between pages (the problem is that when a lot of people play at the same time, it pauses for much more time than I set).
So do you think that "time.sleep" could be the problem? is there another way to set a pause between pages without using this command?

Really appreciate your help!
Guy

[Chris @ oTree]

Don't use time.sleep(). It will pause your entire server. if you want a pause between pages, do it with javascript code.

[Experiment Verteiler]

Could you describe how you put locks on the database? I thought they would run in single threads, so that race conditions would not occur. That would be very helpful, thank you!

[Chris @ oTree]

the server is single threaded, and time.sleep() pauses the current thread, so therefore it freezes the entire server process. definitely not recommended.

[Chris @ oTree]

(oh i guess you were not replying to my post...anyway you are right, since 3.0 oTree is single threaded so you will not get race conditions with database access.)

[Guy Lichtinger]

Thanks Chris! that explains a lot.

Do you know in which JavaScript's command I need to use to have a pause between pages (after the player clicks on 'Next' and before the next page is loaded)?
I tried to use the 'SetTimeout' functions, but the page is submitted immediately anyway, and I couldn't find online another way to have the pause without the page submitted immediately.

Thanks a lot!
Guy

[feb...@free.fr]

I recommend upgrading to version >=3.0 and one Perfomance M Dyno for the web server with one timeout worker ( standard 1X  maybe enough?) . But test and try.

best

[Marlene B.]

Okay, thank you!