All participant links blocked with mturk deployment

[Corinne D.]

Hello,

 

I’ve encountered  a strange problem when deploying my otree app to Mturk:

The app works at first, mturkers are able to join and start playing. But after some time, all the participant links are taken at once… But it’s not actual Mturker that have entered the game, there are no Mturkers ID attached to these players. And those “ghost” participants do not play the game, on my monitoring screen they just appear as having started and stay on the welcome page. When deploying to mturk, we have 2 times the number of links than the number of participants, and all of those get blocked. So for a session of 72 participants for example, I had 40 real players, and 104 “ghosts”.

 

After the appearance of ghosts, real mturkers are no longer able to join our experiment, because it shows that it is full. The real mturkers that entered the experiment before the “ghosts” are still able to proceed with the experiment.

 

We tried to deploy to Mturk 3 times, and every time we had the same problem. We tried with a smaller number of participants (32), and with several different configurations of the heroku add-ons (postgres, redis, dyno type), but none of it helped.

I should also say that we have tested the app a lot, both on the local host, and on the Heroku server. And this problem never happened in our testing. It is only when deploying to mturk that we have this strange problem.

 

Any idea where the problem might come from? And more importantly, how could we fix it?

 

Thanks a lot,

 

Corinne

[Chris @ oTree]

hm, i haven't seen this one before. anyone else?

[tharg...@gmail.com]

Hi,

I think I'm having the same problem right now. From the Manage HITs individually tab, it looks like there are 40 HITs out there that are not being completed and it has been like this for more than an hour (while HITs should go back to the pool after 1 hour):

When I look at the oTree monitor, it looks like all the links have been opened (120 x 2). 

In the meantime I received an email from a worker saying that they accepted the HIT but they received a message that the session was full. So it seems that HITs are being sent back to the pool but that no new worker can join the session...

I can see that new workers are taking the HIT as the number of participants with the qualification that prevents them to retake the study keeps growing but nothing happens.

Should I expire the HIT? What can I do to prevent this?

I'm using oTree 5.1.7

Thank you in advance,

Alice

[Chris @ oTree]

Can you look at the data export for this session? That should list the mturk_assignment_id and mturk_worker_id for each participant. That might give useful info about who is clicking all these links. (This data might also be in the session mturk payments tab)

[tharg...@gmail.com]

Dear Chris,

As you can see below, no one is opening these links (starting at participant 102, the Mturk id column (K) is empty all the way down):

It already happened to me in the lab that when participants' links are not opened after a while, they automatically open all at once in the the monitor tab. It is not a problem there, because participants are provided with the link but it's a big issue on MTurk since the session appears full because links cannot be re-used.

 This is what I suspect happened here as well. As you can see on the heroku metrics tab below, there is a spike in response time at 5.26pm. I suspect this corresponds to the time where all the un-used links openened all at once, asit was very quiet before (the experiment started about 1 hour or so before):

Is there a way to prevent the participants from starting when the links are not used after a while? Did I set up something wrong for this to be happening?

Thank you so much for your assistance with this!

Alice

[Chris @ oTree]

Did you click the "advance slowest participants" button? Because that would explain this (and what you saw in the lab). Do you have the logs from that time when all the links were open? if you have papertrail enabled, it would be there. then you could see the IP addresses that were opening these pages.

[Corinne D.]

Hello,

I tried running my study on Prolific yesterday evening and had the same issue again: after some time, Otree blocks all the available links. New participants cannot enter, they see a "session is full" screen. It's the same as with Mturk: the blocked links do not have a prolific ID.

I did not push the "advance slowest participants" button. However i noted that the blocking of links happened right after one participant had a time-out on a wait page (I have a "wait too long" that allows individual players to proceed if they're not matched into a group). Could it be related?.

I searched the papertrail for one of the "blocked" links and here's what I get:

 Apr 27 09:16:10 dyncoop1 app/web.1 Auto-submitting timed out page: /InitializeParticipant/3ew2mmo4
Apr 27 09:16:10 dyncoop1 app/web.1 INFO:     127.0.0.1:49928 - "GET /InitializeParticipant/3ew2mmo4 HTTP/1.1" 307 Temporary Redirect
Apr 27 09:16:10 dyncoop1 app/web.1 INFO:     127.0.0.1:49936 - "GET /p/3ew2mmo4/Welcome/WelcomePage/1 HTTP/1.1" 200 OK
Apr 27 09:39:23 dyncoop1 app/web.1 Auto-submitting timed out page: /p/3ew2mmo4/Welcome/WelcomePage/1
Apr 27 09:39:24 dyncoop1 app/web.1 INFO:     127.0.0.1:48008 - "GET /p/3ew2mmo4/Welcome/WelcomePage/1 HTTP/1.1" 200 OK
Apr 27 09:43:47 dyncoop1 app/web.1 Auto-submitting timed out page: /p/3ew2mmo4/Welcome/WelcomePage/1
Apr 27 09:43:47 dyncoop1 app/web.1 INFO:     127.0.0.1:38560 - "GET /p/3ew2mmo4/Welcome/WelcomePage/1 HTTP/1.1" 200 OK
Apr 27 09:46:53 dyncoop1 app/web.1 Auto-submitting timed out page: /p/3ew2mmo4/Welcome/WelcomePage/1
Apr 27 09:46:53 dyncoop1 app/web.1 INFO:     127.0.0.1:46712 - "GET /p/3ew2mmo4/Welcome/WelcomePage/1 HTTP/1.1" 200 OK

Any idea why this happens and how to fix it?

Many thanks !!

Best regards,

Corinne

[Chris @ oTree]

OK thanks for sending this; this narrows it down somewhat. Though I still can't reproduce this issue myself. Could you send me your project code and tell me which session config this issue is happening for? Then I could try running it myself.

[Chris @ oTree]

Hi, I have now fixed this. It was a bug in oTree related to group_by_arrival_time. Thanks to those who reported this issue and especially Corinne for providing her code for me to test.

Chris

[Chris @ oTree]

Yes just upgrade otree. 

Sent from my phone

On Apr 30, 2021, at 7:42 AM, Corinne D. <corinne....@gmail.com> wrote:



Great! Thanks a lot!

Do I need to update oTree or something like this for the fix to become effective?

Best,

Corinne

--
You received this message because you are subscribed to the Google Groups "oTree help & discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otree+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/otree/97cbc4a8-d554-4173-895a-9513060d5c11n%40googlegroups.com.