mturk - how to check if task is correctly loading?

[Abhilasha Kumar]

Hi, I have uploaded a single batch of my game on MTurk. So far, only three workers have tried to do the task but all of them have not typed in the correct completion code they should get at the end of the study, and I have no record of their data on the server. 

Is there a way to check that the task is actually working and they are able to correctly access the link? 

The task does work normally on https://IP/with auth/ID/?id. I followed the instructions here https://github.com/nodeGame/nodegame/wiki/nodeGame-Mturk-v7 to change the survey page content on Mturk, where I changed the "host" variable to https://IP.  

When I look at the "preview" on Mturk, I see the page loading but the survey link does not appear (because it says it will be generated when they accept the HIT). 

I'm just not sure if participants are able to even see the link or not. Let me know if you have ideas on how to check this.

Thanks!

Abhilasha

[Abhilasha Kumar]

Alright, I have an update. One turker just emailed me saying they cannot get the link to the experiment.  Attached is the HTML code I pasted into the source page in the survey. Please let me know if you see any problems! Thank you. 

--
You received this message because you are subscribed to a topic in the Google Groups "nodeGame" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodegame/LWiOmHTa5O8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodegame+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/96e75f6e-2eec-4f72-9156-6ac9f06a8582n%40googlegroups.com.

[Abhilasha Kumar]

Another update - attached is a screenshot from the participant. Seems like there is an error in generating the link correctly. Could this be an issue in my actual game files or is this to do with the html? 

[Stefano Balietti]

Hi,

Usually, you should test your task on the Sandbox first: https://requestersandbox.mturk.com/

There you can check if your game runs correctly by logging in the worker sandbox: https://workersandbox.mturk.com/

Make sure that:

- claimid is enabled in auth: https://github.com/nodeGame/nodegame/wiki/nodeGame-Mturk-v7#running-the-hit

- you changed the host in this file: https://github.com/nodeGame/nodegame-mturk/blob/master/public/mturklinkpage.html (you said you did, but double check, also https)

Let me know if it works for you.

Cheers,
Stefano

You received this message because you are subscribed to the Google Groups "nodeGame" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodegame+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqm%2BBttrTmUi0%3D-UFtKmE1RngJJe4i4jVhoQ2_Zodqcbpg%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano, thanks - I just tried the sandbox and I'm getting the same error that the participant sent me a screenshot of. I did make sure that the claimID is enabled inside the auth file and also that the host variable is correctly referencing the https link. 

The game is available at https://github.com/abhilasha-kumar/helping-behavior-game and I believe you have access to it. I'd be grateful if you could point me to where there might be an issue - I'm guessing there is something off in the auth file but not sure what exactly - let me know if you have thoughts, thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMnV0xVQDKOGaPr_sFcA1AiOSpkmPdYhSOCPYFm8pDWa0A%40mail.gmail.com.

[Stefano Balietti]

Will have a careful look later. For now, I just noticed you are using 'external' mode. Have you tried the local mode? You should generate the code beforehand (e.g., starting the server once with mode 'auto' and then switching to local). 

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqnf076Tiosgsujy7C_qpOQ8Kv5eyYprs1QYy4x%2BwWtwSg%40mail.gmail.com.

[Abhilasha Kumar]

Alright that seemed to be the problem - I changed the mode to 'auto' and seems like it is working now - thank you! On a more general note, thanks so much for the careful documentation and responsive help group! This has really been helpful in trying to learn how to build a game from scratch :) 

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMnj2SwQ8VWTG9TijVhBbVKisdPFGsSvd849rKV%2BhqdYkA%40mail.gmail.com.

[Stefano Balietti]

Great! And thanks for your feedback.

 When you run your game, I suggest switching back to 'local' mode. In the eventuality that you need to restart the server, you want to keep the same codes, otherwise you might lose some participants.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqnrXka9qtdo3DxEmBT4P_ncM96BckT8j9MUMzcH43ik3g%40mail.gmail.com.

[Abhilasha Kumar]

Yes, will do - thanks! 

---

Abhilasha Kumar

Research Associate, Dept. of Psychological & Brain Sciences

Indiana University, Bloomington

M +1 (314) 662-5443 | abhilas...@wustl.edu

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMkRMYFUoWwGE8mO6CN%3DPd6ARgnJ7hOYCzTABJzd8CTf%3Dg%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano, I have another question for you. I did a quick pilot run of a few participants, and most of the data is coming in correctly, but the Mturk IDs are not getting recorded in the excel file. I did put in the headers in my result file but don't have any node.set statements in my player.js file for the mturk IDs - is that what will fix this issue? I was assuming the code in the auth file would take care of this but I guess not:

Would it simply be adding something like below to the player.js file?

node.set({WorkerId: WorkerId});

My auth.settings.js file contains the following:

claimIdPostProcess: function(clientObj, query, headers) {

clientObj.WorkerId = query.id;

clientObj.AssignmentId = query.a;

clientObj.HITId = query.h;

},

Let me know what you think, thanks! 

Abhilasha

[Stefano Balietti]

Hi,

Which file is that? How did you create it? You should use:

gameRoom.computeBonus({

   amt: true,

   // other options depending on your game situation.

});

For instance this will NOT work:

memory.save('filename.csv', {

    amt: true

     // ...

});

In general, 

node.set({WorkerId: WorkerId});

will not work unless you have that WorkerId variable saved globally.

Best,

Stefano

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqmz3yxZv3LKXUxSBaz-qyJ3wH7fXhkGLoDvR3bBgj8A%3DA%40mail.gmail.com.

[Stefano Balietti]

To add to my previous answer. WorkerId, HitID, and AssignmentId are stored in the registry, not in the memory db. If you would like to have everything in one place you should copy them over somehow.

For instance:

memory.forEach(p => {

   let { WorkerId, AssignmentId, HITId } = channel.registry.getClient(p.id);

   memory.insert({

        player: p.id,

        stage: node.game.getCurrentGameStage(),

        wid: WorkerId,

        aid: AssignmentId,

        hid: HITid

   });

});

But I would just use gameRoom.computeBonus.

[Abhilasha Kumar]

Hi, thanks for your input! So far, I've been saving results via the memory db in logic.js, like below:

exit: function(){

db.gameplayprac.save('gameplayprac.csv', {

// Custom header.

header: ["access","exit","WorkerId","hid","AssignmentId","bonus","Approve","Reject", "helperRandCode", "architectRandCode", "config", "goalnumber", "goal", "helperChoice", "helperChoiceTime","helperMove", "architectMove", "goalSuccess", "completioncode"],

// Saves only updates from previous save command.

updatesOnly: true,

flatten: true

});

I am not familiar with the computeBonus function - is there documentation on this somewhere that I can read? I will try getting the values from the registry in the meantime. 

Also - is there a way to test the experiment locally with the Mturk stuff (i.e., claimID enabled etc.) or do I have to upload the changes to server and then try it via the sandbox? 

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMn3W8Yke8k%3DKfbCsOxrhXyS99NWTGwUgASkXJkyP3Fe7w%40mail.gmail.com.

[Stefano Balietti]

Hi,

Here is some documentation: 

https://github.com/nodeGame/nodegame/wiki/Server-API-v7

Also it is often used together with the EndScreen:

https://github.com/nodeGame/nodegame/wiki/EndScreen-Widget-v7

Mturk stuff should run on the Sandbox. But if all you want is checking if those values are saved you can do this:

- add that mturk task file the public directory of your game,

- set the host to localhost

- add random variables for HITID, AssignmentID, and TurkerID

Notice, you can point to localhost even in the file uploaded on MTurk.

Let me know.

Best,

Stefano 

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqmohPedDuTRTqs7UzUJQvfQfd-hx9xbkG-SfZn_3vOvLA%40mail.gmail.com.

[Abhilasha Kumar]

Great, thank you - I will try this and see if it works!

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMn231_eUt6pkTvnB9TJfL2CeKE-Qu2j%3D8k4UvEmcFaCaQ%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano,

So I think I have ended up confusing myself a lot with the mturk ID stuff. Basically, based on your instructions above, I moved the content of the mturk landing page to the public folder of my 

local machine and then changed the host to 'https://localhost:8080/gamename'But i don't understand why we do this, because when I launch the game via node launcher.js this landing page is never visited. Or am I supposed to add that page to an initial  stage/step - I also tried this but don't fully understand how the ID stuff works in this case.

In any case, what should the auth.settings.js file consist of at this point? I set enabled and claimID to true, and then did this:

claimIdPostProcess: function(clientObj, query, headers) {

clientObj.WorkerId = 12345//query.id;

clientObj.AssignmentId = 78901 //query.a;

clientObj.HITId = 11111 // query.h;

},

Is this not what you meant by setting the HITId, etc. to random numbers? On a separate note, I am unable to launch the game locally with authorization on because it gives me the "unauthorized user/pwd" error - any ideas on how to fix this?

Finally, the gameroom.computeBonus stuff goes in logic.js? I put it there but when I try to launch the game, this is the error I get:

Please let me know if you have any thoughts on all of this - sorry for all the questions, some of these may be very basic but clearly I am missing something! 

Thanks,

Abhilasha

[Stefano Balietti]

Hi,

Sorry for the slow reply, I missed this last msg somehow. To visit the landing page you need the pass the full path in the address bar:

http://localhost/yourgame/landingpage.html

Your modification in auth.settings.js will work, but it makes sense testing the value sent from the landing page. On line 395 of file 

https://github.com/nodeGame/nodegame-mturk/blob/master/public/mturklinkpage.html

there are these values to change:

// Testing values.
	// host = 'http://localhost:8080';
	// wid = 30;
	// aid = 2;
	// hid = 1;

After authorization is on you need to connect using the codes you (or auto mode) created. They are saved in the same auth directory, you can check those. You can also use "dummy" mode for testing, so it will be just localhost/game/auth/1/1 and ... 2/2/ , 3/3/ and so on. But if claimid is working, you should not worry about the codes at all.

Finally, GameRoom.computeBonus goes into logic, but it gives you an error on a strange line 1302, which does not match the master branch. Did you make some custom changes? Anyway, I assume the error is from this line: 

https://github.com/nodeGame/nodegame-server/blob/master/lib/rooms/GameRoom.js#L1315

Perhaps you are calling the computeBonus too early without players?

Let me know if you can solve these issues, sorry for the slow response again.

Stefano

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqmwBTZKGpUPZF6M8iBzohqQZVtpbNwo7ke5U9OpyCgeFg%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano, 

Thanks for getting back to me - this is helpful. Ok so I am able to get to the landing page via http://localhost/yourgame/auth/1/1/landingpage.html. However, once I do get there, I am not able to advance because the game link is not generated. So I am seeing the "do not accept this HIT" message, see below:

So maybe something is off in my landing page or auth file? 

Separately, this happens even if I comment out the gameRoom code in logic.js. When the gameRoom code is uncommented, I see the same error as before - I thought it might have something to do with me calling the function too early (as you suggested) so I moved it towards the end of the logic file but that didnt work. I have uploaded the revised code to the repository in case that is helpful for you to take a look at: abhilasha-kumar/helping-behavior-game.

Please let me know what you think! 

Thanks,

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMnNEDRR6WWH-n7P4fnyr1%3DcBtSnFrVv7RXV4pKVERSNfg%40mail.gmail.com.

[Stefano B]

Hi,

I have checked your code and I was able to make it work after adjusting the landing page. Sorry, my instructions were not correct! To make it work in localhost you need to comment out some Mturk specific code, otherwise it will not run. Here is the tested instructions:

- Make sure you run ssl

- Set host to: host = 'https://localhost:8080/moveablock' // make sure no trailing slash here.

- Set auth to dummy and login once

- Then connect to your landing page https://localhost:8080/moveablock/xyz.html

I will send you the updated file privately, otherwise it will take too long to update the repo.

Cheers,

Stefano

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeqnBf9teNLT3SOPN_iW0kf53thTQNFaOKfSG6cnzVSFXVg%40mail.gmail.com.

[Stefano B]

(actually pushed to your repo)

[Abhilasha Kumar]

Hi Stefano,

Thanks for looking into this. So I am having trouble using HTTPS locally, but I tried running the experiment using http with the changes you suggested. First, will the local testing not work without https, and if not, is there a straightforward way to do use https locally? I was looking into this online but all the solutions seemed a bit of work. 

Second, when I do this via http, I am able to get to the xyz.html page, and it does generate a link to the experiment  (http://localhost:8080/moveablock/auth/0) however, the link generated is identical for both tabs so I am unable to actually start the experiment because of the "wrong auth/pwd" message in the second window/tab. Also - one thing I noticed is that if I don't have a separate step with the xyz.html page and try to do https://localhost:8080/moveablock/xyz.html, then I am immediately taken to the game, entirely bypassing the landing page, which is confusing. And in cases where I am actually able to advance to the game portion, I am not seeing the data file load in any values for the mturk ID. 

So, it seems like testing this locally is a lot of trouble - maybe it's not worth it? I can possibly just run it on sandbox and test it that way - I suppose one question is, if I want to test the game with 2 players on sandbox to make sure all the data (including IDs) is getting recorded will I need two worker sandbox accounts? 

Thanks for your help on this! 

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CANJPQLPqO4N0UvE-fq9MWMgQyvBnD5bmfxx5Pu6PxeQybsOYBw%40mail.gmail.com.

[Stefano Balietti]

Hi  Abhilasha,

You can generate a self-signed certificate with this instructions:

https://github.com/nodeGame/nodegame-server/tree/master/ssl

You can copy the files generated inside a directory of choice, then start the server with the path to that dir.

node launcher.js --ssl /path/to/ssl

E.g., if you create the ssl inside the nodegame directory:

node launcher.js --ssl ssl

I thought the purpose of this setup is to test whether codes are generated correctly, which is indeed the case. Testing with multiple players requires changing the simulated Worker Id that is sent to the server (it was fixed in the current file). I changed that and pushed. I am able to get different codes and start the game, however as soon as it starts the game is redirected to the landing page. Did you implement this feature?

I generally test this online with the sandbox, so the whole localhost procedure was not properly tested/documented. However, it feels like it's quite close to being completed. Anyway, even if the game runs smoothly on localhost, I would recommend testing again in the sandbox.

Cheers,
Stefano

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeq%3DuWGbuazvMj%3DrSmbkAvxZY%3DPuX_8AQ_AOv1MVLOrmDtg%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano,

Thanks - I was able to run the game via SSL on localhost now. Regarding the redirect to the landing page, yes I had implemented that initially thinking that it was the only way to get to the landing page but it seems like it is unnecessary now so I have removed it and am able to get to the actual game now. 

The main purpose for doing this exercise was to see if I can get the Mturk ID of the worker into my gameplayprac.csv file, which I create in logic.js. That is still not happening - if I uncomment the gameRoom.computeBonus code in logic.js I still get the error message and the game doesn't run, see below. I've pushed the updated code to the repo just to make sure you have the latest version to look at (without the redirect to the landing page). I may be getting confused about something again. 

Also - while the game runs, I am not taken to the landing page, but directly to the first stage of the game when I do  https://localhost:8080/moveablock/xyz.html - is that expected? 

Let me know what you think, thanks!

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMnnAfSWNCdCEn1pgSReNZ4Xhp%3DSy42jsPr56S%2BxPLZ%2BcQ%40mail.gmail.com.

[Stefano Balietti]

Hi,

gameRoom.computeBonus must run during the game, that is inside a step callback. I tried this:

    stager.extendStep('instructions', {
        cb: function() {//create view for results storage
            gameRoom.computeBonus({
                amt: true,
                dump: true,
                addDisconnected: true

                // other options depending on your game situation.
             });
         }

    });

and it works.

--> Also - while the game runs, I am not taken to the landing page, but directly to the first stage of the game when I do  https://localhost:8080/moveablock/xyz.html - is that expected?

Yes, but you can access the landing page by opening the page from the file system (instead of localhost). So the address bar starts with File:// something.

Cheers,

Stefano

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAGGPeq%3DwEBCuv0D%3DpmXkZfJhi840mbBOWh4TZdjAZ%2Brqm_tTqA%40mail.gmail.com.

[Abhilasha Kumar]

Hi Stefano, thank you - I didn't realize that computeBonus needs to be inside a step callback! The bonus.csv file is now being generated, whew! I will test this on the sandbox without the default values just to make sure it still runs, but I think I have a good idea of the different moving parts now. Thank you so much for your help! 

Best,

Abhilasha

To view this discussion on the web visit https://groups.google.com/d/msgid/nodegame/CAMSNNMmBhaxDmAgBSwvs0d%3DLt7kGj9hu%3DDcCt0Yk0Fkx2B9H5g%40mail.gmail.com.