Component result data says 'none' but state is FINISHED

[beckyann...@gmail.com]

Hi JATOS team,

We're running an experiment and finding that sometimes the component result data says 'none', but the component's state is FINISHED. Here's an example of what it looks like:

So far this is only happening occasionally with one particular study component, and in all cases, the participant doesn't finish the study. (In this example, I think the participant must've used the back button to go back to the first component.) We have several participants who did the whole study, and the data is saved successfully for this component. Some of our participants have slow/unreliable connections, so it might be related to that, or to reloading the page or using the back button. 

What we're confused about is why the component state is FINISHED even though there's no data? I thought that using the 'jatos.submitResultData' function would guarantee that the results are saved before marking the component as finished? At first I thought this must be the JavaScript problem I ran into before, where the success callback function was being being called immediately, which caused the next component to load before the data saved. But I don't think that's the problem this time - here's how we're sending the data back:

jatos.onLoad(function () {

        jsPsych.init({

            timeline:timeline, 

            on_finish: function() {              

                var interaction_results = jsPsych.data.getInteractionData().json();             

                var task_results = jsPsych.data.get().json();            

                var all_results = "["+task_results+","+interaction_results+"]";                

                jatos.submitResultData(all_results, jatos.startNextComponent);

            }

        })

    })

Does anyone have any thoughts on why we're getting component state = FINISHED with no data?

Thanks!

Becky

[Kristian Lange]

Hi Becky!

Your JavaScript looks fine. And according to your code the result data should be at least "[]" even if task_results and interaction_results are empty.

But which version of JATOS do you run? There were some changes how jatos.js deals with the sending of result data about a year ago.

Best,

Kristian

[beckyann...@gmail.com]

Hi Kristian!

Sorry, I should've mentioned that we're still using JATOS v3.3.3 (planning to upgrade soon!). I did look through the release notes for all of the releases since v3.3.3, but didn't see anything that looked relevant to this, but of course I might've missed it.

Do you mind explaining what might be happening here, or pointing me to the relevant GH issues or changes to jatos.js?

Thanks very much!

Becky

[Kristian Lange]

Hi Becky,

Sorry for taking some time to answer, it's a bit chaotic here in the moment.

In v3.3.3 jatos.js didn't have the 'HTTP Loop' yet. This takes care of sending result data and other HTTP requests to the JATOS server while providing some guarantees like keeping the order of the result data sent (important with jatos.attachResultData) and that all data being sent before the component finishes. It's a really useful thing to have, especially if you use OpenSesame/OSWeb experiments.

But I had a look at the old version (v3.3.3) of jatos.js and even there your example cannot lead to a FINISHED state and 'none' result data. Extremely simplified it looks like this (in v3.3.3):

jatos.submitResultData = function (resultData, onSuccess, onError) {

jatos.jQuery.ajax({

...

success: function () {

// call success callback function

},

error: function (err) {

// call error callback function

}

});

};

It waits for the submitting of the result data and only then calls the jatos.startNextComponent callback function. And since your result data are at least '[]' I can't see a way for the result data to be empty. But maybe you have another jatos.submitResultData in your code somewhere? This one can overwrite the data. Or you have another jatos.start*Component somewhere that finishes the component prematurely? Another thing: is there something weird in the logs around this time?

Anyway, updating your JATOS is a good idea. This 'HTTP Loop' takes extra care for result data sending.

Best,
Kristian

[beckyann...@gmail.com]

Hi Kristian,

No problem at all - I can completely relate to things being chaotic right now :/

> But I had a look at the old version (v3.3.3) of jatos.js and even there your example cannot lead to a FINISHED state and 'none' result data.

Interesting! I hadn't seen this bit of code but assumed it worked this way, which is why I'm so confused.

> But maybe you have another jatos.submitResultData in your code somewhere? This one can overwrite the data. Or you have another jatos.start*Component somewhere that finishes the component prematurely? Another thing: is there something weird in the logs around this time?

Thanks very much for these suggestions for things to check. I'll look into this and post here if I have any updates.

Best,

Becky

[beckyann...@gmail.com]

Follow-up for anyone else who has this problem: we're pretty sure this is caused by the participant using the browser's back button to go backwards through components. Apparently when the participant clicks the back button, the component will be marked as FINISHED even though there's no data for the component (it says 'none' in the results preview box). We mistakenly thought that leaving the 'allow reloads' checkbox unticked would prevent participants from going backwards via the back button, but it really just prevents page reloads (as the name suggests!). In later versions of JATOS, there's a 'linear study flow' checkbox that can be used to prevent participants from using the back button to go back to previous components.

Beck

[Kristian Lange]

Sorry for not thinking of the browser's back button earlier - I was too much focused on the reload that I forgot about this possibility :) .

And I want to add that in newer JATOS version there is also a warning popup whenever the participant wants to leave the current component unexpectedly, including via the back button. (http://www.jatos.org/jatos.js-Reference.html#jatosshowbeforeunloadwarning).

But I can see that the state FINISHED is kind of misleading. JATOS automatically sets this state although the component was never 'officially' finished (e.g. with jatos.startNextComponent or jatos.endStudy). I'm thinking about whether we should introduce a new state for those circumstances, something like ABANDONED. It's definitely not a FAIL since nothing went wrong and not an ABORTED since it wasn't actively canceled. Or we do not set any state at all and just leave the last state, which is probably DATA_RETRIEVED or RESULTDATA_POSTED.

Best,

Kristian

[Elisa Filevich]

I vote to keep the last state, like DATA_RETRIEVED etc

Elisa

-- 
You received this message because you are subscribed to the Google Groups "JATOS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jatos+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jatos/17500fb4-10a8-4be1-b13c-958c9cba2f67n%40googlegroups.com.

[beckyann...@gmail.com]

No problem Kristian! And yes, I think the fact that the component has the FINISHED state is what was so confusing for us. So it would be great if that could be changed in a future version. It would be nice if the state could indicate that the participant used the browser's back/forward button, in the same way that reloaded components are flagged. But if that's not possible then I think keeping the last state also makes sense.

Thanks very much!